How to encode video for mobile use

The bad news first. There are hundreds of mobile devices out there, and it's basically impossible to support 100.0% of them. The good news is that mobile devices are getting better. Gone are the days when 176x144 3GP was synonymous with mobile video. Modern smartphones can actually play high quality video, and smartphone use is increasing. That's not to say that 3GP is over, or that everyone has a smartphone. But smartphone use is growing (at a 64% annualized rate as of Q2 2010), and, not surprisingly, smartphone users are more likely to watch video on their phones than their, er, dumb-phone-using friends. So if you want to support 90%+ of mobile devices, you need at least two video types: 3GP + MPEG-4 for less sophisticated devices, and H.264 + MP4 for smartphones. That's good news, really. One output video can cover all of your smartphone users - iPhone/iPad/iPod, Android, and (for the most part) Blackberry too. Toss in PSP, PS3, and Xbox 360 for good measure.  (If you want to learn more about encoding for iOS specifically, check out our comprehensive iOS encoding guide.) Of course, while one universal smartphone output can take care of most smartphone users, you can do better with multiple mobile outputs. For example, the iPad has a native resolution of 1024x768, fully five times higher than the 480x320 on earlier iPhones. So if you encode your video at 480x320, you'll be missing out on the near-high-def capabilities of the iPad. Fortunately, you can target mobile devices well using a handful of standard encoding profiles. Start with the Universal Smartphone Profile for wide compatibility; add in an Advanced Smartphone Profile version for the more advanced devices; and round out your mobile list with a legacy profile for widest compatibility - either our Legacy Smartphone Profile (below), or even a 3GP video for even wider compatibility. Note that the following defaults are the starting point for these profiles. Zencoder uses these settings by default, but you can replicate them easily enough in whatever encoding tool you're using. Defaults: Video: H.264, Level 3.0, Baseline profile Audio: AAC, 1-2 channels

1. Universal Smartphone Profile

This is a great starting profile for wide compatibility with modern smartphones. Plays on just about everything, though it doesn't take advantage of the higher resolutions and codec complexity possible on the newest crop of devices.

Plays on:

iOS: iPhone, iPad, Apple TV, iPod Touch, iPod Classic, iPod 5.5G Blackberry: Bold 9000, Curve 8910, 8900, 8520, Pearl 9XXX, Storm, Storm 2, Torch, Tour, Bold 9650 + 9700 Android: All (?) Other: PSP (3.30+), PS3, Xbox 360, web

Doesn't play on:

iPod 5G, PSP (pre-3.30), Blackberry Curve 9330, 9300, 8530, 83XX, Pearl 8XXX, 88XX

Settings:

Defaults, plus: audio_bitrate: 128 (or less) audio_sample_rate: 44100 (or less) size: 480x320 max_frame_rate: 30 video_bitrate: 1500 (or less)

1b. Universal Smartphone Profile B: higher resolution

This profile plays better on iPhone 4g, iPad, Apple TV, new iPod Touch, Droid, PS3, and Xbox, by increasing the video resolution. The extra pixels are wasted on older iPhones though, and make for a video that won't play on Blackberry and some Android phones.

Plays on:

Everything above, minus Blackberry and maybe weaker Android devices

Settings:

Universal Smartphone Profile (above), plus: size: 640x480

2. Advanced Smartphone Profile

Newer iOS devices allow higher resolutions and higher encoding complexity (which means better compression). In particular, iPad and Apple TV users shouldn't have to watch 480x320 video on their beautiful screens, so it makes sense to provide a higher quality version if you want to provide a good experience to these users.

Plays on:

iOS: iPhone 4G, iPad, Apple TV*, newer iPod Touch Android: Nexus One, Droid, maybe others. (YMMV on these, though. Some users report trouble with 720p video.) Other: PS3, web

Doesn't play on:

iOS: iPod 5G/5.5G/Classic, iPhone 3GS and before, older iPod Touch PSP, old Apple TV* Blackberry: all Android: others Other: PSP, PS3, Xbox 360, web

Settings:

Defaults, plus: h264_profile: main h264_level: 3.1 audio_bitrate: 160 (or less) audio_sample_rate: 48000 size: 1280x720 (max) or 960x640 (iPhone 4 native) max_frame_rate: 30 video_bitrate: 5000 (or less)

*2b. Advanced Smartphone Profile B: with old Apple TV compatibility

To support older Apple TV devices, use the Advanced Smartphone Profile setting, plus one of the following:

Settings:

Advanced Smartphone Profile (above), plus either one of the following: size: 960x540 OR max_frame_rate: 24

3. Legacy Smartphone Profile

This profile plays on the last major set of H.264-based mobile devices: notably, older iPods and some Blackberries. The tradeoff is significantly smaller video: 320x240, at no more than 768kbps.

Plays on:

Everything above, plus: iPod 5G, PSP (pre 3.30), Blackberry Curve 9330, 9300, 8530, 83XX, Pearl 8XXX, 88XX

Settings:

Defaults, plus: audio_bitrate: 128 (or less) audio_sample_rate: 44100 (or less) size: 320x240 max_frame_rate: 30 video_bitrate: 768 (or less) h264_level: 1.3

4. Legacy 3GP Profile A and B

Finally, a 3GP profile or two will extend support to many remaining mobile devices. Notably, you can use these on most of the same devices supported above under the Legacy Smartphone Profile. So if you're encoding a 3GP video at 320x240, you might not need to encode another H.264 video at 320x240. Note that 3GP video support is still in beta at Zencoder. Finally, note that these videos will look terrible, but that's the cost of supporting 3GP phones.

Plays on:

Hard to say. There are thousands of types of 3GP devices, and each one is a little different. Consider these a starting point.

Profile A

format: 3gp video_codec: mpeg4 size: 320x240 aspect_mode: pad frame_rate: 15 upscale: true video_bitrate: 192 bitrate_cap: 192 audio_bitrate: 24 audio_channels: 1 audio_sample_rate: 16000

Profile B

format: 3gp video_codec: mpeg4 size: 176x144 aspect_mode: pad frame_rate: 5 upscale: true video_bitrate: 52 bitrate_cap: 58 buffer_size: 16 audio_bitrate: 16 audio_channels: 1 audio_sample_rate: 16000

Summary

If you want to create mobile video, start with the Universal Smartphone Profile. For better quality, supplement this with Advanced Smartphone Profile video. For wider compatibility, add a Legacy profile or two using either MP4 or 3GP. That's it. It only takes 1-3 profiles to support most mobile devices.

Edits

Older iPhone/iPod devices ask for the "H.264 Baseline Low Complexity" profile. "Low Complexity" isn't actually a H.264 standard - it actually just means "only 1 reference frame". The jury is out on how much Apple devices really enforce this, but for true compatibility, you should probably use Baseline profile and limit reference frames to 1. You can do this at Zencoder with the new h264_reference_frames setting. November 23: A few people have asked about Palm Pre video. The published specs for Palm Pre are very similar to other smartphones: 480x320 native resolution (with 640x480 supported); H.264, H.263, or MPEG-4 video; MP3 and AAC audio (along with a few other codecs). If these specs are accurate and comprehensive, then the Universal and the Legacy profiles above should work on Palm Pre. I don't have easy access to a Pre, so if anyone wants to test this for us, we'd be happy to give you some free encoding at Zencoder to use for testing. Get in touch if you want to help out. January 24: In order to deliver 3GP video as an RTMP stream it needs to be "hinted". Add "hint": 1 to your API request to enable it.