Encoding Video For Mobile Devices?

MadGeek007 writes "I am developing an app for Android that will use many short (averaging 10-20 minutes) instructional videos. Unfortunately, I know next to nothing about encoding video. I'd like to use a codec that is supported by Android and iOS out-of-the-box. I need the videos to look decent on large mobile displays (IPhone 4, HTC EVO, etc.), and still be able to stream well on a good 3G connection. The sound quality is also important. With so many different display resolutions on mobile devices, do I need to encode multiple copies of the same video? Or can I get away with a one-size-fits-all video? Can anyone recommend encoding software, codecs, resolutions, and bitrates that would work best for this application?"

Comments

Handbrake

[jacoplane]

Handbrake is what I use:

http://handbrake.fr/

Re:Handbrake

[SquarePixel]

That and output as H.264 which is really the only choice.

Re:Handbrake

[Anonymous Coward]

I also use handbrake. I don't own a smartphone so I'm going from guess work here.

Video codec i would go with h264, that is pretty much the standard right now. Handbrake uses x264 by default. You can get advanced and pass specific options to x264 via handbrake. However some players don't support advanced x264 features. I would start with the apple presets and work from there.

Audio codec mp3 or aac. The rate varies, i would say ~160 for mp3 and ~128 for aac. Again trial and error will work best. Mixdown stereo, i wouldn't putz with any of the dolby features.

Screen resolution is an issue I am completley guessing at. I'm assuming That the players scale resolution up or down. So I would render one low def, and one high def. I don't know the phones aspect ratios. High def being near evo 4g quality low def being errm low. You could possibly just do one medium definition but I don't know if the players will choke on higher resolutions.

For encoding video you have several choices
Target Size, Bitrate, or constant Quality.

Target Size usually looks like crap in my experience.

Bitrate allows finer grained control but you really should do 2 passes on it for better quality. So in turn it takes longer.

Constant quality works at variable bit rates to my understanding, and it only requires one pass. This is what I use. I rip all my dvds at RF=23. Lower number is higher quality. Due a couple small clips to figure out end size. But i would recommend starting with constant quality.

Re:Handbrake

[imbaczek]

mod parent up! also, use a nightly build for extra x264 goodness.

Re:Handbrake

[krick-zero]

Fair Use Wizard is pretty decent.

It costs money, but they released a free full version 2.8 a while back that you can still find on this page...

http://www.videohelp.com/tools/FairUse_Wizard

Re:Handbrake

[bemymonkey]

Android's support of H.264 is surprisingly good. As a content provider, it's hard to go wrong for any iPhone or Android phone with H.264.

Consumers, on the other hand, get the short end of the stick... those hundreds of gigs of DivX everyone has laying around are useless on many Android devices, and even software-based decoder-players aren't enough because they need max CPU permanently, draining the battery nearly as fast as Flash on an iPhone (at least according to Steve Jobs :p)...

Re:Handbrake

[cynyr]

only reason i disagree with you is i have 2 devices at home that will play mkv's, both are computers(PCs running vlc/mplayer). All of my portables and other devices(most importantly my ps3) won't play anything inside a mkv even if it is a compatible video/audio stream. Also i've had no luck doing "ffmpeg -i dumb.mkv -ac copy -vc copy useful.mp4". I always get some sort of error, about not being able to pack the h264 video into the mp4. I haven't tried, ripping them out, and then using mp4box, or ffmpeg to put them back together.

Re:Handbrake

[teh31337one]

Consumers, on the other hand, get the short end of the stick... those hundreds of gigs of DivX everyone has laying around are useless on many Android devices, and even software-based decoder-players aren't enough because they need max CPU permanently, draining the battery nearly as fast as Flash on an iPhone (at least according to Steve Jobs :p)...

My Samsung Galaxy S i9000 android phone (European version) plays DivX natively, without any dropped frames. :D

Re:Handbrake

[JavaBear]

I prefer MKV, simply because it is one of the few to support an entire video package, with multiple audio tracks, subtitles and chapter marks.

Multiple versions

[mikael_j]

While it does mean you'll use up a little extra storage it is probably best to encode one version for each resolution, h.264 tends to be the standard in video these days (especially for mobile devices since they tend to have h.264 decoding hardware).

By using one resolution per device (or at least for the more common devices and then a couple of fallback resolutions) you ensure the best possible quality for the largest number of users while also avoiding wasting a lot of bandwidth streaming high-res iPhone 4-res video to some other phone just because you didn't want your video to look like crap on the iPhone.

Re:Multiple versions

[mikael_j]

Well, 480p isn't high enough to match the resolution of the iPhone 4 and it would have to be upscaled which looks significantly worse than native resolution (it does, really, there are plenty of users who dislike this even if you happen to be one of the loudmouths who claim it looks "just fine").

Obviously higher res video needs higher bitrates to look good.

And finally you don't have to waste bandwidth by streaming the same 960x640 high bitrate video to every user just to make sure that the iPhone 4 users don't get a video that looks like crap on their device. You also lower the risk of having lower end devices choke on videos with a bitrate that's too high for them to handle.

Re:Multiple versions

[Kylock]

Going lower than this is silly. The OP is asking about iOS and Andriod phones specifically. If were going to talk about the newest hardware, I'm not sure about the iPhone 4, but the HTC Evo and the Moto Droid X both support 720p video. HTC Just recently announced a new phone where they plan to support 1080, although it seems like overkill for such a small screen, I think they are thinking more people will use the hdmi outputs available on those phones.

Maybe consider doing 720p and 360p versions if you decide to do 2 encodes instead of just one.

MPEG-4, H.264

[dorzak]

For iOS pretty much the only choice is MPEG-4, H.264.

I believe Android supports it as well.

Apple actually has a decent section in their web developer section on developer.apple.com about what is supported for iOS.

Supported media

[LiENUS]

The android developers site has an excellent list of supported media formats. http://developer.android.com/guide/appendix/media-formats.html The iphone 4 specifications http://www.apple.com/iphone/specs.html claims that the iPhone 4 supports AAC-LC and h.264 which android supports as well. So looks like you have an easy match for high quality as well.

mediacoder

[ceraphis]

As others mentioned, make it an h264 video with aac audio. I suggest using mediacoder, a free encoder with a billion options, including preconfigured iphone profiles I believe. Others suggest handbrake but I've found in the past that mediacoder looks like it has a lot more options to fiddle with. YMMV though, I've read handbrake has come a long way since ditching the "only encode DVDs" thing It used to do exclusively.

Re:mediacoder

[Warll]

It should be mentioned that MediaCoder is in the ffmpeg hall of shame: http://www.ffmpeg.org/shame.html (I cannot link any more direct because ffmpeg's bug tracker uses a self signed cert)

Judging from the related bug tracker the author appears to almost be playing dumb.
I agree that it is a fine program and I myself have suggested it to non-tech savy friends, but that doesn't mean I feel good about doing so.

Re:mediacoder

[dotancohen]

Here's the bug:
https://roundup.ffmpeg.org/issue1162

The MediaCoder author is asking what he violated and the FFmpeg dev is just telling him to RTFL (license), all of it. The FFmpeg dev is being a jerk, too, and calling him names. The MediaCoder author even mentioned that he asked on the mailing list before distributing and got told that him usage was fine.

MediaCoder might be violating the license, but it looks like an honest mistake that the author wants to rectify. The FFmpeg dev would rather call him names than show him what he did wrong.

Re:Android: WebM; iOS: H.264

[Anonymous Coward]

Or better just use H.264 for both.

Audio: speech or more?

[dingen]

The sound quality is also important.

You say you are making instructional videos, which implies to me the audio will contain mostly speech. If that is indeed the case, then a low bitrate like 64 kbps in mono will probably suffice. Encoders like MP3 or AAC are very good at keeping speech intelligible at lower bitrates.

Re:h.264 takes too much juice to play

[willy_me]

It will use less power to play h264 in hardware then MPEG2 in software. And all of the phones will have hardware h264 decoders.

QT-AVIdemux

[gagol]

I am under Ubuntu and uses QT-AVIdemux and uses MENU: Auto-> Apple-> Apple Ipod

Works everytime.

DO NOT USE THE GTK VERSION as the auto facilities are not included.

other concerns

[YrWrstNtmr]

As others have suggested, handbrake + h.264

But my thought is, 10-20 minute instructional videos? Especially on a mobile device?
Break it up into 2-4 minute segments. No one is going to watch a 20 minute video, and retain what was in minute 0-6. zzzzzzzzzz

Re:other concerns

[Anonymous Coward]

Not to mention he can get more advertising revenue this way.

Re:It's a shame, the out-of-the-box requirement.

[pslam]

This is what I mean by misinformation:

I think he wants to make it through the entire 20 minute instructional video on battery... Enjoy your horribly wasteful software decoding, or use h.264 which has very liberal and cheap licensing requirements (in fact, you won't have to pay anything as the hardware decoder is already paid for by the hardware manufacturer, and you don't owe anything for encoding the video, look it up)

Do you realize how little CPU it takes to do QVGA decode software-only? Depends on the handset, but 10-30% is realistic. Do you know how much battery impact that has? Ball-park 100mW. Very little compared to the backlight or OLED (0.5W) and an order of magnitude less than the power a continuous 3G link is taking (1-2W).

H.264 has additional license fees for professional use. Yes, most people ignore that.

You continue to use the fallacy that Theora is worse as a reason not to use it. QVGA is not horrible on a modern smartphone - it is perfectly acceptable on a screen that's barely 4 inches across. The different between Theora and H.264 everyone overstates is for high bitrate, high profile, high resolution videos. This is none of the above, and even if it was, under fair analysis (not H.264 high profile which even a 3GS doesn't do) it's about 20% for "HD" video. Ths isn't HD.

48kbit MP3 is perfectly reasonable when it's mono. Next you'll tell my 128kbit MP3 stereo isn't acceptable. Come on, this is coming out of a handset speaker. You obviously haven't tried 48kbit or you wouldn't make such a ridiculous statement. Me, I've been in the codec business (and writing them) for over a decade.

I'm not even suggesting anyone use Theora/Vorbis as a solution. My gripe is that it could so easily be a neat solution. But isn't because, well, there's a ton of misinformation like yours around.

The truth - and if you'd ever tried it you wouldn't even question this - is that QVGA resolution is fine, 48kbit mono audio is fine, Theora is more than fine, and battery impact is negligibly different between codecs at these settings. I would love to know WHY people even think otherwise, because I'm trying hard to combat the spread of overkill like this. Who is "educating" folks with this?

Re:It's a shame, the out-of-the-box requirement.

[pslam]

This generalization is wrong. Verses H.264, Theora does not perform well at any bitrate or resolution. Those who claim otherwise simply don't like H.264 because it's proprietary or have been given wrong information, like yourself.

Define "well". That's suitably vague. I put a figure on it: 20%. What's 20%? That's error power difference.

That's still perfectly usable. In some cases you can just use a little more bit rate to make up the difference, and depending on what that costs to you (bandwidth/storage/battery) that's not a problem. In some cases you don't even need that extra 20% and you can live with - or not notice - the difference.

Again, bear in mind the subject matter: these are instructional videos, NOT some glorious multimedia entertainment experience.

I'm dismayed that the consensus seems to be he needs H.264 720p 256kbit 5.1 AAC multi-megabit videos. What a waste. 240p mono would do just fine, and has a better chance of working over 3G even in marginal areas. Has everyone lost touch with reality, and not to mention pragmatism?

ISO MPEG-4 H.264 Baseline Profile

[gig]

Encode your video in H.264 Baseline Profile and it will play on everything. Baseline Profile is DVD-quality, 640x480. Players with smaller screens will scale it down. The bigger HD H.264 profiles will not play on everything, because not all devices can play HD yet. Devices with smaller screens will scale the video down.

You don't get to express your individuality with a choice of codec, because video consumers only have one: H.264. If your video is not H.264, the consumer cannot see it. That is the reason H.264 exists, that is why it has that ISO/IEC name starting with an H instead of its previous name, which was Advanced Video Coding. Making a universal meeting place for video content is why we have ISO standardization of video codecs, so there is a common playback and capture codec for consumers. In the same way that you had to use MPEG-2 video on a DVD Player, you have to use MPEG-4 H.264 video on the video players has succeeded it: iPhone, iPad, Blackberry, Android, Palm, Blu-Ray, iTunes, iPod, Zune, YouTube (although they will transcode nonstandard codecs to H.264 automatically), QuickTime Player, FlashPlayer, Safari, Chrome, IE9, Mac OS, Ubuntu, various set-tops and other devices.

To stream well on 3G it will have to be very low-bandwidth. Typically, a version is encoded for Wi-Fi and a separate version for 3G.

Apple has a lot of information about video authoring and encoding for mobile devices on their developer site. Apple has forgotten more about this topic than most companies will ever know: QuickTime is the backbone of audio video authoring, MPEG-4 is a standardization of the QuickTime file format, Final Cut Pro is the most popular pro video editor, iMovie is the most popular consumer video editor, WebKit the most video-savvy browser core, and of course they run iTunes Store are the maker of the iPod. So you can follow their advice and get the job done right. Their advice will also apply to Android and other smartphones because when it comes to video they are all iPods.

http://developer.apple.com/