HTML 5 video is coming, but which codec is going to be used to deliver it?

Internet Explorer, Safari and Chrome have chosen on H.264, while Firefox and Opera are going for Ogg Theora. For us developers this sucks, because there is no one codec we can bet on to work in any browser. There's flash, which is more ubiquitous than any single browser, but it's a whole new toolchain to learn, and in my opinion it's a declining technology which will one day join the ranks of shockwave and >applet<. The future is in HTML, so what to do? Unfortunately there's no easy answer, so the most I can do is give an overview of what is going on today.

Choose multiple

First and foremost, keep in mind that the standard for the <video> tag allows multiple codes. You can easily specify a video tag with both an ogg vorbis and an H.264 source. This works as follows:

<video controls="true">  
   <source src="video.ogg" type="video/ogg" />  
   <source src="video.mp4" type="video/mp4" />
  
   Alternative content

</video>

There are big drawbacks to this approach though. Having 2 different files means it will take (about) twice as much diskspace, cpu, energy and time to encode. So there is definitely benefit in having a single filetype that works everywhere.

H.264

H.264/MPEG-4 AVC seems to have the most backing at the moment, getting support from Chrome, Safari and Internet Explorer. H.264 is widely regarded as technically superior. What this means beyond anything else is that it has the best quality/compression ratio. It's encoders are better, there's plenty of competition and it's already widely in use.

The big issue is that H.264 is owned by the MPEG LA organization. Technology within the H.264 is patented by many different companies, which all came together in MPEG-LA. There's very little chance H.264 is becoming an open standard.

This doesn't mean it's not free to use today. MPEG-LA initially planned to expire the current royalty-free license this year, but it's been extended until 2015. A very smart move. Now H.264 uptake is going through the roof, it's a much smarter idea to wait until everybody has bought into H.264 and then charge everybody for using it.

This can lead to similar situations to what happened to the GIF image format (reference), in which Unisys wanted to charge intranets $5000 for using gif images.

OGG Theora

The alternative is choosing for Theora. If you choose for Theora there's no risk (provided there's no future patent claims on the technology) you will get sued 5 years in the future for putting up video on the web. Theora was developed as an open standard.

From this perspective it's understandable why an open source company like Mozilla would go for Theora over H.264. It might be technically not as great as H.264, but at least the entire internet won't get a huge bill from MPEG LA 5 years in the future.

What completely amazes me is that the other browser vendors do not provide support for OGG. I understand there's a preference for H.264, but there is absolutely no reason to not provide support for multiple video codecs. Even if the browsers just interfaced with the media layers on the respective operating systems (GStreamer, Core Video, DirectShow). it would allow users themselves to install the codec. You can disagree with Mozilla, but they're making an ideological standpoint. The other vendors however, I can't think of a reason why they wouldn't also want to support ogg.

And then there's VP8

VP8 (and VP1 - 7) has been developed by On2. On2 has quite a reputation. VP3 was released as an open codec, which ended up becoming the basis for Ogg Theora. VP6 is one of the major codecs in the Flash Player since 2005, and VP7 is the video codec used by Skype. Technically it's supposed to match H.264.

Google 'finalized' buying On2 in february, and since then it controls VP8. There's been many speculations about google buying them to open source the codec and make it the winning standard. Even the FSF has written an open letter asking them to do just this. This would be great news for the web, because it would provide a codec that meets all the interested party's requirements. Just yesterday newteevee reports this might actually come to happen, and it should be announced at Google's I/O conference.

I'm worried though. Microsoft tried to make it's own codec (VC-1) the big video standard. It actually believed at one point it completely owned the standard, until the MPEG-LA members got an eye on it. It turned out Microsoft infringed patents of over 16 different patent holders (reference). I'm willing to bet microsoft has more lawyers on staff than On2 had employees. I can't imagine On2 has not made similar mistakes.

If Google indeed tries to release VP8 as an open, royalty free standard, I would be surprised it doesn't turn out that it didn't even own it in the first place.

One last word

Don't forget that the only reason we are in this mess in the first place, is because it's legal in most countries to claim ownership of ideas presented as software. Whether or not software patents could make sense from a theoretical point of view, in practice it's preventing innovation because large corporations have the legal advantage.

This problem would not exist if it weren't for software patents.