The patent thicket problem
A couple of weeks ago I stated my conviction that there's no such thing as an open video codec that can be guaranteed to be unencumbered by patents. I regret to say this, but like I wrote then, the field of multimedia formats is a true patent thicket. Ed Bott, a ZDNet blogger, counted 1,135 patents from 26 companies just in the H.264 pool. That is only one of the multimedia standards MPEG LA commercializes, and there are patent holders who don't work with MPEG LA but who may also have rights that are relevant to Theora (or VP8, for that matter).
Those 1,135 patents refer to registrations in a total of 44 different countries (with different numbers in each country). So there are certainly many duplicates in terms of the scope of the patent claims. Nevertheless, even just a fraction of 1,135 is still a huge number considering that it's about a single codec.
Contrary to popular misbelief, patent law doesn't stipulate a 1-to-1 relationship between patents and products. In the pharma sector there is sometimes only one new patent on a given product, or maybe two or three. In software, every little step of the way is, at least potentially, patentable. That's why even a codec like Theora or VP8 might, although it has a different background, infringe on some MPEG LA patents.
The Xiph.Org Foundation's president, Christopher 'Monty' Montgomery, wrote that if Steve Jobs' email was real, it would "strengthen the pushback against software patents". I'm afraid there isn't enough of a pushback out there that would really be needed to bring about political change in that regard (because small and medium-sized IT companies aren't truly committed to the cause), but now that more and more people do look into the threat that patents pose to FOSS (and other software), there will be greater awareness for the patent thicket problem and for the fact that patent law creates huge numbers of little monopolies as opposed to serving to protect completely functional products or technologies.
The question of relative safety in patent terms
I agree with the FSFE's president, Karsten Gerloff, that "[j]ust because a standard calls for licensing fees does not mean that the users are safe from legal risk". It's true that there might even be patents that could be asserted against H.264 but aren't under the control of MPEG LA, for reasons such as the ones I outlined recently.
However, given the extremely widespread commercial use of H.264, including some of the prime targets of patent trolls, the fact that no such patents have been asserted against H.264 licensees so far is a fact that certainly makes a number of people reasonably comfortable. While there is also significant use of Theora (and related technologies), its adopters aren't nearly as attractive targets for patent trolls as the users of H.264. Besides patent trolls, there are those large commercial holders, and as I explained before, the MPEG LA pool is so big that problems for Theora are in my opinion not outside the realm of plausibility.
The question isn't how attractive a target Theora has been so far. If it was elevated from its current status to a part of the HTML 5 standard, we'd be talking about a commercial relevance that is easily 100 times greater.
The need for consensus in the HTML 5 standard-setting discussion
HTML 5 is an extremely important leap forward and the W3C certainly wants to achieve consensus that results in consistent support by the major browser makers. This is also in the interest of web developers and website operators. If the W3C imposed Theora as the standard video format against the concerns the leading proprietary browser makers voice, this could result in inconsistent implementations of what should become a common basis for all browsers. That, in turn, would mean a lot of potential hassle for the web community.
The market relevance of Apple and Microsoft is significant enough that even without the patent uncertainty argument the preferences and positions of those vendors must be taken into account by the W3C. Their support for H.264 doesn't mean they leverage their relevance as platform companies in order to push another product of their own. H.264 is a multi-vendor patent pool, and of the 1,135 patents in the H.264 pool, Apple contributed only one and Microsoft only 65 (less than 6% of the total), according to Ed Bott's count. Both companies are also H.264 licensees and, quite plausibly, net payers (getting charged more license fees for their own use than the share of MPEG LA income that they receive). They may very well have strategic reasons for which they favor H.264, but that would be another story.
The burden of proof in the HTML 5 standard-setting discussion
I can understand the frustration of FOSS advocates and, especially, the Xiph.Org Foundation that some companies make references to uncertainty surrounding patents Theora may infringe without telling the public which those patents are. At the same time, I don't think anyone could have expected Steve Jobs to include a list of patents in that email about open-source codecs, which was just a high-level explanation of his views.
Unfortunately for Theora's developers and other supporters, there is no such thing as a burden of proof on browser makers saying they're uncomfortable with Theora because of patent-related uncertainties.
If the proponents of Theora want to disprove the "uncertainty" argument, they can't just refer to the fact that nothing has happened yet. If Theora was elevated to a part of the HTML 5 standard, the resulting adoption would represent a fundamental change of the situation.
Unfortunately, it's easier to make the case for than against a possible infringement of patents by a given piece of software. If a patent holder wants to document an infringement, there are different formats, the most popular one being a so-called claim chart. If the Xiph.Org Foundation and its allies now wanted to show that Theora doesn't infringe on any patents, they'd have to theoretically look at every patent out there.
That wouldn't be possible, but how much of an effort would be reasonable?
Under normal circumstances I believe one couldn't expect an open-source project to undertake any patent clearance of this scale. However, if companies such as Google and Opera and a formal non-profit with very deep pockets such as the Mozilla Foundation push for a standards decision with far-reaching implications for the whole industry, then I don't think it would be unreasonable to expect that they should at least look at the patents in the MPEG LA pool and perform patent clearance for Theora with respect to those.
As long as they don't make that kind of reasonable best effort, their argument about Theora being patent-safe amounts to "trust us". I said that I agree with the FSFE that the availability of a patent pool doesn't guarantee that the pool is complete. Nor does the opposite situation (developers electing not to take out patents) guarantee anything.
I don't know what Theora's proponents and opponents laid on the table in internal discussions at the W3C level. What I just wrote is based on the public debate. Also, what I wrote about Theora would equally apply to VP8 if Google proposed its inclusion in HTML 5 (after possibly open-sourcing it).
Is H.264 licensing a practical alternative for FOSS?
I asked MPEG LA, the patent pool firm that manages H.264 and other codecs, whether it would -- hypothetically speaking -- be possible for Mozilla (the maker of Firefox) to license H.264 and then make it available to everyone on Free and Open Source Software terms including the right for users to include the code in derived works. This is the answer MPEG LA gave me:
MPEG LA’s purpose is to provide voluntary licenses of convenience to users enabling them to have coverage under the essential patents of many different patent holders as an alternative to negotiating separate licenses with each. The licenses are nonexclusive and limited to coverage in connection with the applicable standard (e.g., AVC/H.264) being licensed. Therefore, although MPEG LA does not regulate this space directly, as you point out, users are not authorized to use the licensed technology beyond these limitations without payment of applicable royalties or other licenses from patent holders permitting such use.That answer doesn't mention Free Software or open source, but it clearly reaffirms that "users are not authorized to use the licensed technology beyond [certain] limitations without payment of applicable royalties or other licenses [...]", and such limitations aren't compatible with FOSS licenses. They go clearly against both the Free Software Definition and the Open Source Definition with their respective prohibition of discrimination against certain types of use and the requirement to allow such use free of charge.
Under our AVC License, the Licensee is the party providing the AVC end product in hardware or software. Therefore, for products where Mozilla is the Licensee, it would be responsible for paying the royalties and notifying users of the License coverage, and where other parties are Licensees, those responsibilities will fall upon them. In normal usage such as personal use, no additional License or royalty is necessary because applicable royalties are paid by the end product supplier, but additional License rights may be required where the codec is used for other purposes such as subscription or title-by-title sale of AVC video.
While it's clear that code made available under a FOSS license couldn't practically implement H.264, the alternative approach would be for a FOSS browser maker such as Mozilla to include a proprietary plugin in a distribution to end users. The proprietary plug-in would be installed automatically but the license terms would make it clear that, unlike the FOSS code that is part of the same distribution, that part can't be incorporated into derived works without obtaining a license to the H.264 patent pool from MPEG LA.
Canonical (Ubuntu) and OpenOffice are comfortable with proprietary extensions to free software
Ubuntu maker Canonical has chosen that mixed free-unfree software approach. This caused some outrage by parts of the community (since it gave the impression of a FOSS company supporting H.264 against Theora), and Canonical had to justify its approach. I interpret Canonical's "clarifications" as a recognition of the fact that H.264 is commercially extremely relevant, but they try to maintain their FOSS image as much as they can.
There's a similar debate now concerning OpenOffice, for which there are free as well as unfree plug-ins and certain FOSS advocates would like unfree ones to be excluded from the project's official list of extensions. Bradley Kuhn, a Free Software Foundation board member, expressed his personal views in a blog post, "Beware of proprietary drift". It seems the Free Software Foundation lost this argument and the OpenOffice project will continue to welcome extensions that aren't Free Software.
While patents aren't explicitly discussed in the OpenOffice context, this is clearly an example of where things may be heading, contrary to the FOSS purism some people advocate. Proprietary extensions to OpenOffice could also contain patented elements.
Will the W3C at some point have to depart from its royalty-free standards policy?
My prediction is that there won't be a solution for an HTML 5 video codec that proprietary and FOSS-oriented vendors can reach consensus on. The current diversity of codecs and plug-ins is suboptimal but acceptable: it certainly hasn't prevented web video from becoming extremely popular. So there isn't really a pressing need to converge on a single standard for now.
In the long run it remains to be seen whether the W3C can maintain is royalty-free standards policy. That approach has been key to the success that web technologies have had so far, but it could, as the situation concerning codecs demonstrates, increasingly impede progress.
In the early days of web technology development, there wasn't much attention by big industry, nor by patent trolls. Hence it was possible to create patent-free standards.
The kind of technology created at that time was also far simpler than today's advances in online media. The field has become very sophisticated, which has many implications including the consequence that patent thickets related to new web technologies will reach previously unseen heights in terms of size and density.
In this earlier blog post I wrote, under the subhead "The FOSS way of innovation exposes all FOSS to patent attacks", that patents reward the first to patent a new idea, while FOSS innovation is usually of a different kind (with a few exceptions). That's also an important kind of innovation, but it's not favored by the patent system and may therefore not be a sufficient basis for future web innovation.
HTML may become like GSM, at some point requiring licenses to large numbers of patents
As the web advances in technological terms, and given that software patents are extremely unlikely to be abolished in the largest markets anytime soon, the W3C may in a matter of only a few years feel forced to revisit its standards policy.
It takes licenses to thousands of patents in order to build a GSM phone, and at some point it may be required to license large numbers of patents to build a fully functional HTML web browser. I'm afraid it's only a question of when, not if it will happen.
If you'd like to be updated on patent issues affecting free software and open source, please subscribe to my RSS feed (in the right-hand column) and/or follow me on Twitter @FOSSpatents.
