<abstract>This document describes implementation considerations related to voice and video codecs for use in Jingle RTP sessions.</abstract>
&LEGALNOTICE;
<number>xxxx</number>
<status>ProtoXEP</status>
<type>Informational</type>
<sig>Standards</sig>
<approver>Council</approver>
<dependencies>
<spec>XMPP Core</spec>
<spec>XEP-0167</spec>
</dependencies>
<supersedes/>
<supersededby/>
<shortname>N/A</shortname>
<discuss>jingle</discuss>
&stpeter;
<revision>
<version>0.0.4</version>
<date>2009-04-04</date>
<initials>psa</initials>
<remark><p>Clarified status of H.264.</p></remark>
</revision>
<revision>
<version>0.0.3</version>
<date>2009-04-02</date>
<initials>psa</initials>
<remark><p>Rewrote document based on developer feedback and Council discussion.</p></remark>
</revision>
<revision>
<version>0.0.2</version>
<date>2009-03-04</date>
<initials>psa</initials>
<remark><p>Added more information about video codecs.</p></remark>
</revision>
<revision>
<version>0.0.1</version>
<date>2009-03-04</date>
<initials>psa</initials>
<remark><p>First draft, copied from XEP-0167 with slight revisions and addition of requirements section.</p></remark>
</revision>
</header>
<section1topic='Introduction'anchor='intro'>
<p>&xep0167; defines the &xep0166; signalling exchanges needed to establish voice and video chat using the Real-time Transport Protocol &rfc3550;; however, it does not discuss the matter of voice and video codecs, since the state of codec technologies is more fluid than the signalling interactions. This document fills that gap by providing guidance to Jingle developers regarding voice and video codecs.</p>
<p>Because codec technologies are typically subject to patents, the topics discussed here are controversial. This document attempts to steer a middle path between (1) specifying mandatory-to-implement technologies that realistically will not be implemented and deployed and (2) providing guidelines that, while realistic, do not encourage the implementation and deployment of patent-clear technologies.</p>
<pclass='box'>Please note that this document is strictly informational and does not (yet) provide binding recommendations to the XMPP developer community regarding mandatory-to-implement technologies; however, it is expected that this document will provide input that the &XSF; could use in making such recommendations. Furthermore, it is expected that any recommendations that might be made by the XSF would need to be modified over time as the technology landscape changes.</p>
<p>The ideal codec would meet the following criteria:</p>
<dl>
<di><dt>Quality</dt><dd>The encoding quality is acceptable for deployment among XMPP users.</dd></di>
<di><dt>Packetization</dt><dd>The specification of the codec clearly defines packetization of data for sending over RTP.</dd></di>
<di><dt>Availability</dt><dd>The codec can be implemented on a wide variety of computing platforms and is commonly used in Internet or other systems.</dd></di>
<di><dt>Patents</dt><dd>The codec is patent-clear. <note>The term patent-clear does not necessarily mean that no patents have ever been applied for or granted regarding a technology, or that the technology is completely free from patents (since such a judgment is nearly impossible to make, and is outside the purview of the XMPP developer community and the XMPP Standards Foundation); the term means only that those who implement the technology are generally understood to be relatively safe from the threat of patent litigation, either because any relevant patents have expired, were filed in a defensive manner, or are made available under suitable royalty-free licenses.</note></dd></di>
</dl>
<p>Unfortunately, not all codecs are ideal. In the next section we discuss the audio and video codecs that are most attractive for implementation in Jingle RTP applications.</p>
<p>Note: In general, audio codecs are more mature than video codecs. As a result, there are more patent-clear options for audio than for video. Although most XMPP developers would prefer to implement codecs that are patent-clear (both for ethical reasons and to produce free or at least affordable software), such options are not always widely implemented and deployed. This document takes these factors into account.</p>
</section1>
<section1topic='Codec Summary'anchor='summary'>
<section2topic='Audio'anchor='audio'>
<section3topic='Speex'anchor='audio-speex'>
<p>According to the speex.org website, the Speex codec is "an Open Source/Free Software patent-free audio compression format designed for speech". Speex was developed by Jean-Marc Valin and is maintained by the <linkurl='http://www.xiph.org/'>Xiph.org Foundation</link>. The following table summarizes the available information about Speex.</p>
<tablecaption='Codec Considerations for Speex'>
<tr>
<th>Quality</th>
<th>Packetization</th>
<th>Availability</th>
<th>Patents</th>
</tr>
<tr>
<td>Good quality; optimized for voice; can be used for wide-band audio.</td>
<td>Freely downloadable under a revised BSD license at <<linkurl='http://speex.org/'>http://speex.org/</link>> and commonly deployed on Internet (VoIP) systems; not commonly deployed on non-Internet systems.</td>
<td>Designed to be patent-clear.</td>
</tr>
</table>
</section3>
<section3topic='G.711'anchor='audio-g711'>
<p>G.711 refers to the Pulse Code Modulation (PCM) codec defined in &ITU; recommendation G.711, which is widely used on the public switched telephone network (PSTN) and by many voice over Internet Protocol (VoIP) providers. There are two versions: the μ-law ("U-law") version is widely deployed in North America and in Japan and the A-law version is widely deployed in the rest of the world. The following table summarizes the available information about G.711.</p>
<tablecaption='Codec Considerations for G.711'>
<tr>
<th>Quality</th>
<th>Packetization</th>
<th>Availability</th>
<th>Patents</th>
</tr>
<tr>
<td>Good quality; no wide-band mode.</td>
<td>See &rfc5391;.</td>
<td>Commonly deployed in both PSTN and VoIP systems.</td>
<td>Developed in 1972; patents have expired.</td>
</tr>
</table>
</section3>
</section2>
<section2topic='Video'anchor='video'>
<section3topic='Theora'anchor='audio-speex'>
<p>According to the theora.org website, the Theora codec is "a free and open video compression format". Theora is based on the VP3 codec originally developed by On2 Technologies and is now maintained by the <linkurl='http://www.xiph.org/'>Xiph.org Foundation</link>. The following table summarizes the available information about Theora.</p>
<tablecaption='Codec Considerations for Theora'>
<tr>
<th>Quality</th>
<th>Packetization</th>
<th>Availability</th>
<th>Patents</th>
</tr>
<tr>
<td>Acceptable quality.</td>
<td>See &rtptheora;.</td>
<td>Freely downloadable under a revised BSD license at <<linkurl='http://theora.org/'>http://theora.org/</link>>; not yet commonly deployed, especially on devices that have deployed H.264 instead.</td>
<td>On2's patents over VP3 were contributed to the Xiph.org Foundation in 2001.</td>
</tr>
</table>
</section3>
<section3topic='H.264'anchor='audio-h264'>
<p>H.264 is a technology for video compression jointly designed by the ITU and the &ISO;. The following table summarizes the available information about H.264.</p>
<tablecaption='Codec Considerations for H.264'>
<tr>
<th>Quality</th>
<th>Packetization</th>
<th>Availability</th>
<th>Patents</th>
</tr>
<tr>
<td>High quality.</td>
<td>See &rfc3984;.</td>
<td>Commonly deployed in commercial video systems. Not freely downloadable; both software implementations and service deployments can be subject to royalty payments for commercial use.</td>
<td>Patented.</td>
</tr>
</table>
</section3>
</section2>
</section1>
<section1topic='Guidance for Implementors'anchor='guidance'>
<p>Given that both Speex and G.711 are patent-clear, freely implementable, and commonly deployed, this document suggests that implementors strongly consider including support for both codecs in audio applications of Jingle RTP sessions.</p>
<p>The situation regarding video codecs is more murky, and implementors face difficult tradeoffs. While Theora is patent-clear and freely implementable, it is not yet commonly deployed. On the other hand, deployment of H.264 is fairly common, but it is not patent-clear or freely implementable. For many open-source / free software projects and smaller technology vendors, implementation of H.264 is either impossible (because of patents and licensing restrictions) or prohibitively expensive (because of royalty payments). These developers are strongly encouraged to implement Theora and also to urge wider adoption of Theora among larger technology vendors. However, this document acknowledges that it may take some time before Theora is commonly deployed (especially on mobile devices) and that systems based on H.264 might be dominant in the marketplace for several years. This situation is unfortunate but cannot be directly changed by the XMPP developer community.</p>
</section1>
<section1topic='Recommendations to the XSF'anchor='recommendations'>
<p>This document suggests that both Speex and G.711 could be recommended as mandatory-to-implement technologies for audio codecs, should the XSF decide to make such recommendations.</p>
<p>This document suggests that at this time it is not possible for the XSF to recommend a mandatory-to-implement technology for video codecs, but that it might be possible for the XSF to recommend Theora in the future if Theora is more widely adopted.</p>
<p>For security considerations related to Jingle RTP sessions, refer to XEP-0167. This document introduces no new security considerations. See also the security considerations sections of the relevant codec specifications.</p>
<p>This document requires no interaction with the ®ISTRAR;.</p>
</section1>
<section1topic='Acknowledgements'anchor='ack'>
<p>Thanks to Olivier Crête, Dave Cridland, Justin Karneges, Tobias Markmann, Jack Moffitt, Jeff Muller, Arc Riley, Kevin Smith, Justin Uberti, and Paul Witty for their feedback.</p>
