<abstract>This specification defines an XMPP protocol extension for the third-party control of telephone calls and other similar media sessions. The protocol includes support for session management/signaling, as well as advanced media resources such as speech recognizers, speech synthesizers and audio/video recorders. The protocol serves a different purpose from that of first-party protocols such as Jingle or SIP, and is compatible with those protocols.</abstract>
<p>Rayo is a protocol to allow third-party remote control over media sessions, audio/video mixers and a variety of advanced media resources such as speech recognizers, speech synthesizers and audio/video recorders. These capabilities can be combined to create a wide variety of applications such as menu-based phone systems, in-game conferencing and anonymous dating services. Unlike <ahref='xep-0166.html'>Jingle</a> or even <ahref='http://en.wikipedia.org/wiki/Session_Initiation_Protocol'>SIP</a>, a Rayo client is not concerned with being a party to either the session negotiation or the media stream itself.</p>
<ul>
<li>A Rayo server takes on the role of negotiating a media session between itself and some other endpoint, or between two external endpoints, by way of an implementation-specific means, be that Jingle, SIP, the public-switched telephone network, or anything else. The server may even bridge multiple networks.</li>
<li>The server then presents the Rayo protocol as an interface to a Rayo client, allowing it to monitor and/or exercise third-party control over the established media sessions.</li>
<li>The client has the option to accept/reject/answer inbound session requests, request the creation of outbound sessions and monitor their progress, execute media operations such as speech synthesis, speech recognition & recording, and to end sessions.</li>
</ul>
<p>The relationship between the calling parties, the Rayo server and the Rayo client looks something like this:</p>
<p>In this example, a call from 'tel:+13058881212' has reached the Rayo server 'shakespeare.lit' by calling 'tel:+18003211212', and been assigned an ID '9f00061'. The server has determined that 'juliet@capulet.lit' is a valid candidate to be the client to whom the server delegates control of the call, and so has directed an <ahref='def-offer'>offer event</a> to her 'balcony' resource.</p>
<p>The client, 'juliet@capulet.lit', then decides that it is able to handle the incoming call, and so accepts it from the server, thus gaining exclusive control and indicating to the calling party that the call will be processed and that it should ring.</p>
<examplecaption="Potential controlling party attempts to become definitive controlling party by sending the call an accept command"><![CDATA[
<iqfrom='juliet@capulet.lit/balcony'
to='9f00061@call.shakespeare.lit'
type='set'
id='hd721'>
<acceptxmlns='urn:xmpp:rayo:1'/>
</iq>
]]></example>
<examplecaption="Call acknowledges accept command to the (now) definitive controlling party"><![CDATA[
<iqfrom='9f00061@call.shakespeare.lit'
to='juliet@capulet.lit/balcony'
type='result'
id='hd721'/>
]]></example>
<p>Following confirmation from the server that the attempt to gain control of the call was successful, the client proceeds to answer the call, opening up the media stream between the caller and the server.</p>
<examplecaption="Controlling party answers the call"><![CDATA[
<iqfrom='juliet@capulet.lit/balcony'
to='9f00061@call.shakespeare.lit'
type='set'
id='43jo3'>
<answerxmlns='urn:xmpp:rayo:1'/>
</iq>
]]></example>
<examplecaption="Call acknowledges answer command to controlling party"><![CDATA[
<iqfrom='9f00061@call.shakespeare.lit'
to='juliet@capulet.lit/balcony'
type='result'
id='43jo3'/>
]]></example>
<p>Once the client has confirmation that the call has been answered, it triggers the start of a media output component in order to play a message to the caller using a Text-to-speech (TTS) engine.</p>
<examplecaption="Controlling party requests a new output component"><![CDATA[
<iqfrom='juliet@capulet.lit/balcony'
to='9f00061@call.shakespeare.lit'
type='set'
id='j9d3j'>
<outputxmlns='urn:xmpp:rayo:output:1'
voice='allison'>
<documentcontent-type="text/plain">
<![CDATA[
You have no new messages. Goodbye!
]]]]><![CDATA[>
</document>
</output>
</iq>
]]></example>
<examplecaption="Call acknowledges request for new output component and provides its ID"><![CDATA[
<p>The client then decides it has no further operations to perform on the call, and that the call should end. It instructs the server to hang up the call gracefully.</p>
<examplecaption="Controlling party hangs up the call"><![CDATA[
<iqfrom='juliet@capulet.lit/balcony'
to='9f00061@call.shakespeare.lit'
type='set'
id='f3wh8'>
<hangupxmlns='urn:xmpp:rayo:1'/>
</iq>
]]></example>
<examplecaption="Call acknowledges hangup command to controlling party"><![CDATA[
<iqfrom='9f00061@call.shakespeare.lit'
to='juliet@capulet.lit/balcony'
type='result'
id='f3wh8'/>
]]></example>
<examplecaption="Controlling party receives notification of the call being terminated"><![CDATA[
<presencefrom='9f00061@call.shakespeare.lit'
to='juliet@capulet.lit/balcony'
type='unavailable'>
<endxmlns='urn:xmpp:rayo:1'>
<hangup-command/>
</end>
</presence>
]]></example>
</section1>
<section1topic='Requirements'anchor='reqs'>
<p>The protocol defined herein is designed to provide the following features:</p>
<ol>
<li><strong>Call Control:</strong> Incoming calls are "offered" to clients at which point they can be answered, rejected, redirected to another destination, etc. Outbound calls may also be made and monitored. Every attempt is made to be shield the Rayo client from the low level telephony protocol (e.g. SIP, Jingle, PSTN, etc).</li>
<li><strong>Audio File Playback:</strong> A compatible Rayo server will fetch a file from a a specified URL and play the containing audio to the caller.</li>
<li><strong>Speech Synthesis / TTS:</strong> In cases where dynamic data must be spoken, a Speech Synthesis engine may be used to play computer generated speech to the caller.</li>
<li><strong>Touch-tone Events / DTMF:</strong> Rayo surfaces real-time event when the caller presses keys on their touch-tone keypad.</li>
<li><strong>Speech Recognition:</strong> Enables the phone application to take spoken queues allowing for sophisticated voice-driven menus and directory services.</li>
<li><strong>Call Recording:</strong> Can be used to capture the caller's voice (e.g. Voicemail) or both sides of the call for auditing and compliance purposes.</li>
<li><strong>Mixing:</strong> Typically referred to as an audio "conference"; calls can be joined together so that the participants can hear each other in real-time.</li>
</ol>
<p>Many third-party call control protocols have preceeded Rayo (see Asterisk's AGI/AMI, FreeSWITCH's eventsocket, Microsoft's TAPI, Java's JTAPI, Novell/AT&T's TSAPI, CSTA, etc). None of these protocols is ideal, and all have one or more of the following drawbacks:</p>
<ul>
<li><strong>Totally ground-up wire protocol</strong> requiring implementation all the way down to the socket.</li>
<li><strong>Platform/vendor/hardware specific</strong> - each system implements its own proprietary protocol (in many cases, without a formal published specification) which does not allow easily porting an application from one back-end to another.</li>
<li><strong>Synchronous interface</strong> - Operations on calls or other entities are often blocking, and one must serialise all control messages.</li>
<li><strong>Inconsistent</strong> - evolved, rather than designed.</li>
<li><strong>Lacking in scalability</strong> - client/server sometimes tied one-to-one, servers rarely clustered, advanced message routing not possible.</li>
<li><strong>Poor security</strong> - lack of wire-level encryption, lack of or sub-standard authentication mechanisms, lack of or limited authorization mechanisms, lack of or poor sandboxing between multiple tenants on one system.</li>
<li><strong>Inextensible</strong> - The specification of extensions to the core protocol is either impossible or very difficult.</li>
</ul>
<p>Rayo has been designed with these failings in mind, and intends to address many concerns not addressed by these earlier attempts. The following considerations were made:</p>
<ul>
<li><strong>Simple client library implementation</strong> - XMPP client libraries exist in all modern languages, and many are of a high standard of quality and maturity.</li>
<li><strong>Cross-platform standard</strong> - The protocol must not expose any platform specifics and all elements should be candidates for implementation on any suitable platform. Addiditionally, the protocol must be ratified as a standard following a community discussion.</li>
<li><strong>Asynchronous interface</strong> - The protocol should present an asynchronous interface for the purposes of performance and flexibility in performing parallel operations.</li>
<li><strong>Consistent</strong> - The protocol must provide a considered, unobtrusive, logically and philisophically consistent interface.</li>
<li><strong>Federated</strong> - The protocol must support communication between client and server entities on separately owned, operated and addressed networks.</li>
<li><strong>Flexible routing</strong> - The protocol must lend itself to routing across wide networks such as the internet, and to potential complex routing such as proxying or redirection. Additionally, the client and server should each be aware of the presence of the other and be able to use such information to make routing decisions.</li>
<li><strong>Extensible</strong> - The protocol must provide for the possibility of extra functionality being added by future specifications or an individual implementation.</li>
<li><strong>Secure</strong> - The protocol should include appropriate measures for authentication and authorization of participants, as well as preventing third-parties from intercepting control messages.</li>
</ul>
<p>Many of the features in the above list are available to Rayo at no specification or implementation cost, since they are core to XMPP itself and thus Rayo inherits these 'for free'.</p>
<p>Additionally, the protocol is required to abstract away the complexity of the back-end negotiation, especially the details of the transport protocols such as SIP or Jingle, but to map conceptually to such protocols.</p>
</section1>
<section1topic='Terminology'anchor='terms'>
<section2topic='Glossary'anchor='terms-glossary'>
<dl>
<di>
<dt>Third-party call control (3PCC)</dt>
<dd>The observation and/or control of a live media session by an entity which is not a direct party to the session.</dd>
</di>
<di>
<dt>Command</dt>
<dd>Commands instruct the receiving entity to perform some atomic action. Commands may be executed against a given call, component or mixer and can be considered completed as soon as they receive a response. Some commands create components, and return a reference to the component in their response.</dd>
</di>
<di>
<dt>Component</dt>
<dd>Components extend the Rayo protocol by providing additional media and call control functionality. Components are created by an appropriate command, which returns a reference to the component. Components are executed asynchronously, and have a lifecycle attached to them, with the ability to trigger events or have commands issued to it. Once a component is stopped or comes to an end naturally, it will issue a special <complete/> event, indicating that it has ceased executing and deliver any required data.</dd>
</di>
<di>
<dt>Potential controlling party (PCP)</dt>
<dd>An XMPP entity to which an offer to control an incoming call may be sent.</dd>
</di>
<di>
<dt>Definitive controlling party (DCP)</dt>
<dd>The XMPP entity which gains a lock on control of a session, either by requesting the session's creation, or being the first respondent to an offer.</dd>
</di>
<di>
<dt>Security Zone</dt>
<dd>A security zone is the conceptual border around a call which defines which parties may interact with the call's media or signaling. A security zone MUST contain the rayo server's internal implementation, the media server to which the call is joined, the DCP, and any JID whose bare form is the same as the DCP. A server MAY relax this definition further, for example to consider all JIDs at the same domain to be in the same security zone.</dd>
<p>A Rayo server is an entity which is capable of receiving and initiating calls and being party to their media stream, while exposing a Rayo interface to a client in order to permit control over its calls. The Rayo server may handle calls in any way supported by the implementation, such as SIP, Jingle, etc, and should expose a full XMPP domain at the root level of the service deployment (eg shakespeare.lit).</p>
<p>The Rayo server is responsible for keeping track of valid clients, routing calls to the correct potential controlling parties, performing authorization measures on received stanzas, etc.</p>
<p>For the purposes of this specification, complex server-side deployments such as clusters, proxies, gateways, protocol translators, etc are not considered. Further details of such concepts may be found in their (present or future) relevant specifications.</p>
<p>A Rayo client is an entity which implements the Rayo protocol for the purpose of asserting control over calls made available by a Rayo server. The method by which such control measures are determined is outside the scope of this document, but may be the result of human interaction or some automated decision-making process.</p>
<p>A Rayo client is responsible for indicating its availability to a Rayo server and responding to offer messages appropriately.</p>
<p>A Rayo call is a short-lived XMPP entity within the scope of the deployment's root domain, perhaps at a sub-domain, with the purpose of representing a single session. It is usually a simple alias for the main server process.</p>
<p>A Rayo call is the entity with which most client interactions are made, and is responsible for sending its events to and receiving commands from a client. Calls may host components.</p>
<p>Calls have separate presence from the root domain of the service and thus appear to be separate entities.</p>
<p>A Rayo mixer is an XMPP entity within the scope of the deployment's root domain, perhaps at a sub-domain, with the purpose of representing a service for the linking of media streams from several calls. It is usually a simple alias for the main server process.</p>
<p>A Rayo mixer is responsible for sending its events to and receiving commands from one or more clients, and can host components.</p>
<p>Mixers have separate presence from the root domain of the service and its calls and thus appear to be separate entities.</p>
<p>A Rayo command is a simple combination of request and response and may be issued directly to the service domain, a call, a mixer or a component attached to any of the former. Commands are executed serially and are generally very short-lived.</p>
<p>Components extend the Rayo protocol by providing additional media and call control functionality.</p>
<p>Components have a lifecycle and are started by sending a specialized command to a call or mixer. Thus, a request for creation of a component will return a reference to the component's ID, and the component will continue to execute until it completes, potentially sending events and processing commands along the way (such as an instruction to pause or terminate), before finally issuing an event indicating its completion and thus unavailability. Multiple components may be active on a call or mixer at any one time, and commands may be executed on any entity during the execution of a component.</p>
<p>A call's Remote Party is the software or device with which the Call's signalling (and optionally media) connection is established. This might be a software or hardware phone, a PBX, a gateway or some other such system.</p>
<p>All of the actors described in the previous section (with the exception of commands) are represented by XMPP entities with a JID of their own. Thus, a scheme for determining the JIDs of each of these entities is required. The following is the required naming scheme for Rayo deployments, where elements in square brackets are optional.</p>
<p>Commands should be addressed to the entity on which they should be enacted. Individual commands only apply to certain object (for example instructing a component to hangup will return an error). In general, commands may be sent from a client to the service, a call, a mixer or a component. Events may be sent from a call, a mixer or a component to a client.</p>
<p>Rayo defines several events and commands which may be executed on one of the above actors. These payloads must be sent within an XMPP primitive element, and the rules are as such:</p>
<li><strong>Events:</strong> Sent as directed presence from the entity producing the event to a client. Rayo servers SHOULD timestamp all events using &xep0203; in order to allow clients to reliably use Rayo events for billing purposes.</li>
<li><strong>Commands:</strong> Sent as an <iq/> with the 'type' attribute 'set' from the client to the entity to be acted on. Responses returned as an <iq/> with the 'type' attribute either 'result' or 'error'.</li>
</ul>
</section2>
</section1>
<section1topic='Session Flow'anchor='session'>
<p>This section describes the form, function and order of Rayo stanzas sent across the wire, and the circumstances in which they apply and/or may arise.</p>
<p>In order for a Rayo client to be considered a potential controlling party for incoming sessions, it MUST first notify the Rayo server that it is available for the receipt of calls. This is done by sending directed presence to the Rayo server with a <show/> element containing 'chat' as in the example:</p>
<examplecaption="Client presents itself as available to the Rayo server"><![CDATA[
<presencefrom='juliet@capulet.lit/balcony'
to='shakespeare.lit'>
<cxmlns='http://jabber.org/protocol/caps'
hash='sha-1'
node='urn:xmpp:rayo:client:1'
ver='QgayPKawpkPSDYmwT/WM94uAlu0='/>
<show>chat</show>
</presence>
]]></example>
<p>Conversely, when a Rayo client wishes not to be considered a potential controlling party, it SHOULD send directed presence to the Rayo server with a <show/> element containing 'dnd' as in the example:</p>
<examplecaption="Client presents itself as unavailable to the Rayo server"><![CDATA[
<p>Sessions may be established either at the request of the Rayo client (an outbound call) or as a result of a 3rd party request (an inbound call). Each scenario differs in the Rayo protocol only up to the point at which the session is established and media begins to flow. First we shall examine the sequence of stanzas passed between server and client in each of these scenarios.</p>
<p>In order for a client to establish a new outbound call, it MUST first send a <linkurl="#def-dial">dial command</link> to the server, indicating the proposed target for the call, its apparent source, and any meta-data to send to the target as headers.</p>
<examplecaption="Client requests establishment of a new outbound session"><![CDATA[
<iqfrom='juliet@capulet.lit/balcony'
to='shakespeare.lit'
type='set'
id='h7ed2'>
<dialxmlns='urn:xmpp:rayo:1'
to='tel:+13055195825'
from='tel:+14152226789'>
<headername="x-skill"value="agent"/>
<headername="x-customer-id"value="8877"/>
</dial>
</iq>
]]></example>
<p>On successfully receiving and parsing the dial command, the server SHOULD perform its own proprietary authorization measures to ensure that only desirable outbound sessions are created. If it is established that the command should not be allowed, the server MUST return an error giving an authorization reason.</p>
<p>If a 'uri' attribute is set on the dial command, the server MUST attempt to create the call at the requested URI. This allows clients to know the URI of the call prior to it coming into existence, for clients where this distinction might be important. Such a URI MUST be a valid Rayo call address.</p>
<p>The specified metadata in the form of the 'from' attribute and any <header/> elements SHOULD be mapped to the underlying signalling protocol for communication to the remote party.</p>
<p>If the client is unknown to the server and the server does not permit session creation by unknown clients, the server MUST return a <registration-required/> error with a type of 'auth'.</p>
<examplecaption="Server indicates client is unknown and the server does not permit session creation by unknown clients"><![CDATA[
<p>If the client is not authorized (as determined by an implementation/deployment-specific algorithm) to create a new outbound session given the parameters provided, the server MUST return a <not-authorized/> error with a type of 'auth'.</p>
<examplecaption="Server indicates client is not authorized to create a new outbound session given the parameters provided"><![CDATA[
<p>If the server does not have sufficient resources to create a new session, the server MUST return a <resource-constraint/> error with a type of 'wait'.</p>
<examplecaption="Server indicates that it does not have sufficient resources to create a new session"><![CDATA[
<p>If the command is successful and the call is queued, however, confirmation of such should be sent to the client, including a reference to the unique ID of the call. This call ID may be used to execute commands and filter events for the duration of the session.</p>
<examplecaption="Confirmation of successful dial request and call ID"><![CDATA[
<p>Once the server receives notification that the session has been accepted by the remote party, it should send a ringing event to the client to indicate such:</p>
<p>Similarly, once the server receives notification that the session has been answered, it should negotiate media between the remote party and its local media server. Once media negotiation is complete, it should send an answered event to the client to indicate such:</p>
<p>In this case, the server MUST negotiate media as specified by the join element, in accordance with the rules defined in <linkurl='#session-joining'>joining calls</link>. Media MUST NOT be negotiated with the local media server, unless the join specifies so. The join operation MUST behave as described in <linkurl='#session-joining'>joining calls</link>.</p>
<p>When the Server receives a call from one of its connected networks, it MUST then expose that requested session to Rayo clients. It SHOULD use an implementation-specific routing mechanism to map incoming calls to some set of registered JIDs which are considered appropriate controlling parties. From this set, it SHOULD then remove any parties whom it can identify as being temporarily inappropriate for control (either unavailable based on presence, under too much load, or any other metric which the server has available). If, as a result, the set of Potentially Controlling Parties is empty, the server MUST reject the call indicating that the requested service was unavailable.</p>
<p>If the server can identify active Potential Controlling Parties, it MUST offer them control of the call according to its particular distribution method, which MAY be simultaneous or staged. The server must broadcast an offer on behalf of the call to all Potential Controlling Parties, using applicable to/from/header data from the incoming session. The server MUST also include entity capabilities information in the presence stanza containing the offer, in order to advertise the fact that the entity is a call, qualified by the node name "urn:xmpp:rayo:call:1".</p>
<p>Once the server has offered control, it MUST wait for a response from a PCP or for the remote party to end the call. The server SHOULD monitor the availability of PCPs to whom offers have been sent. If they all cease to be PCPs (eg by going offline) then the call should be rejected in the same way as if there had not been any available PCPs to begin with.</p>
<p>If an offered PCP executes a command against the call, by sending a command node to the call's JID inside an IQ 'set', the server should execute the following routine:</p>
<ol>
<li>If the call does not have a DCP, set it to the PCP from which the command originated.</li>
<li>If the call has a DCP, and the command did not originate from the DCP, return a conflict (cancel) error in response to the command of the following format:
<examplecaption="Server indicates that the call already has another DCP and that control of the call is no longer available."><![CDATA[
<li>If the command is an accept command, notify the remote calling party that the call has been accepted. Return an empty IQ result to the command issuing party to confirm successful execution.</li>
<li>If the command is an answer command, notify the remote calling party that the call has been answered and negotiate media between the calling party and the server's local media server. Return an empty IQ result to the command issuing party to confirm successful execution.</li>
<li>If the command is any other, handle it in the manner detailed in the following sections.</li>
<p>Calls on a Rayo Server are capable of having their media streams moved/manipulated. Once such manipulation is to join the media streams of two calls. In a scenario where callA and callB should be joined, the client MUST send a join command to either call (not both) specifying the call ID of the other call, and optionally media attributes (direction and media) specified in the schema, like so:</p>
<p>If the calls to be joined to each other are in the same security zone, the server MUST join the media streams of the two calls and return an empty IQ result to confirm that the operation has been successful. If the parties to be joined are not within the same security zone, an error should be returned as detailed below.</p>
<p>When calls are joined to each other by any mechanism, each call MUST dispatch a joined event specifying who they have been joined to:</p>
<examplecaption="Call A and B were joined, both calls emit joined events"><![CDATA[
<p>By default, the server MUST join the calls by bridging their audio through its local media server, with bidirectional media. In order to specify alternative behaviour, the client MAY specify a media option (either 'bridge' or 'direct') and/or a direction option (either 'duplex', 'send' or 'recv'), and the server MUST bridge accordingly.</p>
<p>There are several reasons why the call might return an error instead of acknowledging a join:</p>
<ul>
<li>The specified join party does not exist or cannot be found.</li>
<li>The specified join party is inaccessible for the purposes of being joined due to security restrictions.</li>
<li>The server does not have sufficient resources to complete the join.</li>
<li>The join command was malformed.</li>
<li>The specified media/direction options or their combination are not possible/supported.</li>
</ul>
<p>If the specified join party does not exist or cannot be found, the server MUST return a <service-unavailable/> error with a type of 'cancel'.</p>
<examplecaption="Call indicates that the specified join party does not exist or cannot be found"><![CDATA[
<p>If the specified join party is inaccessible for the purposes of being joined due to security restrictions, the server MUST return a <not-allowed/> error with a type of 'cancel'.</p>
<examplecaption="Call indicates that the specified join party is inaccessible for the purposes of being joined due to security restrictionss"><![CDATA[
<p>If the server does not have sufficient resources to complete the join, the server MUST return a <resource-constraint/> error with a type of 'wait'.</p>
<examplecaption="Call indicates that there are not sufficient resources to complete the join"><![CDATA[
<p>If the specified media/direction options or their combination are not possible/supported, the server MUST return a <feature-not-implemented/> error with a type of 'modify'.</p>
<examplecaption="Call indicates that the specified media/direction options or their combination are not possible/supported."><![CDATA[
<p>The server MUST unjoin the media streams of the two calls, rejoin both to the media server and return an empty IQ result to confirm that the operation has been successful:</p>
<examplecaption="CallA acknowledges unjoin from callB"><![CDATA[
<iqfrom='callA@call.shakespeare.lit'
to='juliet@capulet.lit/balcony'
type='result'
id='h7ed2'/>
]]></example>
<p>Optionally, if no join is specified on the unjoin command, all existing joins must be broken:</p>
<examplecaption="Client instructs callA to unjoin from every existing join"><