Rayo is a protocol to allow third-party remote control over media sessions, audio/video mixers and a variety of advanced media resources such as speech recognizers, speech synthesizers and audio/video recorders. These capabilities can be combined to create a wide variety of applications such as menu-based phone systems, in-game conferencing and anonymous dating services. Unlike Jingle or even SIP, a Rayo client is not concerned with being a party to either the session negotiation or the media stream itself.

• A Rayo server takes on the role of negotiating a media session between itself and some other endpoint, or between two external endpoints, by way of an implementation-specific means, be that Jingle, SIP, the public-switched telephone network, or anything else. The server may even bridge multiple networks.
• The server then presents the Rayo protocol as an interface to a Rayo client, allowing it to monitor and/or exercise third-party control over the established media sessions.
• The client has the option to accept/reject/answer inbound session requests, request the creation of outbound sessions and monitor their progress, execute media operations such as speech synthesis, speech recognition & recording, and to end sessions.

The relationship between the calling parties, the Rayo server and the Rayo client looks something like this:

This document defines the core Rayo protocol, and contains provisions for its extension by further specifications.

In order to understand the nature of a Rayo interaction, here we show a simple example of a control session.

In this example, a call from 'tel:+13058881212' has reached the Rayo server 'shakespeare.lit' by calling 'tel:+18003211212', and been assigned an ID '9f00061'. The server has determined that 'juliet@capulet.lit' is a valid candidate to be the client to whom the server delegates control of the call, and so has directed an offer event to her 'balcony' resource.

The client, 'juliet@capulet.lit', then decides that it is able to handle the incoming call, and so accepts it from the server, thus gaining exclusive control and indicating to the calling party that the call will be processed and that it should ring.

Following confirmation from the server that the attempt to gain control of the call was successful, the client proceeds to answer the call, opening up the media stream between the caller and the server.

Once the client has confirmation that the call has been answered, it triggers the start of a media output component in order to play a message to the caller using a Text-to-speech (TTS) engine.

After confirmation that the output component was successfully created, the client then awaits notification of its completion.

The client then decides it has no further operations to perform on the call, and that the call should end. It instructs the server to hang up the call gracefully.

The protocol defined herein is designed to provide the following features:

1. Call Control: Incoming calls are "offered" to clients at which point they can be answered, rejected, redirected to another destination, etc. Outbound calls may also be made and monitored. Every attempt is made to be shield the Rayo client from the low level telephony protocol (e.g. SIP, Jingle, PSTN, etc).
2. Audio File Playback: A compatible Rayo server will fetch a file from a a specified URL and play the containing audio to the caller.
3. Speech Synthesis / TTS: In cases where dynamic data must be spoken, a Speech Synthesis engine may be used to play computer generated speech to the caller.
4. Touch-tone Events / DTMF: Rayo surfaces real-time event when the caller presses keys on their touch-tone keypad.
5. Speech Recognition: Enables the phone application to take spoken queues allowing for sophisticated voice-driven menus and directory services.
6. Call Recording: Can be used to capture the caller's voice (e.g. Voicemail) or both sides of the call for auditing and compliance purposes.
7. Mixing: Typically referred to as an audio "conference"; calls can be joined together so that the participants can hear each other in real-time.

Many third-party call control protocols have preceeded Rayo (see Asterisk's AGI/AMI, FreeSWITCH's eventsocket, Microsoft's TAPI, Java's JTAPI, Novell/AT&T's TSAPI, CSTA, etc). None of these protocols is ideal, and all have one or more of the following drawbacks:

• Totally ground-up wire protocol requiring implementation all the way down to the socket.
• Platform/vendor/hardware specific - each system implements its own proprietary protocol (in many cases, without a formal published specification) which does not allow easily porting an application from one back-end to another.
• Synchronous interface - Operations on calls or other entities are often blocking, and one must serialise all control messages.
• Inconsistent - evolved, rather than designed.
• Lacking in scalability - client/server sometimes tied one-to-one, servers rarely clustered, advanced message routing not possible.
• Poor security - lack of wire-level encryption, lack of or sub-standard authentication mechanisms, lack of or limited authorization mechanisms, lack of or poor sandboxing between multiple tenants on one system.
• Inextensible - The specification of extensions to the core protocol is either impossible or very difficult.

Rayo has been designed with these failings in mind, and intends to address many concerns not addressed by these earlier attempts. The following considerations were made:

• Simple client library implementation - XMPP client libraries exist in all modern languages, and many are of a high standard of quality and maturity.
• Cross-platform standard - The protocol must not expose any platform specifics and all elements should be candidates for implementation on any suitable platform. Addiditionally, the protocol must be ratified as a standard following a community discussion.
• Asynchronous interface - The protocol should present an asynchronous interface for the purposes of performance and flexibility in performing parallel operations.
• Consistent - The protocol must provide a considered, unobtrusive, logically and philisophically consistent interface.
• Federated - The protocol must support communication between client and server entities on separately owned, operated and addressed networks.
• Flexible routing - The protocol must lend itself to routing across wide networks such as the internet, and to potential complex routing such as proxying or redirection. Additionally, the client and server should each be aware of the presence of the other and be able to use such information to make routing decisions.
• Extensible - The protocol must provide for the possibility of extra functionality being added by future specifications or an individual implementation.
• Secure - The protocol should include appropriate measures for authentication and authorization of participants, as well as preventing third-parties from intercepting control messages.

Many of the features in the above list are available to Rayo at no specification or implementation cost, since they are core to XMPP itself and thus Rayo inherits these 'for free'.

Additionally, the protocol is required to abstract away the complexity of the back-end negotiation, especially the details of the transport protocols such as SIP or Jingle, but to map conceptually to such protocols.

A complete Rayo deployment has several elements and interacting entities which must be understood.

This section describes the form, function and order of Rayo stanzas sent across the wire, and the circumstances in which they apply and/or may arise.