<abstract>This document defines an XMPP protocol extension for establishing an out-of-band bytestream between any two XMPP users, mainly for the purpose of file transfer. The bytestream can be either direct (peer-to-peer) or mediated (though a special-purpose proxy server). The typical transport protocol used is TCP, although UDP may optionally be supported as well.</abstract>
<remark><p>Incorporated errata: specified format for SHA1 output; specified BND.ADDR and BND.PORT for SOCKS5 reply; removed extraneous SOCKS5 acknowledgement example from Section 4.9; clarified rules for creation of SOCKS5 connection request in Section 4.6; added examples to Section 4.8; specified that ATYP value is hardcoded to 3 in this usage.</p></remark>
<remark><p>Cleaned up narratives to reflect current practices and removed unnecessary authentication references; fixed mismatch SOCKS5 parameter table values.</p></remark>
<remark><p>Removed SIDs from the result queries, you should key off the IQ 'id' attribute instead. Added the disco exchange for finding available proxies.</p></remark>
<remark><p>Clarified that this proposal uses an adaptation of the SOCKS5 protocol, not the full protocol; replaced DTD with schema; added security considerations.</p></remark>
<remark><p>Added service discovery example; added 'srvid' attribute to streamhost element and required inclusion of either 'srvid' or 'port' attribute; improved the algorithms for generating SOCKS5 UNAME and PASSWD parameters; specified that the DST.ADDR and DST.PORT parameters may be ignored; removed references to connected/disconnected notification, bidirectional bytestreams, and multiple targets; updated implementation notes.</p></remark>
<remark><p>Specified option of "reversing the connection" (Target becomes Initiator); added more error cases; resurrected and cleaned up formal use case.</p></remark>
<remark><p>Added section on connected/disconnected notifications sent from Proxy to Initiator; cleaned up several examples; specified more error conditions; clarified the formal descriptions; added implementation notes and future considerations.</p></remark>
<p>XMPP is designed for sending relatively small fragments of XML between network entities (see &xmppcore;) and is not designed for sending binary data. However, sometimes it is desirable to send binary data to another entity that one has discovered on the XMPP network (e.g., to send a file). Therefore it is widely recognized within the Jabber community that it would be valuable to have a generic protocol for streaming binary data between any two entities on the network. The main application for such a bytestreaming technology would be file transfer, for which there are currently a number of incompatible protocols (resulting in a lack of interoperability). However, other applications are possible, which is why it is important to develop a generic protocol rather than one that is specialized for a particular application such as file transfer. This document defines a protocol that meets the following conditions:</p>
<li>Bytestreams are established over standard TCP connections (&rfc0793;) or UDP associations (&rfc0768;), where TCP support is REQUIRED and UDP support is OPTIONAL</li>
<li>Sockets may be direct (peer-to-peer) or mediated (established through a bytestreaming service)</li>
<li>Where possible, standard wire protocols are used</li>
<p>Specifically, this document proposes that the Jabber community make use of the SOCKS 5 protocol, which is an IETF-approved, IPv6-ready technology for bytestreams. (Note: Because this proposal uses a subset of the SOCKS5 protocol that is specially adapted for Jabber bytestreams, existing SOCKS5 proxies cannot be used to implement this proposal without modifications.)</p>
<td>A Jabber Entity that wishes to establish a bytestream with another Entity</td>
</tr>
<tr>
<td>Target</td>
<td>The Entity with which the Initiator is attempting to establish a bytestream</td>
</tr>
<tr>
<td>Proxy</td>
<td>A Jabber entity which is not NAT/Firewalled and is willing to be a middleman for the bytestream between the Initiator and the Target</td>
</tr>
<tr>
<td>StreamHost</td>
<td>The system that the Target connects to and that is "hosting" the bytestream -- may be either the Initiator or a Proxy</td>
</tr>
<tr>
<td>StreamID</td>
<td>A relatively unique Stream ID for this connection; this is generated by the Initiator for tracking purposes and MUST be less than 128 characters in length</td>
</tr>
</table>
</section1>
<section1topic='Narrative'anchor='narrative'>
<p>There are two scenarios addressed by this protocol:</p>
<ol>
<li>direct connection (i.e., the StreamHost is the Initiator)</li>
<li>mediated connection (i.e., the StreamHost is a Proxy)</li>
</ol>
<p>The "happy paths" for these scenarios are described separately below for ease of understanding. A full description of these scenarios is captured in the Formal Use Case. This narrative describes TCP connections only; UDP associations are described in the <linkurl='#udp'>Optional UDP Support</link> section of this document.</p>
<p>Direct connection is the simpler case. In this situation, the StreamHost is the Initiator (StreamHost/Initiator), which means that the Initiator knows the network address of the StreamHost and knows when to activate the bytestream. The process for establishing bytestreams in this case is as follows:</p>
<ol>
<li><p>Initiator sends IQ-set to Target specifying the full JID and network address of StreamHost/Initiator as well as the StreamID (SID) of the proposed bytestream.</p></li>
<li><p>Target opens a TCP socket to the specified network address.</p></li>
<li><p>Target requests connection
via SOCKS5, with the DST.ADDR and DST.PORT parameters set to the values defined below.</p></li>
<li><p>StreamHost/Initiator sends acknowledgement of successful connection to Target via SOCKS5.</p></li>
<li><p>Target sends IQ-result to Initiator, preserving the 'id' of the initial IQ-set.</p></li>
<li><p>StreamHost/Initiator activates the bytestream.</p></li>
<li><p>Initiator and Target may begin using the bytestream.</p></li>
<p>Mediated connection is slightly more complicated. In this situation, the StreamHost is not the Initiator but a Proxy, which means that the Initiator must discover the network address of the StreamHost before sending the initial IQ-set, must negotiate a connection with the StreamHost in the same way that the Target does, and must request that the StreamHost activate the bytestream before it can be used. The process for establishing bytestreams in this case is as follows:</p>
<ol>
<li><p>Optionally, Initiator discovers the network address of StreamHost in-band.</p></li>
<li><p>Initiator sends IQ-set to Target specifying the full JID and network address of StreamHost as well as the StreamID (SID) of the proposed bytestream.</p></li>
<li><p>Target opens a TCP socket to the selected StreamHost.</p></li>
<li><p>Target establishes connection via SOCKS5, with the DST.ADDR and DST.PORT parameters set to the values defined below.</p></li>
<li><p>StreamHost sends acknowledgement of successful connection to Target via SOCKS5.</p></li>
<li><p>Target sends IQ-result to Initiator, preserving the 'id' of the initial IQ-set.</p></li>
<li><p>Initiator opens a TCP socket at the StreamHost.</p></li>
<li><p>Initiator establishes connection via SOCKS5, with the DST.ADDR and DST.PORT parameters set to the values defined below.</p></li>
<li><p>StreamHost sends acknowledgement of successful connection to Initiator via SOCKS5.</p></li>
<li><p>Initiator sends IQ-set to StreamHost requesting that StreamHost activate the bytestream associated with the StreamID.</p></li>
<li><p>StreamHost activates the bytestream. (Data is now relayed between the two SOCKS5 connections by the proxy.)</p></li>
<li><p>StreamHost sends IQ-result to Initiator acknowledging that the bytestream has been activated (or specifying an error).</p></li>
<li><p>Initiator and Target may begin using the bytestream.</p></li>
<p>Before attempting to initiate a bytestream, the Initiator may want to know if the Target supports the bytestream protocol. It may do so using &xep0030; as follows:</p>
<p>In this case, the bytestreams proxy is hosted on a third party server "example.net", where the proxy itself has an address of "streamhostproxy.example.net".</p>
<section2topic='Initiator Queries Proxy to Find Out if it is a Proxy'anchor='proto-proxyinfo'>
<p>For each item in the disco#items result, the Initiator must query to determine if it is a bytestreams proxy. It may do so using Service Discovery as follows:</p>
<examplecaption='Initiator Sends Service Discovery Request to Proxy'><![CDATA[
<p>The proxy will return its information. The Initiator SHOULD inspect each identity to see if it contains an identity of category "proxy" and type "bytestreams".</p>
<examplecaption='Server Replies to Service Discovery Request'><![CDATA[
<section2topic='Initiator Discovers Network Address of StreamHost'anchor='proto-address'>
<p>If the StreamHost is a Proxy, the Initiator must first request the full network address used for bytestreaming (obviously this is not required if the StreamHost is the Initiator). This is done by sending an IQ-get to the proxy in the bytestreams namespace, as follows:</p>
<examplecaption='Initiator Requests Network Address from Proxy'><![CDATA[
<p>The <streamhost/> element specifying a network address MUST possess the following attributes:</p>
<ul>
<li><cite>jid</cite> = the JID of the StreamHost for communications over Jabber</li>
</ul>
<p>In addition, the <streamhost/> element MUST include:</p>
<p>EITHER</p>
<ul>
<li><cite>host</cite> = the hostname or IP address of the StreamHost for SOCKS5 communications over TCP</li>
<li><cite>port</cite> = a port associated with the hostname or IP address for
SOCKS5 communications over TCP</li>
</ul>
<p>OR</p>
<ul>
<li><cite>zeroconf</cite> = a zeroconf <note>Zeroconf is a set of protocols that enable IP networking without the need for configuration. For further information, refer to <<linkurl='http://www.zeroconf.org/'>http://www.zeroconf.org/</link>>.</note> identifier to which an entity may connect, for which the service identifier and protocol name SHOULD be "_jabber.bytestreams".</li>
</ul>
<examplecaption='Proxy Informs Initiator of Network Address'><![CDATA[
<p>If the Initiator does not have permissions to initiate bytestreams on the Proxy for whatever reason (e.g., a proxy implementation may enable administrators to ban JIDs or domains from using the Proxy), the Proxy MUST return a &forbidden; error to the Initiator (for information about error syntax, refer to &xep0086;):</p>
<section2topic='Initiator Informs Target of StreamHosts'anchor='proto-inform'>
<p>In order to establish a bytestream between the Initiator and the Target, the Initiator must provide network address information for the StreamHost(s) to the Target. This happens in-band via a single IQ-set, which must contain the following information:</p>
<ul>
<li>The network address of at least one StreamHost to which the Target may attempt to connect</li>
<li>The Stream ID for this connection</li>
<li>The "mode" to use, normally "tcp" but optionally "udp" (see the <linkurl='#udp'>Optional UDP Support</link> section of this document)</li>
</ul>
<p>The protocol format is shown below.</p>
<examplecaption='Initiation of Interaction'><![CDATA[
<section2topic='Target Establishes SOCKS5 Connection with StreamHost'anchor='proto-establish'>
<p>If the Target is willing to accept the bytestream, it MUST attempt to open a standard TCP socket on the network address of the StreamHost communicated by the Initiator. If the Initiator provides more than one StreamHost, the Target SHOULD try to connect to them in the order they occur.</p>
<p>If the Target tries but is unable to connect to any of the StreamHosts and it does not wish to attempt a connection from its side, it MUST return a ¬found; error to the Initiator.</p>
<examplecaption='Target Is Unable to Connect to Any StreamHost and Wishes to End Transaction'><![CDATA[
<p>If the Target is able to open a TCP socket on a StreamHost, it MUST utilize the SOCKS5 protocol specified in &rfc1928; to establish the connection with the StreamHost. In accordance with the SOCKS5 RFC, the Target MAY have to authenticate in order to use the proxy. However, any authentication required is beyond the scope of this document.</p>
<p>Once the Target has successfully authenticated with the Proxy (even anonymously), it SHOULD send a CONNECT request to the appropriate host in order to continue the negotiation. The following rules apply:</p>
<ol>
<li>The hostname MUST be SHA1(SID + Initiator JID + Target JID) where the definition of the SHA1 hashing algorithm is as specified by &rfc3174; and the output is hexadecimal-encoded (not binary).</li>
<li>The port MUST be 0 (zero).</li>
<li>The JIDs provided MUST be the JIDs used for the IQ exchange, which MAY be full JIDs (&FULLJID;) or bare JIDs (&BAREJID;).</li>
<li>The appropriate stringprep profiles (as specified in &xmppcore;) MUST be applied to the JIDs before application of the SHA1 hashing algorithm.</li>
<p>When replying to the client in accordance with Section 6 of <cite>RFC 1928</cite>, the StreamHost SHOULD set the BND.ADDR and BND.PORT to the values provided by the client in the connection request.</p>
<p>If the StreamHost used is a Proxy, the Initiator MUST authenticate and establish a connection with the StreamHost before requesting that the StreamHost activate bytestream. The Initiator will establish a connection to the SOCKS5 proxy in the same way the Target did (passing the same value for the CONNECT request), as shown in the following examples.</p>
<examplecaption='Initiator Connects to StreamHost'><![CDATA[
<section2topic='Activation of Bytestream'anchor='proto-activation'>
<p>In order for the bytestream to be used, it MUST first be activated by the StreamHost. If the StreamHost is the Initiator, this is straightforward and does not require any in-band protocol. However, if the StreamHost is a Proxy, the Initiator MUST send an in-band request to the StreamHost. This is done by sending an IQ-set to the Proxy, including an <activate/> element whose XML character data specifies the full JID of the Target.</p>
<examplecaption='Initiator Requests Activation of Bytestream'><![CDATA[
<p>Using this information, with the SID and from address on the packet, the Proxy is able to activate the stream by hashing the SID + Initiator JID + Target JID. This provides a reasonable level of trust that the activation request came from the Initiator.</p>
<p>If the Proxy can fulfill the request, it MUST then respond to the Initiator with an IQ-result.</p>
<examplecaption='Proxy Informs Initiator of Activation'><![CDATA[
<p>If the Proxy cannot fulfill the request, it MUST return an IQ-error to the Initiator; the following conditions are defined:</p>
<ul>
<li>¬found; error if the from address does not match that of the Initiator's full JID</li>
<li>¬allowed; error if only one party (either Initiator or Recipient, but not both) is connected to the Proxy</li>
<li>&internalserver; error if the proxy cannot activate the bytestream because of some internal malfunction</li>
</ul>
</section2>
</section1>
<section1topic='Formal Use Case'anchor='usecase'>
<p>This is a formal representation of the narrative information provided above. The primary actor is the Initiator and the goal is to establish a bytestream between the Initiator and the Target. (Note: "UCE" stands for "Use Case Ends" (success is assumed unless otherwise specified), "P" stands for "Primary Flow", and "A" stands for "Alternate Flow".)</p>
<p>The <query/> element is the container for all in-band communications. This element MUST be in the namespace "http://jabber.org/protocol/bytestreams". This element has a single attribute for the stream session identifier, and contains multiple <streamhost/> elements, a single <streamhost-used/> element, or a single <activate/> element.</p>
<p>The "sid" specifies the bytestream session identifier. This attribute MUST be present. The value of this attribute is any character data.</p>
<p>The "mode" specifies the mode to use, either 'tcp' or 'udp'. If this attribute is missing, the default value of "tcp" MUST be assumed.</p>
<p>The <streamhost/> element conveys the network connection information. At least one instance MUST be present in the initial IQ-set from the Initiator to the Target. If multiple instances of this element are present, each one MUST be a separate host/port combination.</p>
<p>The <streamhost-used/> element transports the out-of-band token. It MUST be present in the IQ-set from the Target to the Initiator, and there MUST be only one instance.</p>
<p>The <activate/> element is used to request activation of a unidirectional or bidirectional bytestream. It MUST be present in the IQ-set sent from the Initiator to the StreamHost after the Initiator receives an IQ-result from the Target, and there MUST be only one instance.</p>
<p>The <streamhost/> element contains the bytestream connection information. This element has attributes for the StreamHost's JID, network host/address, and network port. This element MUST NOT contain any content nodes.</p>
<p>The "jid" attribute specifies the StreamHost's JID. This attribute MUST be present, and MUST be a valid JID for use with an <iq/>.</p>
<p>The "host" attribute specifies the host to connect to. This attribute MUST be present. The value MUST be either a resolvable domain name or the "dotted decimal" IP address (e.g. "1.2.3.4").</p>
<p>The "port" attribute specifies the port to connect to. This attribute MAY be present. The value MUST be a valid port number in decimal form.</p>
<p>The "zeroconf" attribute specifies the zero-configuration service available for bytestreaming. This attribute SHOULD be present. The value SHOULD be '_jabber.bytestreams'.</p>
<p>When communicating the available hosts, the Initiator MUST include EITHER the host and port OR the zeroconf information.</p>
<p>The <streamhost-used/> element indicates the StreamHost connected to. This element has a single attribute for the JID of the StreamHost to which the Target connected. This element MUST NOT contain any content node.</p>
<p>The "jid" attribute specifies the full JID of the StreamHost. This attribute MUST be present, and MUST be a valid JID for use with an <iq/>.</p>
<p>Support for UDP associations is strictly OPTIONAL. However, implementations that support UDP associations MUST adhere to the profile described in this section.</p>
<p>If an implementation supports UDP associations, it MUST advertise that separately by returning a feature of 'http://jabber.org/protocol/bytestreams#udp' in response to <cite>Service Discovery</cite> information requests.</p>
<examplecaption='Initiator Sends Service Discovery Request to Target'><![CDATA[
<p>There is one main difference between UDP mode and TCP mode: rather than simply establishing a TCP connection, the Target and/or Initiator MUST (1) establish a UDP association and then (2) initialize the UDP channel. In particular:</p>
<ul>
<li>If direct connection is followed, Target MUST complete UDP association and initialization of the UDP channel before informing Initiator of success via the <streamhost-used/> element.</li>
<li>If mediated connection is followed, (1) Target MUST complete UDP association and initialization of the UDP channel before informing Initiator of success via the <streamhost-used/> element, and (2) Initiator MUST complete UDP association and initialization of the UDP channel before asking StreamHost to activate the bytestream.</li>
</ul>
<p>The processes for establishing the UDP association and for initializing the UDP channel are described below.</p>
<section3topic='Establishing the UDP Association'anchor='udp-process-assoc'>
<p>Once the Target has successfully authenticated with the Proxy (as described under <linkurl='#proto-establish'>Target Establishes SOCKS5 Connection with StreamHost</link>), it MUST send a UDP ASSOCIATE (rather than CONNECT) request to the host identified by the algorithm defined above.</p>
<examplecaption='Target Requests UDP Association with StreamHost'><![CDATA[
<section3topic='Initializing the UDP Channel'anchor='udp-process-init'>
<p>After connecting to the StreamHost, the Target (direct connection) or both Target and Initiator (mediated connection) MUST initialize the UDP channel. In order to do so, each sending entity MUST send a SOCKS5 UDP packet to the StreamHost on the same port used for the initial TCP connection (in the foregeoing example, a host of 192.168.4.1 and port of 5086), with DST.PORT set to '1' and DATA containing the sending entity's JID (i.e, the JID of either the Target or Initiator).</p>
<examplecaption='Target or Initiator Sends UDP Initialization Packet to StreamHost'><![CDATA[
<udpsuccessxmlns='http://jabber.org/protocol/bytestreams'dstaddr='Value of Hash'/>
</message>
]]></example>
<p>The <udpsuccess/> element indicates that the StreamHost has received a UDP initialization packet. This element has a single attribute containing the DST.ADDR that was used in the UDP packet.</p>
<p>If Target is unable to initialize the UDP channel, it MUST return a &remoteserver; error to Initiator.</p>
<p>Note: Since UDP is not reliable, the Target SHOULD resend the UDP packet if the reply notification is not received within a short time (a 5-second retry is RECOMMENDED). The StreamHost SHOULD ignore duplicate UDP initialization packets once it has replied with a notification.</p>
<p>Once the UDP association is established, UDP packets can be exchanged with the StreamHost. When a UDP packet is sent by either party, it MUST contain a 4-byte header (in addition to other possible headers, such as that of SOCKS5), which consists of the source virtual port and then the destination virtual port of the packet, both 16-bit values in network byte order. This allows the peers to multiplex many packets for different purposes over one session. The actual application data should follow this header, and thus the payload size will always be "Application Data Size + 4".</p>
<p>For all packets sent to the StreamHost, DST.PORT is set to 0, and DATA contains the payload.</p>
<examplecaption='Sending UDP to StreamHost'><