<abstract>This document defines an XMPP protocol extension for establishing an out-of-band bytestream between any two XMPP users, mainly for the purpose of file transfer. The bytestream can be either direct (peer-to-peer) or mediated (though a special-purpose proxy server). The typical transport protocol used is TCP, although UDP can optionally be supported as well.</abstract>
<remark><p>Incorporated errata: specified format for SHA1 output; specified BND.ADDR and BND.PORT for SOCKS5 reply; removed extraneous SOCKS5 acknowledgement example from Section 4.9; clarified rules for creation of SOCKS5 connection request in Section 4.6; added examples to Section 4.8; specified that ATYP value is hardcoded to 3 in this usage.</p></remark>
<remark><p>Cleaned up narratives to reflect current practices and removed unnecessary authentication references; fixed mismatch SOCKS5 parameter table values.</p></remark>
<remark><p>Removed SIDs from the result queries, key off the IQ 'id' attribute instead. Added the disco exchange for finding available proxies.</p></remark>
<remark><p>Clarified that this proposal uses an adaptation of the SOCKS5 protocol, not the full protocol; replaced DTD with schema; added security considerations.</p></remark>
<remark><p>Added service discovery example; added 'srvid' attribute to streamhost element and required inclusion of either 'srvid' or 'port' attribute; improved the algorithms for generating SOCKS5 UNAME and PASSWD parameters; specified that the DST.ADDR and DST.PORT parameters can be ignored; removed references to connected/disconnected notification, bidirectional bytestreams, and multiple targets; updated implementation notes.</p></remark>
<remark><p>Specified option of "reversing the connection" (Target becomes Initiator); added more error cases; resurrected and cleaned up formal use case.</p></remark>
<remark><p>Added section on connected/disconnected notifications sent from Proxy to Initiator; cleaned up several examples; specified more error conditions; clarified the formal descriptions; added implementation notes and future considerations.</p></remark>
<p>XMPP is designed for sending relatively small fragments of XML between network entities (see &xmppcore;) and is not designed for sending binary data. However, sometimes it is desirable to send binary data to another entity that one has discovered on the XMPP network (e.g., to send a file). Therefore it is valuable to have a generic protocol for streaming binary data between any two entities on an XMPP network. The main application for such a bytestreaming technology is file transfer as specified in &xep0096; and &xep0234;. However, other applications are possible, which is why it is important to develop a generic protocol rather than one that is specialized for a particular application such as file transfer.</p>
<p>This document defines a protocol that meets the following conditions:</p>
<li>Bytestreams are established over standard TCP connections (&rfc0793;) or UDP associations (&rfc0768;), where TCP support is REQUIRED and UDP support is OPTIONAL</li>
<p>Specifically, this document makes use of the SOCKS 5 protocol, which is an IETF-approved, IPv6-ready technology for bytestreams defined in &rfc1928;. However, because this proposal uses a subset of the SOCKS5 protocol that is specially adapted for bytestreaming over XMPP, existing SOCKS5 proxies cannot be used to implement this proposal without modifications.</p>
<p>Early versions of this specification documented only the use of TCP connections. In version 1.6 (approved in November 2004), optional UDP associations were added, as described in the <linkurl='#udp'>Optional UDP Support</link> section of this document. However, the main body of this document describes the use of TCP, which is the primary method of SOCKS5 Bytestreams ("S5B").</p>
</section1>
<section1topic='Terminology'anchor='terms'>
<p>The following terms are used throughout this document.</p>
<dl>
<di>
<dt>Requester</dt>
<dd>The entity that starts a bytestream negotiation with a Target. <note>Before version 1.8 of this document a Requester was known as an Initiator.</note></dd>
</di>
<di>
<dt>Target</dt>
<dd>The entity with which the Requester is attempting to establish a bytestream.</dd>
</di>
<di>
<dt>Proxy</dt>
<dd>An entity that is willing to be a middleman for the bytestream between the Requester and the Target.</dd>
</di>
<di>
<dt>StreamHost</dt>
<dd>The system that the Target connects to and that is "hosting" the bytestream; the Streamhost can be either the Requester or a Proxy.</dd>
</di>
<di>
<dt>StreamID</dt>
<dd>A relatively unique Stream ID for this connection; this is generated by the Requester for tracking purposes and MUST be less than 128 characters in length.</dd>
</di>
</dl>
<p>Note: Because either party can attempt to establish a bytestream (this is formalized in &xep0260;), the Requester and the Target roles apply to a particular S5B negotiation, and do not map to the Initiator and Responder roles from &xep0166; in a fixed way. For example, during a Jingle negotiation the Initiator might first take on the role of an S5B Requester but if that first bytestreams negotiation fails then the Jingle Responder might take on the role of an S5B Requester.</p>
<p>In the protocol flow diagrams, the line types have the following meaning:</p>
<ul>
<li>"----" ... communications over XMPP</li>
<li>"____" ... communications over TCP</li>
<li>"\\\\" and "////" ... communications over SOCKS 5</li>
<li>"====" ... communications over the bytestream itself</li>
</ul>
<p>In the examples, "streamer.example.com" is a Proxy that services bytestreams on port 7625.</p>
<p>If an entity supports this protocol, it MUST advertise that fact in its responses to &xep0030; information ("disco#info") requests by returning a feature of "http://jabber.org/protocol/bytestreams".</p>
<examplecaption='Requester Sends Service Discovery Request to Target'><![CDATA[
<p>Before attempting to initiate a bytestream, the Requester might need to find a proxy (e.g., if it has not been configured to know about a proxy). It can do so using Service Discovery by communicating with its server.</p>
<examplecaption='Requester Sends Service Discovery Request to Server'><![CDATA[
<p>The proxy returns its information and the Requester inspects it to determine if it contains an identity of category "proxy" and type "bytestreams".</p>
<examplecaption='Server Replies to Service Discovery Request'><![CDATA[
<p>Next the Requester needs to request the full network address to be used for bytestreaming through the Proxy. This is done by sending an IQ-get to the proxy containing a &QUERY; element qualified by the bytestreams namespace. <note>Before version 1.8 of this specification, the &QUERY; element in this use case possessed a 'sid' attribute; however, it is unnecessary for the Requester to specify the StreamID here and it would be harmful for the Proxy to reserve the StreamID at this point because the StreamID might never be used and because the Requester might use the Proxy's services for multiple different streams.</note></p>
<examplecaption='Requester Requests Network Address from Proxy'><![CDATA[
<p>The Proxy replies by returning an IQ-result that contains its network address, structured using the <streamhost/> child of the &QUERY; element; the <streamhost/> element MUST possess the following attributes:</p>
<ul>
<li><cite>host</cite> = the hostname or IP address of the StreamHost for SOCKS5 communication over TCP</li>
<li><cite>jid</cite> = the JabberID of the StreamHost for communication over XMPP</li>
<li><cite>port</cite> = a port associated with the hostname or IP address for SOCKS5 communication over TCP</li>
</ul>
<examplecaption='Proxy Informs Requester of Network Address'><![CDATA[
<p>If the Requester does not have permissions to initiate bytestreams on the Proxy for whatever reason (e.g., a proxy implementation might enable administrators to ban JIDs or domains from using the Proxy), the Proxy MUST return a &forbidden; error to the Requester.</p>
<examplecaption='Requester is Forbidden to use Proxy'><![CDATA[
<p>In this situation, the StreamHost is the Requester, which means that the Requester knows the network address of the StreamHost and knows when to activate the bytestream.</p>
<section2topic='Process'anchor='direct-process'>
<p>For direct connections, the process for establishing a bytestream is as follows:</p>
<ol>
<li><p>Requester initiates S5B negotiation with Target by sending an IQ-set that includes the full JID &LOCALFULL; and network address of StreamHost/Requester as well as the StreamID (SID) of the proposed bytestream.</p></li>
<li><p>Target opens a TCP socket to the specified network address at the StreamHost/Requester.</p></li>
<li><p>Target requests SOCKS5 connection at StreamHost/Requester.</p></li>
<li><p>StreamHost/Requester sends acknowledgement of successful connection to Target via SOCKS5.</p></li>
<li><p>Target accepts the S5B stream by returning an IQ-result to the Requester, preserving the 'id' of the initial IQ-set.</p></li>
<li><p>Requester and Target exchange data over the bytestream.</p></li>
<p>To initiate an S5B negotiation with the Target, the Requester sends the Target network address information about one or more StreamHosts. In the case of a direct connection, the Requester might include information only about itself (as shown in the following example) or about itself and a Proxy. The &QUERY; element MUST possess a 'sid' attribute the specifies the Stream ID for this bytestream and MAY possess a 'mode' attribute whose value is "tcp" (the default) or "udp"; the &QUERY; element also MUST contain one or more <streamhost/> elements, each of which MUST possess the 'host', 'jid', and 'port' attributes.</p>
<p>If the Target is willing to negotiate a bytestream, it proceeds as shown in the following sections.</p>
</section3>
<section3topic='Target Establishes SOCKS5 Connection with StreamHost/Requester'anchor='direct-proto-establish'>
<p>Next the Target attempts to open a standard TCP socket on the network address of the StreamHost/Requester.</p>
<p>Note: If the Requester provides more than one StreamHost, the Target SHOULD try to connect to them in the order of the <streamhost/> children within the &QUERY; element. &xep0260; modifies this rule by providing explicit priorities for each streamhost candidate.</p>
<p>If the Target is able to open a TCP socket on a StreamHost/Requester, it MUST use the SOCKS5 protocol to establish a SOCKS5 connection. In accordance with <cite>RFC 1928</cite>, the Target might need to authenticate in order to use the proxy. However, any authentication required is beyond the scope of this document.</p>
<p>Once the Target has successfully authenticated with the StreamHost/Requester, it sends a CONNECT request to the appropriate host in order to continue the negotiation. The following rules apply:</p>
<ol>
<li>The hostname MUST be SHA1(SID + Requester JID + Target JID) where the definition of the SHA1 hashing algorithm is as specified by &rfc3174; and the output is hexadecimal-encoded (not binary).</li>
<li>The port MUST be 0 (zero).</li>
<li>The JIDs provided MUST be the JIDs used for the IQ exchange, which MAY be full JIDs &FULLJID; or bare JIDs &BAREJID;.</li>
<li>The appropriate stringprep profiles (as specified in &xmppcore;) MUST be applied to the JIDs before application of the SHA1 hashing algorithm.</li>
</ol>
<examplecaption='Target Establishes SOCKS5 Connection with StreamHost'><![CDATA[
<p>When replying to the Target in accordance with Section 6 of <cite>RFC 1928</cite>, the StreamHost MUST set the BND.ADDR and BND.PORT to the values provided by the client in the connection request.</p>
<p>If the Target tries but is unable to connect to any of the StreamHosts and it does not wish to attempt a connection from its side, it MUST return an ¬found; error to the Requester.</p>
<examplecaption='Target Is Unable to Connect to Any StreamHost and Wishes to End Negotiation'><![CDATA[
<p>After the Target has authenticated with the StreamHost/Requester, it replies to the initiate request with an IQ-result whose &QUERY; element contains a <streamhost-used/> child that specifies which StreamHost was used (in this case, the StreamHost/Requester).</p>
<examplecaption='Target Notifies Requester of Bytestream'><![CDATA[
<p>At this point, the Requester knows which StreamHost was used by the Target and the parties are able to use the StreamHost/Requester to exchange data over the bytestream.</p>
<p>In this situation, the StreamHost is not the Requester but a Proxy, which means that the Requester needs to discover the network address of the StreamHost before sending the initiation request to the Target, needs to negotiate a connection with the StreamHost in the same way that the Target does, and needs to ask the StreamHost to activate the bytestream before it can be used.</p>
<li><p>As a precondition, the Requester optionally discovers the network address of StreamHost over XMPP as discussed in the <linkurl='#disco'>Service Discovery</link> section of this document.</p></li>
<li><p>Requester initiates S5B negotation with Target by sending IQ-set that includes the JabberID and network address of StreamHost as well as the StreamID (SID) of the proposed bytestream.</p></li>
<li><p>Target opens a TCP socket to the selected StreamHost.</p></li>
<li><p>Target requests SOCKS5 connection at StreamHost/Proxy.</p></li>
<li><p>StreamHost sends acknowledgement of successful connection to Target via SOCKS5.</p></li>
<li><p>Target sends IQ-result to Requester, preserving the 'id' of the initial IQ-set.</p></li>
<li><p>Requester opens a TCP socket at the StreamHost.</p></li>
<li><p>Requester establishes connection via SOCKS5, with the DST.ADDR and DST.PORT parameters set to the values defined below.</p></li>
<li><p>StreamHost sends acknowledgement of successful connection to Requester via SOCKS5.</p></li>
<li><p>Requester sends IQ-set to StreamHost requesting that StreamHost activate the bytestream associated with the StreamID.</p></li>
<li><p>StreamHost activates the bytestream. (Data is now relayed between the two SOCKS5 connections by the proxy.)</p></li>
<li><p>StreamHost sends IQ-result to Requester acknowledging that the bytestream has been activated (or specifying an error).</p></li>
<li><p>Requester and Target can begin using the bytestream.</p></li>
<p>To initiate an S5B negotiation with the Target, the Requester sends the Target network address information about one or more StreamHosts. In the case of a mediated connection, the Requester might include information only about the Proxy (as shown in the following example) or about the Proxy and itself. The &QUERY; element MUST possess a 'sid' attribute the specifies the Stream ID for this bytestream and MAY possess a 'mode' attribute whose value is "tcp" (the default) or "udp"; the &QUERY; element also MUST contain one or more <streamhost/> elements, each of which MUST possess the 'host', 'jid', and 'port' attributes.</p>
<p>If the Target is willing to negotiate a bytestream, it proceeds as shown in the following sections.</p>
</section3>
<section3topic='Target Establishes SOCKS5 Connection with Proxy'anchor='mediated-proto-establish'>
<p>Next the Target attempts to open a standard TCP socket on the network address of the Proxy.</p>
<p>If the Target is able to open a TCP socket on the Proxy, it uses the SOCKS5 protocol to establish a SOCKS5 connection. In accordance with <cite>RFC 1928</cite>, the Target might need to authenticate in order to use the proxy. However, any authentication required is beyond the scope of this document.</p>
<p>Once the Target has successfully authenticated with the Proxy, it sends a CONNECT request to the appropriate host in order to continue the negotiation. The following rules apply:</p>
<ol>
<li>The hostname MUST be SHA1(SID + Requester JID + Target JID) where the definition of the SHA1 hashing algorithm is as specified by &rfc3174; and the output is hexadecimal-encoded (not binary).</li>
<li>The port MUST be 0 (zero).</li>
<li>The JIDs provided MUST be the JIDs used for the IQ exchange, which MAY be full JIDs &FULLJID; or bare JIDs &BAREJID;.</li>
<li>The appropriate stringprep profiles (as specified in &xmppcore;) MUST be applied to the JIDs before application of the SHA1 hashing algorithm.</li>
</ol>
<examplecaption='Target Establishes SOCKS5 Connection with StreamHost'><![CDATA[
<p>When replying to the Target in accordance with Section 6 of <cite>RFC 1928</cite>, the Proxy MUST set the BND.ADDR and BND.PORT to the values provided by the client in the connection request.</p>
<p>After the Target has established a SOCKS5 connection with the Proxy, it replies to the initiate request with an IQ-result whose &QUERY; element contains a <streamhost-used/> child that specifies which StreamHost was used (in this case, the Proxy).</p>
<examplecaption='Target Notifies Requester of Bytestream'><![CDATA[
<p>At this point, the Requester knows which StreamHost was used by the Target.</p>
</section3>
<section3topic='Requester Establishes SOCKS5 Connection with StreamHost'anchor='mediated-proto-initiator'>
<p>Here, unlike the direct connection case described above, the Requester also needs to establish a SOCKS5 connection to the Proxy before the parties are able to use the Proxy to exchange data over the bytestream. Therefore the Requester will establish a connection to the SOCKS5 proxy in the same way the Target did (passing the same value for the CONNECT request), as shown in the following examples.</p>
<examplecaption='Requester Connects to StreamHost'><![CDATA[
<section3topic='Activation of Bytestream'anchor='mediated-proto-activation'>
<p>Next the Requester needs to activate the bytestream with the Proxy. This is done by sending an IQ-set to the Proxy, including an <activate/> element whose XML character data specifies the full JID of the Target.</p>
<examplecaption='Requester Requests Activation of Bytestream'><![CDATA[
<p>Using this information, with the SID and from address on the packet, the Proxy is able to activate the stream by hashing the SID + Requester JID + Target JID. This provides a reasonable level of trust that the activation request came from the Requester.</p>
<p>If the Proxy can fulfill the request, it MUST respond to the Requester with an IQ-result.</p>
<examplecaption='Proxy Informs Requester of Activation'><![CDATA[
<iqfrom='streamer.example.com'
id='oqx6t1c9'
to='requester@example.com/foo'
type='result'/>
]]></example>
<p>At this point the parties can begin exchanging data over the bytestream.</p>
<p>If the Proxy cannot fulfill the request, it MUST return an IQ-error to the Requester; the following conditions are defined:</p>
<ul>
<li>¬found; error if the 'from' address does not match that of the Requester's full JID</li>
<li>¬allowed; error if only one party (either Requester or Recipient, but not both) is connected to the Proxy</li>
<li>&internalserver; error if the proxy cannot activate the bytestream because of some internal malfunction</li>
<p>The <query/> element is the container for all in-band communications. This element MUST be qualified by the "http://jabber.org/protocol/bytestreams" namespace. Depending on the use case, this element contains multiple <streamhost/> elements, a single <streamhost-used/> element, or a single <activate/> element.</p>
<p>The "sid" specifies the bytestream session identifier. The value of this attribute is any character data.</p>
<p>The <streamhost/> element conveys the network connection information. At least one instance MUST be present in the initial IQ-set from the Requester to the Target. If multiple instances of this element are present, each one MUST be a separate host/port combination.</p>
<p>The <streamhost-used/> element transports the out-of-band token. It MUST be present in the IQ-set from the Target to the Requester, and there MUST be only one instance.</p>
<p>The <activate/> element is used to request activation of a unidirectional or bidirectional bytestream. It MUST be present in the IQ-set sent from the Requester to the StreamHost after the Requester receives an IQ-result from the Target, and there MUST be only one instance.</p>
<p>The <streamhost/> element contains the bytestream connection information. This element has attributes for the StreamHost's JID, network host/address, and network port. This element MUST NOT contain any content nodes.</p>
<p>The "jid" attribute specifies the StreamHost's JID. This attribute MUST be present, and MUST be a valid JID for use with an <iq/>.</p>
<p>The "host" attribute specifies the host to connect to. This attribute MUST be present. The value MUST be either a resolvable domain name or the "dotted decimal" IP address (e.g. "1.2.3.4").</p>
<p>The "port" attribute specifies the port to connect to. This attribute MAY be present. The value MUST be a valid port number in decimal form.</p>
<p>The <streamhost-used/> element indicates the StreamHost connected to. This element has a single attribute for the JID of the StreamHost to which the Target connected. This element MUST NOT contain any content node.</p>
<p>Support for UDP associations is strictly OPTIONAL. However, implementations that support UDP associations MUST adhere to the profile described in this section.</p>
<p>If an implementation supports UDP associations, it MUST advertise that separately by returning a feature of 'http://jabber.org/protocol/bytestreams#udp' in response to <cite>Service Discovery</cite> information requests.</p>
<p>There is one main difference between UDP mode and TCP mode: rather than simply establishing a TCP connection, the Target and/or Requester MUST (1) establish a UDP association and then (2) initialize the UDP channel. In particular:</p>
<li>If direct connection is followed, Target MUST complete UDP association and initialization of the UDP channel before informing Requester of success via the <streamhost-used/> element.</li>
<li>If mediated connection is followed, (1) Target MUST complete UDP association and initialization of the UDP channel before informing Requester of success via the <streamhost-used/> element, and (2) Requester MUST complete UDP association and initialization of the UDP channel before asking StreamHost to activate the bytestream.</li>
<p>The processes for establishing the UDP association and for initializing the UDP channel are described below.</p>
<section3topic='Establishing the UDP Association'anchor='udp-process-assoc'>
<p>Once the Target has successfully authenticated with the Proxy (as described under <linkurl='#proto-establish'>Target Establishes SOCKS5 Connection with StreamHost</link>), it MUST send a UDP ASSOCIATE (rather than CONNECT) request to the host identified by the algorithm defined above.</p>
<examplecaption='Target Requests UDP Association with StreamHost'><