From 8e4f9a5d8206beaaaf8ac1a0af06bb8730e71853 Mon Sep 17 00:00:00 2001 From: Peter Saint-Andre Date: Wed, 19 Jun 2013 20:27:42 -0600 Subject: [PATCH] 0.0.7 --- inbox/http-over-xmpp.xml | 1039 +++++++++++++++++++++++--------------- 1 file changed, 619 insertions(+), 420 deletions(-) diff --git a/inbox/http-over-xmpp.xml b/inbox/http-over-xmpp.xml index 274fb509..e6c317f2 100644 --- a/inbox/http-over-xmpp.xml +++ b/inbox/http-over-xmpp.xml @@ -11,6 +11,7 @@ TODO: Inform: * Scientists (XMPP, CoAP, MQTT) * W3C Technical Architecture Group (TAG) (dev of REST) * SPARQL WG +* Special feature if server contains SPARQL Endpoing (separate XEP?). --> @@ -39,6 +40,8 @@ TODO: Inform: XMPP Core XEP-0001 XEP-0030 + XEP-0047 + XEP-0131 XEP-0137 XEP-0166 @@ -52,6 +55,51 @@ TODO: Inform: peter.waher@jabber.org http://se.linkedin.com/pub/peter-waher/1a/71b/a29/ + + 0.0.6 + 2013-06-17 + pw + +

Removed the streamBase64 transfer mechanism and replaced it with In-band bytestreams (IBB).

+

Created new examples, avoiding re-use of known public examples.

+

Extended the descriptions of the different transfer mechanisms.

+

Ability to state capabilities of the client in the request.

+ + + + 0.0.5 + 2013-05-22 + pw + +

Changed IQ stanza type from 'get' to 'set' for all HTTP methods.

+ + + + 0.0.4 + 2013-05-14 + pw + +

Updated format of encoding table, and made it into a definition list.

+

Language corrections.

+ + + + 0.0.3 + 2013-05-10 + pw + +

Added more information about chunking.

+

Added implementation notes regarding bandwidth and stanza size limitations in XMPP Servers.

+ + + + 0.0.2 + 2013-05-08 + pw + +

Added support for XEP-0131, Stanza Headers and Internet Metadata.

+ + 0.0.1 2013-05-05 @@ -63,11 +111,11 @@ TODO: Inform:

- Many documents have been written about how to transport XMPP datagrams using HTTP. The motivation behind such solutions has often been to be able to use XMPP in + Many documents have been written on how to transport XMPP datagrams using HTTP. The motivation behind such solutions has often been to be able to use XMPP in scripting languages such as Java Script running in web browsers.

- But very little has been written up to this point about the reverse: How to transport HTTP methods and HTTP responses over an XMPP-based peer-to-peer network. + But up to this point very little has been written about the reverse: How to transport HTTP methods and HTTP responses over an XMPP-based peer-to-peer network. Here, the motivation is as follows: There are multitudes of applications and APIs written that are based on HTTP over TCP as the basic communication transport protocol. As these are moving closer and closer to the users, problems arise when the users want to protect their data and services using firewalls. Even though there are methods today to open up firewalls manually or automatically permit communication with such devices and applications, you still open up the application for everybody. This @@ -80,7 +128,7 @@ TODO: Inform:

Furthermore, with the advent of semantic web technologies and its use in web 3.0 and Internet of Things applications, such applications move even more rapidly into - the privte spheres of the users, where security and privacy is of paramount importance, the need to use more secure transport protocols than HTTP over TCP is necessary. + the private spheres of the users, where security and privacy is of paramount importance, it is necessary to use more secure transport protocols than HTTP over TCP.

There are many different types of HTTP-based communication that one would like to be able to transport over XMPP. A non-exhaustive list can include: @@ -109,13 +157,17 @@ TODO: Inform: Here HTTP Client authentication to resources on the HTTP Server is made by a third party, an XMPP Server.

  • - &xep0072;: This speification handles execution of SOAP-based web services specifically. This specification have some benefits regarding to Web Service calls over XMPP, - but is only one example of all types of HTTP-based communucation one would desire to transport over XMPP. + &xep0072;: This specification handles execution of SOAP-based web services specifically. This specification has some benefits regarding to Web Service calls over XMPP, + but is only one example of all types of HTTP-based communication one would desire to transport over XMPP.
  • - &xep0124;: This specification handles XMPP-based communication over HTTP sessions (BOSH), allowing for instance, java script to communicate using XMPP using the + &xep0124;: This specification handles XMPP-based communication over HTTP sessions (BOSH), allowing for instance, XMPP communication in java script using the XML HTTP Request object. This is in some way the reverse of what this document proposes to do.
    • +
  • + &xep0131;: While not directly related to HTTP, it is used to transport headers in the form of collections of key-value pairs, exactly as is done in HTTP. The format + for encoding headers into XMP defined by this XEP will be re-used in this XEP. +
  • &xep0147;: This informational specification proposes ways to define XMPP-actions using URL's. This document will propose a different URI scheme for HTTP-based resources over an XMPP transport. @@ -125,7 +177,7 @@ TODO: Inform:

    This document presupposes the server already has a web server (HTTP Server) implementation, and that it hosts content through it, content which can be both - dynamic (i.e. generated) or static (for example files) in nature. Content, which it wants to + dynamic (i.e. generated) or static (e.g. files) in nature. Content, which it wants to publish to XMPP clients as well as HTTP clients. It also presupposes that the client is aware of HTTP semantics and MIME encoding.

    @@ -135,9 +187,9 @@ TODO: Inform:
    HTTP
    - Hyper Text Transfer Protocol. Version 1.1 of HTTP is described in + Hyper Text Transfer Protocol. Version 1.1 of HTTP is described in RFC 2616 RFC 2616: Hypertext Transfer Protocol -- HTTP/1.1 <http://tools.ietf.org/html/rfc2616> - . The PATCH method is described in + . The PATCH method is described in RFC 5789 RFC 5789: PATCH Method for HTTP <http://tools.ietf.org/html/rfc5789>
    @@ -156,11 +208,11 @@ TODO: Inform:
    HTTP Request
    -
    An HTTP Request consists of a Method, version information, headers and optional body.
    +
    An HTTP Request consists of a HTTP Method, version information, headers and optional body.
    HTTP Resource
    -
    A resouce on an HTTP Server identified by a path. Each path begins with a separator character (/).
    +
    A resource on an HTTP Server identified by a path. Each path begins with a separator character (/).
    HTTP Response
    @@ -178,9 +230,8 @@ TODO: Inform:

    - All HTTP communication is done usig the Request/Response paradigm. Each HTTP Request is made sending an iq-stanza - containing a req element to the server. Each iq-stanza sent is of type get (not to be confused by the HTTP Method - GET). + All HTTP communication is done using the Request/Response paradigm. Each HTTP Request is made sending an iq-stanza + containing a req element to the server. Each iq-stanza sent is of type set.

    When the server responds, it does so by sending an iq-stanza response (type result) back to the client containing a resp @@ -189,67 +240,125 @@ TODO: Inform:

    Requests or responses containing data must also consider how this data should be encoded within the XML telegram. Normally in HTTP, content and headers are separated - by a blank line, and the transfer of the content is made in the same stream. Specific HTTP headers are used to define how the content is transfered and encoded within - the stream (Content-Type, Content-Length, Content-Encoding, Content-Transfer-Encoding). This approach is not possible if the response is to be inbedded in an XML telegram, + by a blank line, and the transfer of the content is made in the same stream. Specific HTTP headers are used to define how the content is transferred and encoded within + the stream (Content-Type, Content-Length, Content-Encoding, Content-Transfer-Encoding). This approach is not possible if the response is to be embedded in an XML telegram, since it can interfere with the encoding of the encompassing XML.

    To solve this, this document specifies additional data transfer mechanisms that are compatible with the XMPP protocol. The normal HTTP-based content transfer headers will still be transported, but do not affect the content encoding used in the XMPP transport. The following content encoding methods are available:

    -
      -
    • -

      - text: Normal text content. The text is encoded as text within XML, using the same encoding used by the XML stream. XML escape characters (<, > and &) - are escaped using the normal &lt;, &gt; and &amp; character escape sequences. -

      -
      • -
    • -

      - xml: Xml content embedded in the XML telegram. Note however, that any processing instructions or XML version statements must be avoided, since it may cause - the XML stream to become invalid XML. If this is a problem, normal text encoding can be used as an alternative. The advantage of xml - instead of text or base64 encodings is when used in conjuncion with EXI compression &xep0322;. EXI compression has the ability to - compress XML efficiently. Text will not be compressed, unless response exists in internal string tables. Base-64 encoded data will be compressed so that the 33% - size gain induced by the encoding is recaptured. -

      -
      • -
    • -

      - base64: Base-64 encoded binary content. Can be used to easily embed binary content in the telegram. -

      -
      • -
    • -

      - chunkedBase64: Chunked Base-64 encoded binary content. The content is not embedded in the telegram. Instead it is sent in chunks, using separate - chunk messages to the client. Chunked transport can be used by the server when it doesn't know the size of the final result. Chunked transport is - similar to the streamBase64 encoding with the (loose) distinction that a stream is indefinite, while chunked transport conveys finitely sized content. -

      -
      • -
    • -

      - sipub: The sender might deem the content to be too large for sending embedded in the XMPP telegram. To circumnavigate this, the sender publishes - the content as a file using &xep0137; (Publishing Stream Initiation Requests), instead of embedding the content directly. This might be the case for instance, when - a client requests a video resource, without using a ranged request. -

      -
      • -
    • -

      - streamBase64: This option may be used to encode indefinite streams, like live audio or video streams (HLS, SHOUTcast, Motion JPeg web cams, etc). It works - as chunkedBase64 with the distinction that the content is indefinite. This option is not available in requests, only in responses. -

      -
      • -
    • -

      - jingle: For demanding multi-media streams alternative methods to transport streaming rather than embedded into the XMPP stream may be - required. Even though the streamBase64 method may be sufficient to stream a low-resolution web cam in the home, or listen to a microphone - or a radio station, it is probably badly suited for high-resolution video streams with multiple video angles and audio channels. If such content is accessed - and streamed, the server can negotiate a different way to stream the content using &xep0166;. -

      -
      • -
    +
    + +
    text
    +
    +

    + Normal text content. The text is encoded as text within XML, using the same encoding used by the XML stream. XML escape characters (<, > and &) + are escaped using the normal &lt;, &gt; and &amp; character escape sequences. +

    +
    +     + +
    xml
    +
    +

    + Xml content embedded in the XML telegram. Note however, that any processing instructions or XML version statements must be avoided, since it may cause + the XML stream to become invalid XML. If this is a problem, normal text encoding can be used as an alternative. The advantage of xml + instead of text or base64 encodings is when used in conjunction with EXI compression &xep0322;. EXI compression has the ability to + compress XML efficiently. Text will not be compressed, unless response exists in internal string tables. Base-64 encoded data will be compressed so that the 33% + size gain induced by the encoding is recaptured. +

    +
    +     + +
    base64
    +
    +

    + Base-64 encoded binary content. Can be used to easily embed binary content in the telegram. +

    +
    +     + +
    chunkedBase64
    +
    +

    + Chunked Base-64 encoded binary content. The content is not embedded in the telegram. Instead it is sent in chunks, using separate + chunk messages to the client. Chunked transport can be used by the server when it doesn't know the size of the final result. + Streaming content, i.e. content of infinite length, must use ibb or jingle transport types to transfer content. + If the content consists of a file, sipub should be used. +

    +

    + Chunked encoding is perfect for dynamic responses of moderate sizes, for instance for API method responses. The server does not know when the response + is begun to be generated what the final size will be, but it will be most probably "manageable". Using the chunked transfer mechanism enables the + server to start sending the content, minimizing the need for buffers, and at the same time minimizing the number of messages that needs to be sent, + increasing throughput. +

    +

    + The client can limit the maximum chunk size to be used by the server, using the maxChunkSize attribute in the request. The chunk + size can be set to a value between 256 and 65536. If not provided in the request, the server chooses an appropriate value. Note that chunks can + be sent containing a smaller amount of bytes than the maximum chunk size provided in the request. +

    +
    +     + +
    sipub
    +
    +

    + The sender might deem the content to be too large for sending embedded in the XMPP telegram. To circumnavigate this, the sender publishes + the content as a file using &xep0137; (Publishing Stream Initiation Requests), instead of embedding the content directly. This might be the case for instance, when + a client requests a video resource, without using a ranged request. +

    +

    + This transfer mechanism is of course the logical choice, if the content is already stored in a file on the server, and the size of the file + is sufficiently large to merit the overhead of sipub. Smaller files can simply be returned using the text, xml + or base64 mechanisms. +

    +

    + The client can disable the use of sipub by the server, by including a sipub='false' attribute in the request. + sipub is enabled by default. On constrained devices with limited support for different XEP's, this can be a way to avoid the + use of technologies not supported by the client. +

    +
    +     + +
    ibb
    +
    +

    + This option may be used to encode indefinite streams, like live audio or video streams (HLS, SHOUTcast, Motion JPeg web cams, etc). + It uses &xep0047; to send the content over an in-band bytestream to the client. This option is not available in requests, only in responses. +

    +

    + Streams must not use any of the above mechanisms. Only ibb and jingle mechanisms can be used. If the content + represents multimedia jingle is preferrable, especially if different encodings are available. +

    +

    + The client can disable the use of ibb by the server, by including a ibb='false' attribute in the request. + ibb is enabled by default. On constrained devices with limited support for different XEP's, this can be a way to avoid the + use of technologies not supported by the client. +

    +
    +     + +
    jingle
    +
    +

    + For demanding multi-media streams alternative methods to transport streaming rather than embedded into the XMPP stream may be + required. Even though the ibb method may be sufficient to stream a low-resolution web cam in the home, or listen to a microphone + or a radio station, it is probably badly suited for high-resolution video streams with multiple video angles and audio channels. If such content is accessed + and streamed, the server can negotiate a different way to stream the content using &xep0166;. +

    +

    + The client can disable the use of ingle by the server, by including a jingle='false' attribute in the request. + jingle is enabled by default. On constrained devices with limited support for different XEP's, this can be a way to avoid the + use of technologies not supported by the client. +

    +
    +     +

    - Note: Content endoded using chunkedBase64 or streamBase64 encoding methods can be terminated, either by the receptor going - off-line, or by sending a close command to the sender. + Note: Content encoded using chunkedBase64 encoding method can be terminated, either by the receptor going off-line, or by + sending a close command to the sender. The transfer methods sipub, ibb and jingle have + their own mechanisms for aborting content transfer.

    @@ -262,12 +371,14 @@ TODO: Inform:

    -
    + +
    clayster.com
    +     @@ -276,9 +387,11 @@ TODO: Inform: to='httpclient@clayster.com/browser' id='1'> -
    -
    -
    + +
    Fri, 03 May 2013 13:52:10 GMT-4
    +
    OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE
    +
    0
    +     ]]> @@ -289,12 +402,14 @@ TODO: Inform:

    - -
    + + +
    clayster.com
    +     @@ -303,27 +418,20 @@ TODO: Inform: to='httpclient@clayster.com/browser' id='2'> -
    -
    -
    -
    -
    + +
    Fri, 03 May 2013 16:39:54GMT-4
    +
    Clayster
    +
    text/turtle
    +
    ...
    +
    Close
    +     - @base <http://example.org/> . -@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . -@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . -@prefix foaf: <http://xmlns.com/foaf/0.1/> . -@prefix rel: <http://www.perceive.net/schemas/relationship/> . + @prefix dc: <http://purl.org/dc/elements/1.1/>. +@base <http://clayster.com/>. -<#green-goblin> - rel:enemyOf <#spiderman> ; - a foaf:Person ; # in the context of the Marvel universe - foaf:name "Green Goblin" . - -<#spiderman> - rel:enemyOf <#green-goblin> ; - a foaf:Person ; - foaf:name "Spiderman", "Человек-паук"@ru . +<xep> dc:title "HTTP over XMPP"; + dc:creator <PeterWaher>; + dc:publisher <XSF>. ]]> @@ -333,9 +441,6 @@ TODO: Inform: that was used. In the HTTP over XMPP case, connections are not handled in the same way, and so the "Connection: Close" header has no meaning in this case. For more information about connection handling in the HTTP over XMPP case, see the section on Connection Handling.

    -

    - Note 2: The above Turtle example was taken from http://www.w3.org/TR/turtle/#sec-intro. -

    @@ -343,12 +448,14 @@ TODO: Inform:

    -
    + +
    clayster.com
    +     @@ -357,10 +464,12 @@ TODO: Inform: to='httpclient@clayster.com/browser' id='3'> -
    -
    -
    -
    + +
    Fri, 03 May 2013 16:57:12GMT-4
    +
    Clayster
    +
    video/mp4
    +
    12345678
    +     ]]> @@ -371,22 +480,25 @@ TODO: Inform:

    - -
    -
    -
    + + +
    clayster.com
    +
    Clayster HTTP/XMPP Client
    +
    application/sparql-query
    +
    ...
    +     - PREFIX foaf: <http://xmlns.com/foaf/0.1/> -SELECT ?name ?mbox ?hpage -WHERE { ?x foaf:name ?name . - OPTIONAL { ?x foaf:mbox ?mbox } . - OPTIONAL { ?x foaf:homepage ?hpage } - } - + PREFIX dc: <http://purl.org/dc/elements/1.1/> +BASE <http://clayster.com/> + +SELECT ?title ?creator ?publisher +WHERE { ?x dc:title ?title . + OPTIONAL { ?x dc:creator ?creator } . + }     @@ -396,33 +508,26 @@ WHERE { ?x foaf:name ?name . to='httpclient@clayster.com/browser' id='4'> -
    -
    -
    -
    + +
    Fri, 03 May 2013 17:09:34-4
    +
    Clayster
    +
    application/sparql-results+xml
    +
    ...
    +     - - - + + - - Alice + + HTTP over XMPP - - <http://work.example.org/alice/> - - - - - Bob - - - <mailto:bob@work.example> + + http://clayster.com/PeterWaher @@ -433,11 +538,7 @@ WHERE { ?x foaf:name ?name . ]]>

    - Note: The above SPARQL example was taken from http://www.w3.org/TR/sparql11-query/#MultipleOptionals - in combination with http://www.w3.org/TR/sparql11-protocol/#select-longpost. -

    -

    - Note 2: If using xml encoding of data, care has to be taken to avoid including the version and encoding information + Note: If using xml encoding of data, care has to be taken to avoid including the version and encoding information (<?xml version="1.0"?>) at the top of the document, otherwise the resulting XML will be invalid.

    @@ -447,14 +548,16 @@ WHERE { ?x foaf:name ?name .

    -
    -
    -
    + +
    clayster.com
    +
    text/html
    +
    ...
    +     <html><header/><body><p>Beautiful home page.</p></body></html> @@ -466,8 +569,10 @@ WHERE { ?x foaf:name ?name . to='httpclient@clayster.com/browser' id='5'> -
    -
    + +
    Fri, 03 May 2013 17:40:41GMT-4
    +
    0
    +     ]]> @@ -478,12 +583,14 @@ WHERE { ?x foaf:name ?name .

    -
    + +
    clayster.com
    +     @@ -492,9 +599,11 @@ WHERE { ?x foaf:name ?name . to='httpclient@clayster.com/browser' id='6'> -
    -
    -
    + +
    Fri, 03 May 2013 17:46:07GMT-4
    +
    text/plain
    +
    ...
    +     You're not allowed to change the home page! @@ -508,12 +617,14 @@ WHERE { ?x foaf:name ?name .

    -
    + +
    clayster.com
    +     @@ -522,10 +633,12 @@ WHERE { ?x foaf:name ?name . to='httpclient@clayster.com/browser' id='7'> -
    -
    -
    -
    + +
    Fri, 03 May 2013 17:55:10GMT-4
    +
    Clayster
    +
    message/http
    +
    ...
    +     GET /rdf/ex1.turtle HTTP/1.1 Host: clayster.com @@ -547,15 +660,17 @@ Host: clayster.com

    -
    -
    -
    -
    + +
    www.example.com
    +
    application/example
    +
    e0023aa4e
    +
    100
    +     [description of changes] @@ -567,22 +682,21 @@ Host: clayster.com to='httpclient@clayster.com/browser' id='8'> -
    -
    + +
    /file.txt
    +
    e0023aa4e
    +     ]]> -

    - Note: The example is taken from §2.1 RFC 5789. -

    - +

    - In the following sub-sections, the different data encoding formats are discussed with corresponding examples to illustrate how they work. + In the following sub-sections, the different data encoding formats are discussed, each with corresponding examples to illustrate how they work. The interesting part of these examples is the data element and its contents.

    - +

    Text responses is a simple way to return text responses (i.e. any MIME Type starting with text/). Since the text is embedded into XML, the characters <, > and & need to be escaped to &lt;, &gt; and &amp; respectively. @@ -590,42 +704,35 @@ Host: clayster.com

    The following example shows how a TURTLE response, which is text-based, is returned using the text encoding:

    - + -
    -
    -
    -
    -
    + +
    Fri, 03 May 2013 16:39:54GMT-4
    +
    Clayster
    +
    text/turtle
    +
    ...
    +
    Close
    +     - @base <http://example.org/> . -@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . -@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . -@prefix foaf: <http://xmlns.com/foaf/0.1/> . -@prefix rel: <http://www.perceive.net/schemas/relationship/> . + @prefix dc: <http://purl.org/dc/elements/1.1/>. +@base <http://clayster.com/>. -<#green-goblin> - rel:enemyOf <#spiderman> ; - a foaf:Person ; # in the context of the Marvel universe - foaf:name "Green Goblin" . - -<#spiderman> - rel:enemyOf <#green-goblin> ; - a foaf:Person ; - foaf:name "Spiderman", "Человек-паук"@ru . +<xep> dc:title "HTTP over XMPP"; + dc:creator <PeterWaher>; + dc:publisher <XSF>. ]]> - +

    - XML is a conveniant way to return XML embedded in the XMPP response. This can be suitable for MIME Types of the form .*/(.*[+])?xml + XML is a convenient way to return XML embedded in the XMPP response. This can be suitable for MIME Types of the form .*/(.*[+])?xml (using regular expression to match them), like text/xml, application/soap+xml or application/sparql-results+xml. Care has to be taken however, since not all XML constructs can be embedded as content to an XML element without invalidating it, like the xml version and encoding declaration (<?xml version="1.0"?> as an example). @@ -635,45 +742,38 @@ Host: clayster.com encode the XML escape characters <, > and &, or use another encoding, like base64.

    - The advantage of xml instead of text or base64 encodings is when used in conjuncion with + The advantage of xml instead of text or base64 encodings is when used in conjunction with EXI compression. EXI compression has the ability to compress XML efficiently. Text will not be compressed, unless response exists in internal string tables. Base-64 encoded data will be compressed so that the 33% size gain induced by the encoding is recaptured.

    - + -
    -
    -
    -
    + +
    Fri, 03 May 2013 17:09:34-4
    +
    Clayster
    +
    application/sparql-results+xml
    +
    ...
    +     - - - + + - - Alice + + HTTP over XMPP - - <http://work.example.org/alice/> - - - - - Bob - - - <mailto:bob@work.example> + + http://clayster.com/PeterWaher @@ -684,26 +784,28 @@ Host: clayster.com ]]> - +

    - Base-64 encoding is a simple way to encode content that is easilly embedded into XML. Apart from the advantage of being easy to encode, + Base-64 encoding is a simple way to encode content that is easily embedded into XML. Apart from the advantage of being easy to encode, it has the disadvantage to increase the size of the content by 33% (unless EXI compression is used at the same time), since it requires 4 bytes to encode 3 bytes of data. Care has to be taken not to send too large items using this encoding.

    The following example shows an image is returned using the base64 encoding:

    - + -
    -
    -
    -
    + +
    Fri, 03 May 2013 16:39:54GMT-4
    +
    Clayster
    +
    image/png
    +
    221203
    +     iVBORw0KGgoAAAANSUhEUgAAASwAAAGQCAYAAAAUdV17AAAAAXNSR0 ... tVWJd+e+y1AAAAABJRU5ErkJggg== @@ -711,7 +813,7 @@ Host: clayster.com ]]> - +

    In HTTP, Chunked Transfer Encoding is used when the sender does not know the size of the content being sent, and to avoid having its buffers overflow, sends the content in chunks with a definite size. @@ -726,17 +828,19 @@ Host: clayster.com predetermined. Chunks are naturally delimited and embedded in the XML stanza. The last chunk in a response must have the last attribute set to true.

    - + -
    -
    -
    -
    + +
    Fri, 04 May 2013 13:43:12GMT-4
    +
    Clayster
    +
    image/png
    +
    221203
    +     @@ -757,11 +861,19 @@ Host: clayster.com

    Note: Chunked encoding assumes the content to be finite. If content is infinite (i.e. for instance live streaming), - the streamBase64 transfer encoding must be used instead. If the sender is unsure if the content is finit or inifinite, - the streamBase64 must be used. + the ibb or jingle transfer encodings must be used instead. If the sender is unsure if the content is + finite or infinite, ibb or jingle must be used. +

    +

    + Note 2: If the web server sends chunked data to the client it uses the HTTP header Transfer-Encoding: chunked, + and then sends the data in chunks but with chunk sizes inserted so the receiving end can decode the incoming data. Note that this data will + be included in the data sent in the XMPP chunks defined by this document. In this case, data will be chunked twice: First by the web server, + and then by the HTTP over XMPP transport layer. When received by the client, it is first reassembled by the HTTP over XMPP layer on the client, + and then by the HTTP client who will read the original chunk size elements inserted into the content. More information about HTTP chunking, + can be found in RDF2616 §3.6.1.

    - +

    Often content being sent can be represented by a file, virtual or real, especially if the content actually represents a file and is not dynamically generated. In these instances, instead of embedding the contents in the response, @@ -769,17 +881,19 @@ Host: clayster.com XEP 0137: Publishing Stream Initiation Requests. This is done using the sipub element.

    - + -
    -
    -
    -
    + +
    Fri, 03 May 2013 16:39:54GMT-4
    +
    Clayster
    +
    image/png
    +
    221203
    +     ]]> - +

    Some web servers provide streaming content, i.e. content where packets are sent according to a timely fashion. Examples are video and audio streams like HLS (HTTP Live Streams), SHOUTcast, ICEcast, Motion JPeg, etc. In all these examples, content is infinite, @@ -804,31 +918,26 @@ Host: clayster.com example.

    - Such content must use the streamBase64 encoding scheme, if used. The streamBase64 is similar to - the chunkedBase64 encoding, except it is imlicitly understood that the stream can be indefinite and needs an - explicit way to close the stream. Closing the stream is done by sending a close command to the sender of the - stream. The close command can also be used to close a chunked stream, and senders of chunked data should - support terminating the output if a close command on the stream is received. + Such content must use the ibb transfer mechanism, if used (or the jingle transfer machanism). + The ibb transfer mechanism uses In-Band Bytestreams + to transfer data from the server to the client. It starts by sending an a ibb element containing a sid + attribute identifying the stream. Then the server sends an ibb:open IQ-stanza to the client according to + XEP-0047. The client can choose to reject, negotiate or acceopt the request + whereby the transfer is begun. When the client is satisified and wants to close the stream, it does so, also according to + XEP-0047. The sid value returned in the HTTP response + is the same sid value that is later used by the IBB messages that follow. In this way, the client can relate + the HTTP request and response, with the corresponding data transferred separately.

    -

    - Note: All streams (or chunked responses) are automatically closed if one of the parties goes off-line. -

    -

    - Note 2: A streamed response does not need to be indefinite. It can be finite. If it is finite, the last chunk - must have the last attribute set to true. -

    -

    - Note 3: The close command can be sent using a message or iq stanza. Use iq stanzas if a - response is desired. The response will be the same close command sent. -

    - + -
    + +
    clayster.com
    +     @@ -837,55 +946,72 @@ Host: clayster.com to='httpclient@clayster.com/browser' id='12'> -
    -
    -
    + +
    Fri, 04 May 2013 15:05:32GMT-4
    +
    Clayster
    +
    multipart/x-mixed-replace;boundary=__2347927492837489237492837
    +     - + + + + + + + - ... + ... ... - - + id='14'> + - - ]]> + id='14'/>]]> - +

    For demanding multi-media streams alternative methods to transport streaming rather than embedded into the XMPP stream may be - required. Even though the streamBase64 method may be sufficient to stream a low-resolution web cam in the home, or listen to a microphone + required. Even though the ibb method may be sufficient to stream a low-resolution web cam in the home, or listen to a microphone or a radio station, it is probably badly suited for high-resolution video streams with multiple video angles and audio channels. If such content is accessed and streamed, the server can negotiate a different way to stream the content using XEP 0166: Jingle.

    - + -
    -
    -
    -
    + +
    Fri, 03 May 2013 16:39:54GMT-4
    +
    Clayster
    +
    video/mp4
    +
    ...
    +

    By creating a new scheme for HTTP over XMPP transport, and implementing support for it in web browsers, XML HTTP request objects and web servers, - Web Applications previously requiring web hosting on the Internet will be able to be hosted privatly behind firewalls instead, by simply switching + Web Applications previously requiring web hosting on the Internet will be able to be hosted privately behind firewalls instead, by simply switching from the http:// scheme to an xmpp:// scheme. All relative URL's within the application, including URL's sent to the XHR object (Ajax) will automatically be directed to use the HTTP over XMPP transport instead.

    @@ -985,7 +1111,7 @@ Host: clayster.com Here, the JID to use in the URL, must not include a resource. Only user name, the @ character and the domain. The / separator between the JID and the Path is actually part of the Part.

    - + multiple connection is canonical.

    - When resolving an URL using the xmpp scheme, the browser needs to extract the JID of the server hosting the resource. If that JID + When resolving an URL using the xmpp:// scheme, the browser needs to extract the JID of the server hosting the resource. If that JID is already in the roster, the request can proceed as usual.

    @@ -1036,7 +1162,7 @@ Host: clayster.com

    • - Only relative URL's are used within references (images, audio, video, links, objekts, etc.). If absolute URL's are used (including scheme), + Only relative URL's are used within references (images, audio, video, links, objects, etc.). If absolute URL's are used (including scheme), the browser might get the first page correctly, but will be unable to get the content with the absolute URL, unless the URL has the same scheme as the principal page.

      @@ -1048,13 +1174,13 @@ Host: clayster.com

    • - Any URL's sent to the XML HTTP Request Object directed to API's or resources hosted by the same application must also be relative, + Any URL's sent to the XML HTTP Request (XHR) Object directed to API's or resources hosted by the same application must also be relative, for the same reasons as above. The XHR Object supports relative URL's.

    - If the above rules are met, which they should under normal conditions as well, typing in the xmpp:// URL in the browser (for instance when + If the above rules are met, which they should under normal conditions, typing in the xmpp:// URL in the browser (for instance when you're at the office) should display the application (hosted for example at home behind a firewall) in the same way as when you use http:// (or https://) when you have access to the server (for instance when you're home), as long as friendship exists between the browser JID and the server JID. @@ -1067,30 +1193,33 @@ Host: clayster.com are mostly HTTP over TCP based, even though there are bindings which are not based on this. The most common APIs today (REST) are however all based on HTTP over TCP. Being HTTP over TCP requires the web server hosting the web services either to be public or directly accessible by the client. But as the services move closer to end users (for instance a Thermostat publishing a REST API for control in your home), problems arise - when you try to access the web service outside of private network in which the API is available. As explained previously, the HTTP over XMPP solves this. + when you try to access the web service outside of private network in which the API is available. As explained previously, the use of HTTP over XMPP + solves this.

    - The following example shows a simple SOAP method call, taken from - http://www.w3schools.com/soap/soap_example.asp: + The following example shows a simple SOAP method call:

    - -
    -
    -
    + + +
    www.example.com
    +
    application/soap+xml; charset=utf-8
    +
    ...
    +     - - - IBM + + + 10 + 20 @@ -1104,16 +1233,18 @@ Host: clayster.com to='httpclient@clayster.com/browser' id='15'> -
    -
    + +
    application/soap+xml; charset=utf-8
    +
    ...
    +     - - - 34.5 - + + + 30 + @@ -1134,12 +1265,14 @@ Host: clayster.com

    -
    + +
    clayster.com
    +     @@ -1148,10 +1281,12 @@ Host: clayster.com to='httpclient@clayster.com/browser' id='16'> -
    -
    -
    -
    + +
    Fri, 05 May 2013 15:01:53GMT-4
    +
    Clayster
    +
    text/xml
    +
    ...
    +     @@ -1177,7 +1312,7 @@ Host: clayster.com

    The Semantic Web was originally developed as a way to link data between servers on the Web, and understand it. However, with the advents of technologies such as SPARQL, the Semantic Web has become a way to - unify API's into a universal form of distributed API to all types of data possible. It also alows for a standardized way to perform + unify API's into a universal form of distributed API to all types of data possible. It also allows for a standardized way to perform grid computing, in the sense that queries can be federated and executed in a distributed fashion ("in the grid").

    @@ -1187,7 +1322,7 @@ Host: clayster.com technologies beyond "The Internet" while at the same time improving security and controllability of the information.

    - As the semantic web comes closed to Internet of Things and the world of XMPP, it can benefit from work done with relation to the Internet of + As the semantic web moves closer to Internet of Things and the world of XMPP, it can benefit from work done with relation to the Internet of Things, such as &xep0324;, which would give automatic control of who (or what) can communicate with whom (or what).

    @@ -1204,34 +1339,24 @@ Host: clayster.com to='httpclient@clayster.com/browser' id='2'> -
    -
    -
    -
    -
    + +
    Fri, 03 May 2013 16:39:54GMT-4
    +
    Clayster
    +
    text/turtle
    +
    ...
    +
    Close
    +     - @base <http://example.org/> . -@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . -@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . -@prefix foaf: <http://xmlns.com/foaf/0.1/> . -@prefix rel: <http://www.perceive.net/schemas/relationship/> . + @prefix dc: <http://purl.org/dc/elements/1.1/>. +@base <http://clayster.com/>. -<#green-goblin> - rel:enemyOf <#spiderman> ; - a foaf:Person ; # in the context of the Marvel universe - foaf:name "Green Goblin" . - -<#spiderman> - rel:enemyOf <#green-goblin> ; - a foaf:Person ; - foaf:name "Spiderman", "Человек-паук"@ru . +<xep> dc:title "HTTP over XMPP"; + dc:creator <PeterWaher>; + dc:publisher <XSF>. ]]> -

    - Note: The above example was taken from http://www.w3.org/TR/turtle/#sec-intro. -

    @@ -1250,29 +1375,27 @@ Host: clayster.com to='httpclient@clayster.com/browser' id='17'> -

    -
    -
    -
    + +
    Fri, 05 May 2013 16:02:23GMT-4
    +
    Clayster
    +
    application/rdf+xml
    +
    ...
    +     - - - Eric Miller - - Dr. - + + + HTTP over XMPP + + + ]]> -

    - Note: The above example was taken from - http://www.w3.org/TR/2004/REC-rdf-primer-20040210/#intro. -

    @@ -1280,22 +1403,25 @@ Host: clayster.com

    - -
    -
    -
    + + +
    clayster.com
    +
    Clayster HTTP/XMPP Client
    +
    application/sparql-query
    +
    ...
    +     - PREFIX foaf: <http://xmlns.com/foaf/0.1/> -SELECT ?name ?mbox ?hpage -WHERE { ?x foaf:name ?name . - OPTIONAL { ?x foaf:mbox ?mbox } . - OPTIONAL { ?x foaf:homepage ?hpage } - } - + @prefix dc: <http://purl.org/dc/elements/1.1/>. +@base <http://clayster.com/>. + +<xep> dc:title "HTTP over XMPP"; + dc:creator <PeterWaher>; + dc:publisher <XSF>.     @@ -1305,33 +1431,26 @@ WHERE { ?x foaf:name ?name . to='httpclient@clayster.com/browser' id='4'> -
    -
    -
    -
    + +
    Fri, 03 May 2013 17:09:34-4
    +
    Clayster
    +
    application/sparql-results+xml
    +
    ...
    +     - - - + + - - Alice + + HTTP over XMPP - - <http://work.example.org/alice/> - - - - - Bob - - - <mailto:bob@work.example> + + http://clayster.com/PeterWaher @@ -1341,28 +1460,24 @@ WHERE { ?x foaf:name ?name . ]]> -

    - Note: The above SPARQL example was taken from http://www.w3.org/TR/sparql11-query/#MultipleOptionals - in combination with http://www.w3.org/TR/sparql11-protocol/#select-longpost. -

    - +

    There are many types of streams and streaming protocols. Several of these are based on HTTP or variants simulating HTTP. Examples of such HTTP-based or - pseudo-HTTP based streaming protocols can include + pseudo-HTTP based streaming protocols can include HLS HLS: HTTP Live Streaming <http://en.wikipedia.org/wiki/HTTP_Live_Streaming> used for multi-media streaming, - + SHOUTcast SHOUTcast <http://en.wikipedia.org/wiki/SHOUTcast> used for internet radio and - + Motion JPeg Motion JPeg <http://en.wikipedia.org/wiki/Motion_JPEG> common format for web cameras.

    Common for all streaming data, is that they are indefinite, but at the same time rate-limited depending on quality, etc. Because of this, the - web server is requried to use the streamsBase64 encoding or the Jingle encoding to + web server is required to use the ibb encoding or the jingle encoding to transport the content to the client.

    @@ -1372,7 +1487,7 @@ WHERE { ?x foaf:name ?name .

    If an entity supports the protocol specified herein, it MUST advertise that fact by returning a feature of "urn:xmpp:http" in response to &xep0030; information requests.

    @@ -1415,6 +1530,49 @@ WHERE { ?x foaf:name ?name . case. However, the corresponding headers should always be transported as is, to maintain the information.

    + +

    + HTTP Headers are serialized to and from XML using the XEP-0131 Stanza Headers and Internet Metadata. + However, this does not mean that the SHIM feature needs to be published by the client, as defined in §3 in + XEP-0131, since the headers will be embedded into the HTTP elements. + Also, if there is any conflicts in how to store header values, when it comes to data types, etc., the original format as used by the original + HTTP request must be used, and not the format defined in Header Definitions or + A Note About date-Related Headers in XEP-0131. +

    +

    + The HTTP over XMPP tunnel is just a tunnel of HTTP over XMPP, it does not know the semantic meaning of headers used in the transport. It does not know + if additional headers added by the myriad of custom applications using HTTP are actually HTTP-compliant. It just acts as a transport, returning the + sime kind of response (being deterministric) as if the original request was made through other means, for example over TCP. It does not add, remove or + change semantic meaning of keys and values, nor change the format of the corresponding values. Such changes will create uncountable problems very difficult + to detect and solve in a general case. +

    +

    + This specification differs from XEP-0131 in that this specification the headers are consumed by web servers and web clients (The XMPP client here only + being a "dumb" gateway), while in XEP-0131 the headers are consumed by the XMPP clients themselves, knowing XML and XML formats. +

    +     + +

    + Some XMPP Servers may limit stanza sizes for various reasons. While this may work well for certain applications, like + Instant Messaging and Chat, implementors of HTTP over XMPP need to know that some server have such stanza size + restrictions. Therefore, an implementation should include configurable size limits, so chunking can be used instead + of sending large stanzas. Another limit could be when streaming should be used instead of chunking. This later limit + should be applied in particular on audio and video content. +

    +

    + The implementor should also consider to send large content in the form of files using file transfer, and large multi-media + content using Jingle. +

    +     + +

    + Some XMPP Servers may also have bandwidth resitrctions enforced. This to limit the possibility of Denial of Service attacks + or similar flooding of messages. Implementors of the HTTP over XMPP extensions must know however, that the bandwidth + limitations for instant messaging and chat may be completely different from that of normal web applications. In chatting, + a 1000 bytes/s limit is in most cases sufficient, while the same limit for even a modest web applications will make the + application completely impossible to use. +

    +

    @@ -1427,9 +1585,9 @@ WHERE { ?x foaf:name ?name . (if the browser also maintains an IM client), or automatically rejected.

    - On the other hand, when the browser wants to access an URL using the xmpp scheme, an automatic friendship request to the - corresponding JID should be done, if not already in the roster. It is assumed thay by entering the URL, or using the URL - of an application already displayed, is the same as giving permission to add that JID as a friend to the roster of the + On the other hand, when the browser wants to access an URL using the xmpp:// scheme, an automatic friendship request to the + corresponding JID should be done, if not already in the roster. It is assumed that by entering the URL, or using the URL + of an application already displayed, this implies giving permission to add that JID as a friend to the roster of the browser.

    @@ -1437,7 +1595,7 @@ WHERE { ?x foaf:name ?name .

    A web server should have different security settings available. The following subsections list possible settings for different scenarios. Note that these settings only reflect roster handling and cannot be set per resource. However, the server can - maintain a set of JIDs with different settings and restrict access to parts of the conted hosted by the server per JID. + maintain a set of JIDs with different settings and restrict access to parts of the content hosted by the server per JID.

    @@ -1450,7 +1608,7 @@ WHERE { ?x foaf:name ?name .

    - All new friendship are shown (or queued) to an administrator for manual acception or rejection. Once accepted, the client + All new friendship are shown (or queued) to an administrator for manual acceptance or rejection. Once accepted, the client can access the corresponding content. During the wait (which can be substantial), the client should display a message that the friendship request is sent and response is pending.

    @@ -1496,29 +1654,44 @@ WHERE { ?x foaf:name ?name . xmlns:xs='http://www.w3.org/2001/XMLSchema' targetNamespace='urn:xmpp:http' xmlns='urn:xmpp:http' + xmlns:shim='http://jabber.org/protocol/shim' xmlns:sipub='http://jabber.org/protocol/sipub' + xmlns:ibb='http://jabber.org/protocol/ibb' xmlns:jingle='urn:xmpp:jingle:1' elementFormDefault='qualified'> - + + + - + + + + + + + + + + + + - + @@ -1527,6 +1700,60 @@ WHERE { ?x foaf:name ?name . + + + + + Used for text responses that are not XML. + + + + + Specifically used for XML-formatted responses. + + + + + + + + + + Short binary responses, base-64 encoded. + + + + + Content is divided into chunks of binary base-64 encoded data. + Used if content is generated dynamically and/or content size is not known. + For streaming data the ibb:open or jingle:jingle transports must be used. + For static data, such as files, sipub:sipub should be used. + + + + + + + + Content available through file transfer. + + + + + Content returned through an in-band bytestream. + + + + + + + + Multi-media content returned through jingle. + + + + + @@ -1544,37 +1771,6 @@ WHERE { ?x foaf:name ?name . - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -1597,4 +1793,7 @@ WHERE { ?x foaf:name ?name . ]]>     + +

    Thanks to Peter Saint-Andre, Karin Forsell and Matthew A. Miller for all valuable feedback.

    +     \ No newline at end of file