From f465e5d1f667becf5a1c8b80f086906f2545a2fc Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Sat, 18 Jul 2015 11:25:15 +0200 Subject: [PATCH 001/106] Remove message sentinel from MAM (XEP-0313) The MAM history already says "Switch the sentinel message back to the iq result.", but this has never happened. This commit does the switch. --- xep-0313.xml | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/xep-0313.xml b/xep-0313.xml index c4fece03..1b9259d5 100644 --- a/xep-0313.xml +++ b/xep-0313.xml @@ -151,7 +151,7 @@

A query consists of an &IQ; stanza of type='set' addressed to the account or server entity hosting the archive, with a 'query' payload. On receiving the query, the server pushes to the client a series of messages from the archive that match the client's given criteria, and finally returns - a &MESSAGE; with a <fin/> tag to indicate that the query is completed.

+ the &IQ; result to indicate that the query is completed.

The final &MESSAGE; response MUST include an RSM <set/> element indicating the UID of the first and last message of the (possibly limited) result set. This allows clients to accurately page through messages.

@@ -160,19 +160,21 @@ ]]> - ]]> - - Hail to thee + + + + + + Hail to thee + + + ]]> - - -]]> -

To ensure that the client knows when the results are complete, the server MUST send a <fin> message. The client can optionally include a 'queryid' attribute in their query, which allows the client to match results to their initiating query, and if present in the client's query the server MUST include it in the <fin> response.

+ ]]> +

To ensure that the client knows when the results are complete, the server MUST send the &IQ; result after the last message retrieved from the archive. The client can optionally include a 'queryid' attribute in their query, which allows the client to match results to their initiating query.

When querying a pubsub node's archive, the 'node' attribute is added to the <query> element.

@@ -290,9 +292,8 @@ pushed to a client in one request. Whether or not the client query included a <set/> element, the server MAY simply return its limited results, modifying the <set/> element it returns appropriately.

- + 28482-98726-73623 @@ -329,11 +330,10 @@ ]]>

Note: There is no concept of an "open query", and servers MUST be prepared to receive arbitrary page requests at any time.

-

When the results returned by the server are complete (that is: when they are the last page of the result set), the server MUST include a 'complete' attribute on the <fin> element, with a value of 'true'. If it is not the last page of the result set, the server MUST either omit the 'complete' attribute, or give it a value of 'false'.

+

When the results returned by the server are complete (that is: when they are the last page of the result set), the server MUST include a 'complete' attribute in the &IQ; result, with a value of 'true'. If it is not the last page of the result set, the server MUST either omit the 'complete' attribute, or give it a value of 'false'.

- + 23452-4534-1 @@ -341,9 +341,9 @@ 16 - + ]]> -

Sometimes (e.g. due to network or storage partitioning, or other transient errors) the server might return results to a client that are unstable (e.g. they might later change in sequence or content). In such a situation the server MUST stamp the <fin> element with a 'stable' attribute with a value of 'false'. If the server knows that the data it's serving are stable it MUST either stamp a 'stable' attribute with a value of 'true', or no such attribute. An example of when unstable might legitimately be returned is if the MAM service uses a clustered data store and a query covers a time period for which the data store has not yet converged; it the server could return best-guess results and tell the client that they may be unstable. A client SHOULD NOT cache unstable results long-term without later confirming (by reissuing appropriate queries) that they have become stable.

+

Sometimes (e.g. due to network or storage partitioning, or other transient errors) the server might return results to a client that are unstable (e.g. they might later change in sequence or content). In such a situation the server MUST stamp the &IQ; result with a 'stable' attribute with a value of 'false'. If the server knows that the data it's serving are stable it MUST either stamp a 'stable' attribute with a value of 'true', or no such attribute. An example of when unstable might legitimately be returned is if the MAM service uses a clustered data store and a query covers a time period for which the data store has not yet converged; it the server could return best-guess results and tell the client that they may be unstable. A client SHOULD NOT cache unstable results long-term without later confirming (by reissuing appropriate queries) that they have become stable.

In order for the client find out about additional fields the server might support, it can send an iq stanza of type='get' addressed to the archive like this:

From bddd8a4177e55ba7f663b6ce113d630b8bad17bc Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Tue, 21 Jul 2015 11:48:40 +0200 Subject: [PATCH 002/106] ProtoXEP.modile-concerns v0.0.1: first draft. --- inbox/mobile-concerns.xml | 203 ++++++++++++++++++++++++++++++++++++++ xep.ent | 1 + 2 files changed, 204 insertions(+) create mode 100644 inbox/mobile-concerns.xml diff --git a/inbox/mobile-concerns.xml b/inbox/mobile-concerns.xml new file mode 100644 index 00000000..c2cdac0f --- /dev/null +++ b/inbox/mobile-concerns.xml @@ -0,0 +1,203 @@ + + +%ents; +]> + + +
+ Mobile Considerations + + This document provides background information for XMPP implementors + concerned with mobile devices operating on an LTE cellular network. + + &LEGALNOTICE; + NOT_YET_ASSIGNED + ProtoXEP + Informational + Standards + Council + + XMPP Core + + + XEP-0286 + + + NOT_YET_ASSIGNED + + Sam + Whited + sam@samwhited.com + sam@samwhited.com + + + 0.0.1 + 2015-07-18 + ssw +

First draft.

 + +
+ +

+ XMPP as a protocol was designed before the wide spread adoption of mobile + devices, and is often cited as not being very mobile friendly as a result. + However, this mostly stems from undocumented lore and outdated notions of + how XMPP works. As the Internet and protocol design have changed to be more + accommodating for mobile, so has XMPP. +

+

+ This XEP aims to provide useful background knowledge of mobile handset + behavior, and those considerations that client and server designers can + take to ensure that bandwidth and battery are used efficiently. +

+ + +

+ The two major constraints on mobile devices are power and bandwidth. With + the wide spread proliferation of 3G and LTE technologies, mobile bandwidth + and speeds have become broadly comparable to broadband. However, they are + still relatively expensive compared to traditional wired networks, and + should therefore still be considered. This XEP mostly focuses on LTE as it + already has a very wide deployment and will only continue to further + replace 3G technologies. +

+ + +

+ XML, and by extension XMPP, is known to be highly compressible. In a simple + test of a small (266089 byte) XMPP stream (connection, stream + initialization, feature discovery, roster loading, several presence stanzas + sent and received, disconnect), the entropy of the stream was found to be + 5.616313 bits per byte. Using the `gzip` tool to apply Lempel-Ziv coding + (LZ77) without concern for server-side CPU usage resulted in a compression + ratio of 21% (a 79% reduction in bandwidth). In one test with a much larger + dataset typical of a corporate environment (many hundreds of users in the + roster), the ratio was as low as 13%, an 87% reduction in bandwidth! +

+

+ Compression of XMPP data can be achieved with the DEFLATE algorithm + (&rfc1951;) via TLS compression (&rfc3749;) or &xep0138;. While the + security implications of stream compression are beyond the scope of this + document (See the aforementioned RFC or XEP for more info), mitigating them + may affect compression ratios. The author does not recommend using TLS + compression with XMPP (or in general). If compression must be used, stream + level compression should be implemented instead. Compressing at the stream + level gives us the benefit of being able to flush the compression stream on + stanza boundaries to help prevent information from leaking. This, however, + may drastically increase compression ratios. +

+

+ While the CPU cost of compression directly translates to higher power + usage, it is vastly outweighed by the benefits of reduced network + utilization, especially on modern LTE networks which use a great deal more + power per bit than 3G networks as will be seen later in this document. + Setting security considerations aside and thinking only of power + consumption and bandwidth, supporting compression is highly recommended. +

+ + +

+ While the wide spread adoption of LTE has dramatically increased available + bandwidth on mobile devices, it has also increased power consumption. + According to one study, early LTE devices consumed 5–20% more power than + their 3G counterparts + LTE Smartphone measurements <http://networks.nokia.com/system/files/document/lte_measurements_final.pdf>. + On some networks that support the legacy SVLTE (Simultaneous Voice and LTE) + instead of the more modern VoLTE (Voice Over LTE) standard, or even CSFB + (Circuit-switched fallback) this number would (presumably) be even higher. +

+

+ XMPP server and client implementers, bearing this increased power usage in + mind, and knowing a bit about how LTE radios work, can optimize their + traffic to minimize network usage. For the downlink, LTE user equipment + (UE) utilizes Orthogonal Frequency Division Multiplexing (OFDM), which is + somewhat inefficient + A Close Examination of Performance and Power Characteristics of 4G LTE Networks <http://www.cs.columbia.edu/~lierranli/coms6998-7Spring2014/papers/rrclte_mobisys2012.pdf>. + On the uplink side a different technology, Single-carrier frequency + division multiple access (SC-FDMA) is used, which is slightly more + efficient than traditional (non linearly-precoded) OFDM, slightly + offsetting the fact that broadcasting requires more power than receiving. + LTE UE also implements a Discontinuous reception (DRX) mode in which the + hardware can sleep until it is woken by a paging message or is needed to + perform some task. LTE radios have two power modes: RRC_CONNECTED and + RRC_IDLE. DRX is supported in both of these power modes. By attempting to + minimize the time which the LTE UE state machine spends in the + RCC_CONNECTED state, and maximize the time it stays in the DRX state (for + RCC_CONNECTED and RRC_IDLE), we can increase battery life without degrading + the XMPP experience. To do so, the following rules should be observed: +

+ +

+ Whenever possible, data that is not strictly needed should not be + transmitted (by the server or client). Supporting &xep0352; is highly + recommended. Most importantly, XMPP pings should be kept as far apart as + possible and only used when necessary. Server operators are encouraged to + set high ping timeouts, and client implementors are advised to only send + pings when absolutely necessary to prevent the server from closing the + socket. +

+ + +

+ If one is on 3G, transmitting a small amount of data will cause the radio + to enter FACH mode which is significantly cheaper than its high power + mode. On LTE radios, however, transmitting small amounts of data is + vastly more expensive per bit due to the significantly higher tail-times + (the time it takes for the radio to change state). On LTE radios, one + should transmit as much data as possible when the radio is already on + (eg. by placing messages in a send queue and executing the queue as a + batch). Similarly, when data is being received the radio is already in a + high power state and therefore any data that needs to be sent should be. +

+

+ These rules also apply to server operators: If you receive data, the + phones radio is already on therefore you should send anything you have. + Otherwise, batching data to be sent and sending it all at once (and as + much as possible) will help reduce power consumption. +

+ + + +

+ This section provides pointers to other documents which may be of interest + to those developing mobile clients, or considering support for them in + servers. +

+

&xep0138; provides stream level compression.

+

+ &xep0115; provides a mechanism for caching, and hence eliding, the + disco#info requests needed to negotiate optional features. +

+

+ &xep0237; provides a relatively widely deployed extension for reducing + roster fetch sizes. +

+

+ &xep0198; allows the client to send and receive smaller keep-alive + messages, and resume existing sessions without the full handshake. Useful + on unstable connections. +

+ + +

+ The original mobile XEP, &xep0286;, was written by Dave Cridland, and parts + of it were used when writing this XEP. +

+ + +

This document introduces no new security considerations.

+ + +

+ This document requires no interaction with the Internet Assigned Numbers + Authority (IANA). +

+ + +

+ No namespaces or parameters need to be registered with the XMPP Registrar + as a result of this document. +

+ + diff --git a/xep.ent b/xep.ent index ca451217..b2146cc3 100644 --- a/xep.ent +++ b/xep.ent @@ -405,6 +405,7 @@ THE SOFTWARE. RFC 1939 RFC 1939: Post Office Protocol - Version 3 <http://tools.ietf.org/html/rfc1939>." > RFC 1945 RFC 1945: Hypertext Transfer Protocol -- HTTP/1.0 <http://tools.ietf.org/html/rfc1945>." > RFC 1950 RFC 1950: ZLIB Compressed Data Format Specification version 3.3 <http://tools.ietf.org/html/rfc1950>." > +RFC 1951 RFC 1951: DEFLATE Compressed Data Format Specification version 1.3 <http://tools.ietf.org/html/rfc1951>." > RFC 2026 RFC 2026: The Internet Standards Process <http://tools.ietf.org/html/rfc2026>." > RFC 2045 RFC 2045: Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies <http://tools.ietf.org/html/rfc2045>." > RFC 2068 RFC 2068: Hypertext Transport Protocol -- HTTP/1.1 <http://tools.ietf.org/html/rfc2068>." > From 4c2052ffccf683f042b29905faf6064d4b318403 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Tue, 21 Jul 2015 14:18:23 +0200 Subject: [PATCH 003/106] ProtoXEP Raft v0.0.1: first draft. --- inbox/raft.xml | 178 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 178 insertions(+) create mode 100644 inbox/raft.xml diff --git a/inbox/raft.xml b/inbox/raft.xml new file mode 100644 index 00000000..3e34e787 --- /dev/null +++ b/inbox/raft.xml @@ -0,0 +1,178 @@ + + +%ents; +]> + + +
+ Raft over XMPP + This specification provides a means for transporting messages from the Raft consensus algorithm over XMPP. + &LEGALNOTICE; + xxxx + ProtoXEP + Standards Track + Standards + Council + + XMPP Core + XEP-0001 + Etc. + + + + NOT_YET_ASSIGNED + + Peter + Membrey + peter@membrey.hk + peter@membrey.hk + + + 0.0.1 + 2015-07-20 + pm +

First draft.

 + +
+ +

This XEP does not attempt to implement Raft. Rather, based on the message exchanges defined in the Raft paper, it defines a complementary set of &MESSAGE; stanza elements to allow Raft messages to be transported cleanly over XMPP. The messages for Node to Node communication are well documented in the Raft paper and are reproduced here with minor additions to leverage the benefits of XMPP. However Client to Node communication is only hinted at and, as it does not use Raft natively it is outside the scope of this XEP.

+ +

Raft is a consensus algorithm that is easy to understand and implement. When you want to keep a distributed system consistent, you will need some form of consensus algorithm. Raft was developed at Stanford University as an alternative to the incumbent consensus algorithm PAXOS. PAXOS is claimed to be very complex, hard to teach and even harder to implement. All of these are discussed in a paper here and additional information (including a graphical simulation of a cluster) can be found on the project's website: https://raftconsensus.github.io/.

+ + +

Raft defines a set of core messages needed to implement the protocol. However, it does not specify the transport layer for this protocol. Most implementations have chosen to use vanilla TCP due to its simplicity. However if you want to run a cluster over the Internet, you are likely going to want more than what vanilla TCP provides. This is where XMPP would really shine as a transport layer for Raft. XMPP offers:

+ +
    +
  1. Encrypted transport (TLS)
    2. +
  2. Authenticated Endpoints
    3. +
  3. Ability to use JIDs to identify cluster nodes
    4. +
  4. Re-use XMPP if the application is already XMPP enabled
    5. +
+ +

These are all things that would traditionally have to be re-engineered each time somebody wanted to use Raft across the public Internet. By supporting Raft in XMPP, developers looking to use Raft would have a transport layer that's as easy to use and understand as the Raft protocol itself. As Raft does not offer its own transport protocol and has deliberately left that to the developer, there is no conflict in standardizing an XMPP based transport layer.

+ + +

Making databases or datastores available over the Internet has been the norm for many years. Databases containing PGP keys, certificates or other information can be found hosted by many different organizations. The problem with these systems is that as they become more critical to users, the impact of a server failing increases dramatically. For example, a server that provides a spam database that clients can verify email against, must be operational for those clients to be able to filter spam. Having a single server in this scenario is not acceptable; to provide redundancy there must be multiple servers.

+ +

The problem then becomes how to keep the servers in sync. Raft partly solves this problem by providing the means to ensure the cluster maintains consensus i.e. maintains a consistent view of the data. As mentioned previously, it does not however provide a means for the nodes in the cluster to actually communicate with each other or clients.

+ +

This is where being able to use Raft over XMPP would be highly beneficial. As it stands now, developers must implement their own transport and security, which although certainly possible, is not ideal. First, most developers are not security experts and a wealth of knowledge and experience is needed to properly design a secure system. Second, even with the required expertise, it takes a considerable amount of time and effort to actually implement and test any new implementation. Third, this extra work takes developers' focus away from the problem that they were trying to solve in the first place.

+ +

So how could Raft over XMPP be applied in this instance? First, XMPP has an excellent history when it comes to security. Considerable time and effort was spent ensuring that XMPP was secure when XMPP Core was being standardized. By using XMPP, this hard work and battle tested approach can be leveraged by Raft. This means developers do not need to concern themselves with securing Raft messages, rather they now only need to concern themselves with using XMPP appropriately.

+ +

Raft over XMPP further simplifies things by allowing developers to think at a higher level of abstraction. Nodes in the cluster can be communicated with simply by knowing their JID. It would not be necessary to know a node's IP address (which could change) or what TCP port the node is running on. In addition developers would not need to worry about which node connects to which, managing multiple TCP sockets and how to multiplex data across them.

+ +

Lastly, integrating a Raft implementation with Raft over XMPP, would be relatively straight forward as Raft over XMPP defines and uses the same names as those provided by the Raft paper with few additions. This means that it could be much easier to get a Raft implementation up and running using Raft over XMPP than it would be to do so even with pure vanilla sockets.

+ + + + + + + +

The author has designed Raft over XMPP with the following requirements in mind:

+ +
    +
  1. The protocol MUST support all messages as defined in the Raft paper
    2. +
  2. It SHOULD leverage the benefits of using XMPP as the transport layer
    3. +
  3. Client to Node interaction is out of scope for this XEP
    4. +
+ + + +
+
Raft
A distributed consensus algorithm designed at Stanford University to be simple and easy to implement. It aims to replace PAXOS as the conensus algorithm of choice for real world use and teaching. The Raft website is https://raftconsensus.github.io/
 +
Log Replication
Log replication is how Raft exchanges commands with the rest of the cluster.
 +
Follower
The default state of a cluster member. It receives and applies updates from the Leader
 +
Candidate
When a Follower has not seen a heartbeat from the Leader for a period of time, it will assume the leader has failed and will look to become the Leader itself.
 +
Leader
The Leader of the cluster is responsible for making all changes to the log and sending them to the other members of the cluster
 + +
VoteRequest
A VoteRequest message is sent by a Candidate in order to solicit votes to become the Leader of a cluster
 +
AppendEntries
An AppendEntries message is sent by a Leader to other nodes in the cluster when it has updates that it needs to replicate.
 + +
+ + + + +

This XEP defines a transport layer for Raft and not an actual implementation. That is, it does not seek to implement the Raft consensus algorithm within XMPP, but instead to simply define the means for Raft messages to be transported over XMPP.

+ +

Node to Node communication is the back-bone of a Raft cluster. In operation, only the Leader or a Candidate will send messages. In all other cases, nodes will only reply to messages received. The two messages are AppendEntries and RequestVote.

+ +

Note: These messages are Request/Response in nature but use the &MESSAGE; stanza rather than &IQ; because the handling of missing, delayed or repeated messages is handled by the Raft implementation and not by the transport layer.

+ + + +

When a Follower has not received a heartbeat from the Leader for a given period of time, it will determine that the Leader has failed and will seek to replace it. To do this it needs the support of the majority of nodes in the cluster. It can solicit support from other nodes by declaring itself a Candidate and sending a RequestVote message to all nodes in the cluster:

+ + + + + ]]> + +

A node will respond with a RequestVoteResponse:

+ + + + + ]]> + +

A node can either vote for a given Candidate (voteGranted="true") or against a Candidate voteGranted="false".

+ +

If a node does not receive a reply, no special handling is required.

+ + + + +

The AppendEntries message is used by the Leader to tell Followers that they should append a new entry (or entries) to their logs. It contains additional information to allow a Follower to determine which log entries have been executed and committed on the Leader and also if it has dropped any messages. These features are implemented in Raft directly.

+ + + + + SET X = 1 + + + U0VUIFggPSAx + + + + ]]> + +

The AppendEntries message is described as a simple array in the Raft paper and this has been expanded on in XMPP to take advantage of structured XML. In addition, Raft is designed to be able to replicate any form of command and this could be binary data rather than textual data. To accommodate this, an attribute has been added to the AppendEntries element to allow a sender to flag when the receiver needs to decode the Entry before passing it to the Raft implementation. The data is encoded using base64.

+ +

When followers receive this message, they send a single AppendEntriesResponse in reply as follows:

+ + + + + ]]> + +

As before, if a message is missed in either direction, the transport layer does not need to take action.

+ + + + + + + +

It is not the responsibility of the transport layer to determine whether a node is a member of a cluster or not before delivering messages to the Raft implementation. The Raft implementation should ignore messages that it receives from nodes that aren't part of the cluster.

+ + + + +

This document requires no interaction with the Internet Assigned Numbers Authority (IANA).

+ + + +

The ®ISTRAR; includes 'urn:xmpp:raft' in its registry of protocol namespaces.

+ + + +

REQUIRED for protocol specifications.

+ + From 8d691de73a3835803a0208a812ba2d9557bcd6b2 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Wed, 22 Jul 2015 22:57:42 -0500 Subject: [PATCH 004/106] Overwrite XEP-0286 instead of submitting a new XEP --- inbox/mobile-concerns.xml | 203 ------------------------- xep-0286.xml | 302 ++++++++++++++++++++++---------------- 2 files changed, 178 insertions(+), 327 deletions(-) delete mode 100644 inbox/mobile-concerns.xml diff --git a/inbox/mobile-concerns.xml b/inbox/mobile-concerns.xml deleted file mode 100644 index c2cdac0f..00000000 --- a/inbox/mobile-concerns.xml +++ /dev/null @@ -1,203 +0,0 @@ - - -%ents; -]> - - -
- Mobile Considerations - - This document provides background information for XMPP implementors - concerned with mobile devices operating on an LTE cellular network. - - &LEGALNOTICE; - NOT_YET_ASSIGNED - ProtoXEP - Informational - Standards - Council - - XMPP Core - - - XEP-0286 - - - NOT_YET_ASSIGNED - - Sam - Whited - sam@samwhited.com - sam@samwhited.com - - - 0.0.1 - 2015-07-18 - ssw -

First draft.

 - -
- -

- XMPP as a protocol was designed before the wide spread adoption of mobile - devices, and is often cited as not being very mobile friendly as a result. - However, this mostly stems from undocumented lore and outdated notions of - how XMPP works. As the Internet and protocol design have changed to be more - accommodating for mobile, so has XMPP. -

-

- This XEP aims to provide useful background knowledge of mobile handset - behavior, and those considerations that client and server designers can - take to ensure that bandwidth and battery are used efficiently. -

- - -

- The two major constraints on mobile devices are power and bandwidth. With - the wide spread proliferation of 3G and LTE technologies, mobile bandwidth - and speeds have become broadly comparable to broadband. However, they are - still relatively expensive compared to traditional wired networks, and - should therefore still be considered. This XEP mostly focuses on LTE as it - already has a very wide deployment and will only continue to further - replace 3G technologies. -

- - -

- XML, and by extension XMPP, is known to be highly compressible. In a simple - test of a small (266089 byte) XMPP stream (connection, stream - initialization, feature discovery, roster loading, several presence stanzas - sent and received, disconnect), the entropy of the stream was found to be - 5.616313 bits per byte. Using the `gzip` tool to apply Lempel-Ziv coding - (LZ77) without concern for server-side CPU usage resulted in a compression - ratio of 21% (a 79% reduction in bandwidth). In one test with a much larger - dataset typical of a corporate environment (many hundreds of users in the - roster), the ratio was as low as 13%, an 87% reduction in bandwidth! -

-

- Compression of XMPP data can be achieved with the DEFLATE algorithm - (&rfc1951;) via TLS compression (&rfc3749;) or &xep0138;. While the - security implications of stream compression are beyond the scope of this - document (See the aforementioned RFC or XEP for more info), mitigating them - may affect compression ratios. The author does not recommend using TLS - compression with XMPP (or in general). If compression must be used, stream - level compression should be implemented instead. Compressing at the stream - level gives us the benefit of being able to flush the compression stream on - stanza boundaries to help prevent information from leaking. This, however, - may drastically increase compression ratios. -

-

- While the CPU cost of compression directly translates to higher power - usage, it is vastly outweighed by the benefits of reduced network - utilization, especially on modern LTE networks which use a great deal more - power per bit than 3G networks as will be seen later in this document. - Setting security considerations aside and thinking only of power - consumption and bandwidth, supporting compression is highly recommended. -

- - -

- While the wide spread adoption of LTE has dramatically increased available - bandwidth on mobile devices, it has also increased power consumption. - According to one study, early LTE devices consumed 5–20% more power than - their 3G counterparts - LTE Smartphone measurements <http://networks.nokia.com/system/files/document/lte_measurements_final.pdf>. - On some networks that support the legacy SVLTE (Simultaneous Voice and LTE) - instead of the more modern VoLTE (Voice Over LTE) standard, or even CSFB - (Circuit-switched fallback) this number would (presumably) be even higher. -

-

- XMPP server and client implementers, bearing this increased power usage in - mind, and knowing a bit about how LTE radios work, can optimize their - traffic to minimize network usage. For the downlink, LTE user equipment - (UE) utilizes Orthogonal Frequency Division Multiplexing (OFDM), which is - somewhat inefficient - A Close Examination of Performance and Power Characteristics of 4G LTE Networks <http://www.cs.columbia.edu/~lierranli/coms6998-7Spring2014/papers/rrclte_mobisys2012.pdf>. - On the uplink side a different technology, Single-carrier frequency - division multiple access (SC-FDMA) is used, which is slightly more - efficient than traditional (non linearly-precoded) OFDM, slightly - offsetting the fact that broadcasting requires more power than receiving. - LTE UE also implements a Discontinuous reception (DRX) mode in which the - hardware can sleep until it is woken by a paging message or is needed to - perform some task. LTE radios have two power modes: RRC_CONNECTED and - RRC_IDLE. DRX is supported in both of these power modes. By attempting to - minimize the time which the LTE UE state machine spends in the - RCC_CONNECTED state, and maximize the time it stays in the DRX state (for - RCC_CONNECTED and RRC_IDLE), we can increase battery life without degrading - the XMPP experience. To do so, the following rules should be observed: -

- -

- Whenever possible, data that is not strictly needed should not be - transmitted (by the server or client). Supporting &xep0352; is highly - recommended. Most importantly, XMPP pings should be kept as far apart as - possible and only used when necessary. Server operators are encouraged to - set high ping timeouts, and client implementors are advised to only send - pings when absolutely necessary to prevent the server from closing the - socket. -

- - -

- If one is on 3G, transmitting a small amount of data will cause the radio - to enter FACH mode which is significantly cheaper than its high power - mode. On LTE radios, however, transmitting small amounts of data is - vastly more expensive per bit due to the significantly higher tail-times - (the time it takes for the radio to change state). On LTE radios, one - should transmit as much data as possible when the radio is already on - (eg. by placing messages in a send queue and executing the queue as a - batch). Similarly, when data is being received the radio is already in a - high power state and therefore any data that needs to be sent should be. -

-

- These rules also apply to server operators: If you receive data, the - phones radio is already on therefore you should send anything you have. - Otherwise, batching data to be sent and sending it all at once (and as - much as possible) will help reduce power consumption. -

- - - -

- This section provides pointers to other documents which may be of interest - to those developing mobile clients, or considering support for them in - servers. -

-

&xep0138; provides stream level compression.

-

- &xep0115; provides a mechanism for caching, and hence eliding, the - disco#info requests needed to negotiate optional features. -

-

- &xep0237; provides a relatively widely deployed extension for reducing - roster fetch sizes. -

-

- &xep0198; allows the client to send and receive smaller keep-alive - messages, and resume existing sessions without the full handshake. Useful - on unstable connections. -

- - -

- The original mobile XEP, &xep0286;, was written by Dave Cridland, and parts - of it were used when writing this XEP. -

- - -

This document introduces no new security considerations.

- - -

- This document requires no interaction with the Internet Assigned Numbers - Authority (IANA). -

- - -

- No namespaces or parameters need to be registered with the XMPP Registrar - as a result of this document. -

- - diff --git a/xep-0286.xml b/xep-0286.xml index 78a31f37..bffe86f3 100644 --- a/xep-0286.xml +++ b/xep-0286.xml @@ -6,8 +6,11 @@
- XMPP on Mobile Devices - This document provides background information for XMPP implementors concerned with mobile devices operating in a cellular network such as 3G. + Mobile Considerations + + This document provides background information for XMPP implementors + concerned with mobile devices operating on an LTE cellular network. + &LEGALNOTICE; 0286 Deferred @@ -26,6 +29,18 @@ dave.cridland@isode.com dave.cridland@isode.com + + Sam + Whited + sam@samwhited.com + sam@samwhited.com + + + 0.2 + 2015-07-22 + ssw +

Overhaul to include LTE.

 + 0.1 2010-09-15 @@ -40,140 +55,179 @@
-

The use of XMPP on mobile devices is little understood, since few XMPP implementors have good mobile knowledge, and few mobile engineers have good XMPP knowledge. In addition, as the mobile landscape has changed, optimal protocol designs and usage patterns have also changed. This has led to the sub-optimal combination of a large amount of mostly undocumented lore, as well as several outdated concepts being discussed as fact.

-

This XEP aims to provide useful background knowledge of mobile handset behaviours, and essentially distills a number of conversations with experienced mobile engineers and XMPP implementors, providing useful background as general suggestions.

+

+ XMPP as a protocol was designed before the wide spread adoption of mobile + devices, and is often cited as not being very mobile friendly as a result. + However, this mostly stems from undocumented lore and outdated notions of + how XMPP works. As the Internet and protocol design have changed to be more + accommodating for mobile, so has XMPP. +

+

+ This XEP aims to provide useful background knowledge of mobile handset + behavior, and those considerations that client and server designers can + take to ensure that bandwidth and battery are used efficiently. +

-

Mobile handsets typically have two constraints - power and bandwidth. The advent of 3G technology and beyond has tended to mean that bandwidth is radically higher, and comparable to broadband speeds - however many operators still charge based on transferred data, hence bandwidth remains an important issue for cost purposes.

-

The major cost of power in the handset for our purposes is the radio - here, too, bandwidth plays a part, but as this document will show, the time the radio is forced to be available to receive also costs substantially.

-

Whilst this document refers to &rfc3920;, implementors are advised to take note of &rfc6120;.

+

+ The two major constraints on mobile devices are power and bandwidth. With + the wide spread proliferation of 3G and LTE technologies, mobile bandwidth + and speeds have become broadly comparable to broadband. However, they are + still relatively expensive compared to traditional wired networks, and + should therefore still be considered. This XEP mostly focuses on LTE as it + already has a very wide deployment and will only continue to further + replace 3G technologies. +

- -

XMPP is known to compress well. Both TLS, part of &xmppcore;, and &xep0138; can provide access to the DEFLATE codec (RFC 1951 RFC 1951 DEFLATE Compressed Data Format Specification version 1.3 - <http://tools.ietf.org/html/rfc1951>.), which provides access to simple stream compression.

-

Compression ratios vary with usage, however, typical usage by a general client appears to show a 20% ratio (an 80% reduction in bandwidth) in longer sessionsFixed-purpose clients, such as the Buddycloud client, do see even lower ratios, approaching 10%.. Server implementors should note that there is a substantial memory cost per codec of 300KB assuming maximum settings - this may be dramatically reduced by reducing the memory level and window bits of the implementation - lowering memory level primarily causes increased CPU usage, whereas lowering the window bits directly degrade compression.

-

At an exemplary point in one experiment, the author found the following figuresThe compression ratio is here given as Original/Compressed, hence a 100% compression ratio is no compression at all, and 0% would represent infinite compression.:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Window BitsCompression Ratio (approx)
1520%
1422%
1325%
1230%
1138%
1043%
960%
-

Although there is an equal cost for the mobile device to compress, it is considered that the compression codec memory and CPU costs - while certainly translating into power cost - are outweighed by two factors. Firstly, they're likely to reduce the transmission cost by a greater amount, and secondly they will also reduce the encryption cost in TLS.

-

Care, however, should be taken not to use XEP-0138 compression when TLS compression is in effect.

+ + +

+ XML, and by extension XMPP, is known to be highly compressible. In a simple + test of a small (266089 byte) XMPP stream (connection, stream + initialization, feature discovery, roster loading, several presence stanzas + sent and received, disconnect), the entropy of the stream was found to be + 5.616313 bits per byte. Using the `gzip` tool to apply Lempel-Ziv coding + (LZ77) without concern for server-side CPU usage resulted in a compression + ratio of 21% (a 79% reduction in bandwidth). In one test with a much larger + dataset typical of a corporate environment (many hundreds of users in the + roster), the ratio was as low as 13%, an 87% reduction in bandwidth! +

+

+ Compression of XMPP data can be achieved with the DEFLATE algorithm + (&rfc1951;) via TLS compression (&rfc3749;) or &xep0138;. While the + security implications of stream compression are beyond the scope of this + document (See the aforementioned RFC or XEP for more info), mitigating them + may affect compression ratios. The author does not recommend using TLS + compression with XMPP (or in general). If compression must be used, stream + level compression should be implemented instead. Compressing at the stream + level gives us the benefit of being able to flush the compression stream on + stanza boundaries to help prevent information from leaking. This, however, + may drastically increase compression ratios. +

+

+ While the CPU cost of compression directly translates to higher power + usage, it is vastly outweighed by the benefits of reduced network + utilization, especially on modern LTE networks which use a great deal more + power per bit than 3G networks as will be seen later in this document. +

+

+ Supporting compression and flushing on stanza boundaries is highly + recommended. +

- -

Mobile handsets have a number of levels for radio activity. 3G radios can be either Idle, or else in an increasingly capable - and increasingly power-hungry - series of levels, through FACH to DCH.

-

For the purposes of investigating this, power consumption (or rather battery depletion rate, as current) and timeouts where measured on the 3UK network, with a Nokia E71, using the Energy Profiler. A "typical handset" mentioned here has a 1000mAh battery - some smartphones have up to 1500mAh. Note that all timeouts are under the control of the network operator, not the handset or application.

- -

Idle state is when the radio is neither receiving nor transmitting. It may have live (although silent) TCP connections. The cost is low. There is also a PCH level, which is similarly low-power, and again is only used when the radio is silent.

-

The current here was measured as 8mA - likely affected more by the energy profiler than much else.

- - -

FACH uses a shared channel for low-bandwidth communications. Packet sizes must be small - around 128 octets maximum, although this is operator controlled. Raising to this state takes around 2.5 seconds before the data can flow - although the power cost rises instantly - and after the session has returned to silence, it will remain at FACH level for some time.

-

Note that this threshold includes the overheads from TCP and TLS, which are 52 and 5 octets respectively, leaving around 70 octets for the payload data.

-

Here, the current was measured as 140mA, and a timeout of 8 seconds - this timeout is at the lower end of the expected range, which could be up to around 2 minutes. On a typical handset, this will exhaust the battery in around 7 hours.

- - -

DCH uses a dedicated channel for high bandwidth communications. Again, raising to this state takes 2.5 seconds at the DCH power level, and there is a timeout before dropping back. Some operators will drop back to FACH for the duration fo the FACH timeout - others will drop back to Idle/PCH.

-

Sending more than the FACH threshold will raise the radio all the way to DCH - taking, again, 2.5 seconds.

-

The measurements here were 380mA and 8 seconds - this is sufficient to flatten a typical handset battery in less than 3 hours, and the figures are considered normal.

- -

Transmission of data can use up to 2WThe author's hazy recollection of P=IV suggests around a 570mA current, and raising the level itself takes between 2 and 3 seconds to take effect - during which time the handset cannot receive or transmit, but still incurs the power cost. There are packets sent from and to the handset during this time.

-

Experimentation suggests that uncompressed XMPP will never trigger the FACH state, leaping directly into the more costly DCH state. However, compression does make FACH possible, if rare.

+ +

+ While the wide spread adoption of LTE has dramatically increased available + bandwidth on mobile devices, it has also increased power consumption. + According to one study, early LTE devices consumed 5–20% more power + than their 3G counterparts + LTE Smartphone measurements <http://networks.nokia.com/system/files/document/lte_measurements_final.pdf>. + On some networks that support the legacy SVLTE (Simultaneous Voice and LTE) + instead of the more modern VoLTE (Voice Over LTE) standard, or even CSFB + (Circuit-switched fallback) this number would (presumably) be even higher. +

+

+ XMPP server and client implementers, bearing this increased power usage in + mind, and knowing a bit about how LTE radios work, can optimize their + traffic to minimize network usage. For the downlink, LTE user equipment + (UE) utilizes Orthogonal Frequency Division Multiplexing (OFDM), which is + somewhat inefficient + A Close Examination of Performance and Power Characteristics of 4G LTE Networks <http://www.cs.columbia.edu/~lierranli/coms6998-7Spring2014/papers/rrclte_mobisys2012.pdf>. + On the uplink side a different technology, Single-carrier frequency + division multiple access (SC-FDMA) is used, which is slightly more + efficient than traditional (non linearly-precoded) OFDM, slightly + offsetting the fact that broadcasting requires more power than receiving. + LTE UE also implements a Discontinuous reception (DRX) mode in which the + hardware can sleep until it is woken by a paging message or is needed to + perform some task. LTE radios have two power modes: RRC_CONNECTED and + RRC_IDLE. DRX is supported in both of these power modes. By attempting to + minimize the time which the LTE UE state machine spends in the + RCC_CONNECTED state, and maximize the time it stays in the DRX state (for + RCC_CONNECTED and RRC_IDLE), we can increase battery life without degrading + the XMPP experience. To do so, the following rules should be observed: +

+ +

+ Whenever possible, data that is not strictly needed should not be + transmitted (by the server or client). Supporting &xep0352; is highly + recommended. Most importantly, XMPP pings should be kept as far apart as + possible and only used when necessary. Server operators are encouraged to + set high ping timeouts, and client implementors are advised to only send + pings when absolutely necessary to prevent the server from closing the + socket. +

+ + +

+ If one is on 3G, transmitting a small amount of data will cause the radio + to enter FACH mode which is significantly cheaper than its high power + mode. On LTE radios, however, transmitting small amounts of data is + vastly more expensive per bit due to the significantly higher tail-times + (the time it takes for the radio to change state). On LTE radios, one + should transmit as much data as possible when the radio is already on + (eg. by placing messages in a send queue and executing the queue as a + batch). Similarly, when data is being received the radio is already in a + high power state and therefore any data that needs to be sent should be. +

+

+ These rules also apply to server operators: If you receive data, the + phones radio is already on therefore you should send anything you have. + Otherwise, batching data to be sent and sending it all at once (and as + much as possible) will help reduce power consumption. +

+ - -

As with anything, there are no hard and fast rules. If there were, they might look like these. First, for devices:

-
- -
Transmit no data.
-
Transmitting costs significant power, and moreover raises the radio state. Not transmitting will allow it to maximize the time spent in the low-cost Idle state.
- - -
If you must transmit, then transmit only a small volume.
-
If there is only a small amount of data transmitted - less than 128 octets typically - the radio will only raise to FACH, which is significantly cheaper than DCH.
- - -
If you must transmit, then compress as hard as possible.
-
Since individual octets have an associate power - and often financial - cost, it's worth maximizing the compression algorithm, even if the volume of traffic will raise to DCH.
- - -
If you have transmit a lot, then do a lot
-
If the radio is raised to DCH anyway, then you may as well go fetch that avatar you were missing, since you're chewing through power anyway.
- - -
If you receive, then transmit
-
If your peer raises the radio state, you may as well use it.
- -
-

And for servers, similar rules apply:

-
- -
Send no data.
-
Sending data will cause the handset to be raised out of Idle. This immediately costs massively higher power.
- - -
If you must send, send tiny bits.
-
Sending small enough data maximizes the likelyhood that the devices radio will only be raised to FACH levels.
- - -
If you receive, then send anything you have.
-
Receiving data indicates that the radio is active - it'll stay active for some time, so sending data doesn't incur the overhead of raising the radio state, and won't increase power drain on the handset.
- - -
If you must send when not receiving, send plenty.
-
Sending data will raise the radio's state - unless you can tell this will only raise it to FACH, it's worth sending as much as possible.
- -
-

Finally, protocol designers should aim to minimize any responses required from the handset, and ensure keepalive traffic, if any, fits inside FACH wherever possible.

- - -

This section provides pointers to other documents which may be of interest to those developing mobile clients, or considering support for them in servers.

-

&xep0138; provides application stream level compression, useful if the device TLS stack does not support TLS-based compression.

-

&xep0115; provides a mechanism for caching, and hence eliding, the disco#info requests needed to negotiate optional features.

-

&xep0237; provides a relatively widely deployed extension for reducing the roster fetch bandwidth, in most cases reducing it to a simple affirmation that the client has the current roster. This saves not only bandwidth, but also reduces local storage writes.

-

&xep0198; provides session resumption over TCP, enabling a client to handle the case where the coverage is patchy. The <r/> and <a/> elements also provide a keepalive facility in a small number of octets.

-

&xep0273; provides a mechanism which, amongst other things, would allow a presence "hush", buffering presence during certain states.

+ +

+ This section provides pointers to other documents which may be of interest + to those developing mobile clients, or considering support for them in + servers. +

+

&xep0138; provides stream level compression.

+

+ &xep0115; provides a mechanism for caching, and hence eliding, the + disco#info requests needed to negotiate optional features. +

+

+ &xep0237; provides a relatively widely deployed extension for reducing + roster fetch sizes. +

+

+ &xep0198; allows the client to send and receive smaller keep-alive + messages, and resume existing sessions without the full handshake. Useful + on unstable connections. +

+

+ &xep0357; implements push notifications (third party message delivery), + which are often used on mobile devices and highly optimized to conserve + battery. Push notifications also allow delivery of notifications to + mobile clients that are currently offline (eg. in an XEP-0198 "zombie" + state). +

+

+ &xep0313; lets clients fetch messages which they missed (eg. due to poor + mobile coverage and a flakey network connection). +

-

The author is not a mobile expert, and relied on the knowledge and patient help of several others. In particular, thanks are due to Jussi Laako, Markku Vampari, and Markus Isomaki of Nokia, and Simon Tennant of Buddycloud.

-

The attribution of any mistakes herein is zealously guarded by the author, however.

+

+ This XEP was originally written by Dave Cridland, and parts of his original + work were used in this rewrite. +

-

This document does not discuss a protocol, thus introduces no new security considerations.

+

This document introduces no new security considerations.

 -

None.

+

+ This document requires no interaction with the Internet Assigned Numbers + Authority (IANA). +

-

None.

+

+ No namespaces or parameters need to be registered with the XMPP Registrar + as a result of this document. +

From dd59711e7ae8f96875ab253515132d4574afac48 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 23 Jul 2015 10:02:55 +0200 Subject: [PATCH 005/106] mark 0286 as Experimental --- xep-0286.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xep-0286.xml b/xep-0286.xml index bffe86f3..c8b3ad54 100644 --- a/xep-0286.xml +++ b/xep-0286.xml @@ -13,7 +13,7 @@ &LEGALNOTICE; 0286 - Deferred + Experimental Informational Standards Council From a7a7fdd92d2879446a4554992ef8c5124c838ef9 Mon Sep 17 00:00:00 2001 From: Peter Membrey Date: Thu, 23 Jul 2015 13:38:00 +0800 Subject: [PATCH 006/106] Updated element names based on Council feedback and usage in XEP-0176 --- inbox/raft.xml | 54 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 33 insertions(+), 21 deletions(-) diff --git a/inbox/raft.xml b/inbox/raft.xml index 3e34e787..945d050a 100644 --- a/inbox/raft.xml +++ b/inbox/raft.xml @@ -28,6 +28,12 @@ peter@membrey.hk peter@membrey.hk + + 0.0.2 + 2015-07-22 + pm +

Updates based on list and council feedback.

 + 0.0.1 2015-07-20 @@ -52,6 +58,15 @@

These are all things that would traditionally have to be re-engineered each time somebody wanted to use Raft across the public Internet. By supporting Raft in XMPP, developers looking to use Raft would have a transport layer that's as easy to use and understand as the Raft protocol itself. As Raft does not offer its own transport protocol and has deliberately left that to the developer, there is no conflict in standardizing an XMPP based transport layer.

+ +

The Raft algorithm can be categorized as a request-response protocol. Normally this would make it a prime candidate for using &IQ; stanzas to handle the communication. However because Raft is designed to cope with message loss, it intrinsically supports automatic recovery. There is no need for the transport layer to report errors as even if the transport layer provided them (such as an &IQ; 'error' response), the Raft implementation cannot use it.

+ +

This has a number of benefits. First, it makes Raft adaptable to lossy transport layers where packets can (and do) get lost. Raft is able to automatically recover in this scenario because the next message the Leader sends will allow a Follower to detect that it has missed a message and ask for it to be sent again. The Leader has no way to deal with an error condition caused by sending a message to a Follower.

+ +

Second, when it comes to implementing Raft over XMPP, using &MESSAGE; instead of &IQ; greatly simplifies the implementation. As &IQ; stanzas require a reply, the implementation would need to handle detecting and reporting errors conditions back to the sender. This could mean adding arbitrary timers to try to determine if a Follower has 'timed out'. This adds complexity and uncertainty to the system, and given that Raft itself cannot make use of this information, using &IQ; does not add any value to the Raft over XMPP protocol.

+ + +

Making databases or datastores available over the Internet has been the norm for many years. Databases containing PGP keys, certificates or other information can be found hosted by many different organizations. The problem with these systems is that as they become more critical to users, the impact of a server failing increases dramatically. For example, a server that provides a spam database that clients can verify email against, must be operational for those clients to be able to filter spam. Having a single server in this scenario is not acceptable; to provide redundancy there must be multiple servers.

@@ -96,59 +111,56 @@ -

This XEP defines a transport layer for Raft and not an actual implementation. That is, it does not seek to implement the Raft consensus algorithm within XMPP, but instead to simply define the means for Raft messages to be transported over XMPP.

+

This XEP defines a transport layer for Raft and not an actual implementation. That is, it does not seek to implement the Raft consensus algorithm within XMPP, but instead to simply define the means for Raft messages to be transported over XMPP. To facilitate this, both the message name used in the Raft spec (shown in camel case) and the corresponding element name are mentioned together where appropriate.

Node to Node communication is the back-bone of a Raft cluster. In operation, only the Leader or a Candidate will send messages. In all other cases, nodes will only reply to messages received. The two messages are AppendEntries and RequestVote.

-

Note: These messages are Request/Response in nature but use the &MESSAGE; stanza rather than &IQ; because the handling of missing, delayed or repeated messages is handled by the Raft implementation and not by the transport layer.

- - -

When a Follower has not received a heartbeat from the Leader for a given period of time, it will determine that the Leader has failed and will seek to replace it. To do this it needs the support of the majority of nodes in the cluster. It can solicit support from other nodes by declaring itself a Candidate and sending a RequestVote message to all nodes in the cluster:

+

When a Follower has not received a heartbeat from the Leader for a given period of time, it will determine that the Leader has failed and will seek to replace it. To do this it needs the support of the majority of nodes in the cluster. It can solicit support from other nodes by declaring itself a Candidate and sending a 'request-vote' (RequestVote) message to all nodes in the cluster:

- + ]]> -

A node will respond with a RequestVoteResponse:

+

A node will respond with a 'vote' (RequestVoteResponse) message:

- + ]]> -

A node can either vote for a given Candidate (voteGranted="true") or against a Candidate voteGranted="false".

+

A node can either vote for a given Candidate (vote-granted="true") or against a Candidate (vote-granted="false").

If a node does not receive a reply, no special handling is required.

 -

The AppendEntries message is used by the Leader to tell Followers that they should append a new entry (or entries) to their logs. It contains additional information to allow a Follower to determine which log entries have been executed and committed on the Leader and also if it has dropped any messages. These features are implemented in Raft directly.

+

The 'append' (AppendEntries) message is used by the Leader to tell Followers that they should append a new entry (or entries) to their logs. It contains additional information to allow a Follower to determine which log entries have been executed and committed on the Leader and also if it has dropped any messages. These features are implemented in Raft directly.

- - - + + SET X = 1 - - + + U0VUIFggPSAx - - + + ]]> -

The AppendEntries message is described as a simple array in the Raft paper and this has been expanded on in XMPP to take advantage of structured XML. In addition, Raft is designed to be able to replicate any form of command and this could be binary data rather than textual data. To accommodate this, an attribute has been added to the AppendEntries element to allow a sender to flag when the receiver needs to decode the Entry before passing it to the Raft implementation. The data is encoded using base64.

+

The AppendEntries message is described as a simple array in the Raft paper and this has been expanded on in XMPP to take advantage of structured XML. In addition, Raft is designed to be able to replicate any form of command and this could be binary data rather than textual data. To accommodate this, an attribute has been added to the 'append-entries' element to allow a sender to flag when the receiver needs to decode the Entry before passing it to the Raft implementation. The data is encoded using base64.

-

When followers receive this message, they send a single AppendEntriesResponse in reply as follows:

+

When followers receive this message, they send a single 'append-response' (AppendEntriesResponse) in reply as follows:

- - + ]]> From e183820d437de3d525c081befca24e45b4f65f77 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 23 Jul 2015 16:27:25 +0200 Subject: [PATCH 007/106] Update copyright dates (fixes #28) and deferred notices (fixes #27). --- xep.xsl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/xep.xsl b/xep.xsl index 616c3ce0..f16914f5 100644 --- a/xep.xsl +++ b/xep.xsl @@ -2,7 +2,7 @@ | | [ XML stream over IBB ] | - |<----------------------------->| + |<=============================>| | terminate | |<------------------------------| | ack | @@ -118,7 +118,7 @@ id='hwd987h' to='romeo@montague.lit/orchard' type='set'> - + From 73a62049e0b7a241ba876573b92f071f7065e641 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 13 Aug 2015 14:57:13 -0600 Subject: [PATCH 027/106] fix missing entity reference --- xep-0247.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xep-0247.xml b/xep-0247.xml index 8f84a70d..f72bc9cc 100644 --- a/xep-0247.xml +++ b/xep-0247.xml @@ -53,7 +53,7 @@
  • Two endpoints want to enforce end-to-end encryption
  • Two endpoints want to send a high volume of XMPP traffic but the intermediate servers enforce rate limits
    • -

    The first situation is addressed by &xep0174;. However, if the endpoints already have client-to-server connections but wish to bypass those connections or leverage those streams for a higher-level application such as end-to-end encryption, it is desirable for the two endpoints to negotiate an end-to-end XML stream. This specification defines methods for doing so, where the application format is an XML stream and the transport method is any direct or mediated streaming transport, such as &xep0261; (mediated), &xep0260; (direct or mediated), or a future ice-tcp Jingle transport (direct or mediated) based on &ice-tcp;.

    +

    The first situation is addressed by &xep0174;. However, if the endpoints already have client-to-server connections but wish to bypass those connections or leverage those streams for a higher-level application such as end-to-end encryption, it is desirable for the two endpoints to negotiate an end-to-end XML stream. This specification defines methods for doing so, where the application format is an XML stream and the transport method is any direct or mediated streaming transport, such as &xep0261; (mediated), &xep0260; (direct or mediated), or a future ice-tcp Jingle transport (direct or mediated) based on &rfc6544;.

     From 1b7a6faf3f4842f20e7398cb81255b3b2810dd95 Mon Sep 17 00:00:00 2001 From: Peter Saint-Andre Date: Tue, 14 Jul 2015 09:25:23 -0600 Subject: [PATCH 028/106] clarify CCYY is expected to be four digits, even though XML Schema allows for more. --- xep-0082.xml | 8 ++++---- xep.ent | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/xep-0082.xml b/xep-0082.xml index 0e2c346f..bc4300a6 100644 --- a/xep-0082.xml +++ b/xep-0082.xml @@ -80,7 +80,7 @@

    The following acronyms and characters are used herein to represent time-related concepts:

    - + @@ -102,7 +102,7 @@ CCYY-MM-DD -

    This profile is equivalent to the 'date' datatype defined in XML Schema The 'date' datatype is defined at <http://www.w3.org/TR/xmlschema-2/#date>.. When an XML schema is used to define an XMPP protocol extension that uses this profile, the datatype MUST be an XML Schema 'date'. If there are differences between the description in this document and those in XML Schema, the latter overrule.

    +

    This profile is equivalent to the 'date' datatype defined in XML Schema The 'date' datatype is defined at <http://www.w3.org/TR/xmlschema11-2/#date>.. When an XML schema is used to define an XMPP protocol extension that uses this profile, the datatype MUST be an XML Schema 'date'. If there are differences between the description in this document and those in XML Schema, the latter overrule.

    1776-07-04 @@ -113,7 +113,7 @@ CCYY-MM-DDThh:mm:ss[.sss]TZD

    The Time Zone Definition is mandatory and MUST be either UTC (denoted by addition of the character 'Z' to the end of the string) or some offset from UTC (denoted by addition of '[+|-]' and 'hh:mm' to the end of the string). The fractions of a second are optional and MAY be ignored if included (although an XMPP protocol extension using the DateTime profile MAY require the fractions of a second).

    -

    This profile is equivalent to the 'dateTime' datatype defined in XML Schema The 'dateTime' datatype is defined at <http://www.w3.org/TR/xmlschema-2/#dateTime>.. When an XML schema is used to define a Jabber protocol that uses this profile, the datatype MUST be an XML Schema 'dateTime'. If there are differences between the description in this document and those in XML Schema, the latter overrule.

    +

    This profile is equivalent to the 'dateTime' datatype defined in XML Schema The 'dateTime' datatype is defined at <http://www.w3.org/TR/xmlschema11-2/#dateTime>.. When an XML schema is used to define a Jabber protocol that uses this profile, the datatype MUST be an XML Schema 'dateTime'. If there are differences between the description in this document and those in XML Schema, the latter overrule.

    1969-07-21T02:56:15Z @@ -127,7 +127,7 @@ hh:mm:ss[.sss][TZD]

    The Time Zone Definition is optional; if included, it MUST be either UTC (denoted by addition of the character 'Z' to the end of the string) or some offset from UTC (denoted by addition of '[+|-]' Inclusion of the '+' character means that the time in that zone is ahead of UTC; e.g., a Time Zone Definition of "+07:00" means that if the UTC time is 12:00 then the local time is 19:00 (typically this is true of an area that is east of 0° degrees latitude and west of 180° latitude, such as Bangkok, Thailand). Inclusion of the '-' character means that the time in that zone is behind UTC; e.g., a Time Zone Definition of "-07:00" means that if the UTC time is 12:00 then the local time is 05:00 (typically this is true of an area that is west of 0° degrees latitude and east of 180° latitude, such as Denver, Colorado, USA). and 'hh:mm' to the end of the string). The fractions of a second are optional and MAY be ignored if included (although a Jabber protocol using the DateTime profile MAY require the fractions of a second).

    -

    This profile is equivalent to the 'time' datatype defined in XML Schema The 'time' datatype is defined at <http://www.w3.org/TR/xmlschema-2/#time>.. When an XML schema is used to define a Jabber protocol that uses this profile, the datatype MUST be an XML Schema 'time'. If there are differences between the description in this document and those in XML Schema, the latter overrule.

    +

    This profile is equivalent to the 'time' datatype defined in XML Schema The 'time' datatype is defined at <http://www.w3.org/TR/xmlschema11-2/#time>.. When an XML schema is used to define a Jabber protocol that uses this profile, the datatype MUST be an XML Schema 'time'. If there are differences between the description in this document and those in XML Schema, the latter overrule.

    16:00:00 diff --git a/xep.ent b/xep.ent index 1094aa4d..9b7c2aaf 100644 --- a/xep.ent +++ b/xep.ent @@ -308,8 +308,8 @@ THE SOFTWARE. XML 1.0Extensible Markup Language (XML) 1.0 (Fourth Edition) <http://www.w3.org/TR/REC-xml/>." > XML EncryptionXML Encryption Syntax and Processing <http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/>." > Namespaces in XMLNamespaces in XML <http://www.w3.org/TR/REC-xml-names/>." > -XML Schema Part 1XML Schema Part 1: Structures <http://www.w3.org/TR/xmlschema-1/>." > -XML Schema Part 2XML Schema Part 2: Datatypes <http://www.w3.org/TR/xmlschema-2/>." > +XML Schema Part 1XML Schema Part 1: Structures <http://www.w3.org/TR/xmlschema11-1/>." > +XML Schema Part 2XML Schema Part 2: Datatypes <http://www.w3.org/TR/xmlschema11-2/>." > XML SignatureXML Signature Syntax and Processing <http://www.w3.org/TR/2002/REC-xmldsig-core-20020212/>." > XML-binary Optimized PackagingXML-binary Optimized Packaging <http://www.w3.org/TR/xop10/>." > XPathXML Path Language <http://www.w3.org/TR/xpath>." > From ccfbb0dcf0fde4fde6b5800ad5e62fd6af20690d Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Tue, 18 Aug 2015 18:33:47 -0600 Subject: [PATCH 029/106] ProtoXEP Jingle HTTP Transport Method --- inbox/jingle-http-transport.xml | 618 ++++++++++++++++++++++++++++++++ 1 file changed, 618 insertions(+) create mode 100644 inbox/jingle-http-transport.xml diff --git a/inbox/jingle-http-transport.xml b/inbox/jingle-http-transport.xml new file mode 100644 index 00000000..f0871682 --- /dev/null +++ b/inbox/jingle-http-transport.xml @@ -0,0 +1,618 @@ + + +%ents; +]> + + +
    + Jingle HTTP Transport Method + This specification defines two Jingle transport methods for establishing HTTP connections for either uploading or downloading data. + &LEGALNOTICE; + xxxx + ProtoXEP + Standards Track + Standards + Council + + XMPP Core + XEP-0166 + + + + NOT_YET_ASSIGNED + &lance; + + 0.0.1 + 2015-07-30 + ljts +

    First draft.

     +     +
    + + +

    &xep0166; defines a framework for negotiating and managing out-of-band data sessions over XMPP. In order to provide a flexible framework, the base Jingle specification defines neither data transport methods nor application formats, leaving that up to separate specifications.

    + +

    The current document defines two transport methods for establishing and managing data exchanges between XMPP entities using the Hyper Text Transfer Protocol (HTTP, see &rfc2616;); one method is for sharing via pulling data from an HTTP URI (http-download), and the other is for sharing via pushing data to an HTTP URI (http-upload).

    +     + + +

    Historically, &xep0066; has been used to trigger downloading files via HTTP, as well as initiating the use of any other known URI scheme. However, it has several limitations:

    +
      +
    • It only allows for the downloading of files via HTTP, not uploading.
      • +
    • It can not specify any additional headers that might be necessary for downloading files (authentication tokens, etc).
      • +
    • The allowance of any URI scheme makes it too unwieldy to be integrated into Jingle the way SOCKS5 Bytestreams and In-Band Bytestreams were (see &xep0260; and &xep0261;).
      • +
    +

    As such, this document defines two Jingle mechanisms designed to meet the following requirements:

    +
      +
    • Allow for both uploading and downloading of data via HTTP.
      • +
    • Allow additional HTTP headers to be specified.
      • +
    • Restrict the URI scheme to HTTP/HTTPS.
      • +
    +     + + +

    In accordance with Section 12 of XEP-0166, this document specifies the following information related to both the Jingle http-download and http-upload transport methods:

    +
      +

    1. The transport negotiation process for http-download is defined in the Negotiating HTTP Download section of this document, and the negotation process for http-upload is defined in the Negotiating HTTP Upload section of this document.

      2. +

    2. The semantics of the &TRANSPORT; element are defined in the Negotiating HTTP Download and Negotiating HTTP Upload sections of this document.

      3. +

    3. Successful negotiation of both the http-download and http-upload methods results in use of a streaming transport method suitable for use in Jingle application types where packet loss cannot be tolerated (e.g., file transfer).

      4. +

    4. Multiple components are not supported by http-download or http-upload.

      5. +
    +     + + +

    Negotiating HTTP downloads is done by using a &TRANSPORT; element with the 'urn:xmpp:jingle:transports:http:0' namespace, &VNOTE;. This element MAY include <candidate/> elements which represent URIs where data can be downloaded. Each <candidate/> element MUST include a 'uri' attribute, and MAY contain <header/> elements whose 'name' attribute is an HTTP header and whose text content is the HTTP header value.

    + + +
    Bearer 7472327205ffb74d10b11363044d8c24e3ddba12
    +     +]]>     +

    Multiple candidates MAY be provided, indicating that there are multiple URIs from which the data can be retrieved (e.g. multiple candidates could be included to list the primary URI of a file along with several known mirrors).

    + + + + +]]> +

    The generation of candidates is based on the Jingle content senders, and only the parties specified to send data SHOULD provide candidates.

    +

    Upon receiving an HTTP download candidate, parties that are to receive data (based on the Jingle content senders) SHOULD use an HTTP GET request to the candidate URI to fetch the data.

    +

    Entities MAY initially provide an empty set of candidates if a suitable download URI is not yet known; advertising candidates later is done with transport-info actions.

    + +
    TermDefinition
    CCYYfour-digit year portion of Date
    CCYYfour-digit year portion of Date According to the XML schema datatypes specification <http://www.w3.org/TR/xmlschema11-2/#dateTime>, the year portion of a Date can be more than four digits (for years after 9999) and can be preceded by a minus sign (for years before 1 BCE); given that Jabber/XMPP technologies did not exist before 1999, the use of the minus sign is not recommended.
    MMtwo-digit month portion of Date
    DDtwo-digit day portion of Date
    -ISO 8601 separator among Date portions
    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Content CreatorContent SendersWho Sends Download CandidatesWho performs HTTP GET
    initiatorinitiatorinitiatorresponder
    responderresponderinitiator
    bothbothboth
    nonenonenone
    responderinitiatorinitiatorresponder
    responderresponderinitiator
    bothbothboth
    nonenonenone
    +     + + +

    Negotiating HTTP uploads is done by using a &TRANSPORT; element with the 'urn:xmpp:jingle:transports:http:upload:0' namespace, &VNOTE;. This element MAY include a <candidate/> element which represents a URI where data can be uploaded. The <candidate/> element MUST include a 'uri' attribute, and MAY contain <header/> elements whose 'name' attribute is an HTTP header and whose text content is the HTTP header value.

    + + +
    Bearer 7472327205ffb74d10b11363044d8c24e3ddba12
    +     +]]>     +

    The generation of candidates is based on the Jingle content senders, and only the parties specified to receive data SHOULD provide candidates.

    +

    Upon receiving an HTTP upload candidate, parties that are to send data (based on the Jingle content senders) SHOULD use an HTTP PUT request to the candidate URI, where the request body is the data to be transferred.

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Content CreatorContent SendersWho Sends Upload CandidatesWho Performs HTTP PUT
    initiatorinitiatorresponderinitiator
    responderinitiatorresponder
    bothbothboth
    nonenonenone
    responderinitiatorresponderinitiator
    responderinitiatorresponder
    bothbothboth
    nonenonenone
    + +

    See Upload Complete for signaling that the upload process has been completed.

    +     + + + +

    A common case for using http-upload is to delegate the storage of the uploaded data to an external hosting service, which means that the receiver might not have the direct ability to know when the uploaded data is ready.

    +

    As such, when an upload transfer is used, the party uploading content SHOULD signal when the upload has completed by sending a Jingle session-info event that includes a <uploaded/> element qualified by the 'urn:xmpp:transports:http:info:0' namespace and SHOULD include 'creator' and 'name' attributes identifying the content for which the upload was completed.

    + + + + +]]> +     +     + + + +

    Here, Romeo is offering to send a file to Juliet, so he includes a download URI candidate with his session-initiate.

    + + + + + + 1969-07-21T02:56:15Z + This is a test. If this were a real file... + text/plain + test.txt + + 6144 + 552da749930852c69ae5d2141d3766b1 + + + + + + + +]]> +

    Juliet accepts the offer, and then performs an HTTP GET to retrieve the file.

    + + + + + + + +]]> +     + + +

    Here Romeo is requesting Juliet to send a file by sharing a download URI.

    + + + + + + 552da749930852c69ae5d2141d3766b1 + + + + + +]]> +

    Juliet accepts the request, and includes a download URI in her session-accept.

    + + + + + + 1969-07-21T02:56:15Z + text/plain + test.txt + + 6144 + 552da749930852c69ae5d2141d3766b1 + + + + + + + +]]> +

    Romeo then retrieves the file using an HTTP GET request.

    +     + + +

    In this case, Romeo is offering a file to Juliet but wishes to upload it to her.

    + + + + + + 1969-07-21T02:56:15Z + This is a test. If this were a real file... + text/plain + test.txt + + 6144 + 552da749930852c69ae5d2141d3766b1 + + + + + +]]> +

    Juliet accepts, and provides a candidate with an upload URI that includes an authorization header.

    + + + + + + +
    Bearer 7472327205ffb74d10b11363044d8c24e3ddba12
    +     +     +     +     +]]>     +

    Romeo now uses an HTTP PUT to upload his file. Once the upload is complete, he informs Juliet so she knows the file is ready for to read.

    + + + + +]]> +     + + +

    Here Romeo asks Juliet to upload a file.

    + + + + + + 552da749930852c69ae5d2141d3766b1 + + + + + + + +]]> +

    Juliet accepts the session, and begins uploading the file data with an HTTP PUT request.

    + + + + + + 1969-07-21T02:56:15Z + text/plain + test.txt + + 6144 + 552da749930852c69ae5d2141d3766b1 + + + + + +]]> + +

    Once the upload is complete, she informs Romeo that she has completed the upload so that he knows he can access the data he requested.

    + + + + +]]> +     +     + + +

    To advertise its support for the Jingle HTTP Transport Method, when replying to &xep0030; information requests an entity MUST return URNs for any version of this protocol that the entity supports -- e.g., "urn:xmpp:jingle:transports:http:0" for this version &VNOTE;.

    + + +]]> + + + + + + +]]> +

    In order for an application to determine whether an entity supports this protocol, where possible it SHOULD use the dynamic, presence-based profile of service discovery defined in &xep0115;. However, if an application has not received entity capabilities information from an entity, it SHOULD use explicit service discovery instead.

    +     + + +

    HTTP URI candidates SHOULD use the "https://" URI scheme instead of "http://", and entities MAY refuse to process URIs that are not "https://".

    +

    Certain HTTP headers can cause unintended behaviour, such as using the 'Upgrade' header to trigger a conversion to WebSocket (&rfc6455;).

    +     + + +

    No interaction with &IANA; is required as a result of this document.

    +     + + + +

    This specification defines the following XML namespaces:

    +
      +
    • urn:xmpp:jingle:transports:http:0
      • +
    • urn:xmpp:jingle:transports:http:upload:0
      • +
    • urn:xmpp:jingle:transports:http:info:0
      • +
    +

    The ®ISTRAR; includes the foregoing namespace in its registry of protocol namespaces at &NAMESPACES;, as described in Section 4 of &xep0053;.

    +     + + &NSVER; + + +

    The ®ISTRAR; includes "http-download" in its registry of Jingle transport methods at &JINGLETRANSPORTS;. The registry submission is as follows:

    + + http-download + + A method for negotiating data exchange via HTTP URI retrieval. + + streaming + XEP-XXXX +]]> +

    The ®ISTRAR; includes "http-upload" in its registry of Jingle transport methods at &JINGLETRANSPORTS;. The registry submission is as follows:

    + + http-upload + + A method for negotiating data exchange via uploading to HTTP URIs. + + streaming + XEP-XXXX +]]> +     +     + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +]]> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +]]> + + + + + + + + + + + + + + + + + + + + + + + + + +]]> + + + + From 441fe8647425706a781dde17d9738d957377b89c Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Tue, 18 Aug 2015 18:36:01 -0600 Subject: [PATCH 030/106] ProtoXEP Message Deletion --- inbox/message-deletion.xml | 114 +++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 inbox/message-deletion.xml diff --git a/inbox/message-deletion.xml b/inbox/message-deletion.xml new file mode 100644 index 00000000..12eda1a9 --- /dev/null +++ b/inbox/message-deletion.xml @@ -0,0 +1,114 @@ + + +%ents; +]> + + +
    + Message Deletion + This specification defines a method for indicating that a message should be removed. + &LEGALNOTICE; + XXXX + Experimental + Standards Track + Standards + Council + + XMPP Core + + + + message-delete + &lance; + + 0.0.1 + 2015-07-07 + ljts +

    First draft.

     +     +
    + +

    Occasionally, a &xep0045; room moderator or admin might wish to remove certain chat messages from the room history as part of an effort to address and remedy issues such as message spam, indecent language for the venue, exposing private third-party personal information, etc. However, as with any content moderation tool, the removal request can only be considered as a hint and by itself can not prevent or undo any potential damage caused by the offending message, as clients which don't support message deletion are not obligated to enforce the deletion request and people could have seen or copied the message content already.

    +     + +

    If a client or service implements message deletion, it MUST specify the 'urn:xmpp:message-delete:0' feature in its service discovery information features as specified in &xep0030; and the Entity Capabilities profile specified in &xep0115;.

    + + +]]> + + +... + +... + +]]> +     + +

    When a user indicates to the client that a sent message (or a received message for MUC room moderators) is meant to be deleted, the client will send a new message containing a <remove /> element with the "urn:xmpp:message-delete:0" namespace, with an id attribute set to the id of the message to be removed.

    + + This message contained information not meant for this room. +]]> + + +]]> +     + +

    A receiving client can choose to remove the indicated message from whatever display is used for messages, from any stored history, or choose to display the fact that a message has been removed in another way.

    +

    A MUC or other service that supports message removal SHOULD prevent further distribution of the message by the service (e.g., by not replaying the message to new occupants joining the room, or omitting the message from history archive requests where possible).

    +

    A client MAY inform the user that a no-longer displayed message did previously exist and has been removed.

    +

    Clients and services MUST set the 'id' attribute on messages if they allow for message deletion.

    +

    The Sender MUST NOT send a removal request for a message with non-messaging payloads. For example, a sender MUST NOT send a removal for a roster item exchange request or a file transfer part.

    +

    A removal MUST only be processed when both the original message and removal request are received from the same full-JID (or from a JID of an appropriate admin or moderator in the case of a MUC room.)

    +     + +

    There can never be a guarantee that a removed message was never seen or otherwise distributed, and it is encouraged for clients and services when possible to inform users that no such guarantee exists.

    +

    When used in a &xep0045; context, removals send by non-moderators must not be allowed (by the receiver) for messages received before the sender joined the room - particularly a full JID leaving the room then rejoining and removing a message SHOULD be disallowed, as the entity behind the full JID in the MUC may have changed.

    +     + +

    None.

    +     + + +

    The ®ISTRAR; includes 'urn:xmpp:message-delete:0' in its registry of protocol namespaces (see &NAMESPACES;).

    +
      +
    • urn:xmpp:message-delete:0
      • +
    +     + + &NSVER; + +     + + + + + + + + + + + + + + + + + ]]> + +     From 9e2d55d700ec374acd171a0bb4f59166257437aa Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Wed, 19 Aug 2015 11:18:14 -0500 Subject: [PATCH 031/106] Fix a typo in my OTR XEP --- inbox/otr-info.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/otr-info.xml b/inbox/otr-info.xml index ddc9750f..3fcce5f1 100644 --- a/inbox/otr-info.xml +++ b/inbox/otr-info.xml @@ -272,7 +272,7 @@ xmpp:feste@allfools.lit?otr-fingerprint=AEA4D503298797D4A4FC823BC1D24524B4C54338 . This puts generating SHA-1 collisions well within the reach of governments and well funded criminal organizations. In this authors opinion, there are - no theoretical vulnerabilities, and SHA-1 should be treated as with extreme + no theoretical vulnerabilities, and SHA-1 should be treated with extreme caution.

    From 117ca176c93ecb655a9c98a59f58c8a1006ec917 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Thu, 20 Aug 2015 09:48:26 +0200 Subject: [PATCH 032/106] Add '' to XEP-174 examples --- xep-0174.xml | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/xep-0174.xml b/xep-0174.xml index 4902726a..b11f1461 100644 --- a/xep-0174.xml +++ b/xep-0174.xml @@ -197,6 +197,7 @@ ver=QgayPKawpkPSDYmwT/WM94uAlu0=

    Other people at the hotspot can also advertise similar DNS records for use on the local link. Essentially, the mDNS daemons running on all of the machines at the hotspot collectively manage the ".local." domain, which has meaning only at the hotspot (not across the broader Internet). Queries and responses for services on the local link occur via multicast DNS over UDP port 5353 instead of via normal DNS unicast over UDP port 53. When a new machine joins the local link, it can send out queries for any number of service types, to which the other machines will reply. For the purpose of serverless messaging we are interested only in the "presence" service, but many other services could exist on the local link (see dns-sd.org for a complete list).

    Now let us imagine that a fine young gentleman named Romeo joins the hotspot and that his chat client (actually his mDNS daemon) sends out multicast DNS queries for services of type "presence". To do this, his client essentially reverses the order of DNS record publication (explained above) by asking for pointers to presence services (i.e., PTR records that match "_presence._tcp.local."), querying each service for its service instance and port (i.e., SRV record), mapping each service instance to an IP address (i.e., A record), and finding out additional information about the entity using the service (i.e., TXT record parameters). As explained in the DNS-SD specification, these queries might all be returned in the same answer. As a result, Romeo's client will discover any number of local presence services, among them a service named "juliet@pronto" (with some intriguing TXT record parameters) at IP address 10.2.1.187 and port 5562. Being a romantic fellow, he then initiates a chat with you by opening an XML stream to the advertised IP address and port.

    Your client then responds with a response stream header.

    In order to exchange serverless messages, the initiator and recipient MUST first establish XML streams between themselves, as is familiar from RFC 6120.

    First, the initiator opens a TCP connection at the IP address and port discovered via the DNS lookup for an entity and opens an XML stream to the recipient, which SHOULD include 'to' and 'from' address:

    + Note: If the initiator supports stream features and the other stream-related aspects of XMPP 1.0 as specified in RFC 6120, then it SHOULD include the version='1.0' flag as shown in the previous example.

    The recipient then responds with a stream header as well:

    +

    As with Entity Capabilities over native XMPP networks, a client might not know the &xep0030; features associated with the 'ver' value advertised by another entity. However, in the case of serverless messaging there is no way for the client to discover the entity's supported features without initiating an XML stream to that entity and then sending a Service Discovery information ("disco#info") request over the negotiated stream.

    Unfortunately, full stream negotiation (including TLS and SASL if appropriate) can require a large number of packets. Therefore, as an optimization, it is RECOMMENDED for the receiving entity in a serverless XML stream negotiation to include its disco#info data (including node) as a stream feature, as shown in the following examples.

    + ]]> + Date: Fri, 21 Aug 2015 10:18:59 +0200 Subject: [PATCH 033/106] Added reference to XEP1 in README --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index d2fdef89..4a876ca3 100644 --- a/README.md +++ b/README.md @@ -23,5 +23,7 @@ page: http://xmpp.org/xmpp-protocols/xmpp-extensions/submitting-a-xep/ +[XEP-0001: XMPP Extension Protocols](http://xmpp.org/extensions/xep-0001.html) defines the standards process followed by the XMPP Standards Foundation. + Thanks! From 2a382b1b9b1e102b8f7d717230ade852545e4a4c Mon Sep 17 00:00:00 2001 From: Kevin Smith Date: Mon, 17 Aug 2015 14:31:32 +0100 Subject: [PATCH 034/106] Change Carbons delivery rules to 'all IM traffic' After discussion on the standards list at the time of Last Call, it became clear that servers were not generally implementing Carbons as specified. This patch updates the text to be more forgiving of such discrepancies, and to try to reflect general consensus between implementations of what should be eligible for carbons delivery. --- xep-0280.xml | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/xep-0280.xml b/xep-0280.xml index 14abeb56..39136cbb 100644 --- a/xep-0280.xml +++ b/xep-0280.xml @@ -124,9 +124,9 @@
  • Clients that do not implement the new protocol MUST NOT receive protocol they do not expect
  • All clients that turn on the new protocol MUST be able to see - all inbound chat-type messages.
    • + all inbound instant messaging messages.
  • All clients that turn on the new protocol MUST be able to see - all outbound chat-type messages from all of the resources of the + all outbound instant messaging messages from all of the resources of the user, regardless of whether the clients for the other resources have implemented the new protocol.
    • @@ -227,10 +227,24 @@

    See the section Handling Multiple Enable/Disable Requests for considerations when a client attempts to disable Carbons multiple times.

     - + + + +

    The focus of this specification is instant messaging applications and so those (and only those) &MESSAGE; stanzas used for instant messaging SHOULD be delivered as Carbons. Defining precisely which messages are used for instant messaging and which are not is difficult, as future specifications may add additional payloads used for, or not used for, instant messaging; as such, the rules for which messages are eligible for carbons delivery is left as an implementation detail for servers. The following is a suggested set of rules a server MAY use, or it MAY use its own; in either case it SHOULD follow the general intent of these rules:

    +

    Possible delivery rules: +

      +
    • A &MESSAGE; is eligible for carbons delivery if it is of type "chat".
      • +
    • A &MESSAGE; is eligible for carbons delivery if it is of type "normal" and it contains a <body> element.
      • +
    • A &MESSAGE; is not eligible for carbons delivery if it is determined to have been sent by a MUC room or service, even if it would be otherwise eligible.
      • +
    • A &MESSAGE; is not eligible for carbons delivery if it does not meet any of these criteria.
      • +
    +

    +

    Future specifications may have more precise requirements on which messages need to be eligible for carbons delivery; such future specifications will provide their own discovery and negotiation mechanisms, such that a client negotiating Carbons using the protocol defined in this specification will cause the server to consider messages eligible for Carbons delivery based on the requirements described herein.

    +

    Note: previous versions of this specification limited eligible messages to those of type "chat" - however, this was generally found to be inadequate due to the proliferation of type "normal" messages used in instant messaging.

    +     -

    When the server receives a &MESSAGE; of type "chat" addressed to a bare JID (localpart@domainpart), it delivers a copy to each Carbons-enabled resource for the bare JID – in addition to delivering according to RFC 6121 § 8.5.2. This process is sometimes called "forking".

    +

    When the server receives a &MESSAGE; eligible for carbons delivery addressed to a bare JID (localpart@domainpart), it first delivers it according to RFC 6121 § 8.5.2, and then delivers a copy to each Carbons-enabled resource for the bare JID that did not receive it due to the RFC 6121 delivery rules. This process is sometimes called "forking".

    The receiving server MUST deliver a copy to every Carbons-enabled resource, even if that resource normally would not receive &MESSAGE; stanzas addressed to the bare JID (e.g., resources which have broadcast &PRESENCE; with a negative priority). A Carbons-enabled resource MUST NOT receive more than one copy of the &MESSAGE;.

     -

    When the server receives a &MESSAGE; of type "chat" addressed to a full JID (localpart@domainpart/resourcepart), it delivers the &MESSAGE; according to RFC 6121 § 8.5.3, and delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID recipient.

    +

    When the server receives a &MESSAGE; eligible for carbons delivery addressed to a full JID (localpart@domainpart/resourcepart), it delivers the &MESSAGE; according to RFC 6121 § 8.5.3, and then delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID recipient that did not receive it under the RFC 6121 delivery rules. Note that the RFC 6121 delivery rules can cause a &MESSAGE; addressed to a full JID to be delivered using bare JID delivery rules; in this case the server should also apply the bare JID rules for Carbons.

    Each forwarded copy is wrapped using &xep0297;. The wrapping message SHOULD maintain the same 'type' attribute value; the 'from' attribute MUST be the Carbons-enabled user's bare JID (e.g., "localpart@domainpart"); and the 'to' attribute MUST be the full JID of the resource receiving the copy. The content of the wrapping message MUST contain a <received/> element qualified by the namespace "urn:xmpp:carbons:2", which itself contains a <forwarded/> element qualified by the namespace "urn:xmpp:forward:0" that contains the original &MESSAGE;.

    ]]> -

    The receiving server MUST NOT send a forwarded copy to the full JID the original &MESSAGE; stanza was addressed to, as that recipient receives the original &MESSAGE; stanza.

    +

    The receiving server MUST NOT send a forwarded copy to the client(s) receiving full JID the original &MESSAGE; stanza was addressed to, as that recipient receives the original &MESSAGE; stanza.

     -

    When a client sends a &MESSAGE; of type "chat", its sending server delivers the &MESSAGE; according to RFC 6120 and RFC 6121, and delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID sender.

    +

    When a client sends a &MESSAGE; eligible for carbons delivery, its sending server delivers the &MESSAGE; according to RFC 6120 and RFC 6121, and delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID sender. Note that this happens irrespective of whether the sending client has carbons enabled.

    Each forwarded copy is wrapped using &xep0297;. The wrapping message SHOULD maintain the same 'type' attribute value; the 'from' attribute MUST be the Carbons-enabled user's bare JID (e.g., "localpart@domainpart"); and the 'to' attribute SHOULD be the full JID of the resource receiving the copy. The content of the wrapping message MUST contain a <sent/> element qualified by the namespace "urn:xmpp:carbons:2", which itself contains a <forwarded/> qualified by the namespace "urn:xmpp:forward:0" that contains the original &MESSAGE; stanza.

    -

    If a client is permitted to enable Carbons during its login session, the server MUST allow the client enable and disable the protocol multiple times within a session. The server SHOULD NOT treat multiple enable requests (without an intermediate disable request) as an error; it SHOULD simply return an IQ-result (if the protocol is already enabled) or an IQ-error (if the client is not permitted to enable Carbons) for any subsequent requests after the first. Similarly, the server SHOULD NOT treat multiple disable requests (without an intermediate enable request) as an error; it SHOULD return an IQ-result (if the protocols is already disabled) or an IQ-error (if the client's request failed previously) for any subsequent requests after the first.

    +

    If a client is permitted to enable Carbons during its login session, the server MUST allow the client to enable and disable the protocol multiple times within a session. The server SHOULD NOT treat multiple enable requests (without an intermediate disable request) as an error; it SHOULD simply return an IQ-result (if the protocol is already enabled) or an IQ-error (if the client is not permitted to enable Carbons) for any subsequent requests after the first. Similarly, the server SHOULD NOT treat multiple disable requests (without an intermediate enable request) as an error; it SHOULD return an IQ-result (if the protocols is already disabled) or an IQ-error (if the client's request failed previously) for any subsequent requests after the first.

    Note that &xep0085; recommends sending chat state From 846803d9b9e95b8ff54a4befc2bf7830f95e8ac3 Mon Sep 17 00:00:00 2001 From: Kevin Smith Date: Mon, 17 Aug 2015 14:43:48 +0100 Subject: [PATCH 035/106] Further tweak Carbons text after community discussion --- xep-0280.xml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/xep-0280.xml b/xep-0280.xml index 39136cbb..d6ac9196 100644 --- a/xep-0280.xml +++ b/xep-0280.xml @@ -235,10 +235,12 @@

    • A &MESSAGE; is eligible for carbons delivery if it is of type "chat".
    • A &MESSAGE; is eligible for carbons delivery if it is of type "normal" and it contains a <body> element.
      • +
    • A &MESSAGE; is eligible for carbons delivery if it is of type "error" and sent in response to a &MESSAGE; that was itself eligible for carbons delivery (Note that as this would require message tracking and correlation on the server, it is unlikely to be generally appropriate for most implementations).
    • A &MESSAGE; is not eligible for carbons delivery if it is determined to have been sent by a MUC room or service, even if it would be otherwise eligible.
    • A &MESSAGE; is not eligible for carbons delivery if it does not meet any of these criteria.

    +

    As this is a implementation detail of servers, clients MUST NOT rely on the server implementing a particular set of rules for which messages are eligible for Carbons delivery.

    Future specifications may have more precise requirements on which messages need to be eligible for carbons delivery; such future specifications will provide their own discovery and negotiation mechanisms, such that a client negotiating Carbons using the protocol defined in this specification will cause the server to consider messages eligible for Carbons delivery based on the requirements described herein.

    Note: previous versions of this specification limited eligible messages to those of type "chat" - however, this was generally found to be inadequate due to the proliferation of type "normal" messages used in instant messaging.

     From c568aeecc3ca0c91b79f10819c98bde4a1b49a0c Mon Sep 17 00:00:00 2001 From: Georg Lukas Date: Mon, 17 Aug 2015 18:45:05 +0200 Subject: [PATCH 036/106] Minor Carbons clarifications --- xep-0280.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xep-0280.xml b/xep-0280.xml index d6ac9196..8f7a2f39 100644 --- a/xep-0280.xml +++ b/xep-0280.xml @@ -236,7 +236,7 @@
  • A &MESSAGE; is eligible for carbons delivery if it is of type "chat".
  • A &MESSAGE; is eligible for carbons delivery if it is of type "normal" and it contains a <body> element.
  • A &MESSAGE; is eligible for carbons delivery if it is of type "error" and sent in response to a &MESSAGE; that was itself eligible for carbons delivery (Note that as this would require message tracking and correlation on the server, it is unlikely to be generally appropriate for most implementations).
    • -
  • A &MESSAGE; is not eligible for carbons delivery if it is determined to have been sent by a MUC room or service, even if it would be otherwise eligible.
    • +
  • A &MESSAGE; is not eligible for carbons delivery if it is determined to have been sent by a MUC room or service, even if it would be otherwise eligible (this also includes private messages from MUC participants).
  • A &MESSAGE; is not eligible for carbons delivery if it does not meet any of these criteria.

    • @@ -311,7 +311,7 @@

    The receiving server MUST NOT send a forwarded copy to the client(s) receiving full JID the original &MESSAGE; stanza was addressed to, as that recipient receives the original &MESSAGE; stanza.

     -

    When a client sends a &MESSAGE; eligible for carbons delivery, its sending server delivers the &MESSAGE; according to RFC 6120 and RFC 6121, and delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID sender. Note that this happens irrespective of whether the sending client has carbons enabled.

    +

    When a client sends a &MESSAGE; eligible for carbons delivery, its sending server delivers the &MESSAGE; according to RFC 6120 and RFC 6121, and delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID sender, excluding the sending client. Note that this happens irrespective of whether the sending client has carbons enabled.

    Each forwarded copy is wrapped using &xep0297;. The wrapping message SHOULD maintain the same 'type' attribute value; the 'from' attribute MUST be the Carbons-enabled user's bare JID (e.g., "localpart@domainpart"); and the 'to' attribute SHOULD be the full JID of the resource receiving the copy. The content of the wrapping message MUST contain a <sent/> element qualified by the namespace "urn:xmpp:carbons:2", which itself contains a <forwarded/> qualified by the namespace "urn:xmpp:forward:0" that contains the original &MESSAGE; stanza.

    Date: Mon, 17 Aug 2015 18:53:57 +0200 Subject: [PATCH 037/106] Carbons: remove bare-JID special case --- xep-0280.xml | 37 +++---------------------------------- 1 file changed, 3 insertions(+), 34 deletions(-) diff --git a/xep-0280.xml b/xep-0280.xml index 8f7a2f39..d2bae197 100644 --- a/xep-0280.xml +++ b/xep-0280.xml @@ -245,39 +245,8 @@

    Note: previous versions of this specification limited eligible messages to those of type "chat" - however, this was generally found to be inadequate due to the proliferation of type "normal" messages used in instant messaging.

     - -

    When the server receives a &MESSAGE; eligible for carbons delivery addressed to a bare JID (localpart@domainpart), it first delivers it according to RFC 6121 § 8.5.2, and then delivers a copy to each Carbons-enabled resource for the bare JID that did not receive it due to the RFC 6121 delivery rules. This process is sometimes called "forking".

    - - - Wherefore art thou, Romeo? - 0e3141cd80894871a68e6fe6b1ec56fa -]]> - - - Wherefore art thou, Romeo? - 0e3141cd80894871a68e6fe6b1ec56fa - - - - Wherefore art thou, Romeo? - 0e3141cd80894871a68e6fe6b1ec56fa -]]> - -

    The receiving server MUST deliver a copy to every Carbons-enabled resource, even if that resource normally would not receive &MESSAGE; stanzas addressed to the bare JID (e.g., resources which have broadcast &PRESENCE; with a negative priority). A Carbons-enabled resource MUST NOT receive more than one copy of the &MESSAGE;.

    -     - -

    When the server receives a &MESSAGE; eligible for carbons delivery addressed to a full JID (localpart@domainpart/resourcepart), it delivers the &MESSAGE; according to RFC 6121 § 8.5.3, and then delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID recipient that did not receive it under the RFC 6121 delivery rules. Note that the RFC 6121 delivery rules can cause a &MESSAGE; addressed to a full JID to be delivered using bare JID delivery rules; in this case the server should also apply the bare JID rules for Carbons.

    + +

    When the server receives a &MESSAGE; eligible for carbons delivery addressed to a client JID (either bare or full), it delivers the &MESSAGE; according to RFC 6121 § 8.5.3, and then delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID recipient that did not receive it under the RFC 6121 delivery rules.

    Each forwarded copy is wrapped using &xep0297;. The wrapping message SHOULD maintain the same 'type' attribute value; the 'from' attribute MUST be the Carbons-enabled user's bare JID (e.g., "localpart@domainpart"); and the 'to' attribute MUST be the full JID of the resource receiving the copy. The content of the wrapping message MUST contain a <received/> element qualified by the namespace "urn:xmpp:carbons:2", which itself contains a <forwarded/> element qualified by the namespace "urn:xmpp:forward:0" that contains the original &MESSAGE;.

    ]]> -

    The receiving server MUST NOT send a forwarded copy to the client(s) receiving full JID the original &MESSAGE; stanza was addressed to, as that recipient receives the original &MESSAGE; stanza.

    +

    The receiving server MUST NOT send a forwarded copy to the client(s) the original &MESSAGE; stanza was addressed to, as these recipients receive the original &MESSAGE; stanza.

    When a client sends a &MESSAGE; eligible for carbons delivery, its sending server delivers the &MESSAGE; according to RFC 6120 and RFC 6121, and delivers a forwarded copy to each Carbons-enabled resource for the matching bare JID sender, excluding the sending client. Note that this happens irrespective of whether the sending client has carbons enabled.

    From 756e38b9c2c9b7eff8edde6bfa3da789e2dd7d99 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Mon, 24 Aug 2015 08:36:34 -0600 Subject: [PATCH 038/106] Update revision and fix typos --- xep-0280.xml | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/xep-0280.xml b/xep-0280.xml index d2bae197..f1d8b79b 100644 --- a/xep-0280.xml +++ b/xep-0280.xml @@ -40,6 +40,15 @@ linuxwolf@outer-planes.net linuxwolf@outer-planes.net + + 0.10 + 2015-08-24 + mm (editor) + +

    Removed distinction between full-JID and bare-JID when receiving messages (Georg Lukas).

    +

    Define rules around "elible messages", and provide reasonable default guidelines (Kevin Smith).

    +     +     0.9 2013-10-17 @@ -341,7 +350,7 @@

    Note: if the private &MESSAGE; stanza is addressed to a bare JID, the receiving server still delivers it according to RFC 6121. This might result in a copy being delivered to each resource for the recipient, which effectively negates the behavior of the <private/> element for recipients.

     - +

    If a client is permitted to enable Carbons during its login session, the server MUST allow the client to enable and disable the protocol multiple times within a session. The server SHOULD NOT treat multiple enable requests (without an intermediate disable request) as an error; it SHOULD simply return an IQ-result (if the protocol is already enabled) or an IQ-error (if the client is not permitted to enable Carbons) for any subsequent requests after the first. Similarly, the server SHOULD NOT treat multiple disable requests (without an intermediate enable request) as an error; it SHOULD return an IQ-result (if the protocols is already disabled) or an IQ-error (if the client's request failed previously) for any subsequent requests after the first.

     From a4ce22fefb45fa6b472c94206faed7ebd1cfb769 Mon Sep 17 00:00:00 2001 From: Steve Kille Date: Fri, 21 Aug 2015 15:49:33 +0100 Subject: [PATCH 039/106] Initial X2X over S5066 --- inbox/s2s-over-s5066.xml | 135 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 inbox/s2s-over-s5066.xml diff --git a/inbox/s2s-over-s5066.xml b/inbox/s2s-over-s5066.xml new file mode 100644 index 00000000..bb11b5ac --- /dev/null +++ b/inbox/s2s-over-s5066.xml @@ -0,0 +1,135 @@ + + +%ents; +]> + + +
    + Server to Server communication over STANAG 50666 ARQ + + This specification defines operation over XMPP over the NATO STANAG 5066 data link service for point to point links (ARQ). This enables optimized XMPP performance over HF Radio (which STANAG 5066 was designed for) and over other data links using STANAG 5066. + + + + This XMPP Extension Protocol is copyright (c) 1999 - 2015 by the XMPP Standards Foundation (XSF). + Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation. + ## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. In no event shall the XMPP Standards Foundation or the authors of this Specification be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification. ## + In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising out of the use or inability to use the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages. + This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which may be found at <http://xmpp.org/extensions/ipr-policy.shtml> or obtained by writing to XSF, P.O. Box 1641, Denver, CO 80201 USA). + + xxxx + ProtoXEP + Informational + Standards + Council + + + XEP 0361 + STANAG 5066 + + + + S5066 + + Steve + Kille + steve.kille@isode.com + steve.kille@isode.com + + + 0.0.1 + 2015-08-19 + sek +

    First draft.

     +     +
    + +

    + This specification arose from requirements to operate over HF Radio, which has exceedingly high latency (sometimes minutes) low data rates (down to 75 bits/second) and poor reliablity. STANAG 5066 is a widely used link level protocol. Direct use of STANAG 5066 enables elimination of all extraneous end to end handshaking, which is important to optimize performance. It also enables use of STANAG 5066 flow control, which is important for reslience. + +

    +

    + The solution is based on XEP-0361 "Zero Handshake Server to Server Protocol" and requires peer configuration to be established according to XEP-0361. The data exchanged between the XMPP servers follows exactly what is specified in XEP-0316. The data is transferred using STANAG 5066 rather than using TCP. + +

    +     + +

    + This specification can be considered as a profile for server to server XMPP communication, to enable XMPP deployment over HF Radio using STANAG 5066. This profile MUST only be used where its use has been pre-agreed and configured for both participating servers. +

    +     + + +

    + An example scenario where this protocol is important is where two ships connected by HF Surface Wave communication only need to exchange XMPP messages. A reliable link (Soft Link) can be established using STANAG 5066 and XMPP communicated efficiently and reliably. + +

    +     + + + +

    + Because of potentially very low bandwidth sending server MAY perform traffic optimisation, such as selective removal of stanzas that are not adding sufficient value, like CSNs, or strip selected elements such as xhtml-im. +

    +

    + Applications sending data over STANAG 5066 need to be aware of increased delays and any application level timers (e.g., IQ response timers) need to be set accordingly. +

    +     + +

    + XEP-0361 transfer of data between a pair of XMPP servers is a byte stream flowing in each direction over TCP. There is no other protocol or hand shaking. + When carried instead over STANAG 5066, these byte streams are transmitted as a sequence of blocks transferred in order + Each block is an XML stanza, holding message, presence or iq. Essentially the stream is broken into blocks (stanzas) at natural boundaries XMPP boundaries, and then reassembled on reception into the original stream. + + +

    +

    + XEP-0198 Stream Management MUST not be used over STANAG 5066, as reliability of stanza transfer is handled by use of STANAG 5066. Application-layer keepalives and timeout detection such as white-space pings and XEP-0199 MUST NOT be used. +

    + +     + + +

    + Each stanza is transferred using the RCOP (Reliable Connection Oriented Protocol) defined in Section F.8 of Annex F of STANAG 5066. This reliably transfers the block of data to the destination. If a soft link needs to be established this will be done by the STANAG 5066 service. The XEP-0361 peer agreement is supported by a flow of stanzas in each direction being transferred by RCOP. The peer agreement will use this flow of stanzas to provide a service equivalent to the TCP connection or connections of XEP-0361. +

    +

    + STANAG 5066 SIS Delivery Confirmation MAY be set to NODE DELIVERY, as this gives optimum network performance. CLIENT DELIVERY MAY be used, which increases reliability as stanza delivery to the peer XMPP server is guaranteed and the sending server will receive acknowledgements equivalent to XEP-0198 support. In the event of delivery failure, the whole RCOP PDU (Stanza) MUST be retransmitted. +

    + +     + +

    + The peer addressing of the STANAG 5066 end points will be configured as part of the XEP-03161 peer agreement. +

    +

    + The STANAG 5066 SAP MAY be set to any mutually agreed value. It is RECOMMENDED that 2 is used. This is the standard SAP for RCOP. +

    +

    + The RCOP connection ID number will be set to a mutually agreed value. It is RECOMMENDED that 0 is used as the preferred value. +

    +     +     + + + +

    + Security Considerations of XEP-0361 apply. STANAG 5066 will frequently be employed in conjunction with link level crypto devices, which SHOULD be done when appropriate to provide data confidentiality. +

    + +     + +

    This specification uses STANAG 5066 Edition 3 "PROFILE FOR HF RADIO DATA COMMUNICATIONS" (December 2010).

    +

    STANAG 5066 is a NATO UNCLASSIFED (Releasable to the Public) document that may circulated freely. The author has requested permission form NATO to enable publishing of this document on the Web.

    +     + + +

    + Curtis King designed and validated the approach documented in this XEP. +

    +

    + Kevin Smith provided useful comments on this specification. +

    +     +     From 43257f6b9a3d179ed5119aea72aaa3f684c8c742 Mon Sep 17 00:00:00 2001 From: Steve Kille Date: Fri, 21 Aug 2015 16:14:22 +0100 Subject: [PATCH 040/106] Fix to Standards Track --- inbox/s2s-over-s5066.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/s2s-over-s5066.xml b/inbox/s2s-over-s5066.xml index bb11b5ac..8ffae14d 100644 --- a/inbox/s2s-over-s5066.xml +++ b/inbox/s2s-over-s5066.xml @@ -20,7 +20,7 @@ xxxx ProtoXEP - Informational + Standards Track Standards Council From 3ec99b2c0491616e08e0fcef76597f1ce22d0457 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Mon, 24 Aug 2015 16:34:49 -0600 Subject: [PATCH 041/106] fix typo in title --- inbox/s2s-over-s5066.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/s2s-over-s5066.xml b/inbox/s2s-over-s5066.xml index 8ffae14d..2dc99ade 100644 --- a/inbox/s2s-over-s5066.xml +++ b/inbox/s2s-over-s5066.xml @@ -6,7 +6,7 @@
    - Server to Server communication over STANAG 50666 ARQ + Server to Server communication over STANAG 5066 ARQ This specification defines operation over XMPP over the NATO STANAG 5066 data link service for point to point links (ARQ). This enables optimized XMPP performance over HF Radio (which STANAG 5066 was designed for) and over other data links using STANAG 5066. From 9bc45ee51ed449eced0702764e26591ca684538e Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Wed, 26 Aug 2015 09:45:54 -0600 Subject: [PATCH 042/106] Last Call of XEP-0320 v0.2 --- xep-0320.xml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/xep-0320.xml b/xep-0320.xml index 828652b4..d2dd17c5 100644 --- a/xep-0320.xml +++ b/xep-0320.xml @@ -10,7 +10,8 @@ This specification defines how to use DTLS-SRTP (RFC 5763) in the Jingle application type for the Real-time Transport Protocol (RTP) as a way to negotiate media path key agreement for secure RTP in one-to-one media sessions. &LEGALNOTICE; 0320 - Experimental + Proposed + 2015-09-07 Standards Track Standards Council From f1d3d763f36532d7c3c2825a30a1d699788e48bc Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 27 Jul 2015 16:20:57 -0500 Subject: [PATCH 043/106] Move XEP-0352 (CSI) to "Proposed" --- xep-0352.xml | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/xep-0352.xml b/xep-0352.xml index aee6331b..cb3aa52e 100644 --- a/xep-0352.xml +++ b/xep-0352.xml @@ -10,7 +10,8 @@ This document defines a way for the client to indicate its active/inactive state. &LEGALNOTICE; 0352 - Experimental + Proposed + 2015-09-07 Standards Track Standards Council @@ -55,10 +56,10 @@

    Juliet has an XMPP client on her phone, which is available to receive messages. However most of the time Juliet has her phone screen turned off and is not interested in the status of her contacts unless they are communicating with her.

    - +

    Juliet's client informs the server when Juliet is not interacting with it. The server uses this information to suppress or reduce stanzas that are unimportant, such as status updates.

    - +

    When Juliet returns to her IM client, the client again informs the server, this time to report that it is active again. The server then disables its traffic optimisations and restores the stream to its normal state.

    @@ -96,12 +97,12 @@ ]]> - +

    As might be anticipated, when the client is active again it sends an <active/> element:

    ]]> - +

    There is no reply from the server to either of these elements (though they may indirectly cause the server to send stanzas, e.g., to update presence information when the client becomes active after a period of inactivity).

    @@ -109,10 +110,10 @@

    As this protocol is for indication only, clients MUST NOT make assumptions about how the server will use the active/inactive state information.

    - +

    The server MUST assume all clients to be in the 'active' state until the client indicates otherwise. Also the CSI active/inactive state is unrelated to the user's presence, the server MUST treat the two independently.

    - +

    This protocol is intended primarily for clients with human interaction. Due to the open-ended nature of the possible optimisations implemented by the server, it may not be suitable for non-IM purposes where the fully standard behaviour of XMPP is required.

    From 4d3a3baacc121483e11f075e17a80c3c53da3053 Mon Sep 17 00:00:00 2001 From: Lance Stout Date: Wed, 26 Aug 2015 13:28:21 -0700 Subject: [PATCH 044/106] Update XEP-0264 to allow non-cid URIs, and expand scope --- xep-0264.xml | 198 ++++++++++++++++++++++++++++----------------------- 1 file changed, 107 insertions(+), 91 deletions(-) diff --git a/xep-0264.xml b/xep-0264.xml index 7ca8722c..68684a6c 100644 --- a/xep-0264.xml +++ b/xep-0264.xml @@ -6,11 +6,11 @@
    - File Transfer Thumbnails - This specification defines a way for a client supply a preview image for a file transfer. + Jingle Content Thumbnails + This specification defines a way for a client to supply a preview image for Jingle content. &LEGALNOTICE; 0264 - Deferred + Experimental Standards Track Standards Council @@ -28,6 +28,17 @@ ml@update.uu.se mlundblad@jabber.org + &lance; + + 0.4 + 2015-08-26 + ljts + +

    Changed format to use generic 'uri' attribute to allow for 'https:' and 'http:' URIs in addition to BoB 'cid:' URIs.

    +

    Indicated that multiple thumbnails may be present.

    +

    Expanded scope of use cases to be Jingle content in general, not just file transfer.

    +     +     0.3 2009-04-27 @@ -69,43 +80,15 @@ Added missing namespace on thumnail elements.
    -

    The current methods for file transfers include basic meta data about the -file being offered (name, size, and date). There currently is no way to provide an image thumbnail for files such as photos.

    +

    When offering a Jingle session, it can be helpful to provide a small preview of the offered content to help the session responder decide whether to accept or reject the session.

    +

    This is particularly useful for file transfer content (especially image files), but can also be used for things such as video (e.g. using a still frame from the stream as the preview thumbnail), and even audio by using a small image of album cover art.

     -

    This documents defines a way to include a thumbnail image as an additional metadata in a file transfer.

    +

    This documents defines a way to include a thumbnail image as an additional metadata in a Jingle content description.

     - -

    When a client wishes to supply a thumbnail in a transfer offer, it can do so by including an extra ]]> element as shown in the following exaples.

    - - - - - - - - - - - - - - - - ]]> - -

    The receiver MAY now request the data using the protocol defined in &xep0231;.

    - +

    When a client wishes to supply a thumbnail in a content offer, it can do so by including an extra <thumbnail/> element as shown in the following example:

    + - - - - This is a test. If this were a real file... - - - + + + image/jpeg + image.jpg + 3032449 + 552da749930852c69ae5d2141d3766b1 + This is a test. If this were a real file... + + - ... + - -]]> - - -

    The following attributes are defined for the <thumbnail/> element.

    - - - - - - - - - - - - - - - - - - - - - - - - - - -
    AttributeDescriptionInclusion
    cidA Content-ID that can be mapped to a cid: URL as specified in &rfc2111;. The 'cid' value SHOULD be of the form algo+hash@bob.xmpp.org, where the "algo" is the hashing algorithm used (e.g., "sha1" for the SHA-1 algorithm as specified in &rfc3174;) and the "hash" is the hex output of the algorithm applied to the binary data itself.REQUIRED
    mime-typeThe value of the 'mime-type' attribute MUST match the syntax specified in &rfc2045;. That is, the value MUST include a top-level media type, the "/" character, and a subtype; in addition, it MAY include one or more optional parameters (e.g., the "audio/ogg" MIME type in the example shown below includes a "codecs" parameter as specified in &rfc4281;). The "type/subtype" string SHOULD be registered in the &ianamedia;, but MAY be an unregistered or yet-to-be-registered value.RECOMMENDED
    widthThe intended display width of the thumbnail image. Used as a hint for the receiving client to prepare i.e. a dialog window.OPTIONAL
    heightThe intended display height of the thumbnail image. Used as a hint for the receiving client to prepare i.e. a dialog window.OPTIONAL
    +]]>     - +

    Thumbnails MAY be included for Jingle content other than file transfer. For example, with Jingle RTP video a thumbnail could be included to show who is calling (either by capturing a still frame from the input stream or by using an existing profile image):

    + + + + + + + + + + + + + + + 128 + + + + + +]]> + +

    Multiple <thumbnail/> elements MAY be included to provide thumbnails of different sizes, media types, or URIs.

    + +

    If the provided URI has the scheme 'cid', then the thumbnail data MAY be requested using &xep0231;.

    +     + + +

    The following attributes are defined for the <thumbnail/> element.

    + + + + + + + + + + + + + + + + + + + + + + + + + + +
    AttributeDescriptionInclusion
    uriA URI where the thumbnail data can be accessed (typically by using a URI scheme of 'cid:', 'https:', or 'http:'). If the URI scheme is 'cid:' then the identifier MUST refer to a bit of binary data as described in &xep0231;REQUIRED
    media-typeThe value of the 'media-type' attribute MUST match the syntax specified in &rfc2045;. That is, the value MUST include a top-level media type, the "/" character, and a subtype; in addition, it MAY include one or more optional parameters. The "type/subtype" string SHOULD be registered in the &ianamedia;, but MAY be an unregistered or yet-to-be-registered value.RECOMMENDED
    widthThe intended display width of the thumbnail image. Used as a hint for the receiving client to prepare the appropriate UI, such as a dialog window.OPTIONAL
    heightThe intended display height of the thumbnail image. Used as a hint for the receiving client to prepare the appropriate UI, such as a dialog window.OPTIONAL

    The inclusion of an image thumbnail may leak information about a transfer otherwise taking place on an e2e encrypted file transfer stream. A client MAY wish to not include a thumbnail.

    -

    A client MUST NOT rely on the values specified for the width and height of a thumbnail to allocate a bitmap data buffer for the thumbnail, to prevent possible DoS attacks. Also a client SHOULD apply implementation-specific limits on the thumbnails, if using these values to pepare a UI element for the thumbnail image, of f.ex. 128x128 pixels, values exceeding these would then be truncated and the thumbnail image scaled down when received.

    +

    A client MUST NOT rely on the values specified for the width and height of a thumbnail to allocate a bitmap data buffer for the thumbnail, to prevent possible DoS attacks. Also a client SHOULD apply implementation-specific limits on the thumbnails, if using these values to pepare a UI element for the thumbnail image, e.g. with dimensions of 128x128 pixels, values exceeding these would then be truncated and the thumbnail image scaled down when received.

    This document requires no interaction with &IANA;.

    @@ -183,7 +199,7 @@ wish to not include a thumbnail.

    This specification defines the following XML namespace:

      -
    • urn:xmpp:thumbs:0
      • +
    • urn:xmpp:thumbs:1

    The ®ISTRAR; includes this namespace in the registry located at &NAMESPACES;, as described in Section 4 of &xep0053;.

     @@ -210,8 +226,8 @@ wish to not include a thumbnail.

    - - + + From 7aa9a1979009e2cbbdcdfa635663c3f0daa76b9f Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 27 Aug 2015 09:49:51 -0600 Subject: [PATCH 045/106] XEP-0363 v0.1 -- Advanced to Experimental --- inbox/http-upload.xml => xep-0363.xml | 10 ++++++++-- xep.ent | 1 + 2 files changed, 9 insertions(+), 2 deletions(-) rename inbox/http-upload.xml => xep-0363.xml (97%) diff --git a/inbox/http-upload.xml b/xep-0363.xml similarity index 97% rename from inbox/http-upload.xml rename to xep-0363.xml index 8a24f2f5..aed6e4c0 100644 --- a/inbox/http-upload.xml +++ b/xep-0363.xml @@ -9,8 +9,8 @@ HTTP File Upload This specification defines a protocol to request permissions from another entity to upload a file to a specific path on an HTTP server and at the same time receive a URL from which that file can later be downloaded again. &LEGALNOTICE; - xxxx - ProtoXEP + 0363 + Experimental Standards Track Standards Council @@ -28,6 +28,12 @@ daniel@gultsch.de daniel@gultsch.de + + 0.1 + 2015-08-27 + XEP Editor (mam) +

    Initial published version approved by the XMPP Council.

     +     0.0.1 2015-07-25 diff --git a/xep.ent b/xep.ent index 9b7c2aaf..6bcb4544 100644 --- a/xep.ent +++ b/xep.ent @@ -1345,3 +1345,4 @@ IANA Service Location Protocol, Version 2 (SLPv2) Templates Nonzas (are not Stanzas) (XEP-0360) XEP-0360: Nonzas (are not Stanzas) <http://xmpp.org/extensions/xep-0360.html>." > Zero Handshake Server to Server Protocol (XEP-0361) XEP-0361: Zero Handshake Server to Server Protocol <http://xmpp.org/extensions/xep-0361.html>." > Raft over XMPP (XEP-0362) XEP-0362: Raft over XMPP <http://xmpp.org/extensions/xep-0362.html>." > +HTTP File Upload (XEP-0363) XEP-0363: HTTP File Upload <http://xmpp.org/extensions/xep-0363.html>." > From 36bc44599796124358c9580c966af44afe2bc606 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Wed, 26 Aug 2015 11:17:20 -0500 Subject: [PATCH 046/106] Fix otr-info construction section Remove unclear opinion from the security section --- inbox/otr-info.xml | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/inbox/otr-info.xml b/inbox/otr-info.xml index 3fcce5f1..544accc5 100644 --- a/inbox/otr-info.xml +++ b/inbox/otr-info.xml @@ -147,14 +147,10 @@

    - When sending a message encrypted with OTR, it is RECOMMENDED to encrypt - only the text node of the <body/> element (the message itself). - However, there are some clients in the wild which will encrypt the entire - contents of the <body/> element, including sub-nodes. Because of - this behavior, it is RECOMMENDED that clients decrypt and expand any OTR - messages inside of the body element before re-processing the element as a - whole. Clients that support OTR MUST tolerate encrypted payloads which - expand to XML, and those which expand to plain text messages. + Some clients in the wild have been known to insert XML in the + <body> node of a message. Clients that support OTR should tolerate + encrypted payloads which expand to unescaped XML, and treat it as plain + text.

    @@ -270,10 +266,8 @@ xmpp:feste@allfools.lit?otr-fingerprint=AEA4D503298797D4A4FC823BC1D24524B4C54338 https://www.schneier.com/blog/archives/2012/10/when_will_we_se.html >     . - This puts generating SHA-1 collisions well within the reach of governments - and well funded criminal organizations. In this authors opinion, there are - no theoretical vulnerabilities, and SHA-1 should be treated with extreme - caution. + This puts generating SHA-1 collisions well within the reach of governments, + malicious organizations, and even well-funded individuals.

    From ae5618890dd1e78c4b8b7746e09c9fe2039400fc Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 27 Aug 2015 10:19:12 -0600 Subject: [PATCH 047/106] XEP-0364 v0.1 - Advance to Experimental --- inbox/otr-info.xml => xep-0364.xml | 10 ++++++++-- xep.ent | 1 + 2 files changed, 9 insertions(+), 2 deletions(-) rename inbox/otr-info.xml => xep-0364.xml (97%) diff --git a/inbox/otr-info.xml b/xep-0364.xml similarity index 97% rename from inbox/otr-info.xml rename to xep-0364.xml index 544accc5..3baa7ca9 100644 --- a/inbox/otr-info.xml +++ b/xep-0364.xml @@ -13,8 +13,8 @@ end user experience.     &LEGALNOTICE; - xxxx - ProtoXEP + 0364 + Experimental Informational Standards Council @@ -30,6 +30,12 @@ sam@samwhited.com sam@samwhited.com + + 0.1 + 2015-08-27 + XEP Editor (mam) +

    Initial published version approved by the XMPP Council.

     +     0.0.1 2015-07-28 diff --git a/xep.ent b/xep.ent index 6bcb4544..09426a06 100644 --- a/xep.ent +++ b/xep.ent @@ -1346,3 +1346,4 @@ IANA Service Location Protocol, Version 2 (SLPv2) Templates Zero Handshake Server to Server Protocol (XEP-0361) XEP-0361: Zero Handshake Server to Server Protocol <http://xmpp.org/extensions/xep-0361.html>." > Raft over XMPP (XEP-0362) XEP-0362: Raft over XMPP <http://xmpp.org/extensions/xep-0362.html>." > HTTP File Upload (XEP-0363) XEP-0363: HTTP File Upload <http://xmpp.org/extensions/xep-0363.html>." > +Current Off-the-Record Messaging Usage (XEP-0364) XEP-0364: Current Off-the-Record Messaging Usage <http://xmpp.org/extensions/xep-0364.html>." > From bcefb30da6c166c438d40b462162d7325e4ac2ff Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Tue, 25 Aug 2015 18:31:22 -0500 Subject: [PATCH 048/106] Add initial draft of Entity Versioning. --- inbox/entityversioning.xml | 404 +++++++++++++++++++++++++++++++++++++ 1 file changed, 404 insertions(+) create mode 100644 inbox/entityversioning.xml diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml new file mode 100644 index 00000000..a895be2c --- /dev/null +++ b/inbox/entityversioning.xml @@ -0,0 +1,404 @@ + + +%ents; +]> + + +
    + Entity Versioning + + A method by which rosters and disco items may be versioned so that servers + will not need to send the entire list if it has not been modified, saving + bandwidth and time during session initialization with minimal state being + stored by the server and client. + + &LEGALNOTICE; + xxxx + ProtoXEP + Standards Track + Standards + Council + + RFC 6120 + RFC 6121 + + + + EV + + Sam + Whited + swhited@atlassian.com + sam@samwhited.com + + + Doug + Keen + dkeen@atlassian.com + + + 0.0.1 + 2015-08-25 + ssw +

    First draft

     +     +
    + + +

    + This problem of "downloading the world" (downloading the entire roster + every time a session is initialized) was partially addressed by &xep0237; + which was later merged into &rfc6121; §2.6. While this solved the problem + for the roster, it didn't account for other entities (eg. MUC + disco#items). Furthermore, roster versioning requires that the server + maintain a great deal of state (multiple versions of the roster) which can + be difficult to implement in a large, distributed system. This XEP defines + a method by which entities other than the roster can be versioned and + cached. +

    +     + + +
      +
    • + An extra round trip MUST NOT be required to initiate entity versioning. +
      • +
    • + Clients that do not implement the protocol (but which use servers that + do) MUST still be able to request and receive entities normally. +
      • +
    • + Servers which implement this protocol MUST NOT be required to store + multiple versions of an entity list or maintain other redundant state. +
      • +
    • + Inconsistant state between servers in a cluster should not cause cache + invalidation for the entire entity list. +
      • +
    • + Large changes SHOULD NOT be required for existing servers / clients. +
      • +
    +     + + +
    +
    Aggregate Token
    +
    + A hash which represents the state of a list of entities, and changes if + any of those entities changes. +
    +
    Versioned Entity
    +
    Any abstract object which may be versioned (eg. rooms, users).
    +
    Version Token
    +
    + A generally short, case sensitive string which represents an entity and + changes if that entity changes. +
    +
    +     + + + +
      +
    • + A client on a mobile device where bandwidth and throughput are limited + has a very large roster which cause connections to take an unacceptable + amount of time. With entity versioning, connections after the first + connection do not take as long, and use less bandwidth. +
      • +
    • + A client often wants to view the list of multi-user chat rooms + available on a servers MUC service. However, the list is very long and + takes a long time to download. After enabling entity versioning the + client can fetch the list, and then poll for changes at a later date + without re-requesting the entire list. +
      • +
    +     + +
      +
    • + A server is running in an environment where storing multiple versions + of each users roster may put too much pressure on the storage backend. + After enabling entity versioning, they only have to store a small token + per user and can calculate the diffs to send to the client afterwards. +
      • +
    • + A server maintains an out-of-band HTTP API for fetching information + about MUC rooms to display on their web page. They wish to use a + reverse proxy to cache API requests based on etags. Instead of + attempting to check if the backend page has changed and generate etags, + the room's entity version token is used as a weakly-validated ETag. +
      • +
    +     +     + + + +

    + If a server supports entity versioning, it MUST inform the connecting + client when returning stream features during the stream negotiation + process. This is done by including a <ver/> element, qualified by + the 'urn:xmpp:features:entityver:0' namespace. At the latest, this SHOULD + be done when informing a client that resource binding is required. For + example: +

    + + + + + + + ]]> +

    + The entity versioning stream feature is merely informative and therefore + is never mandatory-to-negotiate. +

    +     + + +

    + Version tokens are short case-sensitive strings which are generated by + the server. Their format is not defined in this spec, but a + recommendation may be found in the Implementation Notes. Version tokens + are akin to a weakly-validated etag for the entity in question. +

    +

    + Servers that implement this protocol must assign such a version token to + each entity that is controlled by the server. The server MUST then update + this version every time any mutable property of the entity changes (eg. + when the subscription status of a user changes). The server MAY choose to + update this token at any time (to force the clients to invalidate their + cached representation fo the object). This version token MUST then be + included with every object representation of that entity sent down in the + stream. This is done by including a sub-node called "version" qualified + by the entity versioning XML namespace defined in this document. + Similarly, clients MAY also add version nodes for each version token they + possess to the request for a list (not specifying a version token will + force the server to send information on that entity to the client). If a + server sends up a list of version tokens, the server MUST then check to + see if those tokens correspond to any entity which it knows about, and + not send down any entities with matching version tokens in the response. +

    +

    For example, a roster request might look like:

    + + + + + 25P2A7H8 + + + VIZSVF0D + + + + + + + + 9ZFZXVP9 + + + + ]]> +

    + Note that in this case there may be three roster items total (and the + client only knows about two of them), or there may be two total roster + items and the server is informing the client about a change to + "bill@shakespeare.lit". Version tokens MUST also be present in roster + pushes: +

    + + + + XWE4MUUP + + + + ]]> +

    A disco request for rooms (as defined in &xep0045;) might look like:

    + + + + 25P2A7H8 + + + 4OLGSVNY + + + + + + + + + VIZSVF0D + + + + ]]> +

    + In this example coven@chat.shakespeare.lit has been modified (eg. the + room name might have been changed), but inverness@chat.shakespeare.lit + has not changed, therefore no update is sent down. +

    +

    + Clients that implement this protocol SHOULD then cache the entity in + question when a version token is received. +

    +     + +

    + While the version token approach to caching does not require a great deal + of state to be stored on the client or the server, it does require a lot + more information to be sent by the client when requesting a list of + entities. For a very large list which is not likely to have changed, it may + be useful know in advance if the roster has changed or not (so that we can + avoid sending the large request entirely). To do this, we can request an + aggregate version token from the server. This aggregate token is calculated + by constructing a string of comma separated "bare JID:version" pairs sorted + in byte-wise order, and taking the MD5 hash of the constructed string. For + example, if the server is calculating the aggregate version token for a + roster, it might end up with the following string: +

    + +

    + Which results in the aggregate token: +

    + +

    + The actual request is an IQ sent to the server, or entity handling the + versioned list which contains a query that specifies the namespace of the + list we want to fetch. Eg. to fetch the aggregate token for the roster one + would query the server: +

    + + + + + + + + + 0514fc90e6c7981b06bbb2173bb8ef03 + + + ]]> +

    + Similarly, to fetch the aggregate token for a list of MUC rooms, one would + query the MUC component directly: +

    + + + + + + + + + 32151d1d01440d5536a7f106afd3f4d8 + + + ]]> +

    + Because aggregate tokens are OPTIONAL to implement, clients MUST fall back + to a normal request if any error is returned in response to an aggregate + token IQ. +

    +

    + Clients are also NOT REQUIRED to check aggregate tokens. However, clients + MAY wish to check aggregate tokens before making a roster or MUC request + when the cached roster or MUC list is very large. When to check aggregate + tokens is left up to the clients. +

    +     +     + + +

    + Version tokens may not provide enough collision resistance across versioned + entities (hereafter simply called "entities"), and may vary from server to + server, and therefore they MUST NOT be used as an entity identifier. +

    +

    + Version tokens SHOULD always be considered opaque to the client (eg. even + if the version token is a derivable and consistant hash on the server side, + clients should not need to know how the server is calculating the token). +

    +

    + The author RECOMMENDS using 8 character (32-bit) random alphanumeric ASCII + strings (eg. AABd7z9T) for version tokens. +

    +

    + If a server which supports this XEP provides an HTTP API which can be used + to fetch information about entities (eg. for listing information about MUC + rooms that a server provides on the providers web page), the entities + version token MAY be used as a weakly validated ETag for any API requests + for that entity. +

    +     + + +

    + Client-side caching of entity information across sessions (rather than + holding them in memory only for the life of a session) could pose a privacy + risk, especially on shared systems. Implementations SHOULD protect cached + entity data with strong encryption or other appropriate means. +

    +     + + +

    This document requires no interaction with &IANA;.

    +     + + + +

    This specification defines the following XML namespace:

    +
      +
    • urn:xmpp:features:entityver:0
      • +
    +

    + Upon advancement of this specification from a status of Experimental to a + status of Draft, the ®ISTRAR; shall add the foregoing namespace to the + registry located at &STREAMFEATURES;, as described in Section 4 of + &xep0053;. +

    +     +     + + +TODO + + + +

    + The original entity versioning proposal was engineered and written by + HipChat's Doug Keen. +

    +     + +     From d6b3f54e00e4d81e2f0771a4b0cfc90c826e6e3b Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Thu, 27 Aug 2015 08:56:37 -0500 Subject: [PATCH 049/106] Remove :features: from namespace --- inbox/entityversioning.xml | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index a895be2c..16dec0d4 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -142,7 +142,7 @@ If a server supports entity versioning, it MUST inform the connecting client when returning stream features during the stream negotiation process. This is done by including a <ver/> element, qualified by - the 'urn:xmpp:features:entityver:0' namespace. At the latest, this SHOULD + the 'urn:xmpp:entityver:0' namespace. At the latest, this SHOULD be done when informing a client that resource binding is required. For example:

    @@ -151,7 +151,7 @@ - + ]]>

    @@ -190,10 +190,10 @@ - 25P2A7H8 + 25P2A7H8 - VIZSVF0D + VIZSVF0D @@ -202,7 +202,7 @@ - 9ZFZXVP9 + 9ZFZXVP9 @@ -219,7 +219,7 @@ XWE4MUUP + XWE4MUUP @@ -233,10 +233,10 @@ type='get'> - 25P2A7H8 + 25P2A7H8 - 4OLGSVNY + 4OLGSVNY @@ -249,7 +249,7 @@ - VIZSVF0D + VIZSVF0D @@ -297,12 +297,12 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8 - + - + 0514fc90e6c7981b06bbb2173bb8ef03 @@ -314,12 +314,12 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8 - + - + 32151d1d01440d5536a7f106afd3f4d8 @@ -379,7 +379,7 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8

    This specification defines the following XML namespace:

      -
    • urn:xmpp:features:entityver:0
      • +
    • urn:xmpp:entityver:0

    Upon advancement of this specification from a status of Experimental to a From b3725e2cf9070cd750bd025bea46da13e433a403 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Thu, 27 Aug 2015 16:22:55 -0500 Subject: [PATCH 050/106] Add a use case to entity versioning Also: Minor typo and wording fixes Fix broken example XML More tabbing fixes --- inbox/entityversioning.xml | 36 ++++++++++++++++++++---------------- 1 file changed, 20 insertions(+), 16 deletions(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index 16dec0d4..45451591 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -50,12 +50,11 @@ This problem of "downloading the world" (downloading the entire roster every time a session is initialized) was partially addressed by &xep0237; which was later merged into &rfc6121; §2.6. While this solved the problem - for the roster, it didn't account for other entities (eg. MUC - disco#items). Furthermore, roster versioning requires that the server - maintain a great deal of state (multiple versions of the roster) which can - be difficult to implement in a large, distributed system. This XEP defines - a method by which entities other than the roster can be versioned and - cached. + for the roster, it didn't account for other entities (eg. disco items). + Furthermore, roster versioning requires that the server maintain a great + deal of state (multiple versions of the roster) which can be difficult to + implement in a large, distributed system. This XEP defines a method by + which entities other than the roster can be versioned and cached.

    @@ -115,6 +114,11 @@ client can fetch the list, and then poll for changes at a later date without re-requesting the entire list. +
  • + A client wishes to cache the features supported by servers of the + contacts in their roster since their disco items is not likely to + change often. +
    • @@ -200,11 +204,11 @@ - + + 9ZFZXVP9 - - + + ]]>

    @@ -218,7 +222,7 @@ - XWE4MUUP @@ -243,13 +247,13 @@ + id='zb8q41fas6yn4' + to='hag66@shakespeare.lit/phone' + type='result'> - VIZSVF0D + VIZSVF0D @@ -346,7 +350,7 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8

    Version tokens SHOULD always be considered opaque to the client (eg. even - if the version token is a derivable and consistant hash on the server side, + if the version token is a derivable and consistent hash on the server side, clients should not need to know how the server is calculating the token).

    From 81cefd2e889673586b168ec66436d5d284ab8736 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Sat, 29 Aug 2015 15:03:50 -0500 Subject: [PATCH 051/106] Initial draft of OTR disco --- inbox/otrdisco.xml | 107 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) create mode 100644 inbox/otrdisco.xml diff --git a/inbox/otrdisco.xml b/inbox/otrdisco.xml new file mode 100644 index 00000000..be091d1e --- /dev/null +++ b/inbox/otrdisco.xml @@ -0,0 +1,107 @@ + + +%ents; +]> + + +

    + OTR Discovery + + This document provides a mechanism by which OTR encryption support can be + discovered in XMPP, without relying on OTRs protocol agnostic discovery + mechanism. + + &LEGALNOTICE; + xxxx + ProtoXEP + Standards Track + Standards + Council + + XMPP Core + XEP-0030 + + + + NOT_YET_ASSIGNED + + Sam + Whited + sam@samwhited.com + sam@samwhited.com + + + 0.0.1 + 2015-08-29 + ssw +

    Initial draft.

     +     +
    + +

    + The Off-the-Record messaging protocol (OTR) is widely layered on top of + XMPP to provide end-to-end encryption. Current use of the protocol is + described in &xep0364;. OTR provides its own discovery mechanism in which + it sends messages with special whitespace characters to indicate support. + While this works when initializing a session, there is no way to query a + client for support and to know in advance that a particular version of + OTR is supported. This XEP provides a mechanism for discovering OTR + support at the XMPP level. This specification aims to solve that by + providing an in-band mechanism for discovering OTR support in XMPP. +

    +

    + It should be noted that newer, more secure encryption protocols exist for + XMPP, and that new implementations of OTR are discouraged. This protocol + is primarily intended to solve issues with existing implementations of + OTR. +

    +     + +

    + If an entity supports OTR it MUST advertise the fact by returning a + feature of 'urn:xmpp:otr:0' &VNOTE; in response to a &xep0030; + information request. It MUST also include '<version>' elements for + each version of the OTR protocol that is supported. Eg. an entity which + supports OTR versions 2 and 3 might include the following feature in its + disco info list: + + + 2 + 3 + + ]]> +

    +     + +

    + Because OTR support is advertised outside of any end-to-end encrypted + stream, it may be subject to downgrade attacks (eg. the server operator + may remove one or more versions of OTR from the list). +

    +     + +

    + This document requires no interaction with the Internet Assigned Numbers + Authority (IANA). +

    +     + +

    This specification defines the following XML namespaces:

    +
      +
    • urn:xmpp:otr:0
      • +
    +

    + The ®ISTRAR; shall include the foregoing namespaces in its disco + features registry as defined in &xep0030;. + + urn:xmpp:otr:0 + Indicates support for Off-the-Record Messaging (OTR) + XEP-xxxx + + ]]> +

    +     + From 50e058ee0312b832c8ae45930fe7f935b1031477 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Sun, 30 Aug 2015 11:05:33 -0500 Subject: [PATCH 052/106] Support OTRv3 only and remove feature subelements --- inbox/otrdisco.xml | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/inbox/otrdisco.xml b/inbox/otrdisco.xml index be091d1e..d9dd19b5 100644 --- a/inbox/otrdisco.xml +++ b/inbox/otrdisco.xml @@ -61,18 +61,24 @@

    If an entity supports OTR it MUST advertise the fact by returning a feature of 'urn:xmpp:otr:0' &VNOTE; in response to a &xep0030; - information request. It MUST also include '<version>' elements for - each version of the OTR protocol that is supported. Eg. an entity which - supports OTR versions 2 and 3 might include the following feature in its - disco info list: - + information request. This indicates support for OTRv3 as defined by + + Off-the-Record Messaging Protocol version 3 + + + (Accessed 2015-08-30). "Off-the-Record Messaging Protocol version 3" + < + https://otr.cypherpunks.ca/Protocol-v3-4.0.0.html + > + . - 2 - 3 - + ]]>

    +

    + If older versions of OTR are required, they may be discovered out of band + using OTRs built in mechanism which is beyond the scope of this document. +

    From 7d8b386dd5e4dc58da3872fb738af7a8c9904be8 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Sun, 30 Aug 2015 11:12:56 -0500 Subject: [PATCH 053/106] Update registrar submission --- inbox/otrdisco.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/otrdisco.xml b/inbox/otrdisco.xml index d9dd19b5..27555167 100644 --- a/inbox/otrdisco.xml +++ b/inbox/otrdisco.xml @@ -104,7 +104,7 @@ urn:xmpp:otr:0 - Indicates support for Off-the-Record Messaging (OTR) + Indicates support for Off-the-Record Messaging (OTR) version 3 XEP-xxxx ]]> From b867a4d5682f9c3bb17a7a0fd20092e3f11d4cc3 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Sun, 30 Aug 2015 11:21:46 -0500 Subject: [PATCH 054/106] Remove redundant sentence --- inbox/otrdisco.xml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/inbox/otrdisco.xml b/inbox/otrdisco.xml index 27555167..802fa2a8 100644 --- a/inbox/otrdisco.xml +++ b/inbox/otrdisco.xml @@ -46,9 +46,8 @@ it sends messages with special whitespace characters to indicate support. While this works when initializing a session, there is no way to query a client for support and to know in advance that a particular version of - OTR is supported. This XEP provides a mechanism for discovering OTR - support at the XMPP level. This specification aims to solve that by - providing an in-band mechanism for discovering OTR support in XMPP. + OTR is supported. This specification aims to solve that by providing an + in-band mechanism for discovering OTR support in XMPP.

    It should be noted that newer, more secure encryption protocols exist for From 796a7a4d94810674898b9d9d9ce4c4665e5d148c Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 31 Aug 2015 20:41:35 -0500 Subject: [PATCH 055/106] Update OTR Disco's security section --- inbox/otrdisco.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/otrdisco.xml b/inbox/otrdisco.xml index 802fa2a8..ed9d2135 100644 --- a/inbox/otrdisco.xml +++ b/inbox/otrdisco.xml @@ -83,7 +83,7 @@

    Because OTR support is advertised outside of any end-to-end encrypted stream, it may be subject to downgrade attacks (eg. the server operator - may remove one or more versions of OTR from the list). + may remove OTR from the features list).

    From f7eb0a8a8ca8d442f9869ac79b0086a2e17b6939 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Tue, 1 Sep 2015 19:54:32 -0500 Subject: [PATCH 056/106] Add cache invalidation to entity versioning --- inbox/entityversioning.xml | 55 +++++++++++++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 3 deletions(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index 45451591..8546d25a 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -52,9 +52,13 @@ which was later merged into &rfc6121; §2.6. While this solved the problem for the roster, it didn't account for other entities (eg. disco items). Furthermore, roster versioning requires that the server maintain a great - deal of state (multiple versions of the roster) which can be difficult to - implement in a large, distributed system. This XEP defines a method by - which entities other than the roster can be versioned and cached. + deal of state (roster items which should be pushed for each entity on + reconnect, or monotonically increasing counters, etc.) which can be + difficult to store or synchronize in a large, distributed system. This XEP + defines a method by which the roster and entities other than the roster can + be versioned and cached, and which is optimized for distributed systems + with large enity lists (but works equally well on small, single server + deployments).

    @@ -268,6 +272,51 @@ question when a version token is received.

    + +

    + When a client syncs with the server and indicates that it has a version + token in its cache that does not match any entity on the server (or when + the server wants to remove an entity from the clients cache for any other + reason), the server MUST reply with an empty <version/> node. When + the client receives such an empty version node it SHOULD purge the entity + from its cache. For example, the following exchange would trigger the + removal of 'inverness@chat.shakespeare.lit' from the cached MUC list: + + + + + 25P2A7H8 + + + 4OLGSVNY + + + + + + + + + + + + + ]]> +

    +

    + If the client receives an indication that it should delete an item from a + list by any other means (eg. via a roster push), it SHOULD remove the + version token associated with that entity from its cache. +

    +

    While the version token approach to caching does not require a great deal From 8c40f321721c811da24b7bbf72d2b5cea190f227 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Wed, 2 Sep 2015 19:55:22 -0500 Subject: [PATCH 057/106] Address some feedback on entity versioning --- inbox/entityversioning.xml | 192 +++++++++++++++++++------------------ 1 file changed, 100 insertions(+), 92 deletions(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index 8546d25a..a0d77cab 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -144,8 +144,7 @@ - - +

    If a server supports entity versioning, it MUST inform the connecting client when returning stream features during the stream negotiation @@ -166,34 +165,34 @@ The entity versioning stream feature is merely informative and therefore is never mandatory-to-negotiate.

    -     + - -

    - Version tokens are short case-sensitive strings which are generated by - the server. Their format is not defined in this spec, but a - recommendation may be found in the Implementation Notes. Version tokens - are akin to a weakly-validated etag for the entity in question. -

    -

    - Servers that implement this protocol must assign such a version token to - each entity that is controlled by the server. The server MUST then update - this version every time any mutable property of the entity changes (eg. - when the subscription status of a user changes). The server MAY choose to - update this token at any time (to force the clients to invalidate their - cached representation fo the object). This version token MUST then be - included with every object representation of that entity sent down in the - stream. This is done by including a sub-node called "version" qualified - by the entity versioning XML namespace defined in this document. - Similarly, clients MAY also add version nodes for each version token they - possess to the request for a list (not specifying a version token will - force the server to send information on that entity to the client). If a - server sends up a list of version tokens, the server MUST then check to - see if those tokens correspond to any entity which it knows about, and - not send down any entities with matching version tokens in the response. -

    -

    For example, a roster request might look like:

    - +

    + Version tokens are short case-sensitive strings which are generated by the + server. Their format is not defined in this spec, but a recommendation may + be found in the Implementation Notes. Version + tokens are akin to a weakly-validated etag for the entity in question. +

    +

    + Servers that implement this protocol must assign such a version token to + each entity that is controlled by the server. The server MUST then update + this version every time any mutable property of the entity changes (eg. + when the subscription status of a user changes). The server MAY choose to + update this token at any time (to force the clients to invalidate their + cached representation fo the object). This version token MUST then be + included with every object representation of that entity sent down in the + stream. This is done by including a sub-node called "version" qualified + by the entity versioning XML namespace defined in this document. + Similarly, clients MAY also add version nodes for each version token they + possess to the request for a list (not specifying a version token will + force the server to send information on that entity to the client). If a + server sends up a list of version tokens, the server MUST then check to + see if those tokens correspond to any entity which it knows about, and + not send down any entities with matching version tokens in the response. +

    +

    For example, a roster request might look like:

    + @@ -271,53 +270,54 @@ Clients that implement this protocol SHOULD then cache the entity in question when a version token is received.

    -     - -

    - When a client syncs with the server and indicates that it has a version - token in its cache that does not match any entity on the server (or when - the server wants to remove an entity from the clients cache for any other - reason), the server MUST reply with an empty <version/> node. When - the client receives such an empty version node it SHOULD purge the entity - from its cache. For example, the following exchange would trigger the - removal of 'inverness@chat.shakespeare.lit' from the cached MUC list: - - +

    + When a client syncs with the server and indicates that it has a version + token in its cache that does not match any entity on the server (or when + the server wants to remove an entity from the clients cache for any other + reason), the server MUST reply with an empty <version/> node. When + the client receives such an empty version node it SHOULD purge the entity + from its cache. For example, the following exchange would trigger the + removal of 'inverness@chat.shakespeare.lit' from the cached MUC list: + + + + + 25P2A7H8 + + + 4OLGSVNY + + + + + + - - 25P2A7H8 - - - 4OLGSVNY - - - - - - - - - - - - - ]]> -

    -

    - If the client receives an indication that it should delete an item from a - list by any other means (eg. via a roster push), it SHOULD remove the - version token associated with that entity from its cache. -

    -     - + to='hag66@shakespeare.lit/phone' + type='result'> + + + + + + + ]]> +

    +

    + If the client receives an indication that it should delete an item from a + list by any other means (eg. via a roster push), it SHOULD remove the + version token associated with that entity from its cache. +

    +     + +

    While the version token approach to caching does not require a great deal of state to be stored on the client or the server, it does require a lot @@ -327,9 +327,11 @@ avoid sending the large request entirely). To do this, we can request an aggregate version token from the server. This aggregate token is calculated by constructing a string of comma separated "bare JID:version" pairs sorted - in byte-wise order, and taking the MD5 hash of the constructed string. For - example, if the server is calculating the aggregate version token for a - roster, it might end up with the following string: + in byte-wise order (because the JID:version pair is constructed before + sorting, if two items in the list have the same JID they can still be + sorted by the version token), and taking the MD5 hash of the constructed + string. For example, if the server is calculating the aggregate version + token for a roster, it might end up with the following string:

    + + + - - - - - - - - - 0514fc90e6c7981b06bbb2173bb8ef03 - - + + + + 0514fc90e6c7981b06bbb2173bb8ef03 + + ]]>

    Similarly, to fetch the aggregate token for a list of MUC rooms, one would - query the MUC component directly: + query the MUC component directly with the type set to the 'disco#items' + namespace:

    @@ -382,13 +385,18 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8 to a normal request if any error is returned in response to an aggregate token IQ.

    +

    + If an aggregate token is requested for a list that may contain more than + one type of entity (eg. MUC rooms and pubsub nodes that live on the same + component), then the server MUST return the aggregate token constructed + with the entire list (rooms and pubsub nodes). +

    Clients are also NOT REQUIRED to check aggregate tokens. However, clients MAY wish to check aggregate tokens before making a roster or MUC request when the cached roster or MUC list is very large. When to check aggregate - tokens is left up to the clients. + tokens (if at all) is left up to the implementation.

    -     From e22172cf39bd9ccfe2e74d9d8e578a96a9240a5e Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Wed, 2 Sep 2015 22:22:03 -0500 Subject: [PATCH 058/106] Fix reference to bare JIDs --- inbox/entityversioning.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index a0d77cab..5ac781ac 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -326,8 +326,8 @@ be useful know in advance if the roster has changed or not (so that we can avoid sending the large request entirely). To do this, we can request an aggregate version token from the server. This aggregate token is calculated - by constructing a string of comma separated "bare JID:version" pairs sorted - in byte-wise order (because the JID:version pair is constructed before + by constructing a string of comma separated "JID:version" pairs sorted in + byte-wise order (because the JID:version pair is constructed before sorting, if two items in the list have the same JID they can still be sorted by the version token), and taking the MD5 hash of the constructed string. For example, if the server is calculating the aggregate version From 0f40c2266831dab0a89da1fad15b8801fa0acf52 Mon Sep 17 00:00:00 2001 From: Daniel Gultsch Date: Tue, 1 Sep 2015 16:44:05 +0200 Subject: [PATCH 059/106] clarify the no-storage/no-store confusion in XEP-0334 and add a new hint to always store message the message hints XEP confusingly talks about no-storage and no-permanent-storage in one place even though no-store was meant added a new message hint to ask the server to store messages even though they do not contain a body. like encrypted messages --- xep-0334.xml | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/xep-0334.xml b/xep-0334.xml index 8183b751..1f381f0f 100644 --- a/xep-0334.xml +++ b/xep-0334.xml @@ -21,6 +21,15 @@ NOT_YET_ASSIGNED &mwild; + + 0.2 + 2015-09-01 + dg + +

    Fixed the wrong use of no-storage instead of no-store

    +

    Added a message hint <store/>

    +     +     0.1 2013-07-11 @@ -45,6 +54,7 @@
    • Whether to store a message (e.g. for archival or as an 'offline message').
    • Whether to copy a message to other resources.
      • +
    • Whether to store a message that would not have been stored under normal conditions
     @@ -74,16 +84,19 @@ recipient is offline at the time of sending). Such a message can be marked with the <no-permanent-store/> hint.

    + +

    Offline storage and &xep0313; can define their own rules on what messages to store and usually only store messages that contain a body element. However a sender may want to indicate that a message is worth storeing even though it might not match those rules (e.g. an encrypted message that carries the payload outside the body element). Such a message can be marked with a <store/> hint.

    +         - -

    The <no-permanent-storage/> hint informs entities that they shouldn't store the message in + +

    The <no-permanent-store/> hint informs entities that they shouldn't store the message in any permanent or semi-permanent public or private archive (such as described in &xep0136; and &xep0313;) or in logs (such as chatroom logs).

    - -

    A message containing a <no-storage/> hint should not be stored by a server either permanently (as above) + +

    A message containing a <no-store/> hint should not be stored by a server either permanently (as above) or temporarily, e.g. for later delivery to an offline client, or to users not currently present in a chatroom.

    @@ -95,6 +108,9 @@ occupants in a &xep0045; room.

    + +

    A message containing the <store/> hint that is not of type 'error' SHOULD be stored by the entity.

    +

    It is important to note that message hints are, as the name implies, just hints. Implementations @@ -122,6 +138,7 @@ + From c20b089c0fc9572da2b9c1a42cedaa409e42816c Mon Sep 17 00:00:00 2001 From: Lance Stout Date: Mon, 24 Aug 2015 14:58:04 -0700 Subject: [PATCH 060/106] Update title and typo fixes --- xep-0357.xml | 14 ++++++++++---- xep.ent | 2 +- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/xep-0357.xml b/xep-0357.xml index 73a1f6df..9a4b95d2 100644 --- a/xep-0357.xml +++ b/xep-0357.xml @@ -6,7 +6,7 @@

    - Push + Push Notifications This specification defines a way for an XMPP servers to deliver information for use in push notifications to mobile and other devices. &LEGALNOTICE; 0357 @@ -29,6 +29,12 @@ lancestout@gmail.com lance@lance.im + + 0.2 + 2015-09-10 + lance +

    Change the name to 'Push Notifications' based on feedback.

     +     0.1 2015-03-18 @@ -52,7 +58,7 @@

    The purpose of push notifications is to inform users of new messages or other pertinent information even when they have no XMPP clients online.

    Typically, these notifications are delivered to a user's mobile device, displaying a notice that can trigger opening an XMPP client to continue a conversation or answer a Jingle session request.

    -

    There have been several push noticiations implementations by mobile XMPP client vendors. However, experience has shown that these implementations carried several drawbacks:

    +

    There have been several push notification implementations by mobile XMPP client vendors. However, experience has shown that these implementations carried several drawbacks:

    • Treated the XMPP client and XMPP server as one unified service, such that push notifications only worked using the "official" client.
    • Proxied a user's session through the client provider's backend services in order to monitor for and trigger push notifications.
      • @@ -76,7 +82,7 @@
    • The third-party (and potentially proprietary or platform-dependent) push service delivers the notification from the client application's backend service to the user's device.
      • -

      This two-tiered push architecture allows the user's XMPP server to deliver notifications to arbitrary third-pary clients, and in turn allows those clients to use the appropriate delivery mechanism for their platforms without having to share any private keys or other credentials with the XMPP server.

      +

      This two-tiered push architecture allows the user's XMPP server to deliver notifications to arbitrary third-party clients, and in turn allows those clients to use the appropriate delivery mechanism for their platforms without having to share any private keys or other credentials with the XMPP server.

      The current state-of-the-art for a generic push notification service requires four actors:

      @@ -335,7 +341,7 @@

      If additional data was provided when enabling the service, the publish request SHOULD include the data as publish options.

      - Customizable Message Routing (XEP-0354) XEP-0354: Customizable Message Routing <http://xmpp.org/extensions/xep-0354.html>." > Namespace Delegation (XEP-0355) XEP-0355: Namespace Delegation <http://xmpp.org/extensions/xep-0355.html>." > Privileged Entity (XEP-0356) XEP-0356: Privileged Entity <http://xmpp.org/extensions/xep-0356.html>." > -Push (XEP-0357) XEP-0357: Push <http://xmpp.org/extensions/xep-0357.html>." > +Push Notifications (XEP-0357) XEP-0357: Push Notifications <http://xmpp.org/extensions/xep-0357.html>." > Publishing Available Jingle Sessions (XEP-0358) XEP-0358: Publishing Available Jingle Sessions <http://xmpp.org/extensions/xep-0358.html>." > Unique and Stable Stanza IDs (XEP-0359) XEP-0359: Unique and Stable Stanza IDs <http://xmpp.org/extensions/xep-0359.html>." > Nonzas (are not Stanzas) (XEP-0360) XEP-0360: Nonzas (are not Stanzas) <http://xmpp.org/extensions/xep-0360.html>." > From 4bba55b8c06c44bf0de04ae5cda75c831e4b3a1d Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 31 Aug 2015 21:41:35 -0500 Subject: [PATCH 061/106] Add a Makefile to build XEPs --- Makefile | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 Makefile diff --git a/Makefile b/Makefile new file mode 100644 index 00000000..38849269 --- /dev/null +++ b/Makefile @@ -0,0 +1,32 @@ +.SILENT: + +OUTDIR?=build +XMLDEPS=xep.xsl xep.xsd xep.ent xep.dtd ref.xsl + +.PHONY: help +help: + @echo 'XEP makefile targets:' + @echo ' ' + @echo ' help - (this message)' + @echo ' all - build all XEPs' + @echo ' clean - recursively unlink the build tree' + @echo ' xep-xxxx - build xep-xxxx.html' + @echo ' ' + @echo 'Output directory: "$(OUTDIR)/"' + +.PHONY: all +all: $(patsubst %.xml, $(OUTDIR)/%.html, $(wildcard *.xml)) + +.PHONY: xep-% +xep-%: $(OUTDIR)/xep-%.html + + +$(OUTDIR)/%.html: %.xml $(XMLDEPS) $(OUTDIR) + xsltproc xep.xsl "$<" > "$@" + +$(OUTDIR): + mkdir -p $(OUTDIR) + +.PHONY: clean +clean: + rm -rf $(OUTDIR) From a2c7826aab72844b966a0e7ebfa0823845163b30 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 31 Aug 2015 21:45:00 -0500 Subject: [PATCH 062/106] Add build instructions to the README Fixes #18 --- README.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 4a876ca3..724c4aca 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Please use this repository to raise issues and submit pull requests: https://github.com/xsf/xeps/issues https://github.com/xsf/xeps/pulls -For in-depth technical discussion, please post to the standards@xmpp.org +For in-depth technical discussion, please post to the standards@xmpp.org email list: http://mail.jabber.org/mailman/listinfo/standards @@ -23,7 +23,18 @@ page: http://xmpp.org/xmpp-protocols/xmpp-extensions/submitting-a-xep/ -[XEP-0001: XMPP Extension Protocols](http://xmpp.org/extensions/xep-0001.html) defines the standards process followed by the XMPP Standards Foundation. +[XEP-0001: XMPP Extension Protocols](http://xmpp.org/extensions/xep-0001.html) +defines the standards process followed by the XMPP Standards Foundation. -Thanks! +Building XEPs +------------- +To build a single XEP as HTML simply run: + + make xep-xxxx + +To change the output directory, set the variable `OUTDIR`, eg. + + OUTDIR=/tmp/xeps make all + +For more information try `make help`. From ba1646cfca0ff24fcaede814510d41a22ceac3e1 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 31 Aug 2015 23:15:10 -0500 Subject: [PATCH 063/106] Add PDF generation support to Makefile --- Makefile | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 38849269..25389fc2 100644 --- a/Makefile +++ b/Makefile @@ -1,32 +1,76 @@ .SILENT: OUTDIR?=build -XMLDEPS=xep.xsl xep.xsd xep.ent xep.dtd ref.xsl +TEMPDIR?=$(TMPDIR)/xepbuild +XMLDEPS=xep.xsl xep.xsd xep.ent xep.dtd ref.xsl $(OUTDIR) +TEXMLDEPS=xep2texml.xsl $(TEMPDIR) $(XMLDEPS) $(TEMPDIR)/xmpp.pdf $(TEMPDIR)/xmpp-text.pdf + .PHONY: help help: @echo 'XEP makefile targets:' @echo ' ' @echo ' help - (this message)' - @echo ' all - build all XEPs' + @echo ' all - build all XEPs (html and pdfs)' + @echo ' pdf - build all XEPs' + @echo ' html - build all XEPs' @echo ' clean - recursively unlink the build tree' - @echo ' xep-xxxx - build xep-xxxx.html' + @echo ' xep-xxxx - build xep-xxxx.html and xep-xxxx.pdf' + @echo ' xep-xxxx.html - build xep-xxxx.html' + @echo ' xep-xxxx.pdf - build xep-xxxx.html' @echo ' ' @echo 'Output directory: "$(OUTDIR)/"' .PHONY: all -all: $(patsubst %.xml, $(OUTDIR)/%.html, $(wildcard *.xml)) +all: html pdfs + +.PHONY: html +html: $(patsubst %.xml, $(OUTDIR)/%.html, $(wildcard *.xml)) + +.PHONY: pdf +pdf: $(patsubst %.xml, $(OUTDIR)/%.pdf, $(wildcard *.xml)) .PHONY: xep-% -xep-%: $(OUTDIR)/xep-%.html +xep-%: $(OUTDIR)/xep-%.html $(OUTDIR)/xep-%.pdf -$(OUTDIR)/%.html: %.xml $(XMLDEPS) $(OUTDIR) +.PHONY: xep-%.html +xep-%.html: $(OUTDIR)/xep-%.html + + +.PHONY: xep-%.pdf +xep-%.pdf: $(OUTDIR)/xep-%.pdf + + +$(OUTDIR)/%.html: %.xml $(XMLDEPS) xsltproc xep.xsl "$<" > "$@" +$(OUTDIR)/%.pdf: %.xml $(TEMPDIR)/%.xml.texml $(TEMPDIR)/%.xml.texml.tex $(TEXMLDEPS) + cd $(TEMPDIR); xelatex $(TEMPDIR)/$<.texml.tex + mv $(TEMPDIR)/$<.texml.pdf $(OUTDIR)/$(patsubst %.xml.pdf,%.pdf,$<.pdf) + +$(TEMPDIR)/%.xml.texml: %.xml $(TEXMLDEPS) $(TEMPDIR) + xsltproc -o $(TEMPDIR)/$<.texml xep2texml.xsl "$<" + +$(TEMPDIR)/%.xml.texml.tex: $(TEMPDIR)/%.xml.texml $(OUTDIR) $(TEMPDIR) + texml -e utf8 $< $<.tex + sed -i 's|\([\s"]\)\([^"]http://[^ "]*\)|\1\\path{\2}|g' $<.tex + sed -i 's|\\hyperref\[#\([^}]*\)\]|\\hyperref\[\1\]|g' $<.tex + sed -i 's|\\pageref{#\([^}]*\)}|\\pageref{\1}|g' $<.tex + +$(TEMPDIR)/xmpp-text.pdf: $(TEMPDIR) + -[ -e $(TEMPDIR)/xmpp-text.pdf ] || curl -o "$(TEMPDIR)/xmpp-text.pdf" https://xmpp.org/images/xmpp-text.pdf + +$(TEMPDIR)/xmpp.pdf: $(TEMPDIR) + -[ -e $(TEMPDIR)/xmpp.pdf ] || curl -o "$(TEMPDIR)/xmpp.pdf" https://xmpp.org/images/xmpp.pdf + +$(TEMPDIR): + mkdir -p $(TEMPDIR) + $(OUTDIR): mkdir -p $(OUTDIR) .PHONY: clean clean: + rm -rf $(TEMPDIR) rm -rf $(OUTDIR) From 4eec7f6324443b300b80ff1f9e0cff21ec8bd96c Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Tue, 1 Sep 2015 19:53:49 -0500 Subject: [PATCH 064/106] Make sure the makefile looks in curdir for files Eg. in case you're generating something in the inbox --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 25389fc2..1598e95e 100644 --- a/Makefile +++ b/Makefile @@ -43,7 +43,7 @@ xep-%.pdf: $(OUTDIR)/xep-%.pdf $(OUTDIR)/%.html: %.xml $(XMLDEPS) - xsltproc xep.xsl "$<" > "$@" + xsltproc --path $(CURDIR) xep.xsl "$<" > "$@" $(OUTDIR)/%.pdf: %.xml $(TEMPDIR)/%.xml.texml $(TEMPDIR)/%.xml.texml.tex $(TEXMLDEPS) cd $(TEMPDIR); xelatex $(TEMPDIR)/$<.texml.tex From b2988569408f0e22288211bbbaf9e0e3a21ba206 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Thu, 10 Sep 2015 13:06:50 -0500 Subject: [PATCH 065/106] Add output when build complete --- Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 1598e95e..e1b98e35 100644 --- a/Makefile +++ b/Makefile @@ -43,11 +43,12 @@ xep-%.pdf: $(OUTDIR)/xep-%.pdf $(OUTDIR)/%.html: %.xml $(XMLDEPS) - xsltproc --path $(CURDIR) xep.xsl "$<" > "$@" + xsltproc --path $(CURDIR) xep.xsl "$<" > "$@" && echo "Finished building $@" $(OUTDIR)/%.pdf: %.xml $(TEMPDIR)/%.xml.texml $(TEMPDIR)/%.xml.texml.tex $(TEXMLDEPS) cd $(TEMPDIR); xelatex $(TEMPDIR)/$<.texml.tex mv $(TEMPDIR)/$<.texml.pdf $(OUTDIR)/$(patsubst %.xml.pdf,%.pdf,$<.pdf) + echo "Finished building $(patsubst %.xml.pdf,%.pdf,$<.pdf)" $(TEMPDIR)/%.xml.texml: %.xml $(TEXMLDEPS) $(TEMPDIR) xsltproc -o $(TEMPDIR)/$<.texml xep2texml.xsl "$<" From 931daefbdaf50987d720112d02af200413fe96eb Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Thu, 10 Sep 2015 13:12:30 -0500 Subject: [PATCH 066/106] Only run sed once and escape possible mistaken label marker --- Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index e1b98e35..2f306c47 100644 --- a/Makefile +++ b/Makefile @@ -55,9 +55,9 @@ $(TEMPDIR)/%.xml.texml: %.xml $(TEXMLDEPS) $(TEMPDIR) $(TEMPDIR)/%.xml.texml.tex: $(TEMPDIR)/%.xml.texml $(OUTDIR) $(TEMPDIR) texml -e utf8 $< $<.tex - sed -i 's|\([\s"]\)\([^"]http://[^ "]*\)|\1\\path{\2}|g' $<.tex - sed -i 's|\\hyperref\[#\([^}]*\)\]|\\hyperref\[\1\]|g' $<.tex - sed -i 's|\\pageref{#\([^}]*\)}|\\pageref{\1}|g' $<.tex + sed -i -e 's|\([\s"]\)\([^"]http\://[^ "]*\)|\1\\path{\2}|g' \ + -e 's|\\hyperref\[#\([^}]*\)\]|\\hyperref\[\1\]|g' \ + -e 's|\\pageref{#\([^}]*\)}|\\pageref{\1}|g' $<.tex $(TEMPDIR)/xmpp-text.pdf: $(TEMPDIR) -[ -e $(TEMPDIR)/xmpp-text.pdf ] || curl -o "$(TEMPDIR)/xmpp-text.pdf" https://xmpp.org/images/xmpp-text.pdf From f845947c314f25e7e15fea078453e7915d3274bc Mon Sep 17 00:00:00 2001 From: Lance Stout Date: Thu, 10 Sep 2015 12:23:55 -0700 Subject: [PATCH 067/106] A better formalization of XEP-0234 * Recast the document to match the registration procedure defined by XEP-0166 for new application types. * Explicitly defined the schema of the <file/> element instead of referencing XEP-0096. * Defined the use of a 'length' attribute on <range/> elements. This attribute was always there because the <file/> and <range/> elements had been defined as the same as the parallel elements in XEP-0096, but the 'length' attribute was not explicitly referenced in this document. * Clarified the need for the 'senders' attribute on Jingle <content/> elements to distinguish between File Offers and File Requests. * Added the <file-not-found/> and <file-too-large/> reason types. * Clarified how to abort a file transfer via 'content-remove' or 'session-terminate' actions. * Added <received/> element for indicating that a transfer was successful, particularly when there are multiple transfers. * Added SDP mapping. --- xep-0234.xml | 916 +++++++++++++++++++++++++++++++++++---------------- xep.ent | 1 + 2 files changed, 635 insertions(+), 282 deletions(-) diff --git a/xep-0234.xml b/xep-0234.xml index 8bd0bbef..15189e96 100644 --- a/xep-0234.xml +++ b/xep-0234.xml @@ -15,15 +15,32 @@ Standards XMPP Core - XEP-0047 - XEP-0065 - XEP-0096 XEP-0166 + XEP-0261 + XEP-0300 NOT_YET_ASSIGNED &stpeter; + &lance; + + 0.17 + 2015-09-08 + ls/psa + +
        +
      • Recast the document to match the registration procedure defined by XEP-0166 for new application types.
        • +
      • Explicitly defined the schema of the <file/> element instead of referencing XEP-0096.
        • +
      • Defined the use of a 'length' attribute on <range/> elements. This attribute was always there because the <file/> and <range/> elements had been defined as the same as the parallel elements in XEP-0096, but the 'length' attribute was not explicitly referenced in this document.
        • +
      • Clarified the need for the 'senders' attribute on Jingle <content/> elements to distinguish between File Offers and File Requests.
        • +
      • Added the <file-not-found/> and <file-too-large/> reason types.
        • +
      • Clarified how to abort a file transfer via 'content-remove' or 'session-terminate' actions.
        • +
      • Added <received/> element for indicating that a transfer was successful, particularly when there are multiple transfers.
        • +
      • Added SDP mapping.
        • +
      +       +       0.16 2014-08-11 @@ -163,54 +180,200 @@ 0.0.2 2008-02-28 psa - Modified negotiation flow to use new content-replace action. +

      Modified negotiation flow to use new content-replace action.

             0.0.1 2008-01-29 psa - First draft. +

      First draft.
    +

    &xep0166; can be used to initiate and negotiate a wide range of peer-to-peer sessions. One session type of interest is file transfer. This document specifies an application format for negotiating Jingle file transfer sessions, where files are exchanged via any available reliable transport.

    +     + +

    &xep0096; was the original XMPP protocol extension for file transfer negotiation. However, that protocol has several drawbacks, most related to the &xep0095; protocol on which it depends:

      -
    1. It does not enable a true, bidirectional negotiation; instead, the initiator sets the terms for the file transfer and the responder either accepts the terms or cancels the negotiation.
      2. -
    2. It is the only technology in the Jabber/XMPP protocol "stack" that uses XEP-095: Stream Initiation. More modern technologies such as voice and video session negotiation use &xep0166;, and it would be helpful if implementors could re-use the same code for all negotiation use cases.
      3. +

    3. It does not enable a true, bidirectional negotiation; instead, the initiator sets the terms for the file transfer and the responder either accepts the terms or cancels the negotiation.

      4. +

    4. It is the only technology in the Jabber/XMPP protocol "stack" that uses XEP-0095: Stream Initiation. More modern technologies such as voice and video session negotiation use &xep0166;, and it would be helpful if implementors could re-use the same code for all negotiation use cases.

    To overcome these drawbacks, this specification defines a file transfer negotiation method that meets the following requirements:

      -
    • Use the session negotiation semantics from XEP-0166.
      • -
    • Use &xep0260; and &xep0261; as transport methods.
      • -
    • Define a file description format that, unlike XEP-0096, enables hash agility (via &xep0300;).
      • -
    • Define a clear upgrade path from SI File Transfer to Jingle File Transfer.
      • +

    • Use the session negotiation semantics from XEP-0166.

      • +
    • +

      Use any reliable Jingle transport mechanism, including but not limited to:

      +
        +
      • ICE-TCP &rfc6544;
        • +
      • SOCKS5 Bytestreams &xep0260;
        • +
      • In-Band Bytestreams &xep0261;
        • +
      +
      • +

    • Define a file description format that, unlike XEP-0096, enables hash agility (via &xep0300;).

      • +

    • Define a clear upgrade path from SI File Transfer to Jingle File Transfer.

    -

    Jingle file transfer is only as reliable as the transports on which it depends. In particular, SOCKS5 Bytestreams ("S5B") does not always result in NAT or firewall traversal. To work around that problem, this specification requires all implementations to support as a fallback mechanism In-Band Bytestreams ("IBB"), which usually results in a successful (if slow) file transfer.

    -

    Note: It is likely that a future version of this specification will also recommend implementation of a Jingle transport method that emulates the IETF's ICE-TCP technology, as specified in &rfc6544;.

    +

    Note that Jingle file transfer is only as reliable as the transports on which it depends. In particular, SOCKS5 Bytestreams ("S5B") does not always result in NAT or firewall traversal. To work around that problem, this specification requires all implementations to support as a fallback mechanism In-Band Bytestreams ("IBB"), which usually results in a successful (if slow) file transfer. A more robust and adaptable option is ICE-TCP (RFC 6455); at the time of writing &xep0176; is being updated to include the ability to negotiate ICE-TCP candidates.

     - -

    This section provides a friendly introduction to Jingle file transfer.

    -

    First, the party that wishes to initiate the file transfer determines the responder's capabilities (via &xep0030; or &xep0115;). Here we assume that the responder supports the following service discovery features:

    -
      -
    • urn:xmpp:jingle:1 as described in XEP-0166
      • -
    • urn:xmpp:jingle:apps:file-transfer:4 as defined in this document &NSVER;
      • -
    • urn:xmpp:jingle:transports:s5b:1 as defined in XEP-0260
      • -
    • urn:xmpp:jingle:transports:ibb:1 as defined in XEP-0261
      • -
    -

    The initiator then sends a Jingle session-initiation request to a potential responder. The content-type of the request specifies two things:

    + +
    + +
    File Offer
    +
    A Jingle File Transfer Content is said to be a File Offer if the content creator is the same as the content sender (see Use of Jingle Content Senders).
    +     + +
    File Request
    +
    A Jingle File Transfer Content is said to be a File Request if the content creator is the opposite of the content sender (see Use of Jingle Content Senders).
    +     + +
    File Sender
    +
    The File Sender is the side of the Jingle session responsible for sending the file data. The File Sender is not necessarily the same entity as the Jingle session initiator, and an entity could be both a File Sender and File Receiver in the context of a single Jingle session with multiple files.
    +     + +
    File Receiver
    +
    The File Receiver is the side of the Jingle session responsible for receiving the file data. The File Receiver is not necessarily the same entity as the Jingle session responder, and an entity could be both a File Receiver and File Sender in the context of a single Jingle session with multiple files.
    +     +
    +     + + +

    In accordance with Section 12 of XEP-0166, this document specifies the following information related to the Jingle File Transfer ("Jingle FT") application type:

      -
    1. An application type of "urn:xmpp:jingle:apps:file-transfer:4". In particular, the <description/> element contains a <file/> elements describing the file to be sent.
      2. -
    2. An appropriate transport method. So far the suggested methods are jingle-s5b (XEP-0260) and, as a fallback, jingle-ibb (XEP-0261).
      3. +

    3. The application format negotiation process is defined in the Negotiating a Jingle File Transfer Session section of this document.

      4. +

    4. The semantics of the &DESCRIPTION; element are defined in the Application Format section of this document.

      5. +

    5. A mapping of Jingle semantics to the Session Description Protocol is provided in the Mapping to Session Description Protocol section of this document.

      6. +

    6. A Jingle File Transfer session SHOULD use a streaming transport method, not a datagram transport method.

      7. +

    7. Transport components are not used in Jingle File Transfer.

      8. +
    8. +

      Content is to be sent and received as follows:

      +

      For streaming transports, outbound content shall be encoded into packets (as defined by the transport mechanism) without any other framing mechanism and sent in succession over the transport. Incoming data received over the transport shall be processed as a stream of packets, where each packet's content payload is entirely composed of the next portion of file data to be processed.

      +
    -

    In this example, the initiator is <romeo@montague.lit>, the responder is <juliet@capulet.lit>, the application type is a file offer, and the transport method is jingle-s5b.

    -

    The flow is as follows.

    + + +

    Jingle File Transfer makes critical use of the 'senders' attribute of Jingle &CONTENT; elements in order to specify which party is responsible for sending the described file. As such, Jingle File Transfer content MUST include a 'senders' attribute, where the allowed values are "initiator" and "responder". The semantics of the values "both" and "none" are undefined in Jingle File Transfer and thus NOT RECOMMENDED for use with Jingle File Transfer content.

    +

    In general, a Jingle File Transfer content is said to be a "File Offer" if the 'senders' attribute is the same as the role of the party adding the content to the session, and a "File Request" if the 'senders' value is the opposite role of the party adding the content.

    +

    Note: The content 'creator' attribute does not specify who created or is sending the file, it only specifies which party to the session added the Jingle content to the session.

    + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Jingle Session RoleContent SendersFile Transfer Type
    initiatorinitiatorFile Offer
    initiatorresponderFile Request
    responderinitiatorFile Request
    responderresponderFile Offer
    +     +     + + +

    A Jingle File Transfer session is described by a content type that contains one application format and one transport method. Each &CONTENT; element defines the details of a single file transfer. A Jingle negotiation MAY result in the establishment of multiple file transfers by including multiple &CONTENT; elements.

    +

    The application format consists of a file description contained within a &DESCRIPTION; element qualified by the "urn:xmpp:jingle:apps:file-transfer:4" namespace &VNOTE;. The file description is a <file/> element specifying metadata such as the name of the file, media type, etc., as illustrated in the following example.

    + + + text/plain + test.txt + 2015-07-26T21:46:00 + 6144 + 552da749930852c69ae5d2141d3766b1 + +]]> +

    The &DESCRIPTION; element is intended to be a child of a Jingle &CONTENT; element as specified in XEP-0166.

    +

    The child elements of the <file/> element are as follows:

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    Element NameDescriptionInclusion
    dateUTC timestamp specifying the last modified time of the file (which MUST conform to the DateTime profile of &xep0082;).OPTIONAL
    descA human readable description of the file. Multiple <desc/> elements MAY be included if different xml:lang values are specified.OPTIONAL
    hashA hash of the file content, using the <hash/> element defined in &xep0300; and qualifed by the 'urn:xmpp:hashes:1' namespace. Multiple hashes MAY be included for hash agility.REQUIRED when offering a file, otherwise OPTIONAL
    media-typeThe media type of the file content, which SHOULD be a valid MIME-TYPE as registered with &IANA; (specifically, as listed at <http://www.iana.org/assignments/media-types>). If not specified, the content is assumed to be "application/octet-stream".RECOMMENDED when offering a file, otherwise OPTIONAL
    nameThe name of the file. The name SHOULD NOT contain characters or character sequences that would be interpreted as a directory structure by the local file system (e.g. "/", "\", "../", etc.). If any such characters or character sequences are present (possibly because the local and remote file systems use different syntax for directory structure), they SHOULD be escaped (e.g., via percent-encoding) before using the name as part of any file system operation. See Security Considerations.OPTIONAL
    sizeThe length of the file's content, in bytes.OPTIONAL, but SHOULD be present when offering a file.
    rangeThe presence of the <range/> element indicates support of ranged transfers, and can be used to control where a transfer starts.OPTIONAL
    +

    One or more <hash/> elements MUST be present when offering a file, but those elements MAY be empty if the hash has not yet been computed. If there is no computed hash value, the <hash/> element(s) MUST possess an 'algo' attribute specifying which hash algorithm will be used. Once a hash has been calculated by the File Sender, the File Sender SHOULD inform the File Receiver of the hash value as described in Checksum.

    +

    Additional elements MAY be included as children of the <file/> element to provide additional metadata about the file, such as &xep0264;.

    +

    The optional <range/> element MAY possess two attributes:

    + + + + + + + + + + + + + + + + +
    AttributeDescriptionInclusion
    offsetSpecifies the position, in bytes, from which to start transferring file data. This defaults to zero (0) if not specified.OPTIONAL
    lengthSpecifies the number of bytes to retrieve starting at offset. This defaults to the length of the file from offset to the end.OPTIONAL
    +

    Inclusion of a <range/> element in a File Offer indicates support of ranged transfers for future File Requests if the transfer is interrupted and needs to be restarted.

    +

    A <range/> element MAY include an 'offset' attribute set to begin the transfer at a point other than the start of the file, and MAY include a 'length' attribute to request a portion of the file smaller than the remaining length of the file. If no 'offset' or 'length' attributes are present then it is the same as if no <range/> element was present, because the default values of the attributes would indicate a requested range of the entire file. In general, the first byte of data to be transferred is at the (zero-indexed) position specified by the 'offset' value, with a total of 'length' bytes sent.

    +     + + +

    In general, the process for negotiating a Jingle File Transfer session is as follows:

    and S5B) | |---------------------------->| | ack | |<----------------------------| @@ -218,25 +381,33 @@ Initiator Responder |<----------------------------| | ack | |---------------------------->| - | [ file transfer ] | + | [optional further | + | negotiation] | + |<--------------------------->| + | File Transfer | |============================>| - | session-terminate | - |<----------------------------| - | ack | - |---------------------------->| - | | - ]]> -

    First the initiator sends a Jingle session-initiate.

    -     + + +

    To start a File Offer, the initiator sends a Jingle session-initiation request to a potential responder. The request specifies three things:

    +
      +
    1. A content 'senders' attribute with the value of 'initiator' to indicate this is a File Offer.
      2. +
    2. An application type of "urn:xmpp:jingle:apps:file-transfer:4". In particular, the <description/> element contains a <file/> elements describing the file to be sent.
      3. +
    3. An appropriate transport method.
      4. +
    +

    In this example, the initiator is <romeo@montague.example>, the responder is <juliet@capulet.example>, the application type is a File Offer, and the transport method is jingle-s5b (XEP-0260).

    +

    The flow is as follows.

    +

    First the initiator sends a Jingle session-initiate.

    + - + 1969-07-21T02:56:15Z @@ -244,50 +415,49 @@ Initiator Responder text/plain test.txt - 1022 - 552da749930852c69ae5d2141d3766b1 + 6144 + 552da749930852c69ae5d2141d3766b1 - - ]]> -

    Note: As in XEP-0096, inclusion of the <range/> child of the <file/> element indicates that the initiatior supports ranged transfers as described below under Ranged Transfers.

    -

    Note: Computing the hash of the file before sending it can slow down the process of file transfer, since the sending application needs to process the file twice. The sender might prefer to send the hash after the file transfer has begun, using a transport-info message as described under Communicating the Hash.

    -

    The responder immediately acknowledges receipt of the Jingle session-initiate.

    - ]]> +

    Note: Inclusion of the <range/> child of the <file/> element indicates that the initiator supports ranged transfers as described below under Ranged Transfers.

    +

    Note: Computing the hash of the file before sending it can slow down the process of file transfer, because the sending application needs to process the file twice. The File Sender might prefer to send the hash after the file transfer has begun, using a transport-info message as described under Checksum.

    +

    The responder immediately acknowledges receipt of the Jingle session-initiate.

    + - ]]> -

    The initiator then attempts to initiate a SOCKS5 Bytestream with the responder as described in XEP-0260 and XEP-0065. In the meantime, the responder returns a Jingle session-accept. In the session-accept message, the <file/> element MAY contain a <range/> element to indicate that the receiver also supports ranged transfers as described below under Ranged Transfers.

    - ]]> +

    The initiator then attempts to initiate a SOCKS5 Bytestream with the responder as described in XEP-0260 and XEP-0065. In the meantime, the responder returns a Jingle session-accept. In the session-accept message, the <file/> element MAY contain a <range/> element to indicate that the receiver also supports ranged transfers as described below under Ranged Transfers.

    + - + 1969-07-21T02:56:15Z @@ -295,122 +465,349 @@ Initiator Responder text/plain test.txt - 1022 - 552da749930852c69ae5d2141d3766b1 + 6144 + 552da749930852c69ae5d2141d3766b1 - - ]]> -

    The initiator acknowledges the Jingle session-accept.

    - ]]> +

    The initiator acknowledges the Jingle session-accept.

    + - ]]> -

    Once one client has successfully created a connection, it sends a <remote-candidate/> element to the peer inside a Jingle transport-info message. If a client receives a remote-candidate notification it SHOULD continue trying to connect to candidates sent by its peer if it has not tried all candidates with a higher priority than the one successfully used by the peer.

    - ]]> +     + + +

    If the File Sender has advertised the existence of a file that it hosts, such as by &xep0358;, or if a previous file transfer attempt has failed and the File Receiver would like to initiate another attempt, the File Receiver can "pull" the file from the File Sender. This is done by sending a Jingle session-initiate to the File Sender which includes a <content/> with the 'senders' attribute set to the opposite Jingle session role of the party requesting the file (see Use of Jingle Content Senders) and a <description/> element qualified by the 'urn:xmpp:jingle:apps:file-transfer:4' namespace and which includes a <file/> element with enough information included to form a "file selector" (see Section 5 of &rfc5547;) to identify the requested file.

    + - + action='session-initiate' + initiator='juliet@capulet.example/yn0cl4bnw0yr3vym' + sid='uj3b2'> + + + + 552da749930852c69ae5d2141d3766b1 + + - + mode='tcp' + sid='xig361fj'> + + + - - ]]> -

    The peer immediately acknowledges receipt.

    - - ]]> -

    (See XEP-0260 for further details.)

    -

    Now the parties exchange the file using the negotiated transport (here, SOCKS5 Bytestreams).

    -

    Once the transfer is completed, either party can acknowledge completion or terminate the Jingle session; preferably this is done by the entity that receives the file to ensure that the complete file (up to the advertised size) has been received.

    - ]]> +

    See File not Available for how to respond if the requester does not have permission to request the file, or if the file cannot be found.

    +     + + +

    While the Jingle File Transfer session is active, either party MAY choose to add additional files (both offers and requests) to the transfer session. To do so, a Jingle content-add action is used, as shown in the following examples.

    + + + + + + second-file.txt + text/plain + 6144 + 552da749930852c69ae5d2141d3766b1 + + + + + + + + +]]> +

    The other party then acks the content-add request.

    + ]]> +

    At this point, the content-add request needs to be either accepted or rejected using Jingle content-accept or content-reject actions.

    +     + + +

    As in XEP-0096, a transfer can include only part of a file (e.g., to restart delivery of a truncated transfer session at a point other than the start of the file). This is done using the <range/> element. The usage is illustrated in the following examples.

    +

    Let us imagine that the parties negotiate a file transfer session using, say, In-Band Bytestreams. During the transfer, the recipient goes offline unexpectedly and IBB stanzas from the File Sender to the File Receiver begin to bounce. When the recipient comes back online, the File Sender could initiate a new Jingle session and specify that it wants to send all chunks after byte 270336 (which might be the 66th chunk of size 4096).

    + + + + + + + 552da749930852c69ae5d2141d3766b1 + + + + + + + + +]]> +     + + +

    At any point, either party MAY choose to abort the transfer of a single file, or end the session entirely to abort all active transfers.

    +

    When there is only a single Jingle content or if a party wishes to abort the transfer of all files in the session, a session-terminate including a Jingle reason of <cancel /> is sent.

    + - + - - ]]> -

    After terminating the session, the parties would close the data transport as described in the relevant specification (e.g., XEP-0260 or XEP-0261).

    -

    For a description of the transport fallback scenario (from SOCKS5 Bytestreams to In-Band Bytestreams), refer to XEP-0260.

    +]]> +

    If a party chooses to abort the transfer of a single file out of several active transfers, a Jingle content-remove action is used, which MAY include a Jingle reason of <cancel/>, as shown in the following example.

    + + + + + + + +]]> +

    The other party then acks the content-remove request.

    + ]]> +

    If after removing the content there are no other Jingle contents the session MUST be terminated as described in the next section.

    +     + + +

    Once all file content in the session has been transfered, either party MAY acknowledge receipt of the received files (see Received) or, if there are no other active file transfers, terminate the Jingle session with a Jingle session of <success/>. Preferably, sending the session-terminate is done by the last entity to finish receiving a file to ensure that all offered or requested files by either party have been completely received (up to the advertised sizes).

    + + + + + + +]]> +     + - -

    At any time during the lifetime of the file transfer session, the hosting entity (i.e., the entity where the file resides) can communicate the checksum of the file to the receiving entity.

    -

    This can be done in the session-initiate message if the sender already knows the checksum, as shown above in Example 1.

    -

    After the session-initiate message, this can also be done by sending a session-info message containing a <checksum/> element qualified by the 'urn:xmpp:jingle:apps:file-transfer:4' namespace, which in turn contains a <file/> element that MUST at least contain a child element of <hash/> qualified by the 'urn:xmpp:hashes:1' namespace and MAY contain other elements qualified by the 'urn:xmpp:jingle:apps:file-transfer:4' namespace (e.g. <name/> and <date/>). Each <hash/> element contains a checksum of the file contents produced in accordance with the hashing function specified by the 'algo' attribute, which MUST be one of the functions listed in the &ianahashes;.

    - +

    &rfc5547; defines the general process for including file transfer information in SDP.

    +

    The SDP media type for Jingle File Transfer can be "message" (e.g. when used with &rfc4975;) or "application"; however, this media value is not reflected in the Jingle File Transfer application format.

    +

    Any combination of <name/>, <size/>, <media-type/>and <hash/> values MAY be used to form a "file selector" (see Section 5 of &rfc5547;), which would be mapped to SDP as follows:

    + "] [size:] [type:] [hash::]]]> +

    (The hash value MUST be encoded as hexadecimal with each byte separated by a colon.)

    +

    The <date/> value is the last modified time of the file, and thus is mapped as follows:

    + "]]> +

    Note: the format used here for <date> is the date-time format defined in &rfc5322;.

    +

    If a range is specified, the SDP mapping requires both a start and stop offset. If no length was specified for the range, the stop offset is "*". If a length was specified, the stop offset is the <range/> offset value plus the length.

    + -<(offset + length) | *>]]> +

    As a full example, given the following Jingle File Transfer content description:

    + + + text/plain + test.txt + 2015-07-26T21:46:00 + 6144 + 552da749930852c69ae5d2141d3766b1 + + +]]> +

    The equivalent SDP would be:

    + +     + + + +

    Once a file has been successfully received, the recipient MAY send a Jingle session-info message indicating receipt of the complete file, which consists of a <received/> element qualified by the 'urn:xmpp:jingle:apps:file-transfer:4' namespace. The <received/> element SHOULD contain 'creator' and 'name' attributes sufficient to identify the content that was received.

    + - + + +]]> +     + + +

    At any time during the lifetime of the file transfer session, the File Sender can communicate the checksum of the file to the File Receiver.

    +

    This can be done in the session-initiate message if the File Sender already knows the checksum, as shown above in Example 3.

    +

    After the session-initiate message, this can also be done by sending a session-info message containing a <checksum/> element qualified by the 'urn:xmpp:jingle:apps:file-transfer:4' namespace. The <checksum/> element SHOULD contain 'creator' and 'name' attributes sufficient to identitfy the content the checksum belongs to. Additionally, the <checksum/> element MUST contain a <file/> element which MUST contain at least one <hash/> element qualified by the 'urn:xmpp:hashes:1' namespace. Each <hash/> element contains a checksum of the file data produced in accordance with the hashing function specified by the 'algo' attribute, which MUST be one of the functions listed in the &ianahashes;.

    + + + - 552da749930852c69ae5d2141d3766b1 + 552da749930852c69ae5d2141d3766b1 - - ]]> -

    If the initiator wishes to communicate only the hashing algorithm as the beginning of the session (e.g., because it has not yet calculated the checksum), it can send an empty <hash/> element (without a checksum in the XML character data as shown in the previous examples) in the session-initiate message; this enables the recipient to check the file during the transfer session (which can be helpful in the case of transfers that are truncated or fail mid-stream).

    - ]]> +

    If a ranged transfer was requested, the <file/> element inside the <checksum/> element MAY include a <range/> element specifying the offset and length of the requested range, which in turn includes <hash/> element(s) with hashes of the data that was transferred for that range.

    + + + + + + 4df403604e746f15062ffdbc29367886 + + + + +]]> +

    If the initiator wishes to communicate only the hashing algorithm at the beginning of the session (e.g., because it has not yet calculated the checksum), it can send an empty <hash/> element (without a checksum in the XML character data as shown in the previous examples) in the session-initiate message; this enables the recipient to check the file during the transfer session (which can be helpful in the case of transfers that are truncated or fail mid-stream).

    + - + 1969-07-21T02:56:15Z @@ -418,146 +815,82 @@ Initiator Responder text/plain test.txt - 1022 + 6144 - - ]]> +]]> +     - -

    As in XEP-0096, a transfer can include only part of a file (e.g., to restart delivery of a truncated transfer session at a point other than the start of the file). This is done using the <range/> element from XEP-0096. The usage is illustrated in the following examples.

    -

    Let us imagine that the parties negotiate a file transfer session using, say, In-Band Bytestreams. During the transfer, the recipient goes offline unexpectedly and IBB stanzas from the sender to the recipient begin to bounce. When the recipient comes back online, the sender could initiate a new Jingle session and specify that it wants to send all chunks after byte 270336 (which might be the 66th chunk of size 4096).

    - + +

    If a requested file cannot be found (or the requester does not have permission to request or know about the existence of the file in question), then the File Sender SHOULD send either a session-terminate or content-reject action in response to the session-initiate or content-add request, and SHOULD include a Jingle reason of <failed-application/> and MAY include an application specific reason of a <file-not-available/> element qualified by the 'urn:xmpp:jingle:apps:file-transfer:errors:0' namespace.

    + - - - - - 552da749930852c69ae5d2141d3766b1 - - - - - - - + + + + + - - ]]> +]]>     + + + +

    There are several situations where a File Receiver might wish to abort a transfer due to an excess of file data, for example:

    +
      +
    • The File Receiver has reached a file system storage quota or other hard limit that prevents continuing to receive file data.
      • +
    • The File Sender has continued sending data past the initially specified size of the file.
      • +
    +

    In such cases, the File Receiver MAY abort the transfer by sending a Jingle session-terminate (or content-remove as appropriate) which includes a Jingle reason of <media-error/> and MAY include an application specific reason of a <file-too-large/> element qualified by the 'urn:xmpp:jingle:apps:file-transfer:errors:0' namespace.

    + + + + + + + + +]]> +

    To prevent denial of service and other attacks, the File Receiver is fully within its rights to drop received data or not send a session-terminate message.

    +     - -

    Jingle file transfer uses only a few of the actions defined in XEP-0166. Jingle usage is summarized in the following table.

    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
    ActionUse
    content-acceptUnused
    content-addUnused
    content-modifyUnused
    content-rejectUnused
    content-removeUnused
    description-infoUnused
    security-infoUnused
    session-acceptAccepting a file offer or request
    session-infoCommunicating the file hash
    session-initiateInitiating a file offer or request
    session-terminateEnding a file transfer session
    transport-acceptAccepting fallback from S5B to IBB
    transport-infoUsed in SOCKS5 Bytestreams
    transport-rejectRejecting fallback from S5B to IBB
    transport-replaceFallback from S5B to IBB
    -     -

    All implementations MUST support the Jingle In-Band Bytestreams Transport Method (XEP-0261) as a reliable method of last resort. An implementation SHOULD support other transport methods as well, especially the Jingle SOCKS5 Bytestreams Transport Method (XEP-0260).

    +

    All implementations MUST support the Jingle In-Band Bytestreams Transport Method (XEP-0261) as a reliable method of last resort. An implementation SHOULD support other transport methods as well, especially ICE-TCP (RFC 6455) and the Jingle SOCKS5 Bytestreams Transport Method (XEP-0260).

    An application MAY present transport methods in any order, except that the Jingle In-Band Bytestreams Transport Method MUST be the lowest preference.

    @@ -570,17 +903,16 @@ Initiator Responder

    To advertise its support for the Jingle File Transfer, when replying to service discovery information ("disco#info") requests an entity MUST return URNs for any version of this protocol that the entity supports -- e.g., "urn:xmpp:jingle:apps:file-transfer:4" for this version &VNOTE;.

    - - ]]> +]]> @@ -588,16 +920,15 @@ Initiator Responder - - ]]> -

    As noted, if an application supports exchange of multiple files, it MUST advertise a service discovery feature of "urn:xmpp:jingle:apps:file-transfer:multi".

    +]]>

    In order for an application to determine whether an entity supports this protocol, where possible it SHOULD use the dynamic, presence-based profile of service discovery defined in &xep0115;. However, if an application has not received entity capabilities information from an entity, it SHOULD use explicit service discovery instead.

     -

    It is RECOMMENDED for implementations to use the strongest hashing algorithm available to both parties. See XEP-0300 for further discussion.

    +

    Caution needs to be exercised when using the <name/> of a file offer or request to control any interaction with a file system. For example, a malicious user could request a file with <name>/etc/passwd</name> or include file system specific control patterns such as <name>../../private.txt</name> to try and access a sensitive file outside of the set of files intended to be shared. Or a malicious user could offer a file named "/etc/passwd" to try and trick the receiver into overwriting that or other sensitive files. Therefore, implementations SHOULD escape any file system path separators in the <name/> before using that value in any file system calls.

    +

    It is RECOMMENDED for implementations to use the strongest hashing algorithm available to both parties. See XEP-0300 for further discussion.

    In order to secure the data stream, implementations SHOULD use encryption methods appropriate to the transport method being used. For example, end-to-end encryption can be negotiated over either SOCKS5 Bytestreams or In-Band Bytestreams as described in XEP-0260 and XEP-0261.

    -

    Refer to XEP-0047, XEP-0065, XEP-0096, XEP-0260, and XEP-0261 for related security considerations.

    +

    Refer to XEP-0047, XEP-0065, XEP-0096, XEP-0176, XEP-0260, XEP-0261, and RFC 6455 for related security considerations.

     @@ -616,17 +947,6 @@ Initiator Responder &NSVER; - -

    The service discovery feature for advertising support for exchange of multiple files is "urn:xmpp:jingle:apps:file-transfer:multi".

    -

    The registry submission is as follows.

    - - urn:xmpp:jingle:apps:file-transfer:multi - Signals support for exchange of multiple files. - XEP-0234 - - ]]> -

    The XMPP Registrar shall include "file-transfer" in its registry of Jingle application formats. The registry submission is as follows:

    Jingle sessions for the transfer of a file streaming XEP-0234 - - ]]> +]]>     - + - + - + - + + + + + + + + + + + + + + + + - - + + + + + + + - - - - - - - - - - - + + + + + + - - ]]> +]]> +     + + + + + + + + + +]]> +     -

    Thanks to Diana Cionoiu, Olivier Crête, Viktor Fast, Philipp Hancke, Waqas Hussain, Justin Karneges, Steffen Larsen, Yann Leboulanger, Marcus Lundblad, Robert McQueen, Joe Maissel, Glenn Maynard, Ali Sabil, Sjoerd Simons, Lance Stout, Will Thompson, Matthew Wild, and Jiří Zárevúcky for their feedback.

    +

    Thanks to Diana Cionoiu, Olivier Crête, Viktor Fast, Philipp Hancke, Waqas Hussain, Justin Karneges, Steffen Larsen, Yann Leboulanger, Marcus Lundblad, Robert McQueen, Joe Maissel, Glenn Maynard, Ali Sabil, Sjoerd Simons, Will Thompson, Matthew Wild, and Jiří Zárevúcky for their feedback.

     diff --git a/xep.ent b/xep.ent index 073fd2b7..3ace61de 100644 --- a/xep.ent +++ b/xep.ent @@ -580,6 +580,7 @@ THE SOFTWARE. RFC 4848 RFC 4848: Domain-Based Application Service Location Using URIs and the Dynamic Delegation Discovery Service (DDDS) <http://tools.ietf.org/html/rfc4848>." > RFC 4854 RFC 4854: A Uniform Resource Name (URN) Namespace for Extensions to the Extensible Messaging and Presence Protocol (XMPP) <http://tools.ietf.org/html/rfc4854>." > RFC 4880 RFC 4880: OpenPGP Message Format <http://tools.ietf.org/html/rfc4880>." > +RFC 4975 RFC 4975: The Message Session Relay Protocol (MSRP) <http://tools.ietf.org/html/rfc4975>." > RFC 4949 RFC 4949: Internet Security Glossary, Version 2 <http://tools.ietf.org/html/rfc4949>." > RFC 5023 RFC 5023: The Atom Publishing Protocol <http://tools.ietf.org/html/rfc5023>." > RFC 5054 RFC 5054: Using the Secure Remote Password (SRP) Protocol for TLS Authentication <http://tools.ietf.org/html/rfc5054>." > From fb6a27747b2adcbb6f24ed5d8ff216263dcf0568 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Sun, 13 Sep 2015 21:40:30 +0200 Subject: [PATCH 068/106] Fix description of XEP-65's element --- xep-0065.xml | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/xep-0065.xml b/xep-0065.xml index c2de88c6..36e716ef 100644 --- a/xep-0065.xml +++ b/xep-0065.xml @@ -29,6 +29,16 @@ &linuxwolf; &stpeter; &infiniti; + + 1.8.1 + 2015-09-17 + fs + +
      +
    • Fixed description of activate element in 9.4.
      • +
    +     +     1.8 2011-04-20 @@ -776,7 +786,7 @@ DATA = (payload)

    The "jid" attribute specifies the JID of the StreamHost. This attribute MUST be present, and MUST be a valid JID for communication over XMPP.

    -

    The <activate/> element is sent from the Requester to the Proxy in order to formally start the bytestream. This element is always empty and has no defined attributes.

    +

    The <activate/> element is sent from the Requester to the Proxy in order to formally start the bytestream. This element has no defined attributes and its XML character data specifies the JID of the target.

    The <udpsuccess/> element is sent from the StreamHost to the Target or Requester to indicate that the StreamHost has received a UDP initialization packet.

    From 61dd46602347ccf6433a4f1a958e7febd55532f0 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 7 Sep 2015 13:48:57 -0500 Subject: [PATCH 069/106] Add profiles / partial sync to entity versioning --- inbox/entityversioning.xml | 510 ++++++++++++++++++++++--------------- 1 file changed, 307 insertions(+), 203 deletions(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index 5ac781ac..07b7097e 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -8,10 +8,9 @@
    Entity Versioning - A method by which rosters and disco items may be versioned so that servers - will not need to send the entire list if it has not been modified, saving - bandwidth and time during session initialization with minimal state being - stored by the server and client. + A method by which lists of items may be versioned so that servers will not + need to send the entire list if it has not been modified, saving bandwidth + and time with minimal state being stored by the server and client. &LEGALNOTICE; xxxx @@ -37,6 +36,12 @@ Keen dkeen@atlassian.com + + 0.0.2 + 2015-09-17 + ssw +

    Add profiles / parcial sync.

     +     0.0.1 2015-08-25 @@ -48,17 +53,17 @@

    This problem of "downloading the world" (downloading the entire roster - every time a session is initialized) was partially addressed by &xep0237; - which was later merged into &rfc6121; §2.6. While this solved the problem - for the roster, it didn't account for other entities (eg. disco items). + every time a session is initialized or receiving an entire disco items + response every time a MUC list is queried, etc.) was partially addressed by + &xep0237; which was later merged into &rfc6121; §2.6. While this solved + the problem for the roster, it didn't account for other entities. Furthermore, roster versioning requires that the server maintain a great deal of state (roster items which should be pushed for each entity on reconnect, or monotonically increasing counters, etc.) which can be difficult to store or synchronize in a large, distributed system. This XEP - defines a method by which the roster and entities other than the roster can - be versioned and cached, and which is optimized for distributed systems - with large enity lists (but works equally well on small, single server - deployments). + defines a method by which generic entity lists can be versioned and cached, + and which is optimized for distributed systems with large entity lists (but + works equally well on small, single server deployments).

    @@ -96,8 +101,8 @@
    Any abstract object which may be versioned (eg. rooms, users).
    Version Token
    - A generally short, case sensitive string which represents an entity and - changes if that entity changes. + A short, case sensitive string which represents an entity and changes if + that entity changes.
    @@ -144,55 +149,103 @@ + +

    + Because entity versioning is designed to be a generic system for syncing + any sort of list in XMPP, and the format and requirements of various entity + lists may vary greatly, no specific wire format is defined in this + specification. Instead, the specifics for various lists will be left up to + separate XEPs which will define entity versioning "profiles" which must be + registered with the XMPP registrar. These profiles will define exactly how + version tokens are represented in the specific list format for which they + wish to use entity versioning. The rest of this document will provide + details about entity versioning which will be common to all entity + versioning profiles and do not need to be redefined in EV profile XEPs. It + will also define an EV profile for fetching the roster. +

    +

    + The roster entity versioning profile which is used as an example throughout + this document will use the namespace 'urn:xmpp:entityver:profile:roster:0' + as described in the XMPP Registrar + Considerations section of this document. +

    +     + -

    - If a server supports entity versioning, it MUST inform the connecting - client when returning stream features during the stream negotiation - process. This is done by including a <ver/> element, qualified by - the 'urn:xmpp:entityver:0' namespace. At the latest, this SHOULD - be done when informing a client that resource binding is required. For - example: -

    - + If a server supports entity versioning, it MUST inform the connecting + client when returning stream features during the stream negotiation + process. This is done by including a <ver/> element, qualified by the + 'urn:xmpp:entityver:0' namespace with child <profile> nodes for each + supported entity versioning profile. At the latest, this SHOULD be done + when informing a client that resource binding is required. For example if + the server only supports versioning of rosters it might return: +

    + - + + + - ]]> + ]]>

    The entity versioning stream feature is merely informative and therefore is never mandatory-to-negotiate.

    +

    + Clients, servers, and other entities that support &xep0030; and entity + versioning must respond to service discovery requests with a feature of + 'urn:xmpp:entityver:0' and with a feature for each EV profile supported by + the responding entity as described in the relavant specifications. Eg. a + response from a server that supports roster versioning for the requesting + entity might look like the following: + + + + + + + ]]> +

    - -

    - Version tokens are short case-sensitive strings which are generated by the - server. Their format is not defined in this spec, but a recommendation may - be found in the Implementation Notes. Version - tokens are akin to a weakly-validated etag for the entity in question. -

    -

    - Servers that implement this protocol must assign such a version token to - each entity that is controlled by the server. The server MUST then update - this version every time any mutable property of the entity changes (eg. - when the subscription status of a user changes). The server MAY choose to - update this token at any time (to force the clients to invalidate their - cached representation fo the object). This version token MUST then be - included with every object representation of that entity sent down in the - stream. This is done by including a sub-node called "version" qualified - by the entity versioning XML namespace defined in this document. - Similarly, clients MAY also add version nodes for each version token they - possess to the request for a list (not specifying a version token will - force the server to send information on that entity to the client). If a - server sends up a list of version tokens, the server MUST then check to - see if those tokens correspond to any entity which it knows about, and - not send down any entities with matching version tokens in the response. -

    -

    For example, a roster request might look like:

    - + +

    + Version tokens are short case-sensitive strings which are generated by + the server. Their format is not defined in this spec, but a + recommendation may be found in the Implementation + Notes. Version tokens are akin to a weakly-validated etag for the + entity in question. +

    +

    + Servers that implement this protocol must assign such a version token to + each entity that is controlled by the server. The server SHOULD then + update this version every time any mutable property of the entity changes + (eg. when the subscription status of a user changes). The server MAY + choose to update this token at any time (to force the clients to + invalidate their cached representation fo the object). This version token + MUST then be included with every object representation of that entity + sent down in the stream. This is done by including a sub-node called + "version" qualified by the entity versioning XML namespace defined in + this document. Similarly, clients MAY also add version nodes for each + version token they possess to the request for a list (not specifying a + version token will force the server to send information on that entity to + the client). If a server sends up a list of version tokens, the server + MUST then check to see if those tokens correspond to any entity which it + knows about, and not send down any entities with matching version tokens + in the response. +

    +

    + For example, a versioned roster request might look like this: + @@ -213,64 +266,25 @@ - ]]> + ]]> +

    Note that in this case there may be three roster items total (and the client only knows about two of them), or there may be two total roster items and the server is informing the client about a change to "bill@shakespeare.lit". Version tokens MUST also be present in roster pushes: -

    - + - - - XWE4MUUP - - - - ]]> -

    A disco request for rooms (as defined in &xep0045;) might look like:

    - - - - 25P2A7H8 - - - 4OLGSVNY + + + XWE4MUUP - - - - - - VIZSVF0D - - - - ]]> -

    - In this example coven@chat.shakespeare.lit has been modified (eg. the - room name might have been changed), but inverness@chat.shakespeare.lit - has not changed, therefore no update is sent down. -

    -

    - Clients that implement this protocol SHOULD then cache the entity in - question when a version token is received. -

    - + ]]>     +

    +

    When a client syncs with the server and indicates that it has a version @@ -278,127 +292,154 @@ the server wants to remove an entity from the clients cache for any other reason), the server MUST reply with an empty <version/> node. When the client receives such an empty version node it SHOULD purge the entity - from its cache. For example, the following exchange would trigger the - removal of 'inverness@chat.shakespeare.lit' from the cached MUC list: - - - - - 25P2A7H8 - - - 4OLGSVNY - - - - - - - - - - - - - ]]> + from its cache. For example, the following would remove the roster item + 'bill@shakespeare.lit' from the cache: + + + + + + + + ]]>

    - If the client receives an indication that it should delete an item from a - list by any other means (eg. via a roster push), it SHOULD remove the - version token associated with that entity from its cache. + Roster pushes that indicate a deleted item MUST also remove the version + from the cache (and need not contain an empty <version/> element).

    -     - -

    - While the version token approach to caching does not require a great deal - of state to be stored on the client or the server, it does require a lot - more information to be sent by the client when requesting a list of - entities. For a very large list which is not likely to have changed, it may - be useful know in advance if the roster has changed or not (so that we can - avoid sending the large request entirely). To do this, we can request an - aggregate version token from the server. This aggregate token is calculated - by constructing a string of comma separated "JID:version" pairs sorted in - byte-wise order (because the JID:version pair is constructed before - sorting, if two items in the list have the same JID they can still be - sorted by the version token), and taking the MD5 hash of the constructed - string. For example, if the server is calculating the aggregate version - token for a roster, it might end up with the following string: -

    - +

    + For very large groups fetching an entire list may not be practical or + necessary. For example, one might imagine a large corporation with a + shared roster that is too large for its version tokens to be sent up to + the server on every sync, or even to download fully the first time. To + solve this, servers MAY choose to send down only a part of an entity list + in response to a query (unless the individual EV profile forbids partial + list sync). How servers choose what items to return is an implementation + detail that is out of the scope of this document. Some suggestions may be + found in the Implementation Notes. On subsequent + requests for the entity list, the server MAY choose to return more + entities (eg. based on changes in its internal selection criteria), + however it MUST NOT invalidate cached entities unless they have actually + been removed from the list. +

    +

    + XEPs defining entity versioning profiles MUST include a section to + indicate if partial sync is supported, and if so, how it will be + indicated to the client (and how the client can request a full list). If + no mechanism is specified, this is done by adding a boolean "full_list" + attribute to the request, eg. a roster request for a partial list looks + like: + + + + + 25P2A7H8 + + + VIZSVF0D + + + + + + + + 9ZFZXVP9 + + + + ]]> +

    +

    + When making a request for a partial list, clients do not need to send up + every entity in their cache. Instead they MAY send up just those entities + for which they wish to check for updates. The server MUST then respond + with any updates for those entities, and MAY also add other entities to + the list if desired. If the client requests a partial list but does not + indicate that it has anything in its cache, what entities to return (if + any) is left up to the server implementation. +

    + + +

    + While the version token approach to caching does not require a great deal + of state to be stored on the client or the server, it does require a lot + more information to be sent by the client when requesting a list of + entities. For a very large list which is not likely to have changed, it + may be useful know in advance if the roster has changed or not (so that + we can avoid sending the large request entirely). To do this, we can + request an aggregate version token from the server. This aggregate token + is calculated by constructing a string of comma separated "ID:version" + pairs sorted in byte-wise order (because the ID:version pair is + constructed before sorting, if two items in the list have the same ID + they can still be sorted by the version token), and taking the MD5 hash + of the constructed string. The ID in the pair is any ID or key that + identifies the entity as defined in its profile (eg. a JID for roster + items and most other entities). For example, if the server is calculating + the aggregate version token for a roster, it might end up with the + following string: +

    + -

    - Which results in the aggregate token: -

    - +

    + Which results in the aggregate token: +

    + -

    - The actual request is an IQ sent to the server, or entity handling the - versioned list which contains a query that specifies the namespace of the - list we want to fetch. Eg. to fetch the aggregate token for the roster one - would query the server with the type set to the `jabber:iq:roster` - namespace: -

    - +

    + The actual request is an IQ sent to the server, or entity handling the + versioned list which contains a query that specifies the namespace of the + list we want to fetch. Eg. to fetch the aggregate token for the roster + one would query the server with the query's XMLNS set to + 'urn:xmpp:entityver:profile:roster:0': +

    + - + - + 0514fc90e6c7981b06bbb2173bb8ef03 - ]]> -

    - Similarly, to fetch the aggregate token for a list of MUC rooms, one would - query the MUC component directly with the type set to the 'disco#items' - namespace: -

    - - - - - - - - - 32151d1d01440d5536a7f106afd3f4d8 - - - ]]> -

    - Because aggregate tokens are OPTIONAL to implement, clients MUST fall back - to a normal request if any error is returned in response to an aggregate - token IQ. -

    -

    - If an aggregate token is requested for a list that may contain more than - one type of entity (eg. MUC rooms and pubsub nodes that live on the same - component), then the server MUST return the aggregate token constructed - with the entire list (rooms and pubsub nodes). -

    -

    - Clients are also NOT REQUIRED to check aggregate tokens. However, clients - MAY wish to check aggregate tokens before making a roster or MUC request - when the cached roster or MUC list is very large. When to check aggregate - tokens (if at all) is left up to the implementation. -

    + ]]>     +

    + Because aggregate tokens are OPTIONAL to implement, clients MUST fall + back to making a normal list request if any error is returned in response + to an aggregate token IQ. +

    +

    + If an aggregate token is requested for a list that may contain more than + one type of entity (eg. MUC rooms and pubsub nodes that live on the same + component), then the server MUST return the aggregate token constructed + with the entire list (rooms and pubsub nodes). +

    +

    + Because aggregate tokens are calculated for the entire list as seen by + the client or server, they will never match if partial lists have been + downloaded by the client. +

    +

    + Clients are also NOT REQUIRED to check aggregate tokens. However, clients + MAY wish to check aggregate tokens before making a roster or MUC request + when the cached roster or MUC list is very large. When to check aggregate + tokens (if at all) is left up to the implementation. +

    +     -

    Version tokens may not provide enough collision resistance across versioned @@ -421,6 +462,22 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8 version token MAY be used as a weakly validated ETag for any API requests for that entity.

    +

    + Servers following this specification may choose to send down partial entity + lists in response to queries. For the case of rosters one or more of the + following may be returned to the requesting entity during the initial + roster sync: +

      +
    • + Users that are grouped with the requester in some way. Eg. for a + company with a large shared roster which places the requesting client + in the "Marketing Department" group, the server may wish to return + roster items that also share that group. +
      • +
    • Users whom the requester has contacted recently or frequently.
      • +
    • Users that should always be returned as part of server policy.
      • +
    +

    @@ -449,6 +506,53 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8 &xep0053;.

    + + &NSVER; + + +

    + The XMPP Registrar shall maintain a registry of entity versioning + profiles. All EV profile registrations shall be defined in separate + specifications (not in this document). Application types defined within + the XEP series MUST be registered with the XMPP Registrar, resulting in + protocol URNs of the form "urn:xmpp:entityver:profile:name:X" (where + "name" is the registered name of the profile and "X" is a non-negative + integer). +

    + ®PROCESS; + + The name of the entity versioning profile. + A natural-language summary of the profile. + + The document in which the original list definition is specified. + + + The document in which the EV profile for the list is specified (may be the + same as <listdev/>). + + + ]]> +     + +

    This specification defines the following entity versioning profile:

    +
      +
    • urn:xmpp:entityver:profile:roster:0
      • +
    +

    + Upon advancement of this specification from a status of Experimental to a + status of Draft, the ®ISTRAR; shall add the following definition to + the entity versioning profiles registry, as described in this document: + + Roster entity versioning + Allows versioning of entities in an XMPP roster. + RFC 6121 + TODO: Insert this document once it is assigned a number + + ]]> +

    +     From f60b582be3ef0883811233473d0d793f185855d6 Mon Sep 17 00:00:00 2001 From: Steve Kille Date: Tue, 1 Sep 2015 08:14:25 +0100 Subject: [PATCH 070/106] Update to reflect S5066 publication on the Web --- inbox/s2s-over-s5066.xml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/inbox/s2s-over-s5066.xml b/inbox/s2s-over-s5066.xml index 2dc99ade..85320483 100644 --- a/inbox/s2s-over-s5066.xml +++ b/inbox/s2s-over-s5066.xml @@ -121,7 +121,7 @@

    This specification uses STANAG 5066 Edition 3 "PROFILE FOR HF RADIO DATA COMMUNICATIONS" (December 2010).

    -

    STANAG 5066 is a NATO UNCLASSIFED (Releasable to the Public) document that may circulated freely. The author has requested permission form NATO to enable publishing of this document on the Web.

    +

    STANAG 5066 is a NATO UNCLASSIFED (Releasable to the Public) document that may circulated freely. It is available on http://nso.nato.int/nso/zPublic/stanags/CURRENT/5066Ed03.pdf

    @@ -131,5 +131,8 @@

    Kevin Smith provided useful comments on this specification.

    +

    + Dave Cridland asked NATO about STANAG 5066 publication, leading to its availability on the Web. +

    From cd2905327cf1ef58b8ac46879ac6fed61d3a5e58 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 17 Sep 2015 10:36:12 -0600 Subject: [PATCH 071/106] promoting s2s-over-s5066 to XEP-0365 --- inbox/s2s-over-s5066.xml | 138 --------------------------------------- xep-0365.xml | 138 +++++++++++++++++++++++++++++++++++++++ xep.ent | 1 + 3 files changed, 139 insertions(+), 138 deletions(-) delete mode 100644 inbox/s2s-over-s5066.xml create mode 100644 xep-0365.xml diff --git a/inbox/s2s-over-s5066.xml b/inbox/s2s-over-s5066.xml deleted file mode 100644 index 85320483..00000000 --- a/inbox/s2s-over-s5066.xml +++ /dev/null @@ -1,138 +0,0 @@ - - -%ents; -]> - - -
    - Server to Server communication over STANAG 5066 ARQ - - This specification defines operation over XMPP over the NATO STANAG 5066 data link service for point to point links (ARQ). This enables optimized XMPP performance over HF Radio (which STANAG 5066 was designed for) and over other data links using STANAG 5066. - - - - This XMPP Extension Protocol is copyright (c) 1999 - 2015 by the XMPP Standards Foundation (XSF). - Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation. - ## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. In no event shall the XMPP Standards Foundation or the authors of this Specification be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification. ## - In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising out of the use or inability to use the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages. - This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which may be found at <http://xmpp.org/extensions/ipr-policy.shtml> or obtained by writing to XSF, P.O. Box 1641, Denver, CO 80201 USA). - - xxxx - ProtoXEP - Standards Track - Standards - Council - - - XEP 0361 - STANAG 5066 - - - - S5066 - - Steve - Kille - steve.kille@isode.com - steve.kille@isode.com - - - 0.0.1 - 2015-08-19 - sek -

    First draft.

     -     -
    - -

    - This specification arose from requirements to operate over HF Radio, which has exceedingly high latency (sometimes minutes) low data rates (down to 75 bits/second) and poor reliablity. STANAG 5066 is a widely used link level protocol. Direct use of STANAG 5066 enables elimination of all extraneous end to end handshaking, which is important to optimize performance. It also enables use of STANAG 5066 flow control, which is important for reslience. - -

    -

    - The solution is based on XEP-0361 "Zero Handshake Server to Server Protocol" and requires peer configuration to be established according to XEP-0361. The data exchanged between the XMPP servers follows exactly what is specified in XEP-0316. The data is transferred using STANAG 5066 rather than using TCP. - -

    -     - -

    - This specification can be considered as a profile for server to server XMPP communication, to enable XMPP deployment over HF Radio using STANAG 5066. This profile MUST only be used where its use has been pre-agreed and configured for both participating servers. -

    -     - - -

    - An example scenario where this protocol is important is where two ships connected by HF Surface Wave communication only need to exchange XMPP messages. A reliable link (Soft Link) can be established using STANAG 5066 and XMPP communicated efficiently and reliably. - -

    -     - - - -

    - Because of potentially very low bandwidth sending server MAY perform traffic optimisation, such as selective removal of stanzas that are not adding sufficient value, like CSNs, or strip selected elements such as xhtml-im. -

    -

    - Applications sending data over STANAG 5066 need to be aware of increased delays and any application level timers (e.g., IQ response timers) need to be set accordingly. -

    -     - -

    - XEP-0361 transfer of data between a pair of XMPP servers is a byte stream flowing in each direction over TCP. There is no other protocol or hand shaking. - When carried instead over STANAG 5066, these byte streams are transmitted as a sequence of blocks transferred in order - Each block is an XML stanza, holding message, presence or iq. Essentially the stream is broken into blocks (stanzas) at natural boundaries XMPP boundaries, and then reassembled on reception into the original stream. - - -

    -

    - XEP-0198 Stream Management MUST not be used over STANAG 5066, as reliability of stanza transfer is handled by use of STANAG 5066. Application-layer keepalives and timeout detection such as white-space pings and XEP-0199 MUST NOT be used. -

    - -     - - -

    - Each stanza is transferred using the RCOP (Reliable Connection Oriented Protocol) defined in Section F.8 of Annex F of STANAG 5066. This reliably transfers the block of data to the destination. If a soft link needs to be established this will be done by the STANAG 5066 service. The XEP-0361 peer agreement is supported by a flow of stanzas in each direction being transferred by RCOP. The peer agreement will use this flow of stanzas to provide a service equivalent to the TCP connection or connections of XEP-0361. -

    -

    - STANAG 5066 SIS Delivery Confirmation MAY be set to NODE DELIVERY, as this gives optimum network performance. CLIENT DELIVERY MAY be used, which increases reliability as stanza delivery to the peer XMPP server is guaranteed and the sending server will receive acknowledgements equivalent to XEP-0198 support. In the event of delivery failure, the whole RCOP PDU (Stanza) MUST be retransmitted. -

    - -     - -

    - The peer addressing of the STANAG 5066 end points will be configured as part of the XEP-03161 peer agreement. -

    -

    - The STANAG 5066 SAP MAY be set to any mutually agreed value. It is RECOMMENDED that 2 is used. This is the standard SAP for RCOP. -

    -

    - The RCOP connection ID number will be set to a mutually agreed value. It is RECOMMENDED that 0 is used as the preferred value. -

    -     -     - - - -

    - Security Considerations of XEP-0361 apply. STANAG 5066 will frequently be employed in conjunction with link level crypto devices, which SHOULD be done when appropriate to provide data confidentiality. -

    - -     - -

    This specification uses STANAG 5066 Edition 3 "PROFILE FOR HF RADIO DATA COMMUNICATIONS" (December 2010).

    -

    STANAG 5066 is a NATO UNCLASSIFED (Releasable to the Public) document that may circulated freely. It is available on http://nso.nato.int/nso/zPublic/stanags/CURRENT/5066Ed03.pdf

    -     - - -

    - Curtis King designed and validated the approach documented in this XEP. -

    -

    - Kevin Smith provided useful comments on this specification. -

    -

    - Dave Cridland asked NATO about STANAG 5066 publication, leading to its availability on the Web. -

    -     -     diff --git a/xep-0365.xml b/xep-0365.xml new file mode 100644 index 00000000..2e5bd412 --- /dev/null +++ b/xep-0365.xml @@ -0,0 +1,138 @@ + + +%ents; +]> + + +
    + Server to Server communication over STANAG 5066 ARQ + + This specification defines operation over XMPP over the NATO STANAG 5066 data link service for point to point links (ARQ). This enables optimized XMPP performance over HF Radio (which STANAG 5066 was designed for) and over other data links using STANAG 5066. + + + &LEGALNOTICE; + 0365 + Experimental + Standards Track + Standards + Council + + XEP 0361 + STANAG 5066 + + + + S5066 + + Steve + Kille + steve.kille@isode.com + steve.kille@isode.com + + + 0.1 + 2015-09-17 + XEP Editor (mam) +

    Initial published version approved by the XMPP Council.

     +     + + 0.0.1 + 2015-08-19 + sek +

    First draft.

     +     +
    + +

    + This specification arose from requirements to operate over HF Radio, which has exceedingly high latency (sometimes minutes) low data rates (down to 75 bits/second) and poor reliablity. STANAG 5066 STANAG 5066 C3B (EDITION 3): PROFILE FOR HF RADIO DATA COMMUNICATIONS <http://nso.nato.int/nso/zPublic/stanags/CURRENT/5066Ed03.pdf>. + is a widely used link level protocol. Direct use of STANAG 5066 enables elimination of all extraneous end to end handshaking, which is important to optimize performance. It also enables use of STANAG 5066 flow control, which is important for reslience. + +

    +

    + The solution is based on &xep0361; and requires peer configuration to be established according to XEP-0361. The data exchanged between the XMPP servers follows exactly what is specified in XEP-0361. The data is transferred using STANAG 5066 rather than using TCP. + +

    +     + +

    + This specification can be considered as a profile for server to server XMPP communication, to enable XMPP deployment over HF Radio using STANAG 5066. This profile MUST only be used where its use has been pre-agreed and configured for both participating servers. +

    +     + + +

    + An example scenario where this protocol is important is where two ships connected by HF Surface Wave communication only need to exchange XMPP messages. A reliable link (Soft Link) can be established using STANAG 5066 and XMPP communicated efficiently and reliably. + +

    +     + + + +

    + Because of potentially very low bandwidth sending server MAY perform traffic optimisation, such as selective removal of stanzas that are not adding sufficient value, like CSNs, or strip selected elements such as xhtml-im. +

    +

    + Applications sending data over STANAG 5066 need to be aware of increased delays and any application level timers (e.g., IQ response timers) need to be set accordingly. +

    +     + +

    + XEP-0361 transfer of data between a pair of XMPP servers is a byte stream flowing in each direction over TCP. There is no other protocol or hand shaking. + When carried instead over STANAG 5066, these byte streams are transmitted as a sequence of blocks transferred in order + Each block is an XML stanza, holding message, presence or iq. Essentially the stream is broken into blocks (stanzas) at natural boundaries XMPP boundaries, and then reassembled on reception into the original stream. + + +

    +

    + &xep0198; MUST not be used over STANAG 5066, as reliability of stanza transfer is handled by use of STANAG 5066. Application-layer keepalives and timeout detection such as white-space pings and &xep0199; MUST NOT be used. +

    + +     + + +

    + Each stanza is transferred using the RCOP (Reliable Connection Oriented Protocol) defined in Section F.8 of Annex F of STANAG 5066. This reliably transfers the block of data to the destination. If a soft link needs to be established this will be done by the STANAG 5066 service. The XEP-0361 peer agreement is supported by a flow of stanzas in each direction being transferred by RCOP. The peer agreement will use this flow of stanzas to provide a service equivalent to the TCP connection or connections of XEP-0361. +

    +

    + STANAG 5066 SIS Delivery Confirmation MAY be set to NODE DELIVERY, as this gives optimum network performance. CLIENT DELIVERY MAY be used, which increases reliability as stanza delivery to the peer XMPP server is guaranteed and the sending server will receive acknowledgements equivalent to XEP-0361 support. In the event of delivery failure, the whole RCOP PDU (Stanza) MUST be retransmitted. +

    + +     + +

    + The peer addressing of the STANAG 5066 end points will be configured as part of the XEP-0361 peer agreement. +

    +

    + The STANAG 5066 SAP MAY be set to any mutually agreed value. It is RECOMMENDED that 2 is used. This is the standard SAP for RCOP. +

    +

    + The RCOP connection ID number will be set to a mutually agreed value. It is RECOMMENDED that 0 is used as the preferred value. +

    +     +     + + + +

    + Security Considerations of XEP-0361 apply. STANAG 5066 will frequently be employed in conjunction with link level crypto devices, which SHOULD be done when appropriate to provide data confidentiality. +

    + +     + +

    This specification uses STANAG 5066 Edition 3 "PROFILE FOR HF RADIO DATA COMMUNICATIONS" (December 2010).

    +

    STANAG 5066 is a NATO UNCLASSIFED (Releasable to the Public) document that may circulated freely. It is available on http://nso.nato.int/nso/zPublic/stanags/CURRENT/5066Ed03.pdf.

    +     + + +

    + Curtis King designed and validated the approach documented in this XEP. +

    +

    + Kevin Smith provided useful comments on this specification. +

    +

    + Dave Cridland asked NATO about STANAG 5066 publication, leading to its availability on the Web. +

    +     +     diff --git a/xep.ent b/xep.ent index 3ace61de..481a2488 100644 --- a/xep.ent +++ b/xep.ent @@ -1348,3 +1348,4 @@ IANA Service Location Protocol, Version 2 (SLPv2) Templates Raft over XMPP (XEP-0362) XEP-0362: Raft over XMPP <http://xmpp.org/extensions/xep-0362.html>." > HTTP File Upload (XEP-0363) XEP-0363: HTTP File Upload <http://xmpp.org/extensions/xep-0363.html>." > Current Off-the-Record Messaging Usage (XEP-0364) XEP-0364: Current Off-the-Record Messaging Usage <http://xmpp.org/extensions/xep-0364.html>." > +Server to Server communication over STANAG 5066 ARQ (XEP-0365) XEP-0365: Server to Server communication over STANAG 5066 ARQ <http://xmpp.org/extensions/xep-0365.html>." > From 353f30abfef5ab403a4e1141ebf33f3309dd497d Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Thu, 17 Sep 2015 12:57:37 -0500 Subject: [PATCH 072/106] Fix escaping in entity versioning cdata --- inbox/entityversioning.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index 07b7097e..b22c9e88 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -529,7 +529,7 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8 The document in which the EV profile for the list is specified (may be the - same as <listdev/>). + same as ). ]]> From 611eb46c65ecc8071426636a3a210575506dc743 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Fri, 11 Sep 2015 19:13:11 +0200 Subject: [PATCH 073/106] Refactored SIDs client-id from attribute to element. --- xep-0359.xml | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/xep-0359.xml b/xep-0359.xml index 2ae633fe..bcee848c 100644 --- a/xep-0359.xml +++ b/xep-0359.xml @@ -19,13 +19,25 @@ - NOT_YET_ASSIGNED (suggested: sid) + stanza-id Florian Schmaus flo@geekplace.eu flo@geekplace.eu + + 0.2 + 2015-09-18 + fs + +
      +
    • Refactored client-id from attribute to element.
      • +
    • Set short name to 'stanza-id'.
      • +
    • Clarified that SID elements must not have additional content.
      • +
    +     +     0.1 2015-07-14 @@ -67,17 +79,17 @@

    - Some use cases require the client to generate the stanza ID. In this case, the client MUST use 'client-id' as attribute name for the ID. + Some use cases require the client to generate the stanza ID. In this case, the client MUST use the 'client-id' element.

    Typical body text - + ]]>

    - The server MAY add an 'id' attribute to the stanza-id element. In that case, it MUST preserve the content of the 'client-id' attribute. + The server or component MAY add a stanza-id element. In that case, it MUST preserve the content of the 'client-id' element.

    Typical body text + by='room@muc.example.org'/> + ]]> -

    - Otherwise, if the server does not override the ID, it MAY omit the 'client-id' attribute. -

      -
    1. XMPP entities, which are routing stanzas, MUST NOT strip stanza-id elements from message stanzas.
      2. -
    2. The values of the 'id' (and 'client-id') attribute SHOULD be unpredictable.
      3. -
    3. Stanza ID generating entities, which encounter a <stanza-id/> element where 'id' (and 'client-id') is already set and where the 'by' attribute matches their own JID, MUST ignore the existing value of 'id' and override it.
      4. +
    4. XMPP entities, which are routing stanzas, MUST NOT strip any elements qualified by the 'urn:xmpp:sid:0' namespace from message stanzas. They SHOULD however ensure that those elements contain only the attributes defined herein, and take approribate countermeasures if this is not the case (e.g. removing those attributes).
      5. +
    5. The values of the 'id' attribute SHOULD be unpredictable.
      6. +
    6. Stanza ID generating entities, which encounter a <stanza-id/> element where 'id' is already set and where the 'by' attribute matches their own JID, MUST ignore the existing value of 'id' and override it.
    7. Stanzas MUST posses, in the direct child level of the stanza, at most one 'stanza-id' extension element with the same JID as value of the 'by' attribute.
      8. -
    8. Stanza ID extension elements which have a 'id' attribute MUST also have the 'by' attribute set.
      9. -
    9. Stanza ID extension elements MUST either posses a 'id' attribute, a 'client-id' attribute or both.
      10. +
    10. Every <stanza-id> extension element MUST have the 'id' attribute and the 'by' attribute set.
      11. +
    11. Every <stanza-id> and <client-id> extension element MUST always posses an 'id' attribute and MUST NOT have any child elements or text content.
    12. The value of the 'by' attribute MUST be a normalized JID as defined in &rfc6122;
     -

    The value of the 'id' attribute should not provide any further information besides the opaque ID itself. Entities observing the value MUST NOT be able to infer any information from it, e.g. the size of the message archive. The value of 'id' and 'client-id' MUST be considered as non-secret values.

    +

    The value of the 'id' attribute should not provide any further information besides the opaque ID itself. Entities observing the value MUST NOT be able to infer any information from it, e.g. the size of the message archive. The value of 'id' MUST be considered as non-secret values.

    This document requires no interaction with &IANA;.

    From f8890be4ad9a437ebab3475d17250b9f0a1d30fc Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Mon, 21 Sep 2015 09:42:28 -0600 Subject: [PATCH 074/106] XEP-0313 v0.4 - See Revision --- xep-0313.xml | 70 +++++++++++++++++++++++++++------------------------- 1 file changed, 36 insertions(+), 34 deletions(-) diff --git a/xep-0313.xml b/xep-0313.xml index 1b9259d5..174eb58e 100644 --- a/xep-0313.xml +++ b/xep-0313.xml @@ -29,10 +29,12 @@ &ksmith; 0.4 - 2015-01-23 + 2015-09-21 ks/mw -

    Switch the sentinel message back to the iq result. - Various small fixes to the document.

     + +

    Switch the sentinel message back to the iq result.

    +

    Various small fixes to the document.

    +     0.3 @@ -157,12 +159,12 @@ allows clients to accurately page through messages.

    - + ]]> - + @@ -178,13 +180,13 @@

    When querying a pubsub node's archive, the 'node' attribute is added to the <query> element.

    - + ]]>

    By default all messages match a query, and filters are used to request a subset of the archived messages. Filters are specified in a &xep0004; data form included with the query. The hidden FORM_TYPE field - MUST be set to this protocol's namespace, 'urn:xmpp:mam:0'. Three further fields are defined by this + MUST be set to this protocol's namespace, 'urn:xmpp:mam:1'. Three further fields are defined by this XEP and MUST be supported by servers, though all of them are optional for the client. These fields are:

    • start
      • @@ -203,10 +205,10 @@ regardless of the to/from addresses on each message.

      - + - urn:xmpp:mam:0 + urn:xmpp:mam:1 juliet@capulet.lit @@ -235,10 +237,10 @@ date/time of the most recent message stored in the archive.

      - + - urn:xmpp:mam:0 + urn:xmpp:mam:1 2010-06-07T00:00:00Z @@ -252,10 +254,10 @@ ]]> - + - urn:xmpp:mam:0 + urn:xmpp:mam:1 2010-08-07T00:00:00Z @@ -273,10 +275,10 @@ stanzas stored in the archive.

      - + - urn:xmpp:mam:0 + urn:xmpp:mam:1 2010-08-07T00:00:00Z @@ -294,7 +296,7 @@ - + 28482-98726-73623 09af3-cc343-b409f @@ -317,9 +319,9 @@ from the previous query.

      - + - urn:xmpp:mam:0> + urn:xmpp:mam:1> 2010-08-07T00:00:00Z @@ -334,7 +336,7 @@ - + 23452-4534-1 390-2342-22 @@ -349,14 +351,14 @@

      In order for the client find out about additional fields the server might support, it can send an iq stanza of type='get' addressed to the archive like this:

      - +       - + - urn:xmpp:mam:0 + urn:xmpp:mam:1 @@ -370,10 +372,10 @@

      If it understands any of the additional fields, it can use them in subsequent queries.

      - + - urn:xmpp:mam:0 + urn:xmpp:mam:1 Where arth thou, my Juliet? @@ -406,7 +408,7 @@

      - + - + A MUC archive MUST NOT include 'private message' results (those sent directly between occupants, not shared in the room) in the results

      -

      A PubSub service offering MAM SHOULD store each of the items published to each node. When responding to MAM requests it MUST construct the message stanza within the <forwarded> element in the same manner as the notifications sent to subscribers for the item, except that specifying the 'from' 'to' and 'id' attributes are OPTIONAL. Pubsub items must be returned one per message stanza (i.e. there MUST NOT be multiple <item> elemetns within the <items> element).

      +

      A PubSub service offering MAM SHOULD store each of the items published to each node. When responding to MAM requests it MUST construct the message stanza within the <forwarded> element in the same manner as the notifications sent to subscribers for the item, except that specifying the 'from' 'to' and 'id' attributes are OPTIONAL. Pubsub items must be returned one per message stanza (i.e. there MUST NOT be multiple <item> elements within the <items> element).

      - + @@ -507,7 +509,7 @@
    - + ]]> @@ -517,7 +519,7 @@ - + @@ -539,7 +541,7 @@ - + romeo@montague.lit @@ -553,7 +555,7 @@ MAY be different to the preferences sent by the client):

    - + romeo@montague.lit @@ -645,7 +647,7 @@

    If a server or other entity hosts archives and supports MAM queries, it MUST advertise - the 'urn:xmpp:mam:0' feature in response to &xep0030; requests made to archiving JIDs + the 'urn:xmpp:mam:1' feature in response to &xep0030; requests made to archiving JIDs (i.e. JIDs hosting an archive, such as users' bare JIDs):

    ... - + ... From b2cf8fb1022b846a9c6566d462a5cbd0a6dd1102 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Tue, 22 Sep 2015 21:03:35 +0200 Subject: [PATCH 075/106] Makefile: Whitespace fixes --- Makefile | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 2f306c47..09d72248 100644 --- a/Makefile +++ b/Makefile @@ -31,16 +31,13 @@ html: $(patsubst %.xml, $(OUTDIR)/%.html, $(wildcard *.xml)) pdf: $(patsubst %.xml, $(OUTDIR)/%.pdf, $(wildcard *.xml)) .PHONY: xep-% -xep-%: $(OUTDIR)/xep-%.html $(OUTDIR)/xep-%.pdf - +xep-%: $(OUTDIR)/xep-%.html $(OUTDIR)/xep-%.pdf ; .PHONY: xep-%.html -xep-%.html: $(OUTDIR)/xep-%.html - +xep-%.html: $(OUTDIR)/xep-%.html ; .PHONY: xep-%.pdf -xep-%.pdf: $(OUTDIR)/xep-%.pdf - +xep-%.pdf: $(OUTDIR)/xep-%.pdf ; $(OUTDIR)/%.html: %.xml $(XMLDEPS) xsltproc --path $(CURDIR) xep.xsl "$<" > "$@" && echo "Finished building $@" From 67beabbbb1a3427dac5bd26a5557647159526418 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Tue, 22 Sep 2015 22:07:41 +0200 Subject: [PATCH 076/106] Use TEMPDIR within OUTDIR in Makefile and some other cleanups: Add XMPPIMAGESURL. Use multiple target rule for "mkdir -p". Use special variables where possible. Remove unnecessary [ -e $FILE ] test in curl recipes. --- Makefile | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/Makefile b/Makefile index 09d72248..f9308d0a 100644 --- a/Makefile +++ b/Makefile @@ -1,9 +1,11 @@ .SILENT: OUTDIR?=build -TEMPDIR?=$(TMPDIR)/xepbuild +RESOURCESDIR=$(OUTDIR)/resources +TEMPDIR?=$(OUTDIR)/xepbuild XMLDEPS=xep.xsl xep.xsd xep.ent xep.dtd ref.xsl $(OUTDIR) -TEXMLDEPS=xep2texml.xsl $(TEMPDIR) $(XMLDEPS) $(TEMPDIR)/xmpp.pdf $(TEMPDIR)/xmpp-text.pdf +TEXMLDEPS=xep2texml.xsl $(TEMPDIR) $(XMLDEPS) $(RESOURCESDIR)/xmpp.pdf $(RESOURCESDIR)/xmpp-text.pdf +XMPPIMAGESURL=https://xmpp.org/images .PHONY: help @@ -56,19 +58,15 @@ $(TEMPDIR)/%.xml.texml.tex: $(TEMPDIR)/%.xml.texml $(OUTDIR) $(TEMPDIR) -e 's|\\hyperref\[#\([^}]*\)\]|\\hyperref\[\1\]|g' \ -e 's|\\pageref{#\([^}]*\)}|\\pageref{\1}|g' $<.tex -$(TEMPDIR)/xmpp-text.pdf: $(TEMPDIR) - -[ -e $(TEMPDIR)/xmpp-text.pdf ] || curl -o "$(TEMPDIR)/xmpp-text.pdf" https://xmpp.org/images/xmpp-text.pdf +$(RESOURCESDIR)/xmpp-text.pdf: $(RESOURCESDIR) + curl -o $@ $(XMPPIMAGESURL)/xmpp-text.pdf -$(TEMPDIR)/xmpp.pdf: $(TEMPDIR) - -[ -e $(TEMPDIR)/xmpp.pdf ] || curl -o "$(TEMPDIR)/xmpp.pdf" https://xmpp.org/images/xmpp.pdf +$(RESOURCESDIR)/xmpp.pdf: $(RESOURCESDIR) + curl -o $@ $(XMPPIMAGESURL)/images/xmpp.pdf -$(TEMPDIR): - mkdir -p $(TEMPDIR) - -$(OUTDIR): - mkdir -p $(OUTDIR) +$(TEMPDIR) $(OUTDIR) $(RESOURCESDIR): + mkdir -p $@ .PHONY: clean clean: - rm -rf $(TEMPDIR) rm -rf $(OUTDIR) From 60c36550a280ed30065d5175ca638280eb91715e Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Mon, 21 Sep 2015 18:53:01 +0200 Subject: [PATCH 077/106] XEP-0359: Minor fixes (typos, s/JID/XMPP address, ...) --- xep-0359.xml | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/xep-0359.xml b/xep-0359.xml index bcee848c..3cb42968 100644 --- a/xep-0359.xml +++ b/xep-0359.xml @@ -26,6 +26,14 @@ flo@geekplace.eu flo@geekplace.eu + + 0.2.1 + 2015-09-22 + fs + +

    Minor fixes (typos, s/JID/XMPP Address, etc.)

    +     +     0.2 2015-09-18 @@ -75,7 +83,7 @@ id='de305d54-75b4-431b-adb2-eb6b9e546013' by='room@muc.xmpp.org'/> ]]>     - In order to create a 'stanza-id' extension, the creating XMPP entity generates and sets the value of the 'id' attribute, and puts its own JID as value of the 'by' attribute. The value of the 'id' attribute must be unique and stable, i.e. it MUST NOT change later for some reason, within the scope of the 'by' value. Thus the IDs defined in this extension MUST be unique and stable within the scope of the generating XMPP entity. It is RECOMMENDED that the ID generating service uses UUID and the algorithm defined in RFC 4122, to generate the IDs. + In order to create a 'stanza-id' extension, the creating XMPP entity generates and sets the value of the 'id' attribute, and puts its own XMPP address as value of the 'by' attribute. The value of the 'id' attribute must be unique and stable, i.e. it MUST NOT change later for some reason, within the scope of the 'by' value. Thus the IDs defined in this extension MUST be unique and stable within the scope of the generating XMPP entity. It is RECOMMENDED that the ID generating service uses UUID and the algorithm defined in RFC 4122 to generate the IDs.

    @@ -97,7 +105,7 @@ type='groupchat'> Typical body text ]]> @@ -105,17 +113,17 @@

      -
    1. XMPP entities, which are routing stanzas, MUST NOT strip any elements qualified by the 'urn:xmpp:sid:0' namespace from message stanzas. They SHOULD however ensure that those elements contain only the attributes defined herein, and take approribate countermeasures if this is not the case (e.g. removing those attributes).
      2. +
    2. XMPP entities, which are routing stanzas, MUST NOT strip any elements qualified by the 'urn:xmpp:sid:0' namespace from message stanzas. They SHOULD however ensure that those elements contain only the attributes defined herein, and take appropriate countermeasures if this is not the case (e.g. removing those attributes).
    3. The values of the 'id' attribute SHOULD be unpredictable.
      4. -
    4. Stanza ID generating entities, which encounter a <stanza-id/> element where 'id' is already set and where the 'by' attribute matches their own JID, MUST ignore the existing value of 'id' and override it.
      5. -
    5. Stanzas MUST posses, in the direct child level of the stanza, at most one 'stanza-id' extension element with the same JID as value of the 'by' attribute.
      6. +
    6. Stanza ID generating entities, which encounter a <stanza-id/> element where 'id' is already set and where the 'by' attribute matches their own XMPP address, MUST ignore the existing value of 'id' and override it.
      7. +
    7. Stanzas MUST posses, in the direct child level of the stanza, at most one 'stanza-id' extension element with the same XMPP address as value of the 'by' attribute.
    8. Every <stanza-id> extension element MUST have the 'id' attribute and the 'by' attribute set.
    9. Every <stanza-id> and <client-id> extension element MUST always posses an 'id' attribute and MUST NOT have any child elements or text content.
      10. -
    10. The value of the 'by' attribute MUST be a normalized JID as defined in &rfc6122;
      11. +
    11. The value of the 'by' attribute MUST be the XMPP address of the entity assigning the unique and stable stanza ID. Note that XMPP addresses are normalized as defined in &rfc6122;
     -

    The value of the 'id' attribute should not provide any further information besides the opaque ID itself. Entities observing the value MUST NOT be able to infer any information from it, e.g. the size of the message archive. The value of 'id' MUST be considered as non-secret values.

    +

    The value of the 'id' attribute should not provide any further information besides the opaque ID itself. Entities observing the value MUST NOT be able to infer any information from it, e.g. the size of the message archive. The value of 'id' MUST be considered as non-secret value.

    This document requires no interaction with &IANA;.

    From 48f8846cee76ce76bf84d95bc55523433a660aec Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Sun, 30 Aug 2015 11:36:46 -0500 Subject: [PATCH 078/106] Clarify that 0030 features must not have children --- xep-0030.xml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/xep-0030.xml b/xep-0030.xml index 1102ba68..e3990d87 100644 --- a/xep-0030.xml +++ b/xep-0030.xml @@ -40,7 +40,10 @@ 2.5rc1 2014-06-20 editor(mam) -

    Errata submitted to standards@ regarding subscriptions and returned items.

     + +

    Errata submitted to standards@ regarding subscriptions and returned items.

    +

    Clarify that feature elements must be childless to make the text match the schema.

    +     2.4 @@ -267,7 +270,7 @@ ]]>

    The target entity then MUST either return an IQ result, or return an error (see the Error Conditions section of this document). The result MUST contain a <query/> element qualified by the 'http://jabber.org/protocol/disco#info' namespace, which in turn contains one or more <identity/> elements and one or more <feature/> elements. (Note: Every entity MUST have at least one identity, and every entity MUST support at least the 'http://jabber.org/protocol/disco#info' feature; however, an entity is not required to return a result and MAY return an error, most likely &feature; or &unavailable;, although other error conditions may be appropriate.)

    Each <identity/> element MUST possess the 'category' and 'type' attributes specifying the category and type for the entity, and MAY possess a 'name' attribute specifying a natural-language name for the entity; the <identity/> element MAY also possess a standard 'xml:lang' attribute, which enables the entity to return localized results if desired (i.e., the <query/> element MAY include multiple <identity/> elements with the same category+type but with different 'xml:lang' values, however the <query/> element MUST NOT include multiple <identity/> elements with the same category+type+xml:lang but with different 'name' values).

    -

    Each <feature/> element MUST possess a 'var' attribute whose value is a protocol namespace or other feature offered by the entity.

    +

    Each <feature/> element MUST possess a 'var' attribute whose value is a protocol namespace or other feature offered by the entity, and MUST NOT have any children.

    Preferably, both the category/type values and the feature values will be registered in a public registry, as described in the XMPP Registrar Considerations section of this document.

    Date: Tue, 22 Sep 2015 19:26:29 -0600 Subject: [PATCH 079/106] XEP-0030 v2.5rc1 update revision date --- xep-0030.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xep-0030.xml b/xep-0030.xml index e3990d87..7afe148b 100644 --- a/xep-0030.xml +++ b/xep-0030.xml @@ -38,7 +38,7 @@ &stpeter; 2.5rc1 - 2014-06-20 + 2015-09-22 editor(mam)

    Errata submitted to standards@ regarding subscriptions and returned items.

    From 0aae416a7cace8be022b7f665a8c2d566df7989a Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 28 Sep 2015 15:36:19 -0500 Subject: [PATCH 080/106] Fix a minor issue with an example in EV --- inbox/entityversioning.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index b22c9e88..27482357 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -349,7 +349,7 @@ - 9ZFZXVP9 From 0abee5101e4780982c3bd5c1099f608a043a4a1c Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 28 Sep 2015 15:37:01 -0500 Subject: [PATCH 081/106] Add search IQ to entity versioning --- inbox/entityversioning.xml | 42 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index 27482357..12a29fe0 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -368,6 +368,47 @@ any) is left up to the server implementation.

    + +

    + When a client has an incomplete versioned list, it may be beneficial to + download more of the list without requesting the full list. To do this, + servers which support entity versioning MUST supply a "search" IQ which + can be used to discover list items matching a certain criteria. What data + to match on (JID, metadata, associated vcards, etc.), and what type of + search are left up to the server implementation and MAY be different + between profiles. +

    +

    + Search queries are qualified by the 'urn:xmpp:entityver:0:search' + namespace and MUST have a 'profile' attribute set to the namespace for + which the search is being performed. For instance, searching the roster + looks like the following: + + + + Search term + + + + + + + + 4YAZ7Y38 + + + + ]]> +

    +

    + Search results SHOULD be added to the given list's cache. In this way, + the full list does not need to be known. +

    +

    While the version token approach to caching does not require a great deal @@ -498,6 +539,7 @@ anne@shakespeare.lit:VIZSVF0D,bill@shakespeare.lit:25P2A7H8

    This specification defines the following XML namespace:

    • urn:xmpp:entityver:0
      • +
    • urn:xmpp:entityver:0:search

    Upon advancement of this specification from a status of Experimental to a From 010fa79105acb652cfdb23df558ac20741c62fba Mon Sep 17 00:00:00 2001 From: Philipp Hancke Date: Mon, 28 Sep 2015 15:02:13 -0700 Subject: [PATCH 082/106] clarify that holdconn is not used --- xep-0320.xml | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/xep-0320.xml b/xep-0320.xml index 828652b4..cef9d5a2 100644 --- a/xep-0320.xml +++ b/xep-0320.xml @@ -32,6 +32,16 @@ --> jingle &fippo; + + 0.3 + 2013-09-28 + ph + +

      +
    • Clarified that holdconn setup role is not mapped.
      • +
    +     +     0.2 2013-10-22 @@ -66,7 +76,7 @@

    &xep0167; recommends the use of the Secure Real-time Transport Protocol (SRTP) for end-to-end encryption of RTP sessions negotiated using &xep0166;. &rfc5763; provides an approach to establish a Secure Real-time Transport Protocol (SRTP) security context using the Datagram Transport Layer Security (DTLS) protocol. A mechanism of transporting the fingerprint attribute that identifies the key that will be presented during the DTLS handshake in Jingle is defined herein. Inclusion of this information is OPTIONAL in both SIP/SDP and Jingle.

    -

    Note that while this specification only describes the use in the context of DTLS-SRTP, the fingerprint transported can be used in other contexts like for example establishing connections using SCTP over DTLS.

    +

    Note that while this specification only describes the use in the context of DTLS-SRTP, the fingerprint transported can be used in other contexts like for example establishing connections using SCTP over DTLS as described in &xep0343;.

    The SDP format (defined in &rfc4572;) is shown below.

    a=fingerprint:hash-func fingerprint @@ -79,6 +89,7 @@ a=fingerprint:sha-256 02:1A:CC:54:27:AB:EB:9C:53:3F:3E:4B:65:2E:7D:46:3F:54:42:C a=setup:role +

    Note that no mapping for the 'holdconn' role is defined herein.

    These SDP attributes can be translated into Jingle as a <fingerprint/> element qualified by the 'urn:xmpp:jingle:apps:dtls:0' namespace, as shown below.

    @@ -248,7 +259,7 @@ a=setup:role     -

    Thanks to Justin Uberti and Lance Stout.

    +

    Thanks to Justin Uberti, Peter Saint-Andre and Lance Stout.

     @@ -292,7 +303,7 @@ a=setup:role - + From 08ae066d4d4006987045cad2c13bbe7f1087dc48 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 28 Sep 2015 19:42:56 -0500 Subject: [PATCH 083/106] Fix a small typo on XEP-0359 --- xep-0359.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xep-0359.xml b/xep-0359.xml index 3cb42968..227e2b52 100644 --- a/xep-0359.xml +++ b/xep-0359.xml @@ -123,7 +123,7 @@ -

    The value of the 'id' attribute should not provide any further information besides the opaque ID itself. Entities observing the value MUST NOT be able to infer any information from it, e.g. the size of the message archive. The value of 'id' MUST be considered as non-secret value.

    +

    The value of the 'id' attribute should not provide any further information besides the opaque ID itself. Entities observing the value MUST NOT be able to infer any information from it, e.g. the size of the message archive. The value of 'id' MUST be considered a non-secret value.

    This document requires no interaction with &IANA;.

    From cd75571f5d42ec3b755ff3e30ed484ea97f01464 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Mon, 21 Sep 2015 19:43:38 +0200 Subject: [PATCH 084/106] XEP-0313: Fix example 9: s;; (PR #90) XEP-0313: Add missing jabber:client namespace in example 2. (PR #93) --- xep-0313.xml | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/xep-0313.xml b/xep-0313.xml index 174eb58e..a60bb3d0 100644 --- a/xep-0313.xml +++ b/xep-0313.xml @@ -27,6 +27,15 @@ &mwild; &ksmith; + + 0.4.1 + 2015-09-29 + XEP Editor (mam) + +

    Fix Example #9 from </message/> to </iq> (Florian Schmaus)

    +

    Add missing jabber:client namespace in Example #2 (Christian Schudt)

    +     +     0.4 2015-09-21 @@ -167,7 +176,7 @@ - + Hail to thee @@ -303,7 +312,7 @@ 20 - + ]]>

    The <first> and <last> elements specify the UID of the first and last returned results (not necessarily of all the messages that matched the query, if the results have been limited).

    @@ -691,7 +700,7 @@ -

    Many thanks to Dave Cridland, Kim Alvefur, Yann Leboulanger and Lance Stout +

    Many thanks to Dave Cridland, Kim Alvefur, Yann Leboulanger, Evgeny Khramtsov, Florian Schmaus and Lance Stout for their input and feedback on this specification.

     From 4c6166fbacf9b467b6d338849af8bf0b8fd5444d Mon Sep 17 00:00:00 2001 From: Philipp Hancke Date: Wed, 30 Sep 2015 10:48:58 -0700 Subject: [PATCH 085/106] use xs:annotation to annotate in schema --- xep-0320.xml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/xep-0320.xml b/xep-0320.xml index cef9d5a2..4bd73164 100644 --- a/xep-0320.xml +++ b/xep-0320.xml @@ -303,7 +303,12 @@ a=setup:role - + + + + the 'holdconn' value is not used and included only for completeness. + + From 2a24caea18a745a80808b1ad7d07720905904d80 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Tue, 11 Aug 2015 18:08:05 +0200 Subject: [PATCH 086/106] Explain how CSI is affected by in order processing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Clarify how the in order processing requirement (RFC 6120 § 10.1) affects CSI. --- xep-0352.xml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/xep-0352.xml b/xep-0352.xml index cb3aa52e..a492889c 100644 --- a/xep-0352.xml +++ b/xep-0352.xml @@ -117,6 +117,28 @@

    This protocol is intended primarily for clients with human interaction. Due to the open-ended nature of the possible optimisations implemented by the server, it may not be suitable for non-IM purposes where the fully standard behaviour of XMPP is required.

    + +

    + XMPP requires stanzas to be processed in order as per &rfc6120; 10.1. Especially "If the server's processing of a particular request could have an effect on its processing of subsequent data it might receive over that input stream..., it MUST suspend processing of subsequent data until it has processed the request.". As a result, all actions triggered by a CSI nonza sent to the server must happen before processing further requests from the same client to the server. +

    +

    + For example: A client sends a CSI active nonza, followed by an XMPP Ping request to the server. The server first changes the CSI state to active and flushes all eventually queued stanzsa. After the state has been restored to 'active' and +all resulting stanzas have been put on the wire, the +server sends the pong. +

    + + + + + + + + +]]> +

    To protect the privacy of users, servers MUST NOT reveal the clients active/inactive state to other From 41e5e6670bdb3dd038cb1c644394a84d7625a28f Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 1 Oct 2015 16:03:54 -0600 Subject: [PATCH 087/106] update revision and add acknowledgement --- xep-0352.xml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/xep-0352.xml b/xep-0352.xml index a492889c..df6e2b15 100644 --- a/xep-0352.xml +++ b/xep-0352.xml @@ -22,6 +22,12 @@ NOT_YET_ASSIGNED &mwild; + + 0.2 + 2015-10-02 + XEP Editor (mam) +

    Explain how SCI is affected by in order procesing (Florian Schmaus).

    +     0.1 2014-08-28 @@ -150,6 +156,9 @@ and responds to the ping with a pong -->

    This document requires no interaction with ®ISTRAR;.

     + +

    Special thanks to Florian Schmaus for providing clarification text.

    +     From a530a679e38f4e3e8a44f6d95495f3222b048107 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Thu, 1 Oct 2015 16:25:47 -0600 Subject: [PATCH 088/106] Revert "Merge pull request #105 from xsf/xep-0352-v0.2" This reverts commit 0258400f59704dadab8e0f509d35f9b07960ca8b, reversing changes made to ae19ece2068a18599cd856461b72e4d3e4abb8f2. --- xep-0352.xml | 31 ------------------------------- 1 file changed, 31 deletions(-) diff --git a/xep-0352.xml b/xep-0352.xml index df6e2b15..cb3aa52e 100644 --- a/xep-0352.xml +++ b/xep-0352.xml @@ -22,12 +22,6 @@ NOT_YET_ASSIGNED &mwild; - - 0.2 - 2015-10-02 - XEP Editor (mam) -

    Explain how SCI is affected by in order procesing (Florian Schmaus).

     -     0.1 2014-08-28 @@ -123,28 +117,6 @@

    This protocol is intended primarily for clients with human interaction. Due to the open-ended nature of the possible optimisations implemented by the server, it may not be suitable for non-IM purposes where the fully standard behaviour of XMPP is required.

    - -

    - XMPP requires stanzas to be processed in order as per &rfc6120; 10.1. Especially "If the server's processing of a particular request could have an effect on its processing of subsequent data it might receive over that input stream..., it MUST suspend processing of subsequent data until it has processed the request.". As a result, all actions triggered by a CSI nonza sent to the server must happen before processing further requests from the same client to the server. -

    -

    - For example: A client sends a CSI active nonza, followed by an XMPP Ping request to the server. The server first changes the CSI state to active and flushes all eventually queued stanzsa. After the state has been restored to 'active' and -all resulting stanzas have been put on the wire, the -server sends the pong. -

    - - - - - - - - -]]> -

    To protect the privacy of users, servers MUST NOT reveal the clients active/inactive state to other @@ -156,9 +128,6 @@ and responds to the ping with a pong -->

    This document requires no interaction with ®ISTRAR;.

     - -

    Special thanks to Florian Schmaus for providing clarification text.

    -     From c3339019c9f22f53a1c12aa0664a1fc80ba97bec Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Fri, 2 Oct 2015 09:02:22 -0600 Subject: [PATCH 089/106] start revision 0.2 --- xep-0352.xml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/xep-0352.xml b/xep-0352.xml index cb3aa52e..4ddab8af 100644 --- a/xep-0352.xml +++ b/xep-0352.xml @@ -22,6 +22,12 @@ NOT_YET_ASSIGNED &mwild; + + 0.2 + 2015-10-02 + XEP Editor (mam) +

    Clarified how CSI is affected by in-order processing (Florian Schmaus).

     +     0.1 2014-08-28 From 4f739c29ee83eb0f4800e4930e2d575aefe02648 Mon Sep 17 00:00:00 2001 From: Florian Schmaus Date: Tue, 11 Aug 2015 18:08:05 +0200 Subject: [PATCH 090/106] Explain how CSI is affected by in order processing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Clarify how the in order processing requirement (RFC 6120 § 10.1) affects CSI. --- xep-0352.xml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/xep-0352.xml b/xep-0352.xml index 4ddab8af..884ae92b 100644 --- a/xep-0352.xml +++ b/xep-0352.xml @@ -123,6 +123,28 @@

    This protocol is intended primarily for clients with human interaction. Due to the open-ended nature of the possible optimisations implemented by the server, it may not be suitable for non-IM purposes where the fully standard behaviour of XMPP is required.

    + +

    + XMPP requires stanzas to be processed in order as per &rfc6120; 10.1. Especially "If the server's processing of a particular request could have an effect on its processing of subsequent data it might receive over that input stream..., it MUST suspend processing of subsequent data until it has processed the request.". As a result, all actions triggered by a CSI nonza sent to the server must happen before processing further requests from the same client to the server. +

    +

    + For example: A client sends a CSI active nonza, followed by an XMPP Ping request to the server. The server first changes the CSI state to active and flushes all eventually queued stanzsa. After the state has been restored to 'active' and +all resulting stanzas have been put on the wire, the +server sends the pong. +

    + + + + + + + + +]]> +

    To protect the privacy of users, servers MUST NOT reveal the clients active/inactive state to other From 3236101fab7b71937bfe8d2c66ad56c5b3df7245 Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Fri, 2 Oct 2015 09:38:17 -0600 Subject: [PATCH 091/106] XEP-0320 v0.3 -- correct date --- xep-0320.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xep-0320.xml b/xep-0320.xml index 85218b09..2d07df79 100644 --- a/xep-0320.xml +++ b/xep-0320.xml @@ -35,7 +35,7 @@ &fippo; 0.3 - 2013-09-28 + 2015-09-28 ph

      From 4988e2709a6de917d858ace7afca1195a357c6b5 Mon Sep 17 00:00:00 2001 From: Christian Schudt Date: Sat, 26 Sep 2015 23:47:05 +0200 Subject: [PATCH 092/106] XEP-0050: element should be attribute --- xep-0050.xml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/xep-0050.xml b/xep-0050.xml index 5d36250b..209ba91e 100644 --- a/xep-0050.xml +++ b/xep-0050.xml @@ -25,6 +25,12 @@ http://www.xmpp.org/schemas/commands.xsd &linuxwolf; + + 1.2.1 + 2015-10-15 + XEP Editor (mam) + Corrected text regarding "xml:lang" is an attribute not an element (Christian Schudt). + 1.2 2005-06-30 @@ -660,7 +666,7 @@ The command should be completed (if possible). -

      The "xml:lang" attribute specifies the language/locale this <command/> is intended for. This element MAY be specified by the requester to request a specific language/locale, and SHOULD be included by the responder to indicate the language/locale in use.

      +

      The "xml:lang" attribute specifies the language/locale this <command/> is intended for. This attribute MAY be specified by the requester to request a specific language/locale, and SHOULD be included by the responder to indicate the language/locale in use.

      The children of a <command/> element (other than <actions/> and <note/>) pertain to the command's execution. The order of these elements denote their precedence, so that those elements earlier in the list have higher precedence.

      From 3020652e01b9882f59a744258db01585bb9bd905 Mon Sep 17 00:00:00 2001 From: Christian Schudt Date: Sun, 27 Sep 2015 00:19:39 +0200 Subject: [PATCH 093/106] XEP-0171: Schema wrongly specifies elements/attributes as required E.g. Example 1 and 2 have no element Many examples have no dictionary attribute. And destination_lang should have type xs:language --- xep-0171.xml | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/xep-0171.xml b/xep-0171.xml index 451b8c07..34ed857f 100644 --- a/xep-0171.xml +++ b/xep-0171.xml @@ -50,6 +50,12 @@ Raymond braymond@echostorm.net + + 1.1.1 + 2015-10-15 + XEP Editor (mam) +

      Corrected XML schema to note a number of constructs are optional not required (Christian Schudt).

       +       1.1 2011-06-15 @@ -416,7 +422,7 @@ - + @@ -439,8 +445,8 @@ - - + + From 29cdac37ae85caf89dd261021f964f5b970ccdff Mon Sep 17 00:00:00 2001 From: Stefan Strigler Date: Mon, 28 Sep 2015 11:18:43 +0200 Subject: [PATCH 094/106] fix bad xml in example fix bad hostname in example fix description in example --- xep-0355.xml | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/xep-0355.xml b/xep-0355.xml index 3a4724a5..3530c4ae 100644 --- a/xep-0355.xml +++ b/xep-0355.xml @@ -28,7 +28,14 @@ goffi@goffi.org goffi@jabber.fr - + + + 0.2.1 + 2015-10-15 + XEP Editor (mam) +

      Corrected examples (Stefan Strigler).

       +       + 0.2 2014-03-21 @@ -48,14 +55,14 @@
    - + 0.1 2014-12-18 XEP Editor (mam)

    Initial published version approved by the XMPP Council.

         - + 0.0.1 2014-11-13 @@ -95,7 +102,7 @@ -

    Namespaces delegations are granted in the server configuration. Only &IQ; stanza namespaces can be delegated.

    +

    Namespaces delegations are granted in the server configuration. Only &IQ; stanza namespaces can be delegated.

    A feature is delegated using:

    1. it's namespace: e.g. 'urn:xmpp:mam:0'
      2. @@ -107,11 +114,11 @@

      Once the managing entity is authenticated and stream is started, the server send it a &MESSAGE; stanza with a <delegation/> elements which MUST have the 'urn:xmpp:delegation:1' namespace. This element contains <delegated/> elements which MUST contain a 'namespace' attribute indicating the delegated namespace. If there is additional attribute filtering, the <delegated/> can have children <attribute/> elements which MUST contain a 'name' attribute with the name of the filtering attribute.

      + - + @@ -166,7 +173,7 @@ ]]>

      The managing entity replies to the stanza by encapsulating its &IQ; result in the same way:

      -In client mode, the managing entity is not certified by the server administrator, so the delegation MUST be explicitly allowed by the managed entity. This is initiated by the managing entity (it can be after an interaction with a managed entity, like a subscription).

      To request delegation for a particular entity, the managing entity MUST have an &IQ; stanza with 'urn:xmpp:delegation:1' namespace. The &QUERY; element MUST have a 'to' attribute which specify the entity it wants to manage.

      Namespace delegations are asked with a <delegate/> element, which MUST contain a 'namespace' attribute set to the requested namespace.

      - +

      If an entity want to manage PEP service for Juliet, it can ask the delegation like this:

      Date: Mon, 5 Oct 2015 21:24:58 -0700 Subject: [PATCH 095/106] add note about whitespace --- xep-0320.xml | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/xep-0320.xml b/xep-0320.xml index 2d07df79..6d8e79cc 100644 --- a/xep-0320.xml +++ b/xep-0320.xml @@ -33,6 +33,16 @@ --> jingle &fippo; + + 0.3.1 + 2015-10-15 + ph + +
        +
      • Noted that whitespace is not significant in examples.
        • +
      +       +       0.3 2015-09-28 @@ -97,7 +107,7 @@ a=setup:role fingerprint ]]>       -

      An example follows.

      +

      An example follows. Note that the whitespace would not appear in actual XML content.

      02:1A:CC:54:27:AB:EB:9C:53:3F:3E:4B:65:2E:7D:46:3F:54:42:CD:54:F1:7A:03:A2:7D:F9:B0:7F:46:19:B2 From bee05d65aafce26f9d279274140cefb88bdd12b7 Mon Sep 17 00:00:00 2001 From: Mili Verma Date: Tue, 22 Sep 2015 11:34:09 +0100 Subject: [PATCH 096/106] Include support for Microsoft SSPI in xep-0233 --- xep-0233.xml | 68 ++++++++++++++++++++++++++++++++++++++++++---------- xep.ent | 1 + 2 files changed, 57 insertions(+), 12 deletions(-) diff --git a/xep-0233.xml b/xep-0233.xml index b5a3ce6a..866b2f13 100644 --- a/xep-0233.xml +++ b/xep-0233.xml @@ -6,8 +6,8 @@
      - Domain-Based Service Names in XMPP SASL Negotiation - This specification defines a method by a connecting client can learn the domain-based service name of a Kerberos acceptor principal for SASL authentication using the GSSAPI mechanism. + XMPP Server Registration for use with Kerberos V5 + This specification defines the Kerberos principal name of an XMPP server. It also details a method by which a connecting client can determine this Kerberos principal name when authenticating using the "GSSAPI" SASL mechanism. &LEGALNOTICE; 0233 Deferred @@ -25,6 +25,12 @@ &linuxwolf; &stpeter; &hildjj; + + 0.5 + 2015-10-20 + mv +

      Included support for Microsoft SSPI (Mili Verma).

       +       0.4 2011-08-26 @@ -64,18 +70,52 @@
      -

      In environments that make use of Kerberos V5 (&rfc4120;) and negotiation of Simple Authentication and Security Layer or SASL (&rfc4422;) over XMPP, a connecting client often needs to know the identity of the Kerberos acceptor principal so that it can obtain a proper ticket for authentication. This scenario was not addressed in &rfc3920; or &rfc6120;. However, the problem can be solved using the concept of domain-based service names (&rfc5178;). In particular, when an XMPP server uses the Kerberos V5 ("GSSAPI") SASL mechanism (&rfc4752;), it can communicate the identity of the acceptor principal as a Kerberos V5 service principal name (&rfc5179;). This document defines an XMPP method for such communication.

      +

      The Kerberos Network Authentication Service (V5) is described in &rfc4120;. An application can call the Kerberos library through the Generic Security Services Application Programming Interface (GSS-API) described in &rfc4121; or the proprietary Microsoft Windows Security Service Provider Interface (SSPI).

      +

      The Simple Authentication and Security Layer or SASL (&rfc4422;) is a framework for adding authentication support to connection-based protocols. The SASL mechanism used in environments that make use of Kerberos V5 is called "GSSAPI" and is described in &rfc4752;. Note that the RFC 4121 API has a hyphen and the SASL mechanism does not.

      +

      Before using the "GSSAPI" SASL mechanism to authenticate to an XMPP server (which is referred to as the "acceptor" in Kerberos terminology), a connecting client needs to obtain a Kerberos ticket from the Key Distribution Centre (KDC). For this the client needs to determine the Kerberos principal name of the XMPP server. This scenario was not addressed in &rfc3920; or &rfc6120;.

      +

      This specification sets out the rules that must be followed when registering the Kerberos principal name of an XMPP server. It also details how a client can determine the hostname of the XMPP server which can then be used to construct the Kerberos principal name.

       - -

      The acceptor principal's hostname is communicated by including a child element of the <mechanisms/> element during SASL negotation, as allowed by RFC 6120 (see Section 6.3.5 and the schema for the 'urn:ietf:params:xml:ns:xmpp-sasl' namespace in Appendix A.4). In the case of the Kerberos V5 SASL mechanism, the child element is a <hostname/> element qualified by the 'urn:xmpp:domain-based-name:1' namespace &NSNOTE;. The XML character data of the <hostname/> element specifies the fully-qualified name of the acceptor principal. The client then generates a domain-based service name from the provided hostname, following the format specified in RFC 5179 (i.e., "protocol/hostname/domainname@REALM") and setting the values as follows:

      + +

      An XMPP client will initiate a connection to the XMPP server.

      +

      The XMPP server will communicate its hostname in a child element of the <mechanisms/> element during SASL negotation, as allowed by RFC 6120 (see Section 6.3.5 and the schema for the 'urn:ietf:params:xml:ns:xmpp-sasl' namespace in Appendix A.4).

      +

      This child element is <hostname/> qualified by the 'urn:xmpp:domain-based-name:1' namespace. &NSNOTE;.

      +

      The XML character data of the <hostname/> element specifies the fully-qualified name of the XMPP server. This should be used for constructing the Kerberos principal name and is independent of the usual rules that an XMPP client uses for establishing a network connection to the XMPP server.

      +       + + +

      When the XMPP server is implemented using GSS-API, the domain-based service name (&rfc5178;, &rfc5179;) is used as the Kerberos principal name. Domain-based service names contain a domain name in addition to a hostname. This allows naming clustered servers after the domain which they service.

      +

      The domain-based service name is mapped to the Kerberos principal name following the format specified in RFC 5179 (i.e., "service/hostname/domain@REALM") and setting the values as follows:

        -
      • The protocol string MUST be "xmpp".
        • -
      • The hostname string MUST be the XML character data of the <hostname/> element.
        • -
      • The domainname string MUST be the canonical name of the service, such as typically communicated in the 'to' address of the initial stream header.
        • +
      • The service string MUST be "xmpp".
        • +
      • The hostname string MUST be the hostname of the XMPP server, as provided by the server in the XML character data of the <hostname/> element during SASL negotiation.
        • +
      • The domain string MUST be the canonical name of the service. This is typically communicated by the client in the 'to' address of the initial stream header.
      • The REALM string SHOULD be determined according to the network policies in effect (usually the domain name, in an uppercase mapping).
      -

      Consider the example of an XMPP service whose canonical name is "example.com". A user might make use of an acceptor principal located at "auth42.us.example.com". The hostname would be communicated as follows.

      +       + + +

      Microsoft Windows provides the proprietary SSPI to support the "GSSAPI" SASL mechanism. This section describes the Windows equivalent of the domain-based service name for an XMPP server.

      +

      In the Microsoft Windows environment, the concept of Service Principal Name (SPN) is used, which is specified in https://msdn.microsoft.com/en-us/library/ms677601%28v=vs.85%29.aspx. This format ("service class/host:port/service name") is similar to the one specified in RFC 5179. The SPN can be generated by setting the values as follows:

      +
        +
      • The service class string MUST be "xmpp".
        • +
      • The host string MUST be the hostname of the XMPP server, as provided by the server in the XML character data of the <hostname/> element during SASL negotiation.
        • +
      • The port is optional. It can be used to differentiate between multiple XMPP servers on a single host computer and should be omitted if the XMPP server uses the default port of 5222 for accepting client connections.
        • +
      • The service name string MUST be the canonical name of the service. This is typically communicated by the client in the 'to' address of the initial stream header.
        • +
      +       + + +

      The goal of this section is to help developers of applications so that clients and servers implemented over SSPI can interoperate with servers and clients implemented over GSS-API.

      +

      Interoperability is achieved by the GSS-API system joining the Windows Active Directory domain or by having a cross-realm trust between the KDCs of the GSS-API and SSPI systems.

      +

      The SPN of the SSPI server does not specify a realm. A GSS-API client constucts the Kerberos principal name according to the rules in the GSS-API environment and adds a realm to the Kerberos principal name, but the Kerberos principal name is mapped to the correct XMPP server on SSPI.

      +

      When the server uses GSS-API, the SPN for the server needs to be created in the SSPI environment. The SPN constructed by the SSPI client according to the rules in the SSPI environment is then mapped to the correct GSS-API XMPP server.

      +

      The domain-based service name of GSS-API does not specify a port, so the port option of the SPN in SSPI should only be used in testing scenarios when both the XMPP client and the XMPP server are implemented using SSPI. The port SHOULD NOT be used in any other scenarios.

      +

      So in effect, whether an endpoint uses SSPI or GSS-API does not affect interoperability as long as the port in SSPI is not used.

      +       + + +

      Consider the example of an XMPP service "example.com" offered by the XMPP server located on the host "auth42.us.example.com", using the default port of 5222 for accepting client connections. When a client connects to the XMPP server, the server communicates its hostname along with supported SASL mechanisms as follows:

      GSSAPI @@ -83,11 +123,15 @@ auth42.us.example.com ]]> -

      The client would then attempt to obtain a ticket for the domain-based principal "xmpp/auth42.us.example.com/example.com@EXAMPLE.COM".

      +

      To use the "GSSAPI" SASL mechanism, the client needs to determine the Kerberos principal name of the XMPP server, which will be:

      +
        +
      • the domain-based service name "xmpp/auth42.us.example.com/example.com@EXAMPLE.COM" if the client is using GSS-API.
        • +
      • the SPN "xmpp/auth42.us.example.com/example.com" if the client is using SSPI.
        • +
      -

      The communication of acceptor principal hostname during SASL negotiation is not known to introduce new security vulnerabilities, as long as it is done after the underlying channel has been secured using Transport Layer Security (TLS; &rfc5246;) as described for XMPP in RFC 6120. For additional security considerations, refer to RFC5178 and RFC 5179.

      +

      The communication of the XMPP server's hostname during SASL negotiation is not known to introduce new security vulnerabilities, as long as it is done after the underlying channel has been secured using Transport Layer Security (TLS; &rfc5246;) as described for XMPP in RFC 6120. For additional security considerations, refer to RFC 5178 and RFC 5179.

       @@ -124,7 +168,7 @@ -

      Thanks to Owen Friel, Shane Hannon, Seamus Kerrigan, Eliot Lear, Alexey Melnikov, and Klaas Wierenga for their comments.

      +

      Thanks to Owen Friel, Shane Hannon, Seamus Kerrigan, Eliot Lear, Alexey Melnikov, Klaas Wierenga and Dave Cridland for their comments.

             diff --git a/xep.ent b/xep.ent index 481a2488..bba8161a 100644 --- a/xep.ent +++ b/xep.ent @@ -537,6 +537,7 @@ THE SOFTWARE. RFC 4103 RFC 4103: RTP Payload for Text Conversation <http://tools.ietf.org/html/rfc4103>." > RFC 4119 RFC 4119: A Presence-based GEOPRIV Location Object Format <http://tools.ietf.org/html/rfc4119>." > RFC 4120 RFC 4120: The Kerberos Network Authentication Service (V5) <http://tools.ietf.org/html/rfc4120>." > +RFC 4121 RFC 4121: The Kerberos Version 5 Generic Security Service Application Program Interface (GSS-API) Mechanism: Version 2 <http://tools.ietf.org/html/rfc4121>." > RFC 4122 RFC 4122: A Universally Unique IDentifier (UUID) URN Namespace <http://tools.ietf.org/html/rfc4122>." > RFC 4145 RFC 4145: TCP-Based Media Transport in the Session Description Protocol (SDP) <http://tools.ietf.org/html/rfc4145>." > RFC 4227 RFC 4227: Using the Simple Object Access Protocol (SOAP) in Blocks Extensible Exchange Protocol (BEEP) <http://tools.ietf.org/html/rfc4227>." > From c892cbc2f673679d4e276258ac0751b3147e1392 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Fri, 16 Oct 2015 09:57:40 -0500 Subject: [PATCH 097/106] Add &sam; to list of common authors --- xep.ent | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/xep.ent b/xep.ent index bba8161a..faaab0f5 100644 --- a/xep.ent +++ b/xep.ent @@ -986,6 +986,15 @@ IANA Service Location Protocol, Version 2 (SLPv2) Templates winfried@tilanus.com " > + + Sam + Whited + sam@samwhited.com + sam@samwhited.com + https://blog.samwhited.com/ + +" > From 974e0fce163683d6f4cbf0d121442e1278947479 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Fri, 16 Oct 2015 15:09:02 -0500 Subject: [PATCH 098/106] Add new "message attaching" spec Proposal to allow one message to be "attached" to another (a hint for the client to display the messages side by side, even if other messages were sent between them). --- inbox/attachto.xml | 179 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 179 insertions(+) create mode 100644 inbox/attachto.xml diff --git a/inbox/attachto.xml b/inbox/attachto.xml new file mode 100644 index 00000000..b2cd60df --- /dev/null +++ b/inbox/attachto.xml @@ -0,0 +1,179 @@ + + +%ents; +]> + + +
      + Message Attaching + + This specification defines a method for indicating that a message contains + content which describes an earlier message in the conversation and should + be grouped with the earlier message. + + &LEGALNOTICE; + XXXX + ProtoXEP + Standards Track + Standards + Council + + XMPP Core + + + + message-attaching + &sam; + + Craig + Petchell + cpetchell@atlassian.com + + + 0.0.1 + 2015-10-16 + ssw +

      First draft.

       +       +
      + +

      + Many messaging applications send special messages which include more + information about a previous message. For example, when sending a link the + service may display the description or an image off the page, or users may + respond to a message with a "sticker" or an "emotion" which they wish to + display alongside the message, even though the conversation has moved on. +

      +

      + This specification defines a way by which one can indicate that a message + is logically (and visibly) "attached" to an earlier message in the + conversation. +

      +       + +

      + If a client implements message attaching, it MUST specify the + 'urn:xmpp:message-attaching:0' feature in its service discovery information + features as specified in &xep0030; and the Entity Capabilities profile + specified in &xep0115;. +

      + + +]]> + + +… + +… + +]]> +       + +

      + Messages that are attached to other messages MUST contain an + <attach-to/> element qualified by the 'urn:xmpp:message-attaching:0' + namespace and with an 'id' attribute set to the ID of the message that we + want to attach to. Messages MAY be attached to any other message, including + those sent by other clients, but clients MAY choose to ignore the attach-to + directive and display the message normally. +

      + + storm.png + +]]> +

      + Note that indicating that a message should be "attached" to an earlier + message in the conversation is merely a suggestion to the client to display + the message next to or below the old message. +

      +       + +

      + A receiving client MAY choose to show the attached message next to or below + the indicated message in whatever display is used for messages or can + choose to display the attachment in another way (including as a normal + message, completely ignoring the attach-to element). +

      +

      + A receiving client SHOULD indicate that the message is an attachment, and + not a part of the original message to prevent confusion. +

      +

      + <attach-to/> elements MUST NOT be put on any stanza type other than + messages. +

      +

      + A server may choose to strip some <attach-to/> messages based on + local policy (eg. a server might have a policy that only it can create + message attachments). +

      +

      Clients MUST send ids on messages if they support attachments.

      +

      + Messages MUST NOT contain more than one >attach-to/< element. +

      +

      + Clients and servers MUST NOT include an <attach-to/> element on + messages with a non-messaging payload unless they are including it on an + error which may be attached to the message that caused the error to be + generated. +

      +       + + Clients that implement message attachments MUST be careful not to display the + attachments in such a way that they could be confused with the original + message and cause someone viewing the conversation to assume they were sent + by the sender of the message being attached to. + + +

      This document requires no interaction with &IANA;.

      +       + +

      This specification defines the following XML namespaces:

      +
        +
      • urn:xmpp:message-attaching:0
        • +
      +

      + The ®ISTRAR; shall include the foregoing namespaces in its disco + features registry as defined in &xep0030;. + + urn:xmpp:message-attaching:0 + Indicates support for attaching one message to another. + XEP-xxxx + + ]]> +

      +       + + + + + + + + + + + + + + + + + ]]> + +       From 7791061e82087e2f99f1438da1c43aba8ee56f5c Mon Sep 17 00:00:00 2001 From: "Matthew A. Miller" Date: Tue, 20 Oct 2015 10:40:10 -0600 Subject: [PATCH 099/106] XEP-0233 Deferred -> Experimental --- xep-0233.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/xep-0233.xml b/xep-0233.xml index 866b2f13..79fe4e64 100644 --- a/xep-0233.xml +++ b/xep-0233.xml @@ -10,7 +10,7 @@ This specification defines the Kerberos principal name of an XMPP server. It also details a method by which a connecting client can determine this Kerberos principal name when authenticating using the "GSSAPI" SASL mechanism. &LEGALNOTICE; 0233 - Deferred + Experimental Standards Track Standards Council From 11cba41cd50a6dd309e90af59e2de122182eb495 Mon Sep 17 00:00:00 2001 From: Lance Stout Date: Tue, 25 Aug 2015 00:17:17 -0700 Subject: [PATCH 100/106] XEP-0215: Try resolving some ambiguities XEP-0215: Add Lance as Co-Author, Bump namespace XEP-0215: Add revision notes --- xep-0215.xml | 91 ++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 70 insertions(+), 21 deletions(-) diff --git a/xep-0215.xml b/xep-0215.xml index 0c05dbb4..e3f9927d 100644 --- a/xep-0215.xml +++ b/xep-0215.xml @@ -23,6 +23,21 @@ &stpeter; &seanegan; &mlundblad; + &lance; + + 0.7 + 2015-10-20 + ls + +
        +
      • Clarified which attributes are needed to identify a service when requesting credentials.
        • +
      • Added 'action' attribute to indicate if a service is being added/removed/updated in a push update.
        • +
      • Added 'expires' attribute to indicate when provided credentials will expire.
        • +
      • Added 'restricted' attribute to indicate the need for fetching credentials for a service.
        • +
      • Bumped namespace version.
        • +
      +       +       0.6 2014-02-27 @@ -111,7 +126,7 @@ -

      In order to learn about external services known to an XMPP server or discovery service, a requesting entity (typically a client) sends an IQ-get containing an empty <services/> element qualified by the 'urn:xmpp:extdisco:1' namespace &NSNOTE;, typically to its own server but perhaps alternatively to a dedicated discovery service.

      +

      In order to learn about external services known to an XMPP server or discovery service, a requesting entity (typically a client) sends an IQ-get containing an empty <services/> element qualified by the 'urn:xmpp:extdisco:2' namespace &NSNOTE;, typically to its own server but perhaps alternatively to a dedicated discovery service.

      The responding entity (XMPP server or discovery service) SHOULD return the list of external services it is aware of, but MAY instead return an appropriate error, such as &unavailable; if the responding entity does not support this protocol or &forbidden; if the requesting entity does not have permission to receive the list of external services. Each service is encapsulated via a <service/> element.

      Note: The processes by which a responding entity discovers external services for "proxying" to XMPP entities are out of scope for this specification.

      The <service/> element MAY be empty or MAY include extended information about the service as described in the Extended Information section of this document.

      @@ -122,6 +137,16 @@ Definition Inclusion + + action + When sending a push update, the action value indicates if the service is being added or deleted from the set of known services (or simply being modified). The defined values are "add", "remove", and "modify", where "add" is the default. + OPTIONAL + + + expires + A timestamp indicating when the provided username and password credentials will expire. The format MUST adhere to the dateTime format specified in &xep0082; and MUST be expressed in UTC. + OPTIONAL + host Either a fully qualified domain name (FQDN) or an IP address (IPv4 or IPv6). @@ -142,6 +167,11 @@ The communications port to be used at the host. RECOMMENDED + + restricted + A boolean value indicating that username and password credentials are required and will need to be requested if not already provided (see Requesting Credentials). + OPTIONAL + transport The underlying transport protocol to be used when communicating with the service (typically either TCP or UDP). @@ -169,7 +199,7 @@ id='ul2bc7y6' to='shakespeare.lit' type='get'> - + ]]> - + - + ]]> - ]]> -

      If a requesting entity requests services of a particular type, the responding service MAY as needed send an updated list of the relevant services by "pushing" the list to a requesting entity that has previously requested the list. However, it MUST NOT push updates to the requesting entity unless it has presence information about the requesting entity (e.g., because the requesting entity is connected to the XMPP server or because the requesting entity has shared presence with a remote discovery service). A push is an IQ set to the requesting entity containing a <service/> payload with updated data about services matching the requested type (e.g., new services or updated credentials)

      +

      If a requesting entity requests services of a particular type, the responding service MAY as needed send an updated list of the relevant services by "pushing" the list to a requesting entity that has previously requested the list. However, it MUST NOT push updates to the requesting entity unless it has presence information about the requesting entity (e.g., because the requesting entity is connected to the XMPP server or because the requesting entity has shared presence with a remote discovery service). A push is an IQ set to the requesting entity containing a <services/> payload with updated data about services matching the requested type (e.g., new services or updated credentials). Each <service/> element SHOULD contain an 'action' attribute indicating if the service is being added, deleted, or modified.

      - - - + ]]>

      Upon receiving a push, the requesting entity would then send an IQ-result to the responding service in accordance with &xmppcore;.

      -

      An entity might know about an external service via DNS or some other means, but still might need short-term credentials to use the service. The entity can request credentials by sending a special request to the server.

      +

      An entity might know about an external service via DNS or some other means, but still might need short-term credentials to use the service. The entity can request credentials by sending a special request to the server composed of a <credentials/> element qualified by the 'urn:xmpp:extdisco:2' namespace and contains a <service/> element which MUST include the 'host' and 'type' attributes to identify the desired service (the 'port' attribute MAY be provided if there are multiple services with the same host and type but different ports).

      - - + + ]]> @@ -284,23 +320,25 @@ id='xi2cax48' to='bard@shakespeare.lit/globe' type='get'> - + ]]>

      For TURN, the server might construct time-limited credentials as described in &turn-rest;.

      +

      There MAY be multiple <service/> elements in the result if more than one service matched the requested service identity (e.g., the same host provides service on multiple ports).

      If the server cannot obtain credentials at the service, it returns an appropriate stanza error, such as ¬found;, &remoteserver;, &timeout;, or ¬authorized;.

             -

      If a server or service needs to include extended information, it SHOULD do so by including each bit of information as the XML character data of the <value/> child of a distinct <field/> element, with the entire set of fields contained within an <x/> element of type "result" qualified by the 'jabber:x:data' namespace (see &xep0004;); this <x/> element SHOULD be a child of the <service/> element qualified by the 'urn:xmpp:extdisco:1' namespace &NSNOTE;. Thus the IQ result SHOULD be of the following form:

      +

      If a server or service needs to include extended information, it SHOULD do so by including each bit of information as the XML character data of the <value/> child of a distinct <field/> element, with the entire set of fields contained within an <x/> element of type "result" qualified by the 'jabber:x:data' namespace (see &xep0004;); this <x/> element SHOULD be a child of the <service/> element qualified by the 'urn:xmpp:extdisco:2' namespace &NSNOTE;. Thus the IQ result SHOULD be of the following form:

      - + @@ -317,7 +355,7 @@       -

      If an XMPP entity supports this protocol, it MUST report that fact by including a service discovery feature of "urn:xmpp:extdisco:1" &NSNOTE; in response to a &xep0030; information request:

      +

      If an XMPP entity supports this protocol, it MUST report that fact by including a service discovery feature of "urn:xmpp:extdisco:2" &NSNOTE; in response to a &xep0030; information request:

      - + ]]> @@ -350,7 +388,7 @@

      This specification defines the following XML namespace:

        -
      • urn:xmpp:extdisco:1
        • +
      • urn:xmpp:extdisco:2

      Upon advancement of this specification from a status of Experimental to a status of Draft, the ®ISTRAR; shall add the foregoing namespace to the registry located at &NAMESPACES;, as described in Section 4 of &xep0053;.

       @@ -394,8 +432,8 @@ - + @@ -424,10 +462,21 @@ + + + + + + + + + + + From f7fc6e0d2d1cb273a1179572eac2112716a1e9db Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Mon, 28 Sep 2015 12:07:47 -0500 Subject: [PATCH 101/106] Add `make preview' to generate HTML on XML save Uses inotifywatch to generate an HTML file anytime an XML file is saved. --- Makefile | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/Makefile b/Makefile index f9308d0a..f9cfa86b 100644 --- a/Makefile +++ b/Makefile @@ -17,6 +17,7 @@ help: @echo ' pdf - build all XEPs' @echo ' html - build all XEPs' @echo ' clean - recursively unlink the build tree' + @echo ' preview - builds html whenever an XEP changes (requires inotify-tools)' @echo ' xep-xxxx - build xep-xxxx.html and xep-xxxx.pdf' @echo ' xep-xxxx.html - build xep-xxxx.html' @echo ' xep-xxxx.pdf - build xep-xxxx.html' @@ -70,3 +71,12 @@ $(TEMPDIR) $(OUTDIR) $(RESOURCESDIR): .PHONY: clean clean: rm -rf $(OUTDIR) + +.PHONY: preview +preview: + inotifywait -m -e close_write,moved_to --format '%e %f' . | \ + while read -r event file; do \ + if [ "$${file: -4}" == ".xml" ]; then \ + xsltproc --path $(CURDIR) xep.xsl "$${file}" > "$(OUTDIR)/$${file: :8}.html" && echo "Built $${file: :8}.html $${event}"; \ + fi \ + done From 8c69d78de4d502111479cbfc208cf9d86b06ddb2 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Sat, 10 Oct 2015 09:25:21 -0500 Subject: [PATCH 102/106] Make sure make preview supports inbox files --- Makefile | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index f9cfa86b..9b17fe29 100644 --- a/Makefile +++ b/Makefile @@ -6,6 +6,7 @@ TEMPDIR?=$(OUTDIR)/xepbuild XMLDEPS=xep.xsl xep.xsd xep.ent xep.dtd ref.xsl $(OUTDIR) TEXMLDEPS=xep2texml.xsl $(TEMPDIR) $(XMLDEPS) $(RESOURCESDIR)/xmpp.pdf $(RESOURCESDIR)/xmpp-text.pdf XMPPIMAGESURL=https://xmpp.org/images +XEPDIRS=. inbox .PHONY: help @@ -74,9 +75,9 @@ clean: .PHONY: preview preview: - inotifywait -m -e close_write,moved_to --format '%e %f' . | \ - while read -r event file; do \ + inotifywait -m -e close_write,moved_to --format '%e %w %f' $(XEPDIRS) | \ + while read -r event dir file; do \ if [ "$${file: -4}" == ".xml" ]; then \ - xsltproc --path $(CURDIR) xep.xsl "$${file}" > "$(OUTDIR)/$${file: :8}.html" && echo "Built $${file: :8}.html $${event}"; \ + xsltproc --path $(CURDIR) xep.xsl "$${dir}/$${file}" > "$(OUTDIR)/$${file%.*}.html" && echo "Built $${file%.*}.html $${event}"; \ fi \ done From 5eda2277f7db808f37052e7bb9b557803e73df8c Mon Sep 17 00:00:00 2001 From: Andreas Straub Date: Thu, 3 Sep 2015 18:22:49 +0200 Subject: [PATCH 103/106] Add new OMEMO spec Proposal for a new multi-end to multi-end encryption protocol --- inbox/omemo.xml | 299 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 299 insertions(+) create mode 100644 inbox/omemo.xml diff --git a/inbox/omemo.xml b/inbox/omemo.xml new file mode 100644 index 00000000..1cb737ac --- /dev/null +++ b/inbox/omemo.xml @@ -0,0 +1,299 @@ + + +%ents; +]> + + +
      + OMEMO Encryption + This specification defines a protocol for end-to-end encryption in one-on-one chats that may have multiple clients per account. + + This XMPP Extension Protocol is copyright (c) 1999 - 2014 by the XMPP Standards Foundation (XSF). + Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation. + ## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. In no event shall the XMPP Standards Foundation or the authors of this Specification be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification. ## + In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising out of the use or inability to use the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages. + This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which may be found at <http://xmpp.org/extensions/ipr-policy.shtml> or obtained by writing to XSF, P.O. Box 1641, Denver, CO 80201 USA). + + xxxx + ProtoXEP + Standards Track + Standards + Council + + XMPP Core + XEP-0163 + + + + NOT_YET_ASSIGNED + + Andreas + Straub + andy@strb.org + andy@strb.org + + + 0.0.1 + 2015-10-25 + as +

      First draft.

       +       +
      + + +

      There are two main end-to-end encryption schemes in common use in the XMPP ecosystem, Off-the-Record (OTR) messaging (&xep0364;) and OpenPGP (&xep0027;). OTR has significant usability drawbacks for inter-client mobility. As OTR sessions exist between exactly two clients, the chat history will not be synchronized across other clients of the involved parties. Furthermore, OTR chats are only possible if both participants are currently online, due to how the rolling key agreement scheme of OTR works. OpenPGP, while not suffering from these mobility issues, does not provide any kind of forward secrecy and is vulnerable to replay attacks. Additionally, PGP over XMPP uses a custom wireformat which is defined by convention rather than standardization, and involves quite a bit of external complexity.

      +

      This XEP defines a protocol that leverages axolotl encryption to provide multi-end to multi-end encryption, allowing messages to be synchronized securely across multiple clients, even if some of them are offline.

      +       + +

      The general idea behind this protocol is to maintain separate, long-standing axolotl-encrypted sessions with each device of each contact (as well as with each of our other devices), which are used as secure key transport channels. In this scheme, each message is encrypted with a fresh, randomly generated encryption key. An encrypted header is added to the message for each device that is supposed to receive it. These headers simply contain the key that the payload message is encrypted with, and they are seperately encrypted using the session corresponding to the counterpart device. The encrypted payload is sent together with the headers as a <message> stanza. Individual recipient devices can decrypt the header item intended for them, and use the contained payload key to decrypt the payload message.

      +

      As the encrypted payload is common to all recipients, it only has to be included once, reducing overhead. Furthermore, axolotl's transparent handling of messages that were lost or received out of order, as well as those sent while the recipient was offline, is maintained by this protocol. As a result, in combination with &xep0280; and &xep0313;, the desired property of inter-client history synchronization is achieved.

      +

      OMEMO version 0 uses v3 messages of the axolotl protocol. Instead of an axolotl key server, PEP (&xep0163;) is used to publish key data.

      +       +       + +
        +
      • Provide forward secrecy
        • +
      • Ensure chat messages can be deciphered by all (capable) clients of both parties
        • +
      • Be usable regardless of the participants' online statuses
        • +
      • Provide a method to exchange auxilliary keying material. This could for example be used to secure encrypted file transfers.
        • +
      +       + + +
      +
      Device
      A communication end point, i.e. a specific client instance
       +
      OMEMO element
      An <encrypted> element in the urn:xmpp:omemo:0 namespace. Can be either MessageElement or a KeyTransportElement
       +
      MessageElement
      An OMEMO element that contains a chat message. Its <payload>, when decrypted, corresponds to a <message>'s <body>.
       +
      KeyTransportElement
      An OMEMO element that does not have a <payload>. It contains a fresh encryption key, which can be used for purposes external to this XEP.
       +
      Bundle
      A collection of publicly accessible data that can be used to build a session with a device, namely its public IdentityKey, a signed PreKey with corresponding signature, and a list of (single use) PreKeys.
       +
      rid
      The device id of the intended recipient of the containing <key>
       +
      sid
      The device id of the sender of the containing OMEMO element
       + +
      +       + +
      +
      IdentityKey
      Per-device public/private key pair used to authenticate communications
       +
      PreKey
      A Diffie-Hellman public key, published in bulk and ahead of time
       +
      PreKeyWhisperMessage
      An encrypted message that includes the initial key exchange. This is used to transparently build sessions with the first exchanged message.
       +
      WhisperMessage
      An encrypted message
       +
      +       +       + + +

      The first thing that needs to happen if a client wants to start using OMEMO is they need to generate an IdentityKey and a Device ID. The IdentityKey is a Curve25519 public/private Key pair. The Device ID is a randomly generated integer between 1 and 2^31 - 1.

      +       + +

      In order to determine whether a given contact has devices that support OMEMO, the devicelist node in PEP is consulted. Devices MUST subscribe to 'urn:xmpp:omemo:0:devicelist' via PEP, so that they are informed whenever their contacts add a new device. They MUST cache the most up-to-date version of the devicelist.

      + + + + + + + + + + + +]]> +       + +

      In order for other devices to be able to initiate a session with a given device, it first has to announce itself by adding its device ID to the devicelist PEP node.

      + + + + + + + + + + + + +]]> +

      This step presents the risk of introducing a race condition: Two devices might simultaneously try to announce themselves, unaware of the other's existence. The second device would overwrite the first one. To mitigate this, devices MUST check that their own device ID is contained in the list whenever they receive a PEP update from their own account. If they have been removed, they MUST reannounce themselves.

      +

      Furthermore, a device MUST announce it's IdentityKey, a signed PreKey, and a list of PreKeys in a separate, per-device PEP node. The list SHOULD contain 100 PreKeys, but MUST contain no less than 20.

      + + + + + + + BASE64ENCODED... + + + BASE64ENCODED... + + + BASE64ENCODED... + + + + BASE64ENCODED... + + + BASE64ENCODED... + + + BASE64ENCODED... + + + + + + + +]]> +       + +

      In order to build a session with a device, their bundle information is fetched.

      + + + + +]]> +

      A random preKeyPublic entry is selected, and used to build an axolotl session.

      +       + +

      In order to send a chat message, its <body> first has to be encrypted. The client MUST use fresh, randomly generated key/IV pairs with AES-128 in Galois/Counter Mode (GCM). For each intended recipient device, i.e. both own devices as well as devices associated with the contact, this key is encrypted using the corresponding long-standing axolotl session. Each encrypted payload key is tagged with the recipient device's ID. This is all serialized into a MessageElement, which is transmitted in a <message> as follows:

      + + +
      + BASE64ENCODED... + BASE64ENCODED... + + BASE64ENCODED... +
      + BASE64ENCODED +       + +]]>       +       + +

      The client may wish to transmit keying material to the contact. This first has to be generated. The client MUST generate a fresh, randomly generated key/IV pair. For each intended recipient device, i.e. both own devices as well as devices associated with the contact, this key is encrypted using the corresponding long-standing axolotl session. Each encrypted payload key is tagged with the recipient device's ID. This is all serialized into a KeyTransportElement, omitting the <payload> as follows:

      + +
      + BASE64ENCODED... + BASE64ENCODED... + + BASE64ENCODED... +
      +]]>       +

      This KeyTransportElement can then be sent over any applicable transport mechanism.

      +       + +

      When an OMEMO element is received, the client MUST check whether there is a <key> element with an rid attribute matching its own device ID. If this is not the case, the element MUST be silently discarded. If such an element exists, the client checks whether the element's contents are a PreKeyWhisperMessage.

      +

      If this is the case, a new session is built from this received element. The client SHOULD then republish their bundle information, replacing the used PreKey, such that it won't be used again by a different client. If the client already has a session with the sender's device, it MUST replace this session with the newly built session. The client MUST delete the private key belonging to the PreKey after use.

      +

      If the element's contents are a WhisperMessage, and the client has a session with the sender's device, it tries to decrypt the WhisperMessage using this session. If the decryption fails or if the element's contents are not a WhisperMessage either, the OMEMO element MUST be silently discarded.

      +

      If the OMEMO element contains a <payload>, it is an OMEMO message element. The client tries to decrypt the base 64 encoded contents using the key extracted from the <key> element. If the decryption fails, the client MUST silently discard the OMEMO message. If it succeeds, the decrypted contents are treated as the <body> of the received message.

      +

      If the OMEMO element does not contain a <payload>, the client has received a KeyTransportElement. The key extracted from the <key> element can then be used for other purposes (e.g. encrypted file transfer).

      +       +       + +

      Before publishing a freshly generated Device ID for the first time, a device MUST check whether that Device ID already exists, and if so, generate a new one.

      +

      Clients SHOULD NOT immediately fetch the bundle and build a session as soon as a new device is announced. Before the first message is exchanged, the contact does not know which PreKey has been used (or, in fact, that any PreKey was used at all). As they have not had a chance to remove the used PreKey from their bundle announcement, this could lead to collisions where both Alice and Bob pick the same PreKey to build a session with a specific device. As each PreKey SHOULD only be used once, the party that sends their initial PreKeyWhisperMessage later loses this race condition. This means that they think they have a valid session with the contact, when in reality their messages MAY be ignored by the other end. By postponing building sessions, the chance of such issues occurring can be drastically reduced. It is RECOMMENDED to construct sessions only immediately before sending a message.

      +

      As there are no explicit error messages in this protocol, if a client does receive a PreKeyWhisperMessage using an invalid PreKey, they SHOULD respond with a KeyTransportElement, sent in a <message> using a PreKeyWhisperMessage. By building a new session with the original sender this way, the invalid session of the original sender will get overwritten with this newly created, valid session.

      +

      If a PreKeyWhisperMessage is received as part of a &xep0313; catch-up and used to establish a new session with the sender, the client SHOULD postpone deletion of the private key corresponding to the used PreKey until after MAM catch-up is completed. If this is done, the client MUST then also send a KeyTransportMessage using a PreKeyWhisperMessage before sending any payloads using this session, to trigger re-keying. (as above) This practice can mitigate the previously mentioned race condition by preventing message loss.

      +

      As the asynchronous nature of OMEMO allows decryption at a later time to currently offline devices client SHOULD include a &xep0334; <store /> hint in their OMEMO messages. Otherwise, server implementations of &xep0313; will generally not retain OMEMO messages, since they do not contain a <body />

      +       + +

      For details on axoltol, see the specification and reference implementation.

      +

      The axolotl library's reference implementation (and presumably its ports to various other platforms) uses a trust model that doesn't work very well with OMEMO. For this reason it may be desirable to have the library consider all keys trusted, effectively disabling its trust management. This makes it necessary to implement trust handling oneself.

      +       + +

      Clients MUST NOT use a newly built session to transmit data without user intervention. If a client were to opportunistically start using sessions for sending without asking the user whether to trust a device first, an attacker could publish a fake device for this user, which would then receive copies of all messages sent by/to this user. A client MAY use such "not (yet) trusted" sessions for decryption of received messages, but in that case it SHOULD indicate the untrusted nature of such messages to the user.

      +

      When prompting the user for a trust decision regarding a key, the client SHOULD present the user with a fingerprint in the form of a hex string, QR code, or other unique representation, such that it can be compared by the user.

      +

      While it is RECOMMENDED that clients postpone private key deletion until after MAM catch-up and this standards mandates that clients MUST NOT use duplicate-PreKey sessions for sending, clients MAY delete such keys immediately for security reasons. For additional information on potential security impacts of this decision, refer to Menezes, Alfred, and Berkant Ustaoglu. "On reusing ephemeral keys in Diffie-Hellman key agreement protocols." International Journal of Applied Cryptography 2, no. 2 (2010): 154-158..

      +

      In order to be able to handle out-of-order messages, the axolotl stack has to cache the keys belonging to "skipped" messages that have not been seen yet. It is up to the implementor to decide how long and how many of such keys to keep around.

      +       + +

      This document requires no interaction with the Internet Assigned Numbers Authority (IANA).

      +       + + +

      This specification defines the following XMPP namespaces:

      +
        +
      • urn:xmpp:omemo:0
        • +
      +

      The ®ISTRAR; shall include the foregoing namespace in its registry at &NAMESPACES;, as goverened by &xep0053;.

      +       + + &NSVER; + +       + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ]]> + + + +

      Big thanks to Daniel Gultsch for mentoring me during the development of this protocol. Thanks to Thijs Alkemade and Cornelius Aschermann for talking through some of the finer points of the protocol with me. And lastly I would also like to thank Sam Whited, Holger Weiss, and Florian Schmaus for their input on the standard.

      +       +       From 1b133e60474bc1f206e97c69c81a18ad01e18690 Mon Sep 17 00:00:00 2001 From: Daniel Gultsch Date: Fri, 4 Sep 2015 11:41:22 +0200 Subject: [PATCH 104/106] introducing a protocol to negotiate omemo encrypted file tranfers over jingle --- inbox/omemo-filetransfer.xml | 182 +++++++++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+) create mode 100644 inbox/omemo-filetransfer.xml diff --git a/inbox/omemo-filetransfer.xml b/inbox/omemo-filetransfer.xml new file mode 100644 index 00000000..7b89b582 --- /dev/null +++ b/inbox/omemo-filetransfer.xml @@ -0,0 +1,182 @@ + + +%ents; +]> + + +
      + OMEMO Encrypted Jingle File Transfer + This specification defines a Jingle application for transfering encrypted files from one entity to another. The protocol is based on the regular Jingle File Transfer specification and diverges from that only in the description of the file. + + This XMPP Extension Protocol is copyright (c) 1999 - 2014 by the XMPP Standards Foundation (XSF). + Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation. + ## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. In no event shall the XMPP Standards Foundation or the authors of this Specification be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification. ## + In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising out of the use or inability to use the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages. + This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which may be found at <http://xmpp.org/extensions/ipr-policy.shtml> or obtained by writing to XSF, P.O. Box 1641, Denver, CO 80201 USA). + + xxxx + ProtoXEP + Standards Track + Standards + Council + + XMPP Core + XEP-0001 + Etc. + + + + NOT_YET_ASSIGNED + + Daniel + Gultsch + daniel@gultsch.de + daniel@gultsch.de + + + 0.0.1 + 2015-09-02 + dg +

      First draft.

       +       +
      + +

      &xep0234; describes a very flexible and powerful method for peer-to-peer file transfer with interchangable transports. Unfortunatly only some of those transports can by encrypted (e.g. &xep0261;) and none integrate into existing end-to-end encryption schemes. This specification defines an approach to encrypt the actual file before transferring it by using the OMEMO encryption. Note that the encryption can and should happen on the fly.

      +       + +
        +
      • Keep the protocol flow as close to Jingle File Transfer as possible and only change the file description
        • +
      • Keep the required changes for a client that already supports Jingle File Transfer and the OMEMO encryption to a minimum
        • +
      +       + +

      Remeo and Juliet are already engaged in a private, OMEMO encrypted conversation and want to negotiate a file transfer. The session-initiate stanza of an OMEMO encrypted file tranfer differs from the session-initiate of a regular Jingle File Transfer in that the 'file' element is wrapped in an 'encrypted' element. This reuses the KeyTransportElement from the OMEMO specification.

      + + + + + +
      + BASE64ENCODED... + BASE64ENCODED... + BASE64ENCODED... +
      + + 2015-08-30T23:24:05Z + This is a test. If this were a real file... + image/png + omemo.png + + 51158 + + +       +       + + + + +       +       + + ]]>       +

      The rest of the negotiation is analogous to a regular file transfer. The file is then encrypted and decrypted respectively using the key/IV pair extracted from the encryption header in AES-GCM. The crypto operation SHOULD happen on the fly.

      +       + +

      Even though AES-GCM comes with build-in integrity protection the hosting entity can—at any time during the lifetime of the session—communicate the checksum of the encrypted file.

      + + + + + 552da749930852c69ae5d2141d3766b1 + + + + + ]]> +       + +
        +
      • AES-GCM being a block cipher will influence the file size. The file size reported in the session intilization MUST be the encrypted one.
        • +
      • The file hash MUST be the hash of encrypted file.
        • +
      • An encryption header MUST only be used for one session. However when doing a rangend tranfer on a previously aborted file the key/IV pair MUST be reused and packed into a new header to keep the integrity of the file.
        • +
      +       + +

      To advertise its support for the OMEMO Encrypted Jingle File Transfer, when replying to service discovery information ("disco#info") requests an entity MUST return URNs for any version of this protocol that the entity supports -- e.g., 'urn:xmpp:jingle:apps:encrypted:file-transfer:0' for this version &VNOTE;.

      + + + + ]]> + + + + + + + + + + ]]> +

      In order for an application to determine whether an entity supports this protocol, where possible it SHOULD use the dynamic, presence-based profile of service discovery defined in &xep0115;. However, if an application has not received entity capabilities information from an entity, it SHOULD use explicit service discovery instead.

      +       + +

      Since the OMEMO Encryption does not provide a direct relation between resources and Device ID’s the initiating entity might not the Device ID of the resource it is creating a session with. Thus it MAY include a key for every trusted device of the foreign entity. However it SHOULD omit the keys for its own devices.

      +       + +
        +
      • Being built upon both Jingle File Transfer and OMEMO encryption, the same security considerations apply.
        • +
      • Only the raw file is being encrypted. The encryption does not apply for the meta data sent with the session initialization including the content-type and the description
        • +
      +       + +

      This document requires no interaction with the Internet Assigned Numbers Authority (IANA).

      +       + + +

      This specification defines the following XML namespace:

      +
        +
      • urn:xmpp:jingle:apps:encrypted:file-transfer:0
        • +
      +

      Upon advancement of this specification from a status of Experimental to a status of Draft, the ®ISTRAR; shall add the foregoing namespace to the registry located at &NAMESPACES;, as described in Section 4 of &xep0053;.

      +       +       + +

      tbd

      +       +       From 76b358907abc5c54252c73acbb6a6f8039e385be Mon Sep 17 00:00:00 2001 From: JC Brand Date: Mon, 26 Oct 2015 07:01:20 +0000 Subject: [PATCH 105/106] Fix typo ("cannot" repeated twice). --- xep-0333.xml | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/xep-0333.xml b/xep-0333.xml index 9ef737cb..77d851dc 100644 --- a/xep-0333.xml +++ b/xep-0333.xml @@ -10,7 +10,7 @@ This specification describes a solution of marking the last received, displayed and acknowledged message in a chat. &LEGALNOTICE; 0333 - Deferred + Experimental Standards Track Standards Council @@ -27,6 +27,12 @@ im@spencermacdonald.com im@spencermacdonald.com + + 0.2.1 + 2015-10-28 + XEP Editor (mam) +

      Fixing typo ("cannot" repeated twice) (JC Brand).

       +       0.2 2013-09-05 @@ -133,7 +139,7 @@ -

      If the sender knows only the recipient's bare JID, it cannot cannot determine (via &xep0030; or &xep0115;) whether the intended recipient supports the Chat Markers protocol. In this case, the sender MAY send a Chat Marker or markable message.

      +

      If the sender knows only the recipient's bare JID, it cannot determine (via &xep0030; or &xep0115;) whether the intended recipient supports the Chat Markers protocol. In this case, the sender MAY send a Chat Marker or markable message.

      If the sender knows a full JID for the recipient (e.g., via presence), it SHOULD attempt to determine (via service disco or entity capabilities) whether the client at that full JID supports the Chat Markers protocol before attempting to send a Chat Marker or markable message.

      From 7d57fe0d09e86345a732921d7e23fbdd9f969dc7 Mon Sep 17 00:00:00 2001 From: Sam Whited Date: Wed, 28 Oct 2015 10:56:49 -0500 Subject: [PATCH 106/106] Typo fix in entity versioning --- inbox/entityversioning.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/inbox/entityversioning.xml b/inbox/entityversioning.xml index 12a29fe0..5b87da92 100644 --- a/inbox/entityversioning.xml +++ b/inbox/entityversioning.xml @@ -40,7 +40,7 @@ 0.0.2 2015-09-17 ssw -

      Add profiles / parcial sync.

       +

      Add profiles / partial sync.

             0.0.1