From 63ef742186fbb7f1bc76b602feb8d5ddf1b892dc Mon Sep 17 00:00:00 2001 From: Dave Cridland Date: Sat, 21 Dec 2019 18:15:45 +0000 Subject: [PATCH 1/3] MAM Fastening Collation initial version --- inbox/mamfc.xml | 154 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 inbox/mamfc.xml diff --git a/inbox/mamfc.xml b/inbox/mamfc.xml new file mode 100644 index 00000000..73c67dd2 --- /dev/null +++ b/inbox/mamfc.xml @@ -0,0 +1,154 @@ + + +%ents; + + +]> + + +
+ MAM Fastening Collation + This specification proposes a mechanism by which MAM results containing fastenings can be collated effectively. + &LEGALNOTICE; + XXXX + ProtoXEP + Standards Track + Standards + + XMPP Core + + + + sasl2 + &dcridland; + &ksmithisode; + + 0.0.1 + 2019-12-19 + dwd + +
    +
  • Initial Revision
    • +
+ + +
+ + +

Fastenings, &xep0422;, provides a mechanism for a sending entity to indicate that a particular message is associated closely to another. But in doing so, this presents the problem that if many messages are not in fact actual human messages, an archive query might end up downloading dozens or even hundreds of messages in order to present just a handful of actual instant messages.

+

This specification tackles the problem by allowing these to be filtered, collated, or presented in full depending on the needs of the client.

+ +

Because this document defines mechanisms for dealing generically with potential types of fastenings, it does not offer any real examples of actual fastenings that might be used.

+

Instead, example fastenings are used within an XML namespace prefixed by &nsx;

+

Pseudo-fastenings are messages that are semantically equivalent to fastenings, but use a different syntax, see Pseudo Fastenings.

+ + + + + +

Support for this protocol is advertised by the Service Discovery protocol defined in &xep0030; using a feature of &ns;.

+ + +

This specification provides for four types of summary listing.

+

The first form, "simplified", is the default, and essentially represents the status quo. MAM queries behave as if the archive contains only traditional IM traffic.

+

The second form, "full", presents every message stanza in the results, including all fastenings, errors, and so on.

+

The third form, "collate", presents each traditional IM message, as "simplified", but within the result includes summary information about the fastenings (and pseudo-fastenings) encountered.

+

Finally a fourth form, "fastenings", returns only the fastenings for a particular message.

+

The "collate" form is the bulk of the specification presented herein.

+ +

The <apply-to/> element of &xep0422; contains a number of fastening elements. These in turn consist of a qualified element, with a number of attributes, and potentially some content within the element.

+

This specification refers to the qualified name (the tuple of XML namespace and local-name) as the "fastening type" (represented as an XML element herein), and the top-level element (including attributes and their values), as the "fastening summary".

+

For example, a hypothetical edit fastening type might be <edit xmlns="&nsx;edit:0"/>, and that would be the fastening summary as well. The full fastening would include any children (text or further XML elements) of the top-level element. But a hypothetical reaction fastening type might be <reaction xmlns="&nsx;reaction:0"/>, but the fastening summary could be <reaction xmlns="&nsx;reaction:0" emoji="Ὁ"/>

+

The summary information against each parent message consists of, for each fastening summary:

+
    +
  • The fastening summary itself.
    • +
  • A count of the number of fastenings with this summary fastened to the parent message.
    • +
  • The full fastening for the last fastening received for the parent message.
    • +
+ + + + + + +

This specification adds an additional field to the form defined in &xep0313; with the field name "{&ns;}summary". This may have only the following values (unless of course further values are advertised by a future specification):

+
    +
  • simplified
    • +
  • full
    • +
  • collate
    • +
  • fastenings
    • +
+ + +

The <result/> element defined in &xep0313; is extended by adding zero or more additional elements with a local name of "applied", qualified by the "&ns;" namespace.

+

Each <applied/> element is associated with precisely one fastening summary.

+

This element contains, as its first child element, the full fastening for the last fastening received by the server for the parent message. This is not included for "shell" fastenings, which are untyped.

+

There is a "count" attribute, consisting of an integral count of the fastenings with the same summary as the first child element (or the count of shell fastenings when this is not included). This count, if absent, defaults to 1. For "shell" fastenings, the attribute "shell" is set to true (or another value with the same semantics for an XML boolean).

+

The <applied/> elements are only included in the <result/> element when the requested summary type is "collate".

+ + +

The latest archive id is usually deductable either from the last message stanza received (and its stanza-id, see &xep0359;) or by the id attribute of the last <result/> element from a query extending to the latest message.

+

Since this specification can cause the latest message to be only in a summarized form when presented in the archive, it also adds an additional element to the <fin/> element which terminates the query, to indicate the latest id held in the archive (which may be that of a fastening).

+

This element, qualified by the "&ns;", has the local name of "latest" and a single attribute, "id", containing the latest archive id.

+ + +

A MAM query where the MAM summary type is "collate", and where "start" and "end" (or the RSM <after/> element) would exclude the parent message but include the fastening, then the MAM result is sent with the <forwarded/> element omitted but the summary present (including all fastenings, not just those that have changed).

+ + + + +

A number of previous specifications exist that - if they were rewritten today - might use fastenings.

+

For the purposes of this specification, it is convenient to treat these as pseudo-fastenings, behaving as if they were a fastening for the purposes of the "collate" and "fastenings" summary types.

+

This specification defines two such types. Both MUST be supported by servers:

+
    +
  • Message Delivery Receipts: &xep0184; "ack messages" - those containing a <received/> element - are considered to be equivalent to a fastening containing just the >received/> element, applying to the message given by the "id" attribute.
    • +
  • Chat Markers: &xep0333; A Chat Marker is similarly equivalent to a fastening containing the Chat Marker, but applying to all previous messages (since previous messages can be assumed to have been read and or displayed, etc).
    • +
+

In both cases, the fastening summary SHOULD omit the id attribute.

+ + + +

A firm TODO.

+ + + + + + + + + + + + + + + + + + + + + ]]> + + + + +

This specification imposes substantial workload for servers.

+ + + +

This XEP requires no interaction with &IANA;.

+ + + +

None.

+ + + +

The authors wish to share any credit with many members of the community, including Marvin Wissfield.

+ + + From 8bb17059ff0472b959b7bd8da8a28794f69e6f1b Mon Sep 17 00:00:00 2001 From: Dave Cridland Date: Mon, 30 Dec 2019 09:28:30 +0000 Subject: [PATCH 2/3] Tidy --- inbox/mamfc.xml | 99 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 74 insertions(+), 25 deletions(-) diff --git a/inbox/mamfc.xml b/inbox/mamfc.xml index 73c67dd2..7b84b7a4 100644 --- a/inbox/mamfc.xml +++ b/inbox/mamfc.xml @@ -17,10 +17,12 @@ Standards XMPP Core + XEP-0422 + XEP-0313 - sasl2 + mamfc &dcridland; &ksmithisode; @@ -36,30 +38,56 @@ -

Fastenings, &xep0422;, provides a mechanism for a sending entity to indicate that a particular message is associated closely to another. But in doing so, this presents the problem that if many messages are not in fact actual human messages, an archive query might end up downloading dozens or even hundreds of messages in order to present just a handful of actual instant messages.

-

This specification tackles the problem by allowing these to be filtered, collated, or presented in full depending on the needs of the client.

+

In XMPP, all messages are not equal. Some are semantically actual human messages; these are referred to in this + document as "instant messages". Others are ancillary messages - reactions, receipts, and so on - that are useful + and important, but do not conform to the concept of an instant message in the sense that a user might reasonably + expect.

+

Fastenings, &xep0422;, provides a generic mechanism for a sending entity to indicate that a particular message is + associated closely to an instant message. But in doing so, this presents the problem that if many messages are not in fact + actual human messages, an archive query might end up downloading dozens or even hundreds of messages in order to + present just a handful of actual instant messages to the user. Much of the information downloaded is not required.

+

For example, to display a message, a client may need to download all the "likes" for it - whereas a simply number of + likes might be sufficient for most users' needs.

+

This specification tackles the problem by allowing these to be filtered, collated, or presented in full depending + on the needs of the client. The client now downloads just the instant messages from the archive, and any likes, + reactions, or receipts are simply summarized.

-

Because this document defines mechanisms for dealing generically with potential types of fastenings, it does not offer any real examples of actual fastenings that might be used.

+

Because this document defines mechanisms for dealing generically with potential types of fastenings, it does not + offer any real examples of actual fastenings that might be used.

Instead, example fastenings are used within an XML namespace prefixed by &nsx;

-

Pseudo-fastenings are messages that are semantically equivalent to fastenings, but use a different syntax, see Pseudo Fastenings.

+

Pseudo-fastenings are messages that are semantically equivalent to fastenings, but use a different syntax, + see Pseudo Fastenings.

+

Nomenclature used for instant messages versus ancillary messages will need to be adjusted to make it consistent + with &xep0422; et al.

 -

Support for this protocol is advertised by the Service Discovery protocol defined in &xep0030; using a feature of &ns;.

+

Support for this protocol is advertised by the Service Discovery protocol defined in &xep0030; using a feature + of &ns;.

This specification provides for four types of summary listing.

-

The first form, "simplified", is the default, and essentially represents the status quo. MAM queries behave as if the archive contains only traditional IM traffic.

-

The second form, "full", presents every message stanza in the results, including all fastenings, errors, and so on.

-

The third form, "collate", presents each traditional IM message, as "simplified", but within the result includes summary information about the fastenings (and pseudo-fastenings) encountered.

+

The first form, "simplified", is the default, and essentially represents the status quo. MAM queries + behave as if the archive contains only traditional IM traffic. No summary is provided.

+

The second form, "full", presents every message stanza in the results, including all fastenings, + errors, and so on.

+

The third form, "collate", presents each traditional IM message, as "simplified", but within + the result includes summary information about the fastenings (and pseudo-fastenings) encountered.

Finally a fourth form, "fastenings", returns only the fastenings for a particular message.

The "collate" form is the bulk of the specification presented herein.

-

The <apply-to/> element of &xep0422; contains a number of fastening elements. These in turn consist of a qualified element, with a number of attributes, and potentially some content within the element.

-

This specification refers to the qualified name (the tuple of XML namespace and local-name) as the "fastening type" (represented as an XML element herein), and the top-level element (including attributes and their values), as the "fastening summary".

-

For example, a hypothetical edit fastening type might be <edit xmlns="&nsx;edit:0"/>, and that would be the fastening summary as well. The full fastening would include any children (text or further XML elements) of the top-level element. But a hypothetical reaction fastening type might be <reaction xmlns="&nsx;reaction:0"/>, but the fastening summary could be <reaction xmlns="&nsx;reaction:0" emoji="Ὁ"/>

+

The <apply-to/> element of &xep0422; contains a number of fastening elements. These in turn consist of a + qualified element, with a number of attributes, and potentially some content within the element.

+

This specification refers to the qualified name (the tuple of XML namespace and local-name) as the "fastening + type" (represented as an XML element herein), and the top-level element (including attributes and their + values), as the "fastening summary".

+

For example, a hypothetical edit fastening type might be <edit xmlns="&nsx;edit:0"/>, and that would + be the fastening summary as well. The full fastening would include any children (text or further XML elements) + of the top-level element. But a hypothetical reaction fastening type might be + <reaction xmlns="&nsx;reaction:0"/>, but the fastening summary could be + <reaction xmlns="&nsx;reaction:0" emoji="Ὁ"/>

The summary information against each parent message consists of, for each fastening summary:

  • The fastening summary itself.
    • @@ -72,7 +100,9 @@ -

    This specification adds an additional field to the form defined in &xep0313; with the field name "{&ns;}summary". This may have only the following values (unless of course further values are advertised by a future specification):

    +

    This specification adds an additional field to the form defined in &xep0313; with the field name + "{&ns;}summary". This may have only the following values (unless of course further values are advertised + by a future specification):

    • simplified
    • full
      • @@ -81,35 +111,54 @@
    -

    The <result/> element defined in &xep0313; is extended by adding zero or more additional elements with a local name of "applied", qualified by the "&ns;" namespace.

    +

    The <result/> element defined in &xep0313; is extended by adding zero or more additional elements with + a local name of "applied", qualified by the "&ns;" namespace.

    Each <applied/> element is associated with precisely one fastening summary.

    -

    This element contains, as its first child element, the full fastening for the last fastening received by the server for the parent message. This is not included for "shell" fastenings, which are untyped.

    -

    There is a "count" attribute, consisting of an integral count of the fastenings with the same summary as the first child element (or the count of shell fastenings when this is not included). This count, if absent, defaults to 1. For "shell" fastenings, the attribute "shell" is set to true (or another value with the same semantics for an XML boolean).

    -

    The <applied/> elements are only included in the <result/> element when the requested summary type is "collate".

    +

    This element contains, as its first child element, the full fastening for the last fastening received by the + server for the parent message. This is not included for "shell" fastenings, which are untyped.

    +

    There is a "count" attribute, consisting of an integral count of the fastenings with the same summary + as the first child element (or the count of shell fastenings when this is not included). This count, if absent, + defaults to 1. For "shell" fastenings, the attribute "shell" is set to true (or another value + with the same semantics for an XML boolean).

    +

    The <applied/> elements are only included in the <result/> element when the requested + summary type is "collate".

     -

    The latest archive id is usually deductable either from the last message stanza received (and its stanza-id, see &xep0359;) or by the id attribute of the last <result/> element from a query extending to the latest message.

    -

    Since this specification can cause the latest message to be only in a summarized form when presented in the archive, it also adds an additional element to the <fin/> element which terminates the query, to indicate the latest id held in the archive (which may be that of a fastening).

    -

    This element, qualified by the "&ns;", has the local name of "latest" and a single attribute, "id", containing the latest archive id.

    +

    The latest archive id can usually be deduced either from the last message stanza received (and its stanza-id, + see &xep0359;) or by the id attribute of the last <result/> element from a query extending to the + latest message.

    +

    Since this specification can cause the latest message to be only in a summarized form when presented in the + archive, it also adds an additional element to the <fin/> element which terminates the query, to + indicate the latest id held in the archive (which may be that of a fastening).

    +

    This element, qualified by the "&ns;", has the local name of "latest" and a single attribute, + "id", containing the latest archive id.

     -

    A MAM query where the MAM summary type is "collate", and where "start" and "end" (or the RSM <after/> element) would exclude the parent message but include the fastening, then the MAM result is sent with the <forwarded/> element omitted but the summary present (including all fastenings, not just those that have changed).

    +

    A MAM query where the MAM summary type is "collate", and where "start" and "end" (or + the RSM <after/> element) would exclude the parent message but include the fastening, then the MAM + result is sent with the <forwarded/> element omitted but the summary present (including all + fastenings, not just those that have changed).

    A number of previous specifications exist that - if they were rewritten today - might use fastenings.

    -

    For the purposes of this specification, it is convenient to treat these as pseudo-fastenings, behaving as if they were a fastening for the purposes of the "collate" and "fastenings" summary types.

    +

    For the purposes of this specification, it is convenient to treat these as pseudo-fastenings, behaving as if they + were a fastening for the purposes of the "collate" and "fastenings" summary types.

    This specification defines two such types. Both MUST be supported by servers:

      -
    • Message Delivery Receipts: &xep0184; "ack messages" - those containing a <received/> element - are considered to be equivalent to a fastening containing just the >received/> element, applying to the message given by the "id" attribute.
      • -
    • Chat Markers: &xep0333; A Chat Marker is similarly equivalent to a fastening containing the Chat Marker, but applying to all previous messages (since previous messages can be assumed to have been read and or displayed, etc).
      • +
    • Message Delivery Receipts: &xep0184; "ack messages" - those containing a <received/> element - are + considered to be equivalent to a fastening containing just the <received/> element, applying to the message + given by the "id" attribute.
      • +
    • Chat Markers: &xep0333; A Chat Marker is similarly equivalent to a fastening containing the Chat Marker, but + applying to all previous messages (since previous messages can be assumed to have been read and or displayed, + etc).

    In both cases, the fastening summary SHOULD omit the id attribute.

     -

    A firm TODO.

    +

    A firm TODO; contributions are welcome - and a good test of whether I've written the rest right!

     From eb5351440e46bcd23e7685a7eda078bb6205dc05 Mon Sep 17 00:00:00 2001 From: Dave Cridland Date: Mon, 30 Dec 2019 12:20:46 +0000 Subject: [PATCH 3/3] UDT Initial ProtoXEP --- inbox/udt.xml | 197 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 inbox/udt.xml diff --git a/inbox/udt.xml b/inbox/udt.xml new file mode 100644 index 00000000..27fbb9c8 --- /dev/null +++ b/inbox/udt.xml @@ -0,0 +1,197 @@ + + +%ents; + + +]> + + +
    + User-defined Data Transfer + This specification proposes a simple mechanism by which applications can transfer data safely, without + needing additional protocol design work. It is intended to provide a protocol that is trivial to implement and can + be driven with a simple API. + &LEGALNOTICE; + XXXX + ProtoXEP + Standards Track + Standards + + XMPP Core + + + + udt + &dcridland; + + 0.0.1 + 2019-12-30 + dwd + +
      +
    • Initial Revision
      • +
    +     +     +
    + + +

    Applications written on top of XMPP often need to exchange data that has no existing standard. Such applications are + often written by developers unfamiliar with best practise in designing new extensions for XMPP, making it hard to achieve + this simple design goal without causing longer term problems.

    +

    This leads to "solutions" such as stuffing JSON directly in the <body/> element, for example, and recognising + this at the receiver either by heuristics or by a special <subject/>. While this works, it's difficult to then + migrate to something else, and enforces that custom clients are always used.

    +

    Therefore this document proposes a very simple (and simplistic) framework for sending such data which - while + very light on features - nevertheless conforms to best practice. Unusually, this specification SHOULD NOT be used + as a base upon which to build other standards, and suggests an API for library developers to implement.

    + +

    Data transferred using this specification is encoded using JSON. The type of the data is given by a URI under + the same rules as an XML namespace, and this specification refers to this as the datatype.

    +

    Because this document defines mechanisms for sending essentially arbitrary data, no real-world examples are + given.

    +

    Instead, example namespaces are used within an XML namespace prefixed by &nsx;

    +     +     + + + +

    Support for this protocol is advertised by the Service Discovery protocol defined in &xep0030; using a feature + of &ns;.

    +

    Support for a particular datatype is given by concatenating the &ns; feature with a hash character + ('#') and the datatype, for example &ns;#&nsx;foo.

    +     + +

    This specification provides for two types of user-defined data transfers. Each uses a similar payload syntax, + the UDT data payload.

    +

    Requests are carried within <iq/> stanzas, either of type "get" or "set", and result in a "result" optionally + containing a second UDT data payload.

    +

    UDT payloads may also be placed within a <message/> stanza. <message/> stanzas MAY contain multiple UDT + payloads, but typical usage is expected to be that there will be only one. The UDT payload may be ancillary data + to another message, or a standalone message in its own right.

    + +

    A UDT payload consists of a single element, <payload/>, qualified by the XML namespace + &ns;. It has a single, mandatory attribute of datatype, which MUST contain a string conformant + to the requirements for XML namespaces (typically a URI under the control of the application developer).

    +

    As with XML namespaces, this URI is never expected to be resolved, and is used solely as an indentifier. + Different strings are considered entirely different datatypes, and common prefixes etc MUST be considered + irrelevant for the purposes of interpreting the data. There are no common or standard datatypes.

    +

    The <payload element contains exactly one mandatory child element, the <json/> element + defined in &xep0335;. This in turns contains the JSON data.

    + + + + { + "annoying-teenager-level": 11 + } + + + +]]> +

    Note that the suggested custom &IQ; query payload of &xep0335; is not used as this would generally require + custom handlers within client libraries:

    + + + + { + "annoying-teenager-percentage": 101 + } + + + +]]> +     +     +     + + +

    In order to satisfy the goals of this protocol, it is necessary to define an API that can be consistently + implemented across APIs. This allows application developers to use the protocol easily, and encourages this + over using the ad-hoc techniques described in the introduction.

    +

    Therefore, while not imposing hard and fast API definitions, this specification proposes naming conventions + for APIs that will hopefully guide application developers toward consistent usage.

    +

    While names are specified in "snake_case", API developers are free to use their own naming. This specification + also defines APIs as taking session arguments which will often be implied by method calls, and omits types.

    +

    Library developers SHOULD make these calls as simple to use as possible. If these are significantly harder to use + for inexperienced developers than ad-hoc techniques, then ad-hoc techniques will be used instead.

    + +
      +
    • session - The session, connection or stream object or handle for the library. In cases where the + library uses a single global connection this is omitted. An OO library is likely to elide this and have the call + be a method call on the object instead.
      • +
    • message - The message object or handle for the library. An OO library is likely to elide this and have the call + be a method call on the object instead.
      • +
    • datatype - Typically a string containing an XML-style namespace.
      • +
    • jid - A string or JID object. Any form of legal jid might be used here.
      • +
    • data - Either a JSON-encoded string, or an object or data structure which can be converted by the + library.
      • +
    • get_or_set - A flag for indicating whether this is a "get" or "set" request.
      • +
    • callback - Some callable object or similar as meets the idiom of the language.
      • +
    +     + +

    Support is advertised on a session by calling udt_advertise(session, datatype). Calling this MUST + explicitly advertise both the &ns; feature and that of the datatype support.

    +

    APIs are free to (and encouraged to) implicitly advertise support when other calls are made.

    +     + +

    Requests are sent to a particular jid by calling udt_request(session, jid, get_or_set, datatype, data). + This might return a UDT payload, or have an additional callback argument to call with the response.

    +

    Applications may register to handle such requests by calling udt_request_callback(session, get_or_set, + datatype, callback). The callback should be called as callback(session, jid, get_or_set, datatype, + data), and returning a payload should send a result containing it. Applications registering this + way SHOULD implicitly advertise support for the datatype.

    +     + +

    Messages are sent to a particular jid by calling udt_message(session, jid, get_or_set, datatype, data).

    +

    Applications may register to handle such requests by calling udt_message_callback(session, get_or_set, + datatype, callback). The callback should be called as callback(session, jid, get_or_set, datatype, + data). Applications registering this way SHOULD implicitly advertise support for the datatype.

    +

    Applications may check to see if any message stanza contains a UDT payload by calling udt_payload(message, + datatype), which returns the payload if it exists.

    +     +     + + + + + + + + + + + + + + + ]]> + + + + +

    All security implications herein are those of the payload.

    +     + + +

    This XEP requires no interaction with &IANA;.

    +     + + +

    None.

    +     + + +

    The authors wish to share any credit with many members of the community, including Florian Schmaus.

    +     + +