xeps/xep-0313.xml

1069 lines
61 KiB
XML
Raw Normal View History

2012-04-18 21:05:41 -04:00
<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE xep SYSTEM 'xep.dtd' [
<!ENTITY % ents SYSTEM 'xep.ent'>
%ents;
]>
<?xml-stylesheet type='text/xsl' href='xep.xsl'?>
<xep>
<header>
<title>Message Archive Management</title>
2013-06-11 16:28:52 -04:00
<abstract>This document defines a protocol to query and control an archive of messages stored on a server.</abstract>
2012-04-18 21:05:41 -04:00
&LEGALNOTICE;
<number>0313</number>
<status>Draft</status>
2021-03-16 16:14:00 -04:00
<lastcall>2021-03-30</lastcall>
<lastcall>2017-11-15</lastcall>
2012-04-18 21:05:41 -04:00
<type>Standards Track</type>
<sig>Standards</sig>
<dependencies>
<spec>XMPP Core</spec>
<spec>XEP-0030</spec>
<spec>XEP-0059</spec>
<spec>XEP-0297</spec>
</dependencies>
<supersedes/>
<supersededby/>
<shortname>mam</shortname>
<schemaloc>
<url>http://www.xmpp.org/schemas/archive-management.xsd</url>
</schemaloc>
&mwild;
&ksmith;
2021-11-02 16:35:43 -04:00
<revision>
<version>1.0.0</version>
<date>2021-11-02</date>
<initials>XEP Editor (jsc)</initials>
<remark>Advance to Stable as per Council Vote from 2021-10-27.</remark>
</revision>
<revision>
<version>0.8.0</version>
<date>2021-10-12</date>
<initials>ks</initials>
<remark>
<p>Update groupchat-messages-in-user-archive advice, introducing fields and disco features to make behaviour explicit in future implementations, in light of Last Call feedback.</p>
</remark>
</revision>
<revision>
<version>0.7.5</version>
<date>2021-09-24</date>
<initials>ka</initials>
<remark>
<p>End some sentences with full stop.</p>
</remark>
</revision>
<revision>
<version>0.7.4</version>
<date>2021-08-23</date>
<initials>ssw</initials>
<remark>
<p>Add implementation note about before/after and before-id/after-id</p>
</remark>
</revision>
2021-03-02 11:23:35 -05:00
<revision>
<version>0.7.3</version>
<date>2021-03-02</date>
<initials>ssw</initials>
<remark>
<p>Add registrar section; fix another broken link to MAM-PUBSUB</p>
</remark>
</revision>
2020-09-29 10:27:44 -04:00
<revision>
<version>0.7.2</version>
<date>2020-09-29</date>
<initials>mar-v-in</initials>
<remark>
<p>Clean up references after MAM split</p>
</remark>
</revision>
2020-08-04 11:43:09 -04:00
<revision>
<version>0.7.1</version>
<date>2020-08-04</date>
<initials>rufferson</initials>
<remark>
<p>Fix missing part of sentence to make more sense</p>
</remark>
</revision>
<revision>
<version>0.7.0</version>
<date>2020-03-20</date>
<initials>mw</initials>
<remark>
<p>Add 'before-id' and 'after-id' fields, flipped pages, single-item retrieval and a new mandatory disco feature</p>
<p>Split preferences protocol into a separate document</p>
<p>Split the details of pubsub archives into a separate document</p>
</remark>
</revision>
<revision>
<version>0.6.3</version>
<date>2018-07-16</date>
<initials>ks</initials>
<remark>Tweak the 'complete' wording to more clearly reflect original intent</remark>
</revision>
2018-07-05 10:08:32 -04:00
<revision>
<version>0.6.2</version>
<date>2018-07-05</date>
<initials>bg</initials>
<remark>Fix invalid statement about 'to' address of archive stanzas</remark>
</revision>
<revision>
<version>0.6.1</version>
<date>2017-02-22</date>
<initials>mw</initials>
<remark>
<p>Warn of spoofing of &lt;x&gt; elements in MUC stanzas</p>
</remark>
</revision>
2017-02-18 14:43:51 -05:00
<revision>
<version>0.6</version>
<date>2017-02-17</date>
2017-02-19 11:21:21 -05:00
<initials>dg, mw</initials>
2017-02-18 14:43:51 -05:00
<remark>
<p>Namespace bump from mam:1 to mam:2</p>
2017-02-18 14:43:51 -05:00
<p>Added method for server to communicate the archive id</p>
<p>Incorporated editorial clarifications based on implementation feedback</p>
<p>Clarifications on the topics of message ordering, message deletion and use of the protocol for synchronization</p>
</remark>
</revision>
<revision>
<version>0.5.1</version>
<date>2016-03-07</date>
<initials>XEP Editor (ssw)</initials>
<remark>
<p>Bump old namespace version in examples (Kevin Smith)</p>
</remark>
</revision>
<revision>
<version>0.5</version>
<date>2016-01-20</date>
<initials>XEP Editor (mam)</initials>
<remark>
<p>Added clarifications and enhancements when interacting with Multi-User Chat (MUC) (Mickaël Rémond)</p>
</remark>
</revision>
<revision>
<version>0.4.1</version>
<date>2015-09-29</date>
<initials>XEP Editor (mam)</initials>
<remark>
<p>Fix Example #9 from &lt;/message/&gt; to &lt;/iq&gt; (Florian Schmaus)</p>
<p>Add missing jabber:client namespace in Example #2 (Christian Schudt)</p>
</remark>
</revision>
2015-01-23 05:16:32 -05:00
<revision>
<version>0.4</version>
2015-09-21 11:42:28 -04:00
<date>2015-09-21</date>
2015-01-23 05:16:32 -05:00
<initials>ks/mw</initials>
2015-09-21 11:42:28 -04:00
<remark>
<p>Switch the sentinel message back to the iq result.</p>
<p>Various small fixes to the document.</p>
</remark>
2015-01-23 05:16:32 -05:00
</revision>
2014-08-15 10:58:59 -04:00
<revision>
<version>0.3</version>
<date>2014-08-14</date>
<initials>ka/ks</initials>
<remark><p>Fetching current preferences,
switch to iq-set for searching,
switch to using a data form,
describe how to fetch that form, remove the archived element and
use a sentinel message instead of iq reply.</p></remark>
</revision>
2013-06-11 16:28:52 -04:00
<revision>
<version>0.2</version>
<date>2013-05-31</date>
<initials>mw</initials>
<remark><p>Document the ability to page through results by message UIDs, define the &lt;archived/&gt; element, and various minor improvements.</p></remark>
</revision>
2012-04-18 21:05:41 -04:00
<revision>
<version>0.1</version>
2012-04-18 21:08:01 -04:00
<date>2012-04-18</date>
2012-04-18 21:05:41 -04:00
<initials>mw</initials>
<remark><p>Initial version, to much rejoicing.</p></remark>
</revision>
</header>
<section1 topic='Introduction' anchor='intro'>
2014-08-15 10:58:59 -04:00
<p>It is a common desire for users of XMPP to want to store their messages in a central archive
2012-04-18 21:05:41 -04:00
on their server. This feature allows them to record conversations that take place on clients that do not
2014-08-15 10:58:59 -04:00
support local history storage, to synchronise conversation history seamlessly between
multiple clients, to read the history of a MUC room, or to view old items in a pubsub node.</p>
2012-04-18 21:05:41 -04:00
</section1>
2013-06-11 16:28:52 -04:00
<section1 topic='Requirements' anchor='requirements'>
2012-04-18 21:05:41 -04:00
<p>As this extension aims to make things easy for client developers, some research was made into the
way clients handle history today. The resulting protocol was designed to allow for the following primary
usage scenarios:</p>
<ul>
<li>Automatic history synchronization between multiple clients.</li>
<li>Calendar-based on-demand display of historic messages in a client that doesn't keep local history.</li>
2014-08-15 10:58:59 -04:00
<li>So-called 'infinite' scrollback, whereby a client automatically fetches and displays historical messages
2012-04-18 21:05:41 -04:00
naturally in the message log as the user scrolls back in time.</li>
</ul>
<p>Another extension for archiving already exists in XMPP, &xep0136;). However implementation experience has
shown that the protocol defined therein supports rather more functionality than is typically needed for the
above uses, and is significantly more effort to implement.</p>
<p>This specification aims to define a much simpler and modular protocol for working with a server-side
message store. Through this it is hoped to boost implementation and deployment of archiving in XMPP. It
should be noted that (although not required) a server is free to implement XEP-0136 alongside this
protocol if it so chooses, though a mapping between both protocols is beyond the scope of this specification.</p>
<p>Notable functionality in XEP-0136 that is intentionally not defined by this specification for simplicity:</p>
<ul>
<li>Support for 'collections'. Few clients even support this concept for local history storage, and
it is possible to apply the logic for splitting a stream of messages into conversations on the
client-side, thus greatly simplifying the protocol.</li>
<li>Support for uploading to the archive. On the assumption that a server automatically archives
messages to and from the client, it is typically rare for a client to end up with messages that are
not already in the archive. Uploading could be useful for bootstrapping an empty archive however, and
may be defined in a future specification if there is demand for such functionality.</li>
<li>Support for 'off the record' chats (OTR; not to be confused with the encryption algorithm of
the same name). This allowed complex negotiation for either the user or contact to command specific
conversations to bypass the archive. In reality the archiving behaviour of a contact's server cannot
be enforced (they could ignore the OTR request and archive the messages anyway without your consent)
so this specification does not try and regulate that. Equivalent functionality can be implemented with
server logic for not including encrypted or specially-tagged messages in the archive (out of scope for
this specification).</li>
<li>Support per-session configuration. This feature was deemed unnecessary for the majority of
implementations, and hence the configuration protocol in this specification is much simplified,
which allows for a simple user interface in clients. Advanced configuration however may be performed
through ad-hoc commands depending on the server implementation.</li>
</ul>
</section1>
2013-06-11 16:28:52 -04:00
<section1 topic='Message archives' anchor='archives'>
<p>An archive contains a collection of messages relevant to a particular XMPP address, e.g. a user, MUC, pubsub node, server. Note: while a service might have many "archives" as defined here (one per JID capable of being queried) this is a conceptual distinction,
2014-08-15 10:58:59 -04:00
and a server is not bound to any particular implementation or arrangement of data stores.</p>
<p>Exactly which messages a server archives is up to implementation and deployment policy,
but it is expected that all messages that hold meaningful content, rather than state changes such as Chat State Notifications, would be archived. Rules are specified later in this document.</p>
<p>A stored message consists of at least the following pieces of information:</p>
<ul>
<li>A timestamp of when the message was sent (for an outgoing message) or received (for
an incoming message).</li>
<li>The remote JID that the stanza is to (for an outgoing message) or from (for an
incoming message).</li>
<li>A server-assigned UID that MUST be unpredictable and unique within the archive.</li>
<li>The message stanza itself. The entire original stanza SHOULD be stored, but at a minimum only the &BODY; tag MUST
be preserved (ie. the server might, at its discretion, strip certain extensions from messages before storage), in
addition to all standard attributes of the stanza (e.g. to, from, type, id).</li>
2014-08-15 10:58:59 -04:00
</ul>
<p>Note that 'incoming' and 'outgoing' messages are viewed within the context of the archived JID, rather than the system as a whole. For example, if romeo@montegue.lit sent a message to juliet@capulet.lit, it would be an outgoing message in the context of archiving for Romeo, and an incoming message in the context of archiving for Juliet.</p>
2017-02-18 14:43:51 -05:00
<section2 topic='Order of messages' anchor='archive_order'>
<p>Order within the archive MUST be preserved, where the order of messages is the same as the order that the client originally received them (or would have received them if online). Throughout this document the term 'chronological order' refers to this order, however implementors should take care not to rely on timestamps alone for
ordering messages, as multiple messages may share the same timestamp.</p>
</section2>
<section2 topic='Message retention and deletion' anchor='archives_deletion'>
<p>A server MAY impose limits on the size of an individual archive. For example a server might begin
to discard old messages once the archive reaches a certain size, or only keep messages until they
reach a certain age. Any such deleted messages MUST be the oldest in the archive, i.e. it is not permitted
to create gaps or "holes" in the archive. The UIDs of deleted messages MUST NOT be reused for new messages.</p>
<p>However a server that wishes to remove messages from the middle of an archive, e.g. to remove accidentally transmitted
2017-02-18 14:43:51 -05:00
sensitive information may omit the &lt;message&gt; stanza from inside the &lt;forwarded&gt; element or replace the
message with an appropriate placeholder when transmitting the result in response to a query. However servers
MUST retain the UID, timestamp and JID of the original message internally to ensure that all queries remain consistent.
It should also be understood that clients maintaining their own local copy of the archive may still retain the original
message locally in this case, and this protocol provides no mechanism for forcibly removing messages from any local archive
or cache that clients may keep.</p>
</section2>
<section2 topic='Archiving entities' anchor='archiving_entities'>
<p>There is no restriction on which services can expose archives, although only user and MUC archives are discussed here.</p>
2017-02-18 14:43:51 -05:00
<section3 topic='User archives' anchor='archives_user'>
2014-08-15 10:58:59 -04:00
<p>The most typical address is that of a user's own bare JID, within which those messages sent to or from that
user's account would generally automatically be stored by the server. The collection
is ordered chronologically by the time each message was sent/received.</p>
2015-01-23 05:34:50 -05:00
2017-02-18 14:43:51 -05:00
<p>Servers that expose archive messages of sent/received messages on behalf of local users MUST expose these archives to the user on the user's bare JID.</p>
</section3>
2018-01-30 07:19:46 -05:00
<section3 topic='MUC archives' anchor='archives_muc'>
<p>A MUC service allowing MAM queries for a room MUST expose the MAM archive on the room's bare JID.</p>
2017-02-18 14:43:51 -05:00
</section3>
2014-08-15 10:58:59 -04:00
</section2>
<section2 topic='Querying Entities' anchor='entities'>
<p>While this document talks about 'clients' and 'servers', as these are the common cases, the querying entity (referred to as a 'client') need not be an XMPP client as defined by RFC6120, but could potentially be any type of entity, and the queried entity (referred to as a 'server') need not be an XMPP server as defined by RFC6120, although access controls might prohibit any given entity from being able to access an archive.</p>
</section2>
2017-02-18 14:43:51 -05:00
<section2 topic='Communicating the archive ID' anchor='archives_id'>
<p>When a message is archived, the server MUST add an &lt;stanza-id/&gt; element as defined in &xep0359; to the message, which informs the recipient of where and under what ID the message is stored. When doing this the server MUST follow the business rules defined in XEP-0359. The 'by' attribute MUST be set to the address of the archive. For regular users thats the bare JID of the account and for MUC thats the bare JID of the room.</p>
<p>Servers MUST NOT include the &lt;stanza-id/&gt; element in messages addressed to JIDs that do not have permissions to access the archive, such as a userss outgoing messages to their contacts. However servers SHOULD include the element as a child of the forwarded message when using &xep0280;</p>
<example caption='Client receives a message that has been archived'><![CDATA[
<message to='juliet@capulet.lit/balcony'
from='romeo@montague.lit/orchard'
2017-02-18 14:43:51 -05:00
type='chat'>
<body>Call me but love, and I'll be new baptized; Henceforth I never will be Romeo.</body>
<stanza-id xmlns='urn:xmpp:sid:0' by='juliet@capulet.lit' id='28482-98726-73623' />
</message>]]></example>
<p>Note: Previous versions of this protocol did not specify any interaction with stanza-id, and clients MUST NOT interpret XEP-0359 IDs in messages as archive IDs unless the server advertises support for 'urn:xmpp:mam:2' specifically.</p>
</section2>
2012-04-18 21:05:41 -04:00
</section1>
2014-08-15 10:58:59 -04:00
<section1 topic='Querying an archive' anchor='query'>
<p>An entity is able to query (subject to appropriate access rights) an archive for all messages within a certain timespan, optionally
2012-04-18 21:05:41 -04:00
restricting results to those to/from a particular JID. To allow limiting the results or paging
2014-08-15 10:58:59 -04:00
through them a client may use &xep0059;, which MUST be supported by both the client and the server.</p>
2017-02-18 14:43:51 -05:00
<p>A query consists of an &IQ; stanza of type 'set' addressed to the account or server entity hosting
2012-04-18 21:05:41 -04:00
the archive, with a 'query' payload. On receiving the query, the server pushes to the client a
2017-02-18 14:43:51 -05:00
series of messages in chronological order from the archive that match the client's given criteria. After the
results it then returns the &IQ; result to indicate that the query is completed.</p>
<p>The final &IQ; result response MUST include an RSM &lt;set/&gt; element, wrapped into a &lt;fin/&gt;
2017-02-18 14:43:51 -05:00
element qualified by the 'urn:xmpp:mam:2' namespace, indicating the
2014-08-15 10:58:59 -04:00
UID of the first and last message of the (possibly limited) result set. This
allows clients to accurately page through messages.</p>
<example caption='A user queries their archive for messages'><![CDATA[
<iq type='set' id='juliet1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2' queryid='f27' />
2015-01-23 05:34:50 -05:00
</iq>]]></example>
2012-04-18 21:05:41 -04:00
2015-01-23 05:34:50 -05:00
<example caption='Their server sends the matching messages'><![CDATA[
<message id='aeb213' to='juliet@capulet.lit/chamber'>
2017-02-18 14:43:51 -05:00
<result xmlns='urn:xmpp:mam:2' queryid='f27' id='28482-98726-73623'>
<forwarded xmlns='urn:xmpp:forward:0'>
<delay xmlns='urn:xmpp:delay' stamp='2010-07-10T23:08:25Z'/>
<message xmlns='jabber:client' from="witch@shakespeare.lit" to="macbeth@shakespeare.lit">
<body>Hail to thee</body>
</message>
</forwarded>
</result>
2015-01-23 05:34:50 -05:00
</message>]]></example>
<example caption='Server returns the result IQ to signal the end'><![CDATA[
<iq type='result' id='juliet1'>
2017-02-18 14:43:51 -05:00
<fin xmlns='urn:xmpp:mam:2'>
<set xmlns='http://jabber.org/protocol/rsm'>
<first index='0'>28482-98726-73623</first>
<last>09af3-cc343-b409f</last>
</set>
</fin>
</iq>]]></example>
2017-02-18 14:43:51 -05:00
<p>To ensure that the client knows when the results are complete, the server MUST send the &IQ; result after last query result has been sent
to the client. The client can optionally include a 'queryid' attribute in their query, which allows the client to match results to their initiating query.</p>
<example caption="A user queries an archive for messages"><![CDATA[
2014-08-15 10:58:59 -04:00
<iq to='pubsub.shakespeare.lit' type='set' id='juliet1'>
<query xmlns='urn:xmpp:mam:2' queryid='f28' />
2014-08-15 10:58:59 -04:00
</iq>
]]></example>
2013-06-11 16:28:52 -04:00
<section2 topic='Filtering results' anchor='filter'>
<p>By default all messages match a query, and filters are used to request a subset of the archived
2017-02-18 14:43:51 -05:00
messages. Filters are specified in a &xep0004; data form included with the query. The hidden FORM_TYPE field
MUST be set to this protocol's namespace, 'urn:xmpp:mam:2'. Six further fields are defined by this
2017-02-19 11:21:21 -05:00
XEP and MUST be supported by servers, though all of them are optional for the client. These fields are:</p>
2014-08-15 10:58:59 -04:00
<ul>
2017-02-18 14:43:51 -05:00
<li>start</li>
<li>end</li>
<li>with</li>
<li>before-id (*)</li>
<li>after-id (*)</li>
<li>ids (*)</li>
2014-08-15 10:58:59 -04:00
</ul>
<p>Servers supporting fields marked with an asterisk (*) MUST advertise the disco feature 'urn:xmpp:mam:2#extended' and clients
that depend on these fields MUST verify that the server advertises this feature before attempting to use them.</p>
2017-02-19 11:21:21 -05:00
<p>Other fields may be used, but are not defined in this document - the naming of new fields MUST be
2017-02-18 14:43:51 -05:00
consistent with the format defined in &xep0068;. Servers MUST NOT mark any fields in the form as
being required (i.e. with the data forms &lt;required/&gt; element), regardless of whether they are
defined in this document or elsewhere.</p>
2013-06-11 16:28:52 -04:00
<section3 topic='Filtering by JID' anchor='filter-jid'>
2014-08-15 10:58:59 -04:00
<p>If a 'with' field is present in the form, it contains a JID against which to match messages. The
server MUST only return messages if they match the supplied JID. A message in a user's archive matches if the JID matches either the to or from of the message. An item in a MUC archive matches if the publisher of the item matches the JID; note that this should only be available to entities that would already have been allowed to know the publisher of the events (e.g. this could not be used by a visitor to a semi-anonymous MUC).</p>
<p>To allow querying for messages the user sent to themselves, the client needs to set the 'with' attribute to the account JID. In that case, the server MUST only return results where both the 'to' and 'from' match the bare JID (either as bare or by ignoring the resource), as otherwise every message in the archive would match.</p>
2014-08-15 10:58:59 -04:00
<p>If 'with' is omitted, the server MUST match all messages in the selected timespan with the query,
2012-04-18 21:05:41 -04:00
regardless of the to/from addresses on each message.</p>
<example caption='Querying for all messages to/from a particular JID'><![CDATA[
2014-08-15 10:58:59 -04:00
<iq type='set' id='juliet1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'>
2015-01-23 05:16:32 -05:00
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
2017-02-18 14:43:51 -05:00
<value>urn:xmpp:mam:2</value>
2014-08-15 10:58:59 -04:00
</field>
<field var='with'>
<value>juliet@capulet.lit</value>
</field>
</x>
2012-04-18 21:05:41 -04:00
</query>
</iq>
2017-02-18 14:43:51 -05:00
]]></example>
2012-04-18 21:05:41 -04:00
<p>If (and only if) the supplied JID is a bare JID (i.e. no resource is present), then
2014-08-15 10:58:59 -04:00
the server SHOULD return messages if their bare to/from address for a user archive, or from address otherwise, would match it. For example,
if the client supplies a 'with' of "juliet@capulet.lit" a query to their own archive would also match messages to
2013-06-11 16:28:52 -04:00
or from "juliet@capulet.lit/balcony" and "juliet@capulet.lit/chamber".</p>
2012-04-18 21:05:41 -04:00
</section3>
2013-06-11 16:28:52 -04:00
<section3 topic='Filtering by time received' anchor='filter-time'>
2014-08-15 10:58:59 -04:00
<p>The 'start' and 'end' fields, if provided, MUST contain timestamps
2012-04-18 21:05:41 -04:00
formatted according to the DateTime profile defined in &xep0082;</p>
2014-08-15 10:58:59 -04:00
<p>The 'start' field is used to filter out messages before a certain date/time.
2012-04-18 21:05:41 -04:00
If specified, a server MUST only return messages whose timestamp is equal to or later
than the given timestamp.</p>
2014-08-15 10:58:59 -04:00
<p>If omitted, the server SHOULD assume the value of 'start' to be equal to the
2012-04-18 21:05:41 -04:00
date/time of the earliest message stored in the archive.</p>
2014-08-15 10:58:59 -04:00
<p>Conversely, the 'end' field is used to exclude from the results messages
2012-04-18 21:05:41 -04:00
after a certain point in time. If specified, a server MUST only return messages whose
2014-08-15 10:58:59 -04:00
timestamp is equal to or earlier than the timestamp given in the 'end' field.</p>
<p>If omitted, the server SHOULD assume the value of 'end' to be equal to the
2012-04-18 21:05:41 -04:00
date/time of the most recent message stored in the archive.</p>
<example caption='Querying the archive for all messages in a certain timespan'><![CDATA[
2014-08-15 10:58:59 -04:00
<iq type='set' id='juliet1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'>
2015-01-23 05:16:32 -05:00
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
2017-02-18 14:43:51 -05:00
<value>urn:xmpp:mam:2</value>
2014-08-15 10:58:59 -04:00
</field>
<field var='start'>
<value>2010-06-07T00:00:00Z</value>
</field>
<field var='end'>
<value>2010-07-07T13:23:54Z</value>
</field>
</x>
2012-04-18 21:05:41 -04:00
</query>
</iq>
2017-02-18 14:43:51 -05:00
]]></example>
2012-04-18 21:05:41 -04:00
<example caption='Querying the archive for all messages after a certain time'><![CDATA[
2014-08-15 10:58:59 -04:00
<iq type='set' id='juliet1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'>
2015-01-23 05:16:32 -05:00
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
2017-02-18 14:43:51 -05:00
<value>urn:xmpp:mam:2</value>
2014-08-15 10:58:59 -04:00
</field>
<field var='start'>
<value>2010-08-07T00:00:00Z</value>
</field>
</x>
2012-04-18 21:05:41 -04:00
</query>
</iq>
2017-02-18 14:43:51 -05:00
]]></example>
2012-04-18 21:05:41 -04:00
</section3>
<section3 topic='Limiting results by id' anchor='query-limit-id'>
<p>If the client has already seen some messages, it may choose to restrict its query to
before and/or after messages it already knows about. This may be done through the 'before-id'
and 'after-id' fields.
In some cases 'before-id' and 'after-id' are the same as using RSM's
'before' and 'after' parameters. For more information, see the
<link url='#impl'>Implementation Considerations</link> section.
</p>
<example caption='Querying the archive for all messages after a certain message'><![CDATA[
<iq type='set' id='juliet1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'>
2015-01-23 05:16:32 -05:00
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
2017-02-18 14:43:51 -05:00
<value>urn:xmpp:mam:2</value>
2014-08-15 10:58:59 -04:00
</field>
<field var='after-id'>
<value>09af3-cc343-b409f</value>
2014-08-15 10:58:59 -04:00
</field>
</x>
2012-04-18 21:05:41 -04:00
</query>
</iq>
2017-02-18 14:43:51 -05:00
]]></example>
2015-01-23 05:34:50 -05:00
<example caption='Querying the archive for all messages between two known messages'><![CDATA[
<iq type='set' id='juliet1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
<value>urn:xmpp:mam:2</value>
</field>
<field var='after-id'>
<value>28482-98726-73623</value>
</field>
<field var='before-id'>
<value>09af3-cc343-b409f</value>
</field>
</x>
2013-06-11 16:28:52 -04:00
</query>
</iq>
2017-02-18 14:43:51 -05:00
]]></example>
<p>If the client already knows the UID of one or more messages it wants to fetch, it can use
the 'ids' field:</p>
<example caption='Fetching a specific message from the archive'><![CDATA[
<iq type='set' id='juliet1'>
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
<value>urn:xmpp:mam:2</value>
</field>
<field var='ids'>
<value>28482-98726-73623</value>
</field>
</x>
</query>
</iq>
2017-02-18 14:43:51 -05:00
]]></example>
2020-07-29 01:38:35 -04:00
<p>If any UID requested by the client in any of the 'before-id', 'after-id' or 'ids' form fields is not present in the archive, the server MUST return an item-not-found error in response to the query.</p>
</section3>
<section3 topic='Including groupchat results in a user archive' anchor='query-include-groupchat'>
<p>If the server advertises that it includes groupchat messages in a user's archive (see <link url='#support'>Determining support</link>), a client may query a user archive and request for them to be included in the result with the 'include-groupchat' field set to 'true'.
</p>
<example caption='Querying the archive and including groupchat messages in results'><![CDATA[
<iq type='set' id='juliet1'>
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
<value>urn:xmpp:mam:2</value>
</field>
<field var='include-groupchat'>
<value>true</value>
</field>
...
</x>
</query>
</iq>
]]></example>
<p>If the server advertises that it includes groupchat messages in the archive, or it advertises that it doesn't, a client may request that they not be included by setting the 'include-groupchat' field to 'false'.</p>
<example caption='Querying the archive and excluding groupchat messages from results'><![CDATA[
<iq type='set' id='juliet1'>
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
<value>urn:xmpp:mam:2</value>
</field>
<field var='include-groupchat'>
<value>false</value>
</field>
...
</x>
</query>
</iq>
]]></example>
<p>Note that where the client doesn't specify the 'include-groupchat' field, it is implementation-defined whether groupchat messages are included in the results (see <link url='#business_rules'>Business Rules</link>). Clients MUST NOT include this field where servers don't advertise support, as the server would reject such a form.</p>
2012-04-18 21:05:41 -04:00
</section3>
<section3 topic='Retrieving form fields' anchor='query-form'>
<p>In order for the client find out about additional fields the server might support, it can send an iq stanza of type 'get' addressed to the archive like this:</p>
<example caption="Client requests supported query fields"><![CDATA[
2014-08-15 10:58:59 -04:00
<iq type='get' id='form1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'/>
2014-08-15 10:58:59 -04:00
</iq>
2017-02-18 14:43:51 -05:00
]]></example>
<p>The server replies with all the form fields it supports in queries, which MUST include the mandatory fields specified in this document.</p>
<example caption="Server returns supported fields"><![CDATA[
2014-08-15 10:58:59 -04:00
<iq type='result' id='form1'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'>
2014-08-15 10:58:59 -04:00
<x xmlns='jabber:x:data' type='form'>
<field type='hidden' var='FORM_TYPE'>
2017-02-18 14:43:51 -05:00
<value>urn:xmpp:mam:2</value>
2014-08-15 10:58:59 -04:00
</field>
<field type='jid-single' var='with'/>
<field type='text-single' var='start'/>
<field type='text-single' var='end'/>
<field type='text-single' var='before-id'/>
<field type='text-single' var='after-id'/>
<field type='list-multi' var='ids'>
<validate xmlns="http://jabber.org/protocol/xdata-validate" datatype="xs:string">
<open/>
</validate>
</field>
<field type='boolean' var='include-groupchat'/>
<field type='text-single' var='{http://example.com/}free-text-search'/>
<field type='text-single' var='{http://example.com/}stanza-content'/>
2014-08-15 10:58:59 -04:00
</x>
</query>
</iq>
]]></example>
<p>If the client understands any of the additional fields it MAY proceed to include any of them in subsequent queries. It is not required to include any or all of the supported fields in queries.</p>
<p>A special note about the 'ids' field: this field is of type 'list-multi' which typically is used to allow the client to select from a provided list of options. In this case the list of all possible ids MUST NOT be provided by the server, as it is likely to be extremely large. Instead the server MUST include a &xep0122; &lt;validate/&gt; element that signals the list is open to arbitrary values provided by the client.</p>
<p>As specified in &xep0068;, names of custom fields SHOULD use Clark notation to avoid conflicts with other extensions.</p>
<example caption="Client uses two discovered query fields in a query"><![CDATA[
2014-08-15 10:58:59 -04:00
<iq type='set' id='query4'>
2017-02-18 14:43:51 -05:00
<query xmlns='urn:xmpp:mam:2'>
2015-01-23 05:16:32 -05:00
<x xmlns='jabber:x:data' type='submit'>
2014-08-15 10:58:59 -04:00
<field type='hidden' var='FORM_TYPE'>
2017-02-18 14:43:51 -05:00
<value>urn:xmpp:mam:2</value>
2014-08-15 10:58:59 -04:00
</field>
<field type='text-single' var='urn:example:xmpp:free-text-search'>
<value>Where arth thou, my Juliet?</value>
</field>
<field type='text-single' var='urn:example:xmpp:stanza-content'>
<value>{http://jabber.org/protocol/mood}mood/lonely</value>
</field>
</x>
</query>
</iq>
]]></example>
<p>Note that as the 'with', 'start' and 'end' fields MUST be implemented by servers, clients are able to submit forms using combinations of only these fields without needing to first fetch the form from the server and the types of these fields MUST be 'jid-single', 'text-single' and 'text-single' respectively. A server MUST NOT rely on a client having first requested the form before submitting queries.</p>
<p>If a client includes a form field that the server does not recognise, the server MUST respond with a 'feature-not-implemented' error.</p>
</section3>
2012-04-18 21:05:41 -04:00
</section2>
2013-06-11 16:28:52 -04:00
<section2 topic='Query results' anchor='results'>
2012-04-18 21:05:41 -04:00
<p>The server responds to the archive query by transmitting to the client all the messages
2014-08-15 10:58:59 -04:00
that match the criteria the client requested, subject to implementation limits. The results are sent as individual stanzas,
2012-04-18 21:05:41 -04:00
with the original message encapsulated in a &lt;forwarded/&gt; element as described in &xep0297;.
</p>
<p>The result messages MUST contain a &lt;result/&gt; element with an 'id' attribute that gives
2017-02-18 14:43:51 -05:00
the current message's archive UID (archived messages MAY also contain a XEP-0359 &lt;stanza-id&gt; element, but clients MUST NOT depend
on it). If the client gave a 'queryid' attribute in its initial query, the server MUST also include that in this result element.
2013-06-11 16:28:52 -04:00
</p>
<p>The &lt;result/&gt; element contains a &lt;forwarded/&gt; element which SHOULD contain the
original message as it was received, and SHOULD also contain a &lt;delay/&gt; element
qualified by the 'urn:xmpp:delay' namespace specified in &xep0203;. The value of the 'stamp'
attribute MUST be the time the message was originally received by the forwarding entity.
2012-04-18 21:05:41 -04:00
</p>
2017-02-18 14:43:51 -05:00
<p>The archive results MUST be sorted in chronological order, both within the returned results and within the ordering of RSM such that if a client were to request the first 10 stanzas in an archive, then use RSM to request the next 10 stanzas (by providing the 'after' element with the UID of the 10th stanza in the first results) all 20 result stanzas would be received in chronological order.
2014-08-15 10:58:59 -04:00
</p>
2012-04-18 21:05:41 -04:00
<example caption='Server returns two matching messages'><![CDATA[
2013-06-11 16:28:52 -04:00
<message id='aeb213' to='juliet@capulet.lit/chamber'>
2017-02-18 14:43:51 -05:00
<result xmlns='urn:xmpp:mam:2' queryid='f27' id='28482-98726-73623'>
2013-06-11 16:28:52 -04:00
<forwarded xmlns='urn:xmpp:forward:0'>
<delay xmlns='urn:xmpp:delay' stamp='2010-07-10T23:08:25Z'/>
2014-08-15 10:58:59 -04:00
<message xmlns='jabber:client'
to='juliet@capulet.lit/balcony'
2013-06-11 16:28:52 -04:00
from='romeo@montague.lit/orchard'
type='chat'>
2013-06-11 16:28:52 -04:00
<body>Call me but love, and I'll be new baptized; Henceforth I never will be Romeo.</body>
</message>
</forwarded>
</result>
2012-04-18 21:05:41 -04:00
</message>
2013-06-11 16:28:52 -04:00
<message id='aeb214' to='juliet@capulet.lit/chamber'>
2017-02-18 14:43:51 -05:00
<result xmlns='urn:xmpp:mam:2' queryid='f27' id='5d398-28273-f7382'>
2013-06-11 16:28:52 -04:00
<forwarded xmlns='urn:xmpp:forward:0'>
<delay xmlns='urn:xmpp:delay' stamp='2010-07-10T23:09:32Z'/>
2014-08-15 10:58:59 -04:00
<message xmlns='jabber:client'
to='romeo@montague.lit/orchard'
2013-06-11 16:28:52 -04:00
from='juliet@capulet.lit/balcony'
type='chat' id='8a54s'>
2013-06-11 16:28:52 -04:00
<body>What man art thou that thus bescreen'd in night so stumblest on my counsel?</body>
</message>
</forwarded>
</result>
2012-04-18 21:05:41 -04:00
</message>
2017-02-18 14:43:51 -05:00
]]></example>
2012-04-18 21:05:41 -04:00
</section2>
<section2 topic='Paging through results' anchor='query-paging'>
<section3 topic='Page limits' anchor='query-paging-limit'>
<p>A client or server will typically want to limit the number of results transmitted at
a time, thereby breaking the result stream into smaller 'pages'. For this purpose a
server MUST support &xep0059; and MUST support the paging mechanism defined therein.
A client MAY include a &lt;set/&gt; element in its query.</p>
<p>For the purposes of this protocol, the UIDs used by RSM correspond with the UIDs of the
stanzas stored in the archive.</p>
<example caption='A query using Result Set Management'><![CDATA[
<iq type='set' id='q29302'>
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
<value>urn:xmpp:mam:2</value>
</field>
<field var='start'>
<value>2010-08-07T00:00:00Z</value>
</field>
</x>
<set xmlns='http://jabber.org/protocol/rsm'>
<max>10</max>
</set>
</query>
</iq>
]]></example>
<p>To conserve resources, a server MAY place a reasonable limit on how many stanzas may be
pushed to a client in one request. Whether or not the client query included a &lt;set/&gt; element, the server MAY simply return
its limited results, modifying the &lt;set/&gt; element it returns appropriately.</p>
<example caption='Server responds to client with limited results using RSM'><![CDATA[
<!-- result messages -->
<iq type='result' id='q29302'>
<fin xmlns='urn:xmpp:mam:2'>
<set xmlns='http://jabber.org/protocol/rsm'>
<first index='0'>28482-98726-73623</first>
<last>09af3-cc343-b409f</last>
<count>20</count>
</set>
</fin>
</iq>
]]></example>
</section3>
<section3 topic='Requesting pages' anchor='query-paging-request'>
<p>The &lt;first&gt; and &lt;last&gt; elements specify the UID of the first and last returned
results (not necessarily of all the messages that matched the query, if the results have been limited).</p>
<p>The RSM &lt;count&gt; element and the 'index' attribute on the RSM &lt;first&gt; element are optional,
a server MAY include them, but a client MUST NOT depend on them being present. Please refer to the RSM
specification for more information surrounding their meaning and use.</p>
<p>Having previously made a query that returned results limited by the server (as described above), a client
can re-send the same request and receive the next 'page' of results. It does this by including a &lt;set&gt;
element with its request, containing an &lt;after/&gt; with the UID of the last message it received
from the previous query.</p>
<example caption='A page query using Result Set Management'><![CDATA[
<iq type='set' id='q29303'>
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'><value>urn:xmpp:mam:2</value></field>
<field var='start'><value>2010-08-07T00:00:00Z</value></field>
</x>
<set xmlns='http://jabber.org/protocol/rsm'>
<max>10</max>
<after>09af3-cc343-b409f</after>
</set>
</query>
</iq>
]]></example>
<p>Note: There is no concept of an "open query", and servers MUST be prepared to receive arbitrary page requests at any time.</p>
<p>RSM does not define the behaviour of including both &lt;before&gt; and &lt;after&gt; in the same request. To retrieve a range of items between two known ids, use before-id and after-id in the query form instead.</p>
<p>If the UID contained within an &lt;after&gt; or &lt;before&gt; element is not present in the archive, the server MUST return an item-not-found error in response to the query.</p>
<example caption='Message id not found in archive'><![CDATA[
<iq type='error' id='q29303'>
<error type='cancel'>
<item-not-found xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
</error>
</iq>
]]></example>
<p>When the results returned by the server are complete (that is: when they have not been limited by the maximum size of the result page (either as specified or enforced by the server)), the server MUST include a 'complete' attribute on the &lt;fin&gt; element, with a value of 'true'; this informs the client that it doesn't need to perform further paging to retreive the requested data. If it is not the last page of the result set, the server MUST either omit the 'complete' attribute, or give it a value of 'false'.</p>
<example caption='Server completes a result with the last page of messages'><![CDATA[
<!-- result messages -->
<iq type='result' id='u29303'>
<fin xmlns='urn:xmpp:mam:2' complete='true'>
<set xmlns='http://jabber.org/protocol/rsm'>
<first index='0'>23452-4534-1</first>
<last>390-2342-22</last>
<count>16</count>
</set>
</fin>
</iq>
]]></example>
<p>Sometimes (e.g. due to network or storage partitioning, or other transient errors) the server might return results to a client that are unstable (e.g. they might later change in sequence or content). In such a situation the server MUST stamp the &lt;fin&gt; element with a 'stable' attribute with a value of 'false'. If the server knows that the data it's serving are stable it MUST either stamp a 'stable' attribute with a value of 'true', or no such attribute. An example of when unstable might legitimately be returned is if the MAM service uses a clustered data store and a query covers a time period for which the data store has not yet converged; it the server could return best-guess results and tell the client that they may be unstable. A client SHOULD NOT cache unstable results long-term without later confirming (by reissuing appropriate queries) that they have become stable.</p>
</section3>
<section3 topic='Requesting the last page'>
<p>To request the page at the end of the archive (i.e. the most recent messages), include just an empty &lt;before/&gt; element in the RSM part of the query. As defined by RSM, this will return the last page of the archive.</p>
<example caption='A request for the last page in an archive'><![CDATA[
<iq type='set' id='q29303'>
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'><value>urn:xmpp:mam:2</value></field>
<field var='start'><value>2010-08-07T00:00:00Z</value></field>
</x>
<set xmlns='http://jabber.org/protocol/rsm'>
<max>10</max>
<before/>
</set>
</query>
</iq>
]]></example>
<p>Within the returned page, all results are still in chronological order, that is, the first result you receive will be the oldest item in the page, and the last result you receive will be the last item in the archive.</p>
</section3>
<section3 topic='Flipped pages' anchor='query-paging-flip'>
<p>When fetching a page, the client may prefer for the server to send the results within that page in reverse order.
For example, if a client implements a user interface that automatically fetches older messages as a user scrolls backward,
it may want to receive and display the newest messages first, instead of waiting for the whole page to be received.</p>
<p>A client wishing for a reversed page should include the &lt;flip-page/&gt; element in its query, like so:</p>
<example caption='Requesting a page that is flipped'><![CDATA[
<iq type='set' id='q29309'>
<query xmlns='urn:xmpp:mam:2'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'><value>urn:xmpp:mam:2</value></field>
<field var='start'><value>2010-08-07T00:00:00Z</value></field>
</x>
<set xmlns='http://jabber.org/protocol/rsm'>
<max>10</max>
<after>09af3-cc343-b409f</after>
</set>
<flip-page/>
</query>
</iq>
]]></example>
<p>It is important to note that flipping a page does not affect what results are returned in response to the query. It only affects the
order in which they are transmitted from the server to the client.</p>
<p>A client that wishes to use flipped pages MUST ensure that the server advertises the 'urn:xmpp:mam:2#extended' feature.</p>
</section3>
</section2>
</section1>
<section1 topic='Archive metadata' anchor='archive-metadata'>
<p>When planning a query, a client may wish to learn the current state of the archive. This includes information about the first/last entries in the archive.</p>
<p>When the archive advertises support for 'urn:xmpp:mam:2#extended' then the archive supports queries for this metadata via an iq of type 'get' to the
archive's address, with a &lt;metadata/&gt; payload in the 'urn:xmpp:mam:2' namespace.</p>
<example caption='Requesting archive metadata'><![CDATA[
<iq type='get' id='jui8921rr9'>
<metadata xmlns='urn:xmpp:mam:2'/>
</iq>
]]></example>
<example caption='Server returns archive metadata'><![CDATA[
<iq type='result' id='jui8921rr9'>
<metadata xmlns='urn:xmpp:mam:2'>
<start id='YWxwaGEg' timestamp='2008-08-22T21:09:04Z' />
<end id='b21lZ2Eg' timestamp='2020-04-20T14:34:21Z' />
</metadata>
</iq>
]]></example>
<p>The server response includes a &lt;metadata/&gt; element containing information about the archive. This element MUST include &lt;start/&gt; and &lt;end/&gt;
elements, which each have an 'id' and XEP-0082 formatted 'timestamp of the first and last messages in the archive respectively.</p>
2012-04-18 21:05:41 -04:00
</section1>
2014-08-15 10:58:59 -04:00
<section1 topic='Business Rules' anchor='business_rules'>
2018-01-30 07:19:14 -05:00
<section2 topic='Storage and Retrieval Rules' anchor='business-storeret'>
2014-08-15 10:58:59 -04:00