<remark><p>All modes allow multiple body children of to and from elements; changed namespace and collection element name to chat; renamed all value of save attribute to message; added stream value of the save attribute, thread attribute, save wrapper element, and Linking Collections and Associating Attributes sections</p></remark>
<remark><p>Added method child elements and expire attribute to pref element</p></remark>
</revision>
<revision>
<version>0.8</version>
<date>2006-09-29</date>
<initials>ip</initials>
<remark><p>Server generates encryption secrets for auto-archiving; specified use of W3C XML Encryption standard; enabled replacement of keys encrypted with an obsolete public key; enabled removal of open collections</p></remark>
<remark><p>Added preferences, results set management and notes; reinstated encryption and replication; simplified auto-archiving and off-the-record (with XEP-0155); many minor changes</p></remark>
<remark><p>Added unset value for save attribute and added service attribute on default element; added source attribute on record element; specified that services should (not must) support save mode for particular contacts.</p></remark>
<remark><p>Integrated text from server-side archiving proposal; added partial support to collection retrieval; harmonized XML formats and namespaces; defined XMPP Registrar considerations and XML schema.</p></remark>
<p>Many XMPP clients implement some form of client-side message archiving. However, it is not always convenient or even possible to archive messages locally, e.g., because it is easier to keep all archives in one universally accessible place (not scattered around on multiple computers or devices) or because the client operates in a web browser or resides on a mobile device that does not have sufficient local storage for message archiving. In addition, server-side archiving makes it possible to offer new services such as integration of IM and email. Therefore it is beneficial to define methods for server-side archiving of XMPP messages.</p>
<p>There are two main approaches to this problem:</p>
<olstart='1'>
<li>Enable the client to send individual messages or entire conversations to the server for archiving (optionally after encryption); we call this manual archiving.</li>
<li>Enable the server (at the client's request) to archive messages as they pass through the server; we call this automated archiving.</li>
</ol>
<p>So that client and server developers can refer to one specification, both approaches are defined in this document. In addition, this document defines common methods for retrieving and managing archived messages.</p>
<p>Complying with <strong>XMPP Core</strong>, the server MUST respond to all &IQ; element of type 'get' or 'set'. However, most successful responses have been omitted from this document in the interest of conciseness.</p>
</section1>
<section1topic='Determining Server Support'anchor='disco'>
<p>For each feature defined herein, if the server supports that feature it MUST return a <feature/> element with the 'var' attribute set to 'http://www.xmpp.org/extensions/xep-0136.html#ns-name', where 'name' is 'auto' for the <linkurl='#auto'>Automated Archiving</link> feature, 'encrypt' for the <em>server-side</em> encryption feature (see <linkurl='#auto'>Automated Archiving</link>), 'manage' for the <linkurl='#manage'>Archive Management</link> feature, 'manual' for the <linkurl='#manual'>Manual Archiving</link> feature, and 'pref' for the <linkurl='#pref'>Archiving Preferences</link> feature.</p>
<p>Not all users want to archive messages. A client SHOULD save its user's default archiving preference (or "Save Mode") to its own server (i.e., specify whether by default all conversations should be archived or not). In addition, a client MAY save different preferences for particular contacts.</p>
<p>Some users may also prefer that the messages they exchange with contacts are "<linkurl='#otr'>Off The Record</link>" (OTR). A client SHOULD save its user's default and contact-specific OTR preferences (or "OTR Modes") to its own server.</p>
<p>Whichever archiving method a client uses (e.g., local file archiving, or automatic or manual archiving to a server), it SHOULD adhere to its user's archiving preferences. However, a client MAY maintain a set of preferences in a local file which takes precedence over the preferences stored on the server for both local archiving and manual archiving.</p>
<p>This section addresses the following use cases:</p>
<olstart='1'>
<li>A client determines its user's current default Save Mode and OTR Mode, and the Modes for particular contacts.</li>
<li>A client sets the default Save Mode and OTR Mode.</li>
<li>A client sets the Save Mode and OTR Mode for a particular contact.</li>
<p>The server responds with the default Save Mode and OTR Mode (a single <default/> element) and any specific Save Modes and OTR Modes for individual contacts (zero or more <item/> elements).</p>
<p>Each <default/> or <item/> element in the response MUST include a 'save' attribute, whose value MAY be 'false' (the saving entity MUST save nothing), 'body' (the saving entity SHOULD save only &BODY; elements), 'message' (the saving entity SHOULD save the full XML content of each &MESSAGE; element) or 'stream' (the saving entity SHOULD save every byte that passes over the stream in either direction). Note: The upload, retrieval and management of 'stream' archives is <em>currently</em> beyond the scope of this document.</p>
<p>Note: Support for the 'message' value is optional and, to conserve bandwidth and storage space, it is RECOMMENDED that client implementations do not specify the 'message' value. <note>Stream compression typically does not mitigate bandwidth and storage issues since collections SHOULD be encrypted, and since clients running in constrained runtime environments typically cannot take advantage of stream compression (no binary data, only XML, may be transfered).</note></p>
<p>Each <default/> or <item/> element in the response whose 'save' attribute is not set to 'false' is RECOMMENDED to also include an 'expire' attribute which indicates how many seconds after messages are archived that the server SHOULD delete them.</p>
<p>Each <default/> or <item/> element in the response MUST include an 'otr' attribute, whose value MAY be 'require', 'prefer', 'approve', 'concede', 'oppose', or 'forbid'. The client MUST be guided by the specified 'logging' attribute value when negotiating (see &xep0155;) whether or not all messages exchanged with a contact will be <linkurl='#otr'>Off The Record</link>. (Note: An 'otr' attribute value of "require" in Message Archiving is equivalent to a 'logging' attribute value of "mustnot" in Chat Session Negotiation; for details, see the <linkurl='#otr-nego'>OTR Negotiation</link> section of this document.)</p>
<p>Note: If the OTR Mode is 'require' then the Save Mode MUST be 'false'.</p>
<p>The server MUST also include <method/> elements that reflect the user's preferences for each of the possible archiving methods. There MUST be at least three such elements for local file archiving (type 'local'), automatic archiving by the user's server (type 'auto'), and manual archiving to a server (type 'manual'). The 'use' attribute of each <method/> element MUST be set to 'prefer', 'concede' or 'forbid' - indicating which archiving methods the user's clients SHOULD, MAY (if it does not support any preferred method) or MUST NOT use.</p>
<p>The server MUST also include an <auto/> element reflecting the current <linkurl='#auto'>Automated Archiving</link> settings for <em>this stream</em>.</p>
<p>If the user has never set the default Modes, the 'save' and 'otr' attributes SHOULD specify the server's default settings, and the 'unset' attribute SHOULD be set to 'true'. Note: The 'unset' attribute defaults to 'false'.</p>
<p>Once it has received a request for archiving preferences from the client, the server MUST send any subsequent changes to any of the user's archiving preferences to the client until the stream is closed (see below). Note: changes to the <auto/> element MUST NOT be replicated in this way.</p>
<li><p>If it does not allow the saving of full message stanza content, and the client set the value of the 'save' attribute to 'message' or 'stream', and any of the user's connected resources have <linkurl='#auto'>Automated Archiving</link> enabled.</p></li>
<li><p>If administrator policies require that at least the <body/> elements (or the full content) of every message are logged automatically, and the client sets the value of the 'save' attribute to 'false' (or 'body').</p></li>
<p>A client may use a similar protocol to set the Modes for a particular contact or domain of contacts (bare JID, full JID or domain). Note: It is STRONGLY RECOMMENDED for the value of the 'jid' attribute to be a bare JID (&BAREJID;).</p>
<examplecaption='Client Sets Modes for a Contact'><![CDATA[
<p>Any client that archives messages SHOULD support <cite>Chat Session Negotiation</cite> and its 'logging' field both to give other contacts the opportunity to indicate this preference, and to negotiate an "Off The Record" (OTR) policy that complies with its user's own <linkurl='#pref'>Archiving Preferences</link>.</p>
<p>Note: A client MUST NOT propose or agree to enable OTR (i.e., disallow message logging) unless it has confirmed that its server will allow it to switch off <linkurl='#auto'>Automated Archiving</link>.</p>
<p>Both parties to a chat session negotiation may have OTR preferences (i.e, the initiating party or "user" and the responding party or "contact"). These preferences will interact in the ways specified below, resulting either in a successful negotiation or an unsuccessful negotiation (naturally, an unsuccessful negotiation can lead to a subsequent negotiation attempt by the user or the contact).</p>
<p>The following table shows what chat session negotiation values the initating party (i.e., the "user") should send for the 'logging' field in the initial data form for a chat session negotiation (note: 'may' means that the receiving party MAY enable message logging and 'mustnot' means that the receiving party MUST NOT enable logging).</p>
<tablecaption='Chat Session Negotiation logging options offered by initiating party (user)'>
<p>** If the user receives no response it MUST NOT send any messages to the contact.</p>
<p>*** Alternatively, the user MAY decide not to <em>initiate</em> an OTR negotiation and to save messages (until the contact initiates a negotiation).</p>
<p>Note: When negotiating a chat session, the user MUST include the <required/> element inside the 'logging' <field/> element. If the user does not receive a successful response to its chat negotiation request (and if the OTR Mode is not 'require'), then it SHOULD proceed as if the contact had responded with the value of the 'logging' <field/> element set to 'may'.</p>
<p>The following table shows what chat session negotiation values the responding party (i.e., "contact") should send for the 'logging' field in its response to a chat session negotiation request from the user.</p>
<tablecaption='Chat Session Negotiation logging value selected by responding party (contact)'>
<p>** The negotiation fails and the parties MUST NOT exchange any messages; however, the recipient MAY attempt to initiate a chat session negotiation with the other party.</p>
<p>Note: If a contact does not include a 'logging' field in its initial Chat Session Negotiation request, and a user's Archiving Preferences indicate that OTR is <em>required</em>, then the client MUST refuse the request. It MAY then send its own Chat Session Negotiation request with a 'logging' field.</p>
<p>If a user's OTR preference for a contact changes during a Chat Session that has been negotiated with the contact, and if the new preference would affect the value of the 'logging' field that was previously negotiated, then the client MUST immediately renegotiate the 'logging' field according to the user's new OTR preference (or terminate the Chat Session).</p>
<p>If a Chat Session Negotiation agreed to enable OTR then the clients MUST NOT allow messages sent in <em>either</em> direction to be archived in any way (including <linkurl='#manual'>Manual Archiving</link> and <linkurl='#auto'>Automated Archiving</link>). <note>If a client (or user) acts in bad faith then its contacts cannot prevent it archiving conversations.</note></p>
<p>If a Chat Session Negotiation agreed to enable OTR then both clients MUST ensure that the Chat Session Negotiation messages themselves are not archived. For example, if <linkurl='#auto'>Automated Archiving</link> was enabled when the client received the initial Chat Session Negotiation request, then the client MUST immediately ask its server to delete its copy of the request (see <linkurl='#manage-remove'>Removing a Collection</link> for a description of how to remove the messages currently being recorded by the server).</p>
<p>While automated archiving is easy for the client and server to implement, there are many contexts in which manual archiving is required. For examples, when:</p>
<p>Therefore, often a client will want to send or receive a sequence of messages, optionally add private notes to the sequence, optionally encrypt the sequence, and then ask the server to archive it. Such messages and notes SHOULD be stored on the server in the form of a "collection".</p>
<p>A "collection" is a set of messages to/from the same user that are received near each other in time or as part of the same conversation thread. A collection is intended to mimic the natural flow of human conversations, which in instant messaging (IM) systems tend to occur in bursts (e.g., a five-minute conversation one day, followed by a ten-minute conversation the next).</p>
<p>A friendly name for the collection MAY be specified with a 'subject' attribute. Note the <linkurl='#security-subject'>Security Considerations</link> regarding the subject attribute.</p>
<p>If an opaque thread ID (found in the &THREAD; children of the &MESSAGE; elements whose content is stored in the collection) is associated with the conversation then it MUST be specified with a 'thread' attribute. Clients SHOULD include a &THREAD; child in each &MESSAGE; element they send that is part of a conversation they expect will be archived (see &xep0201;).</p>
<p>Note: The content of &MESSAGE; elements that have different thread IDs SHOULD be archived in separate collections. The content of &MESSAGE; elements that have the same thread IDs SHOULD be archived in the same collection. The thread attribute SHOULD NOT be set to any value other than the exact content of the &THREAD; elements. If no &THREAD; elements appeared in the conversation the <chat/> element SHOULD have no thread attribute. Implementations SHOULD use the thread attribute for cross-referencing purposes only, within the archive each collection MUST be uniquely identified by the combination of its 'with' and 'start' attributes.</p>
<p>Each collection MAY contain <note/>, <to/> or <from/> elements (or <EncryptedData/> and <EncryptedKey/> elements - see <linkurl='#crypt'>Encryption</link>).</p>
<p>The text of each individual private note MUST be encapsulated in a <note/> element. The absolute time the note was created SHOULD be specified with a 'utc' attribute (which MUST be UTC and adhere to the DateTime format specified in <cite>Jabber Date and Time Profiles</cite>).</p>
<p>The content of each individual message MUST be encapsulated in a <to/> or <from/> element. The time in whole seconds of the message relative to the previous message in the collection (or, for the first message, relative to the start of the collection) SHOULD be specified with a 'secs' attribute. Note: When deciding whether to round up or down to a number of whole seconds, entities MUST ensure that the sum of the 'secs' attribute and the 'secs' attributes of the preceeding messages will accurately reflect the absolute time of the message. (e.g., if a sequence of messages occur at exactly 0.51-second intervals then the 'secs' attributes should generally alternate between '0' or '1'.)</p>
<p>The content of each <to/> or <from/> element SHOULD depend on the user's <linkurl='#pref'>Archiving Preferences</link>. <to/> or <from/> elements MUST NOT be empty. Note: A server MAY be configured to return a <feature-not-implemented/> error if any <to/> or <from/> element contains anything other than &BODY; elements.</p>
<p>If the collection does not exist then the server MUST create a new collection. If the collection already exists then the server MUST append the messages to the existing collection.</p>
<p>Note: Clients MUST take care to append each sequence of messages to the collection before the sequence becomes so large that uploading it may violate common rate limiting restrictions (in Jabber systems, often called "karma").</p>
<section2topic='Changing the Subject of a Collection'anchor='impl-subject'>
<p>If the client specifies a new value for the 'subject' attribute of any existing collection then the server MUST update the existing value. Note: The client cannot specify new values for the 'with' or 'start' attributes. The only way to change these values is to delete the collection (see <linkurl='#manage-remove'>Removing a Collection</link>) and then create a new one.</p>
<examplecaption='Changing the subject of a collection without appending messages'><![CDATA[
<p>The client MAY specify an absolute time for any message by providing a longer 'utc' attribute (which MUST be UTC and adhere to the DateTime format specified in <cite>Jabber Date and Time Profiles</cite>) instead of a 'secs' attribute. The absolute time MAY be before the start time of the collection:</p>
<examplecaption='Storing offline messages in a collection'><![CDATA[
<p>A client MAY archive messages that it receives from &xep0045; rooms. The 'with' attribute MUST be the bare JID of the room. The client MUST include a 'name' attribute for each <from/> element to specify the room nickname (and, if available, bare JID) of the message sender:</p>
<p>Collections MAY be linked together by including a <previous/> and/or <next/> element. Each such element MUST include both a 'with' and a 'start' element to identify the other collection to which the collection is linked. For example, the <previous/> and <next/> elements in the two examples below are being used to link a groupchat between Romeo, Benvolio and Mercutio to a private chat that Romeo was having with Benvolio before they invited Mercutio to join them. Note: Collections MAY be linked in only one direction, they are not required to be double-linked in the way the examples below are.</p>
<examplecaption='Private chat linked to later groupchat'><![CDATA[
<p>A collection MUST NOT contain more than one <previous/> and one <next/> element. If a <previous/> element is uploaded to a collection that already contains one then the older <previous/> element MUST be discarded. The same requirement applies for <next/> elements.</p>
<p>When a collection is retrieved (see <linkurl='#manage-retrieve'>Retrieving a Collection</link>) the <previous/> and <next/> elements MUST appear as the first elements in the collection, whatever order they were uploaded in.</p>
<p><previous/> and <next/> elements MAY be removed from a collection simply by uploading a <previous/> and/or <next/> element without any 'with' or 'start' attributes. Note: The server SHOULD NOT return an error if it finds that a link to be deleted does not exist.</p>
<examplecaption='Deleting any links to other collections'><![CDATA[
<section2topic='Associating Attributes with a Collection'anchor='impl-form'>
<p>A client MAY append attributes to a collection by including an x:data form of type 'submit' (see &xep0004;) when it uploads to a collection.</p>
<p>A collection MUST NOT contain more than one x:data form. If a form is uploaded to a collection that already contains one then the older form element MUST be discarded. When a collection is retrieved (see <linkurl='#manage-retrieve'>Retrieving a Collection</link>) the x:data form MUST appear as the first element in the collection after any <previous/> or <next/> elements, whatever order it was uploaded in. Upon retrieval the 'type' attribute of the form MAY be 'submit' or 'form'.</p>
<examplecaption='Private chat with attributes form'><![CDATA[
<p>The x:data form MAY be removed from a collection simply by uploading an empty form. Note: The server SHOULD NOT return an error if it finds that the form to be deleted does not exist.</p>
<examplecaption='Deleting the attributes form'><![CDATA[
<p>The examples above are not encrypted for clarity. However, clients SHOULD encrypt manually-archived collections (although early implementations of this protocol MAY prefer to defer encryption and decryption to later releases). Servers MUST support the manual-archiving of encrypted collections.</p>
<p>Before uploading a sequence of messages to a collection, the client SHOULD select a symmetric data encryption algorithm, generate a suitable random encryption key, give the key a unique (for the user) name, encrypt the symmetric key with one of the user's public keys, and wrap the result inside one or more <EncryptedKey/> elements, as specified in &w3xmlenc;.</p>
<p>To ensure that all its user's clients will be able to decrypt the collection, the client SHOULD create one <EncryptedKey/> element for each of its user's public keys that are being published using &xep0189;. However, the client MUST NOT create an <EncryptedKey/> element for any public key until it has confirmed that it belongs to the user. Note: The fact that a public key is being published using <cite>Public Key Publishing</cite> is <em>not</em> sufficient proof of ownership, since the user's server may have been compromised at some stage. The method of confirmation is beyond the scope of this document.</p>
<p>The client SHOULD use the symmetric key to encrypt the joined sequence of <to/>, <from/> and <note/> elements, base64 encode the resulting sequence of bytes, and wrap it inside an <EncryptedData/> element, as described in <cite>XML Encryption</cite>.</p>
<p>Clients may add one or more <EncryptedData/> or <EncryptedKey/> elements to a collection using exactly the same method as for <to/>, <from/> and <note/> elements (see <linkurl='#manual-upload'>Uploading Messages to a Collection</link>). One collection may contain <EncryptedData/> elements encrypted with different symmetric keys.</p>
<p>When appending <EncryptedData/> elements to a collection, the client MAY reuse a symmetric KEY that has already been uploaded to the collection. In this case the client SHOULD NOT resend <EncryptedKey/> elements.</p>
<p>Note: A collection that contains <EncryptedData/> or <EncryptedKey/> elements MUST NOT contain <to/> or <from/> or <note/> elements.</p>
<examplecaption='Storing encrypted messages and keys in a collection'><![CDATA[
<p>The <CipherData/> child of each <EncryptedData/> element contains the base64-encoded symmetric-encrypted messages. The <EncryptionMethod/> and <KeyInfo/> children specify the symmetric encryption algorithm and the name of the symmetric key used to encrypt the messages.</p>
<p>The <CarriedKeyName/> child of each <EncryptedKey/> element contains the name of the symmetric key it contains. The name is referenced by the <KeyName/> child of the <KeyInfo/> child of an <EncryptedData/> element. The <CipherData/> child of each <EncryptedKey/> element contains the base64-encoded public-key-encrypted symmetric key. The <EncryptionMethod/> and <KeyInfo/> children specify the public key encryption algorithm and the name of the public key used to encrypt the symmetric key. The name of the public key MAY refer to the name in the <KeyName/> child of one of the <KeyInfo/> elements that are being published using <cite>Public Key Publishing</cite>.</p>
<p>If server administration policies <em>require</em> that every message is logged automatically (see <linkurl='#security'>Security Considerations</link>) then:</p>
<ul>
<li>The server MUST enable automatic archiving when each stream is opened.</li>
<li>Clients MUST NOT be allowed to disable automatic archiving.</li>
<li>Automatic archiving MUST NOT be subject to users' <linkurl='#pref'>Archiving Preferences</link>.</li>
<li>If the server has not received a request from a client for its user's archiving preferences (see <linkurl='#pref-determine'>Determining Preferences</link>) within a few seconds of authenticating the client then the server MUST send a warning message to the client:</li>
</ul>
<examplecaption='Server warns user of a legacy client about compulsory archiving'><![CDATA[
<messageto='juliet@capulet.com/chamber'>
<body>WARNING: All messages that you send or
receive will be recorded by the server.</body>
</message>
]]></example>
<p>Otherwise:</p>
<ul>
<li>Automatic archiving MUST default to disabled when each stream is opened.</li>
<li>A client MAY enable or disable automatic archiving for messages sent over its stream at any time. Note: If the client switches off all auto-archiving then the server MUST close and archive all active collections.</li>
<li>Once automatic archiving is switched on then the server MUST automatically archive messages only according to the user's <linkurl='#pref'>Archiving Preferences</link>.</li>
<li>Note: Both parties to an ESession (see &xep0116;) SHOULD either disable archiving or use an archiving method other than automatic, since ESession decryption keys are short-lived - making it impossible to decrypt automatically archived messages.</li>
<p>Servers (and clients) SHOULD support the encryption (and decryption) of automatically-archived collections (although early implementations of this protocol MAY prefer to defer encryption and decryption to later releases).</p>
<p>Whenever the client enables auto-archiving it SHOULD set the optional 'encrypt' attribute to 'true'. After receiving such a request, if the server supports encryption (see <linkurl='#disco'>Determining Server Support</link>), it MUST encrypt all the messages that it archives automatically (including any message collections that are currently being recorded) by following exactly the same procedure as clients use when manually archiving collections (see <linkurl='#crypt'>Encryption</link>).</p>
<p>The client MAY also specify one or more public keys (in addition to any public keys that the user may be publishing using <cite>Public Key Publishing</cite>). The server MUST use them all to encrypt all the symmetric keys it generates and add these to the collection wrapped in <EncryptedKey/> elements.</p>
<examplecaption='Client enables auto archiving with encryption'><