SslHelper has been removed, and its functionality has been transferred
into TrustedSocketFactory. The added layer of indirection wasn't really
simplifying anything. It's now easier to see what happens when
createSocket() is invoked.
A new instance of SecureRandom is no longer passed to SSLContext.init().
Instead, null is passed.
The (default) provider of the TLS SSLContext used is OpenSSLProvider,
which provides an SSLSocket instance of type OpenSSLSocketImpl. The only
use of SecureRandom is in OpenSSLSocketImpl.startHandshake(), where it is
used to seed the OpenSSL PRNG with additional random data. But if
SecureRandom is null, then /dev/urandom is used for seeding instead.
Meanwhile, the default provider for the SecureRandom service is
OpenSSLRandom, which uses the OpenSSL PRNG as its data source. So we were
effectively seeding the OpenSSL PRNG with itself. That's probably okay
(we trust that the OpenSSL PRNG was properly initialized with random data
before first use), but using /dev/urandom would seem like a better source
(or at least as good a source) for the additional seed data added with
each new connection.
Note that our PRNGFixes class replaces the default SecureRandom service
with one whose data source is /dev/urandom for certain vulnerable API
levels anyway. (It also makes sure that the OpenSSL PRNG is properly
seeded before first use for certain vulnerable API levels.)
This is done when the SASL EXTERNAL mechanism isn't advertised (indicating
the possibility that the server did not accept the client certificate) or
when the command for authenticating with SASL EXTERNAL fails.
The CertificateValidationException will trigger a notification to the user
that there's an authentication problem that needs addressing.
Also, there were instances where CertificateValidationException was being
thrown with a new CertificateException as the cause for the purpose of
notifying the user when STARTTLS is not available. This has been slightly
simplified by eliminating the need to include a new CertificateException
as a cause.
Some IMAP servers are broken and don't correctly handle string
literals with the LOGIN command.
This switches to using quoted strings instead.
This is what Thunderbird does.
These options originated in the AOSP email client from which K-9 Mail was
forked. They provide an odd combination of 2 features:
1. Don't bother to authenticate the server's certificate (applies to both
SSL/TLS and STARTTLS); i.e., blindly accept all certificates. This is
generally a bad security policy which is susceptible to MITM attacks.
2. If STARTTLS is selected but the server doesn't claim to support
STARTTLS, then proceed without using encryption. This, too, is a bad
security policy which is susceptible to MITM attacks.
Since the time that K-9 Mail was forked, a couple things have changed:
> K-9 Mail has implemented the ability for users to review and permanently
accept individual certificates that would otherwise fail authentication.
With this ability, there is no need for a user to subject themselves to
the ongoing risks of feature 1. above. Hence, this commit removes feature
1.
> The AOSP email client has changed its behavior and no longer permits a
security downgrade to an unencrypted connection if the server doesn't
claim to support STARTTLS (i.e., they eliminated feature 2. above). K-9
Mail should do the same. It's unlikely that a server is going to provide
STARTTLS on an intermittent basis, so providing a contingency for such
unusual behavior is an unnecessary risk. Hence, this commit removes that
feature as well.
Effect on existing users:
If the old connection security setting was "SSL/TLS (if available)" (which
now gets remapped to "SSL/TLS"), and the server does not provide a
certificate that can be authenticated, then a "Certificate error for
<account name>" notification is generated telling the user to check their
server settings. Tapping the notification takes the user to the relevant
server settings, where the user can tap "Next" to review the certificate
and choose to permanently accept it. This process would occur during the
first syncing of folders after application upgrade or (in the case of
SMTP) during the first attempt to send a message.
If the connection security setting was "STARTTLS (if available)" (which
now gets remapped to "STARTTLS"), and the server does not provide a
certificate that can be authenticated, then the same process as above
would occur.
If the old connection security setting was "STARTTLS (if available)", and
the server doesn't claim to support STARTTLS, then the user would get a
certificate error notification which would lead them to the server's
settings. There they would need to choose a different connection security
-- most likely "NONE". If they didn't change anything but instead just
tapped "Next", the server settings would be checked again and a dialog
would pop up saying, "Cannot connect to server. (STARTTLS connection
security not available)". (The implementation of notifications when
STARTTLS is not available is not actually included here -- it's in the
commit that follows.)
Regarding the changes to providers.xml: in cases where the scheme ended
with "+ssl", the schemes were simply updated by appending "+". In cases
where the scheme ended with "+tls", a check of the server was made to
assure that STARTTLS was available before appending "+" to the scheme.
Domains paran.com and nate.com failed the check and were removed because
no current information could be found. Domains me.com and mac.com also
failed and were updated based on http://support.apple.com/kb/ht4864.
Previously, if a server's certificate failed authentication while
connecting for push (if, for example, the certificate had expired), then
the attempt to connect would fail, and another attempt would be tried
later. After a certain number of failed attempts, no further attempts
would be made. Meanwhile, the user is oblivious to the failures, and it
could be quite some time before the user realizes that they are not
getting email. Even when they do realize it, they would not know the
cause.
With this commit, users receive a notification when such failures occur
while connecting for push. (These notifications are already generated
with failures while polling.) Tapping the notification will take the user
to the relevant server settings where they can choose to accept the
certificate.
Applicable for strings not intended for user consumption.
A %d string format code can generate eastern-arabic numerals
for users with an arabic locale.
V/k9 (20763): conn1103774136>>> 5 UID SEARCH ٦٤٦:٦٧٠ NOT DELETED
V/k9 (20763): conn1103774136<<<#5# [BAD, Invalid Search criteria]
E/k9 (20763): synchronizeMailbox
E/k9 (20763): com.fsck.k9.mail.store.ImapStore$ImapException: Command: UID SEARCH ٦٤٦:٦٧٠ NOT DELETED; response: #5# [BAD, Invalid Search criteria]
When sending a command it would be sent like this:
PKG1: 1 STARTTLS
PKG2: \r\n
Some imap proxys (maybe from Fortinet?) don't accept commands across packets:
PKG1: 1 STARTTLS\r\n
AUTOMATIC = "Automatic"
PLAIN = "Normal password"
CRAM_MD5 = "Encrypted password"
SMTP also uses LOGIN. No localized text was associated with that because
a future commit will remove that option.
(The text is similar to that of Thunderbird's)
In AccountSetupCheckSettings.onCreate(Bundle), the account settings are
checked.
If an AuthenticationFailedException occurs, A dialog saying "Username or
password incorrect." pops up. We don't want to say this if the cause is
not related to an incorrect user name or password. Instead we want to say
the more generic "Cannot connect to server" which pops up for other
exception types.
This commit attempts to eliminate the use of AuthenticationFailedException
in instances where it could not be due to "Username or password
incorrect."
Previously, the user name and password were being transmitted as IMAP (RFC
3501) quoted strings.
The problem is that quoted strings are only permitted to carry 7-bit
(ASCII) data, whereas user names and passwords entered in K-9 Mail may not
be ASCII, so K-9 was violating the RFC by sending them as quoted strings.
The solution is to transmit the credentials as IMAP literal strings, which
are permitted for user names and passwords, and which permit the
transmission of 8-bit data.
This is only a partial attempt for fixing the LOGIN command for users with
non-ASCII credentials. The problem is that IMAP permits 8-bit data for
user names and passwords (if transmitted as literals), but the RFC says
nothing about the character encoding for 8-bit data. This commit encodes
them as UTF-8.
The RFC author's comments on the subject:
http://mailman2.u.washington.edu/pipermail/imap-protocol/2008-February/000822.html
Ideally, users should avoid the LOGIN command and use the SASL PLAIN
mechanism (within TLS) which explicitly permits UTF-8. (K-9 Mail always
chooses PLAIN over LOGIN, when PLAIN is available.)
See Issue 4492
This method made way too many assumptions about server responses and
should not have been attempting to read and parse them. That should be
left to ImapResponseParser.
Changes:
Implement the PLAIN SASL mechanism. IMAPv4rev1 assures its availability
so long as the connection is encrypted. The big advantage of PLAIN over
IMAP "LOGIN" is that PLAIN uses UTF-8 encoding for the user name and
password, whereas "LOGIN" is only safe for 7-bit US-ASCII -- the encoding
of 8-bit data is undefined.
(Note that RFC 6855 says that IMAP "LOGIN" does not support UTF-8, and
clients must use IMAP "AUTHENTICATE" to pass UTF-8 user names and
passwords.)
Honor the "LOGINDISABLED" CAPABILITY (RFC 2595) when the server declares
it. There's no sense transmitting a password in the clear when it is
known that it will be rejected.
No attempt is made to try CRAM-MD5 if the server doesn't profess to
support it in its CAPABILITY response. (This is the same behavior as
Thunderbird.)
Extract code from ImapConnection.open into new method
ImapConnection.login.
Extract code from ImapConnection.executeSimpleCommand into new method
ImapConnection.readStatusResponse.
Related issues: 6015, 6016
Under certain circumstances it's possible that the 'push state' isn't
updated to contain the most recent 'UIDNEXT' value. In that case
ImapFolderPusher.start() would execute the same code path through its
main loop over and over again, preventing the device from going to
sleep.
Rather than changing the code to update the 'push state' in the corner
case that triggers the behavior described above, this commit introduces
another mechanism to track the 'UIDNEXT' value. This should also catch
as of yet unknown cases where the 'push state' isn't properly updated.
At some point in the future I hope we get to a point where we only
persist the 'push state' when we manually stop/restart the service.
During normal operation there's no need to read from/write to storage
all the time.
Fixes issue 4907
The classes are just as much related to com.fsck.k9.mail.transport
as com.fsck.k9.mail.store, so having them in
com.fsck.k9.mail.store doesn't seem appropriate.
Move LocalKeyStore to com.fsck.k9.security
Move TrustManagerFactory and TrustedSocketFactory to com.fsck.k9.net.ssl
Proper host name validation was not being performed for certificates
kept in the local keystore. If an attacker could convince a user to
accept and store an attacker's certificate, then that certificate
could be used for MITM attacks, giving the attacker access to all
connections to all servers in all accounts in K-9.
This commit changes how the certificates are stored. Previously, an
entire certificate chain was stored for a server (and any of those
certificates in the chain were available for validating signatures on
certificates received when connecting). Now just the single
certificate for the server is stored.
This commit changes how locally stored certificates are retrieved.
They can only be retrieved using the host:port that the user
configured for the server.
This also fixes issue 1326. Users can now use different certificates
for different servers on the same host (listening to different ports).
The above changes mean that users might have to re-accept certificates
that they had previously accepted and are still using (but only if the
certificate's Subject doesn't match the host that they are connecting
to).
This commit modifies AccountSetupBasics so that it now calls
AccountSetupCheckSettings twice -- once for checking the incoming
settings and once for the outgoing settings. Otherwise, an exception
could occur while checking incoming settings, the user could say
continue (or the user could accept a certificate key), and the
outgoing settings would not be checked. This also helps with
determining if a certificate exception was for the incoming or
outgoing server, which is needed if the user decides to add the
certificate to the keystore.
The TrustedSocketFactory, which provides goodies like better cipher suites and
TLSv1.2, was only being used for tunnelled connections. Use it for STARTTLS
connections as well.
As Georg Lukas wrote in his blog post about how Android handles TLS
handshake (http://op-co.de/blog/posts/android_ssl_downgrade/), an
explicit order of cipher suites and TLS versions must be supplied to
avoid having the weak (presumably broken) RC4 cipher at the top of the
preference list.
This commit adds the list included in the blog post to every TLS socket
creation, including IMAP, POP3 and SMTP, see Wireshark screenshots done
during testing at http://vsza.hu/k9mail-tls-hardening/
This builds upon the efforts started 2 commits back where \r\n is used for
all message text and \n is only used when the text is inside an
EolConvertingEditText widget.
The preceding commit resulted in attachments of type message/rfc822 being
sent with 8bit encoding even when the SMTP server did not support
8BITMIME. This commit assures that messages will be converted to 7bit
when necessary.
A new interface CompositeBody was created that extends Body, and classes
Message and Multipart were changed from implementing Body to
CompositeBody. Additional classes BinaryTempFileMessageBody and
LocalAttachmentMessageBody were created (by extending BinaryTempFileBody
and LocalAttachmentBody, respectively), and they too implement
CompositeBody.
A CompositeBody is a Body containing a composite-type that can contain
subparts that may require recursive processing when converting from 8bit
to 7bit. The Part to which a CompositeBody belongs is only permitted to
use 8bit or 7bit encoding for the CompositeBody.
Previously, a Message was created so that it was 7bit clean by default
(even though that meant base64 encoding all attachments, including
messages). Then, if the SMTP server supported 8BITMIME,
Message.setEncoding("8bit") was called so that bodies of type TextBody
would been transmitted using 8bit encoding rather than quoted-printable.
Now, messages are created with 8bit encoding by default. Then, if the
SMTP server does not support 8BITMIME, Message.setUsing7bitTransport is
called to recursively convert the message and its subparts to 7bit. The
method setUsing7bitTransport was added to the interfaces Part and
CompositeBody.
setEncoding no longer iterates over parts in Multipart. That task belongs
to setUsing7bitTransport, which may in turn call setEncoding on the parts.
MimeUtility.getEncodingforType was created as a helper function for
choosing a default encoding that should be used for a given MIME type when
an attachment is added to a message (either while composing or when
retrieving from LocalStore).
setEncoding was implemented in MimeBodyPart to assure that the encoding
set in the Part's headers was the same as set for the Part's Body. (The
method already existed in MimeMessage, which has similarities with
MimeBodyPart.)
MimeMessage.parse(InputStream in, boolean recurse) was implemented so that
the parser could be told to recursively process nested messages read from
the InputStream, thus giving access to all subparts at any level that may
need to be converted from 8bit to 7bit.
The commit that introduced those notifications also introduced a rather
... interesting design pattern: The CertificateValidationException
notified the user of its pure existance - it's no longer a 'message'
only, but defines policy. As this is more than unusual, replace this
pattern by the MessagingController treating
CertificateValidationException specially when accessing remote folders.
Also make clear which account failed when constructing the notification.
With this fix, a CertPathValidatorException or CertificateException will
create a "Certificate error: Check your server settings" notification
in the status bar. When the user clicks on the notification, they are
taken to the appropriate server settings screen where they can review their
settings and can accept a different server certificate.
Missing:
- UI support for threading when polling
- code to upgrade existing installations
- UI elements to switch from/to threaded display mode
- threading of messages with same subject
* imapsearch:
Change settings version to 18 to match what's currrently on master.
Handle aborted imap searches by nuking in-progress connections.
Move IMAP search into the Folder level.
Remove duplicate notification on remote search start.
Rename variables
changed PREFERENCE_CLOUD_SEARCH_ENABLED from "cloud_search_enabled" to "remote_search_enabled" in activity/setup/AccountSettings.java to resolve FC.
Add cloud search icon to local search result screen. Implement pull-to-remote-search.
Log remote search exceptions in addition to toasting them.
Add settings export for remote search settings.
Whitespace; no functional changes.
Handle implicit vs. explicit searches in ActionBar home button behavior.
Whitespace fix; no functional changes.
Add remote search actionbar icons.
IMAP Search: log exceptions on remote search, properly dispatch MessageList changes.
modified loadMessageForView() to dowload message if neither X_DOWNLOADED_FULL nor X_DOWNLOADED_PARTIAL.
Add remote IMAP search support.
Conflicts:
res/menu/message_list_option.xml
res/values/attrs.xml
res/values/themes.xml
src/com/fsck/k9/activity/MessageList.java
src/com/fsck/k9/preferences/Settings.java
* rbayer/IMAPsearch: (21 commits)
More cleanup
Code Cleanup getRemoteSearchFullText -> isRemoteSearchFullText line wraps for preference items
Refactor to allow fetching of extra search results beyond original request. Most code moved out of ImapStore and ImapFolder and into MessagingController.searchRemoteMessagesSynchronous. Should make it easier to add remoteSearch for other server types.
Prevent delete of search results while search results open
remove duplicated code block
Don't hide Crypto when IMAPsearch disabled
Code Style Cleanup: Tabs -> 4 spaces Remove trailing whitespace from blank lines
tabs -> spaces (my bad...)
Fix opening of folders to be Read-Write when necessary, even if they were previously opened Read-Only.
add missing file
Working IMAP search, with passable UI.
UI improvements
Simple help info when enabling Remote Search
Dependency for preferences
Basic IMAP search working
Joe Steele