Commit Graph

971 Commits

Author SHA1 Message Date
Mike Perry d67a59b77f Fixes for cketti's code review on pull req #472
Leave the hostname == null checks so we can fall back if a hostname is not
found. Also convert message-id to upper case to match Apple Mail (for
privacy).
2014-06-13 17:49:26 -07:00
Dominik Schürmann aad171ff7e Client Certificate Authentication 2014-06-05 21:03:18 +02:00
Mike Perry 38c90146e1 Issue 6372: Add preference for setting timezone to UTC.
This is a privacy preference to avoid leaking your current location while
replying to email.
2014-05-27 07:53:50 -07:00
Mike Perry 7ac7fe2cfe Issue 3559: Use From or ReplyTo hostname in Message-ID if available.
I wrote this fix to avoid obviously specifying that I am using a mobile device
to reply to an email.

Others want this for ease of filtering messages from their host by Message-ID.
2014-05-27 07:51:23 -07:00
cketti e2cffa074f Update/remove comments 2014-05-03 06:33:33 +02:00
cketti a9cfa9ae68 Code style
No functional changes
2014-05-03 06:13:38 +02:00
Koji Arai 215ada2e77 astyle 2014-05-02 13:21:35 +09:00
Koji Arai 68850b1dc9 Use setter for quoted text like signature 2014-05-02 09:43:14 +09:00
Koji Arai e163fb87e1 set signature parameter when it is only necessary 2014-05-02 09:06:23 +09:00
Koji Arai 421879e9cf Remove unused getter 2014-05-02 08:39:07 +09:00
Koji Arai 56a48476e4 Rename mDraft to mInsertSeparator. 2014-05-02 08:36:46 +09:00
Koji Arai 6155a65f65 Refactoring: extract a class TextBodyBuilder 2014-05-02 01:33:39 +09:00
cketti 5c93f105ea Avoid NullPointerException reported via Google Play 2014-03-23 00:39:10 +01:00
Joe Steele bd4b7d3664 Issue 6280 -- SMTP Setup: ArrayIndexOutOfBoundsException: length=0; index=0
When the outgoing server settings don't require
authentication, userInfoParts.length == 0.
2014-03-20 10:56:02 -04:00
Joe Steele 95f62785fc Eliminate unused field/parameter 2014-03-20 09:47:43 -04:00
Joe Steele 01d2247ffd Change POP3 error response detection
Instead of interpreting a "-" at the beginning of a line as
an error response, consider the absence of a "+" at the
beginning of a line as an error response.

This is what Thunderbird does.

http://hg.mozilla.org/releases/comm-esr24/file/55e96a433bd1/mailnews/local/src/nsPop3Protocol.cpp#l1177

The problem arises with godaddy servers spewing additional
lines of data upon login failure.  The login was being
interpreted as successful, and a STAT commanded was subsequently
being sent, resulting in a dialog saying 'Cannot connect to
server. (Invalid int: "auth_error:")'.

$ openssl s_client -quiet -crlf -connect pop.secureserver.net:995
...
+OK <24984.1394317012@pop.secureserver.net>
user testuser
+OK
pass testpass
testuser not found in the auth database
warning: auth_error: authorization failed (no such object)
-ERR authorization failed  Check your server settings.
2014-03-11 19:10:14 -04:00
Joe Steele e475e51731 Rework handling of certificate errors while pushing
Eliminate import of MessagingController in ImapStore.
2014-03-11 19:08:09 -04:00
Joe Steele a7898fa2eb Fix issue 6269: IMAP LOGIN failure
Some IMAP servers are broken and don't correctly handle string
literals with the LOGIN command.

This switches to using quoted strings instead.

This is what Thunderbird does.
2014-03-11 19:06:00 -04:00
cketti ff5edf43d4 Merge branch 'pr/453'
Authentication changes

 message to explain why this merge is necessary,
2014-03-05 06:19:55 +01:00
cketti 617123c58b Remove SimpleX509TrustManager because it's no longer used 2014-03-05 06:03:06 +01:00
cketti 0a63466704 Add missing import 2014-03-05 04:16:57 +01:00
Joe Steele 094318dacb Use buffered output streams for all output
POP3 already does this.

This is a more general solution to the problem
addressed in commit 8bfd6ca.
2014-03-03 17:31:26 -05:00
Joe Steele 9dc5338501 Eliminate WebDAV STARTTLS security choice
STARTTLS doesn't really apply to WebDAV and should never have been made
available as an option.

Pre-existing settings will be re-mapped to SSL/TLS.
2014-03-03 17:29:51 -05:00
Joe Steele 14a0a7a2a7 Provide notification if STARTTLS is not available 2014-03-03 17:29:48 -05:00
Joe Steele daea7f1ecd Eliminate the 'if available' connection security options
These options originated in the AOSP email client from which K-9 Mail was
forked.  They provide an odd combination of 2 features:

1. Don't bother to authenticate the server's certificate (applies to both
SSL/TLS and STARTTLS); i.e., blindly accept all certificates.  This is
generally a bad security policy which is susceptible to MITM attacks.

2. If STARTTLS is selected but the server doesn't claim to support
STARTTLS, then proceed without using encryption.  This, too, is a bad
security policy which is susceptible to MITM attacks.

Since the time that K-9 Mail was forked, a couple things have changed:

> K-9 Mail has implemented the ability for users to review and permanently
accept individual certificates that would otherwise fail authentication.
With this ability, there is no need for a user to subject themselves to
the ongoing risks of feature 1. above.  Hence, this commit removes feature
1.

> The AOSP email client has changed its behavior and no longer permits a
security downgrade to an unencrypted connection if the server doesn't
claim to support STARTTLS (i.e., they eliminated feature 2. above). K-9
Mail should do the same.  It's unlikely that a server is going to provide
STARTTLS on an intermittent basis, so providing a contingency for such
unusual behavior is an unnecessary risk.  Hence, this commit removes that
feature as well.

Effect on existing users:

If the old connection security setting was "SSL/TLS (if available)" (which
now gets remapped to "SSL/TLS"), and the server does not provide a
certificate that can be authenticated, then a "Certificate error for
<account name>" notification is generated telling the user to check their
server settings.  Tapping the notification takes the user to the relevant
server settings, where the user can tap "Next" to review the certificate
and choose to permanently accept it.  This process would occur during the
first syncing of folders after application upgrade or (in the case of
SMTP) during the first attempt to send a message.

If the connection security setting was "STARTTLS (if available)" (which
now gets remapped to "STARTTLS"), and the server does not provide a
certificate that can be authenticated, then the same process as above
would occur.

If the old connection security setting was "STARTTLS (if available)", and
the server doesn't claim to support STARTTLS, then the user would get a
certificate error notification which would lead them to the server's
settings.  There they would need to choose a different connection security
-- most likely "NONE".  If they didn't change anything but instead just
tapped "Next", the server settings would be checked again and a dialog
would pop up saying, "Cannot connect to server. (STARTTLS connection
security not available)". (The implementation of notifications when
STARTTLS is not available is not actually included here -- it's in the
commit that follows.)

Regarding the changes to providers.xml:  in cases where the scheme ended
with "+ssl", the schemes were simply updated by appending "+".  In cases
where the scheme ended with "+tls", a check of the server was made to
assure that STARTTLS was available before appending "+" to the scheme.
Domains paran.com and nate.com failed the check and were removed because
no current information could be found.  Domains me.com and mac.com also
failed and were updated based on http://support.apple.com/kb/ht4864.
2014-03-03 17:23:00 -05:00
Joe Steele 39590d49bd Notify user of certificate errors while pushing
Previously, if a server's certificate failed authentication while
connecting for push (if, for example, the certificate had expired), then
the attempt to connect would fail, and another attempt would be tried
later.  After a certain number of failed attempts, no further attempts
would be made.  Meanwhile, the user is oblivious to the failures, and it
could be quite some time before the user realizes that they are not
getting email.  Even when they do realize it, they would not know the
cause.

With this commit, users receive a notification when such failures occur
while connecting for push.  (These notifications are already generated
with failures while polling.)  Tapping the notification will take the user
to the relevant server settings where they can choose to accept the
certificate.
2014-03-03 17:18:49 -05:00
Joe Steele 8d0901a178 Delete old journals when moving the database 2014-03-03 11:03:16 -05:00
Joe Steele 0f991b434e Use Locale.US where appropriate
Applicable for strings not intended for user consumption.

A %d string format code can generate eastern-arabic numerals
for users with an arabic locale.

V/k9      (20763): conn1103774136>>> 5 UID SEARCH ٦٤٦:٦٧٠ NOT DELETED
V/k9      (20763): conn1103774136<<<#5# [BAD, Invalid Search criteria]
E/k9      (20763): synchronizeMailbox
E/k9      (20763): com.fsck.k9.mail.store.ImapStore$ImapException: Command: UID SEARCH ٦٤٦:٦٧٠ NOT DELETED; response: #5# [BAD, Invalid Search criteria]
2014-03-03 10:40:23 -05:00
Joe Steele 0509e1541c Use Locale.US where appropriate 2014-03-03 10:08:07 -05:00
cketti 47e09c92ea Merge pull request #456 from asdil12/single_pkg_command
Fix sendCommand line splitup
2014-03-01 07:39:52 +01:00
Dominik Heidler 8bfd6ca3e0 Fix sendCommand line splitup for some imap proxys
When sending a command it would be sent like this:
PKG1: 1 STARTTLS
PKG2: \r\n

Some imap proxys (maybe from Fortinet?) don't accept commands across packets:
PKG1: 1 STARTTLS\r\n
2014-02-28 18:17:11 +01:00
Miltos Allamanis 2df205874e Renamed "s" to "sizeParam".
The naturalize tool detected that using "sizeParam" is more consistent
with the current codebase state:
* "sizeParam" in LocalStore is 22.89% probable ("s" 10.52%)
2014-02-26 14:27:35 +00:00
Miltos Allamanis be2b3b1ec2 Renamed "usee" to "uee".
The naturalize tool detected that using "uee" is more consistent with
the current codebase state:
* "uee" in LocalStore is 28.47% probable ("usee" 5.01%)
* "uee" in TextBody is 45.02% probable ("usee" 9.10%)
2014-02-26 14:23:33 +00:00
Joe Steele 540de158a0 Change the PLAIN auth. option text based on encryption
If the user chooses a connection security option which assures the use of
encryption, then the PLAIN auth. option is labeled "Normal password",
otherwise it is labeled "Password, transmitted insecurely".

This is similar to Thunderbird's behavior.
2014-02-25 15:22:44 -05:00
Joe Steele f7d397ea09 Eliminate SMTP AUTOMATIC authentication
The server settings for IMAP and POP3 have no such AUTOMATIC setting.
(Nor does Thunderbird have any such setting.)

The AUTOMATIC option is no longer offered to users as a choice.  A
pre-existing setting will continue to be honored, but only to the extent
that it doesn't result in insecure password transmission.  Users in such a
situation will get a "Failed to send some messages" notification
containing the exception text that says to update their outgoing server
authentication setting.

One of the problems with "AUTOMATIC" is that users may not fully
understand its security implications.  For example, a MITM attack could
mask a server's support for STARTTLS and CRAM-MD5, resulting in password
disclosure in certain configurations.

This commit also makes changes to the SMTP authentication process.  No
attempt is made to authenticate using methods that the server does not
profess to support in its EHLO response.  This is the same behavior as
found in Thunderbird.
2014-02-25 15:22:43 -05:00
Joe Steele c7e46faf0a Simplify code with better use of enum ConnectionSecurity 2014-02-25 15:22:38 -05:00
Joe Steele 90fedf7125 Eliminate the LOGIN authentication option from SMTP
The LOGIN option is no longer offered to users as a choice.  This does
*not* eliminate the SASL LOGIN authentication mechanism.  Any pre-existing
LOGIN setting or any imported LOGIN setting will still be recognized.  In
all cases, a user setting of either "Normal password" or "LOGIN" will
result in the SASL PLAIN mechanism being tried first if available,
otherwise SASL LOGIN will be tried if available.

This mirrors similar behavior that exists for IMAP.
2014-02-25 15:22:37 -05:00
Joe Steele dc9720ca13 Use localized strings for authentication type
AUTOMATIC = "Automatic"
PLAIN = "Normal password"
CRAM_MD5 = "Encrypted password"

SMTP also uses LOGIN.  No localized text was associated with that because
a future commit will remove that option.

(The text is similar to that of Thunderbird's)
2014-02-25 15:22:35 -05:00
Joe Steele f24ac67e4d Ignore case for SMTP extension keywords
The server is permitted to use mixed case keywords.
This converts them to upper case on receipt.
2014-02-25 15:22:33 -05:00
Joe Steele 64fd04ece2 POP3 authentication improvements
Changes:

Extract code and create login() and authCramMD5() methods.

Implement the SASL PLAIN authentication mechanism.  Its primary benefit is
the explicit support for UTF-8.  If the user has configured "PLAIN"
authentication, then SASL PLAIN will be used, if available, otherwise
login() will be used.

Implement POP3 APOP authentication (issue 3218).  If the user has
configured "CRAM_MD5" authentication (a future commit will change this
user option to a localized string "Encrypted password"), then SASL
CRAM-MD5 will be used, if available, otherwise the availability of POP3
APOP will be checked and used (per RFC 2449, there is no APOP
"capability").

Extend getCapabilities() to check for available authentication methods by
sending the "AUTH" command with no arguments
(http://tools.ietf.org/html/draft-myers-sasl-pop3-05).  This never became
a standard, but there are servers that support it, and Thunderbird
includes this check.

The SASL PLAIN and CRAM-MD5 authentication methods are not attempted
unless the server professes to have the appropriate capability.
(Previously, CRAM-MD5 was tried regardless of capability.)  No check is
made for the USER capability prior to use of that method.  All this is the
same behavior as in Thunderbird.

Eliminate the testing for capabilities in cases where the test results are
never used (PIPELINING, USER).

Change when getCapabilities() is called.  It is called once upon
connection.  If STARTTLS is negotiated (POP3 STLS), then
getCapabilities() is called again after the connection is encrypted (and
the server is authenticated), but before user authentication is attempted.
2014-02-25 15:22:32 -05:00
Joe Steele 26491676fa Retrict use of AuthenticationFailedException
In AccountSetupCheckSettings.onCreate(Bundle), the account settings are
checked.

If an AuthenticationFailedException occurs, A dialog saying "Username or
password incorrect." pops up.  We don't want to say this if the cause is
not related to an incorrect user name or password.  Instead we want to say
the more generic "Cannot connect to server" which pops up for other
exception types.

This commit attempts to eliminate the use of AuthenticationFailedException
in instances where it could not be due to "Username or password
incorrect."
2014-02-25 15:22:30 -05:00
Joe Steele 3877f58515 (partially) Fix IMAP LOGIN
Previously, the user name and password were being transmitted as IMAP (RFC
3501) quoted strings.

The problem is that quoted strings are only permitted to carry 7-bit
(ASCII) data, whereas user names and passwords entered in K-9 Mail may not
be ASCII, so K-9 was violating the RFC by sending them as quoted strings.

The solution is to transmit the credentials as IMAP literal strings, which
are permitted for user names and passwords, and which permit the
transmission of 8-bit data.

This is only a partial attempt for fixing the LOGIN command for users with
non-ASCII credentials.  The problem is that IMAP permits 8-bit data for
user names and passwords (if transmitted as literals), but the RFC says
nothing about the character encoding for 8-bit data.  This commit encodes
them as UTF-8.

The RFC author's comments on the subject:
http://mailman2.u.washington.edu/pipermail/imap-protocol/2008-February/000822.html

Ideally, users should avoid the LOGIN command and use the SASL PLAIN
mechanism (within TLS) which explicitly permits UTF-8. (K-9 Mail always
chooses PLAIN over LOGIN, when PLAIN is available.)
2014-02-25 15:22:28 -05:00
Joe Steele 6f49ebd975 Permit UTF-8 names & passwords with CRAM-MD5 authentication
CRAM-MD5 (RFC 2195) permits 8-bit data but does not identify its encoding.
Since ASCII does not permit 8-bit data, this commit changes the encoding
to UTF-8.

There is an expired Internet-Draft that proposed that the RFC be changed
to explicitly require UTF-8 encoding of user names and shared secrets.
(But then there's also an expired draft proposing that CRAM-MD5 be retired
to historic status.)

Instead of CRAM-MD5, a better option for users is the SASL PLAIN mechanism
(within TLS) which explicitly permits UTF-8.
2014-02-25 15:22:26 -05:00
Joe Steele 1d1b14da21 Fix ImapStore$ImapConnection.authCramMD5()
See Issue 4492

This method made way too many assumptions about server responses and
should not have been attempting to read and parse them.  That should be
left to ImapResponseParser.
2014-02-25 15:22:24 -05:00
Joe Steele 871ee1cc6c IMAP authentication improvements
Changes:

Implement the PLAIN SASL mechanism.  IMAPv4rev1 assures its availability
so long as the connection is encrypted.  The big advantage of PLAIN over
IMAP "LOGIN" is that PLAIN uses UTF-8 encoding for the user name and
password, whereas "LOGIN" is only safe for 7-bit US-ASCII -- the encoding
of 8-bit data is undefined.

(Note that RFC 6855 says that IMAP "LOGIN" does not support UTF-8, and
clients must use IMAP "AUTHENTICATE" to pass UTF-8 user names and
passwords.)

Honor the "LOGINDISABLED" CAPABILITY (RFC 2595) when the server declares
it.  There's no sense transmitting a password in the clear when it is
known that it will be rejected.

No attempt is made to try CRAM-MD5 if the server doesn't profess to
support it in its CAPABILITY response. (This is the same behavior as
Thunderbird.)

Extract code from ImapConnection.open into new method
ImapConnection.login.

Extract code from ImapConnection.executeSimpleCommand into new method
ImapConnection.readStatusResponse.

Related issues:  6015, 6016
2014-02-25 15:22:22 -05:00
András Veres-Szentkirályi ab3044c9fa use Set instead of implementation type 2014-02-15 23:59:24 +01:00
András Veres-Szentkirályi cbbd0bc405 use more idiomatic String.isEmpty() instead of comparing length to 0 2014-02-15 23:59:24 +01:00
András Veres-Szentkirályi df75853c64 replaced entrySet + getKey with keySet 2014-02-15 23:59:23 +01:00
András Veres-Szentkirályi 3d327763b5 replaced for with for-each loop 2014-02-15 23:59:23 +01:00
András Veres-Szentkirályi e75dd7df39 replaced for with for-each loop 2014-02-15 23:59:23 +01:00
András Veres-Szentkirályi 8627a3e702 replaced for with for-each loop 2014-02-15 23:59:23 +01:00
András Veres-Szentkirályi ac1e68af78 use built-in TextUtils.join instead of StringBuilder 2014-02-15 23:59:23 +01:00
András Veres-Szentkirályi 0c7740ce86 removed unread field mAccount 2014-02-15 21:30:44 +01:00
cketti 3527930f89 Fix 'endless' loop in ImapFolderPusher
Under certain circumstances it's possible that the 'push state' isn't
updated to contain the most recent 'UIDNEXT' value. In that case
ImapFolderPusher.start() would execute the same code path through its
main loop over and over again, preventing the device from going to
sleep.
Rather than changing the code to update the 'push state' in the corner
case that triggers the behavior described above, this commit introduces
another mechanism to track the 'UIDNEXT' value. This should also catch
as of yet unknown cases where the 'push state' isn't properly updated.

At some point in the future I hope we get to a point where we only
persist the 'push state' when we manually stop/restart the service.
During normal operation there's no need to read from/write to storage
all the time.

Fixes issue 4907
2014-02-11 20:17:47 +01:00
Joe Steele 03925fb409 Fix POP3 STLS command
The server response needed to be retrieved.

Thanks to Paul Durrant:
https://groups.google.com/d/msg/k-9-mail/0XHNNMR1TQ4/yExsr7nvJQwJ
2013-12-29 18:47:02 -05:00
cketti 79a5bc9c7e Revert "Make IMAP autoconfig recognize "Draft" as drafts folder"
This reverts commit 453f10128c.
See https://github.com/k9mail/k-9/pull/429
2013-12-17 18:45:57 +01:00
erlendorf 453f10128c Make IMAP autoconfig recognize "Draft" as drafts folder
Yahoo names it "Draft" instead of the more common "Drafts".
2013-12-16 17:14:56 +01:00
Joe Steele 40404c3700 Move some classes out of com.fsck.k9.mail.store
The classes are just as much related to com.fsck.k9.mail.transport
as com.fsck.k9.mail.store, so having them in
com.fsck.k9.mail.store doesn't seem appropriate.

Move LocalKeyStore to com.fsck.k9.security

Move TrustManagerFactory and TrustedSocketFactory to com.fsck.k9.net.ssl
2013-12-02 14:07:57 -05:00
Joe Steele 76605f7d86 Extract code into new LocalKeyStore class
Also, implement the ability to configure an alternate key store
file location. This permits the running of unit tests without
clobbering the live key store file.

Also, add a test to confirm that the key store file is being written
out and reread correctly.
2013-12-02 14:04:40 -05:00
cketti 4b57d79acf Only check against the certificate stored for a server, not all of them 2013-11-29 14:06:02 +01:00
cketti c5c195d243 Add unit tests for TrustManagerFactory 2013-11-29 10:49:52 +01:00
Joe Steele a4440b4042 Fix inadequate certificate validation
Proper host name validation was not being performed for certificates
kept in the local keystore.  If an attacker could convince a user to
accept and store an attacker's certificate, then that certificate
could be used for MITM attacks, giving the attacker access to all
connections to all servers in all accounts in K-9.

This commit changes how the certificates are stored.  Previously, an
entire certificate chain was stored for a server (and any of those
certificates in the chain were available for validating signatures on
certificates received when connecting).  Now just the single
certificate for the server is stored.

This commit changes how locally stored certificates are retrieved.
They can only be retrieved using the host:port that the user
configured for the server.

This also fixes issue 1326.  Users can now use different certificates
for different servers on the same host (listening to different ports).

The above changes mean that users might have to re-accept certificates
that they had previously accepted and are still using (but only if the
certificate's Subject doesn't match the host that they are connecting
to).

This commit modifies AccountSetupBasics so that it now calls
AccountSetupCheckSettings twice -- once for checking the incoming
settings and once for the outgoing settings.  Otherwise, an exception
could occur while checking incoming settings, the user could say
continue (or the user could accept a certificate key), and the
outgoing settings would not be checked.  This also helps with
determining if a certificate exception was for the incoming or
outgoing server, which is needed if the user decides to add the
certificate to the keystore.
2013-11-23 13:26:57 -05:00
brian m. carlson 1bfb78ee51 Use TrustedSocketFactory for STARTTLS.
The TrustedSocketFactory, which provides goodies like better cipher suites and
TLSv1.2, was only being used for tunnelled connections.  Use it for STARTTLS
connections as well.
2013-11-10 00:27:01 +00:00
cketti a036e4d2f9 Merge branch 'tls-hardening' 2013-10-29 04:40:06 +01:00
cketti a97705ffa9 Refactor TrustedSocketFactory 2013-10-29 04:37:52 +01:00
cketti 8f45d76b5c Extract WebDavSocketFactory 2013-10-29 03:42:37 +01:00
András Veres-Szentkirályi 105d3b3c4e filter TLS protocol versions by support as well 2013-10-20 10:42:39 +02:00
András Veres-Szentkirályi 77d43fb7bd extracted method filterBySupport 2013-10-20 10:30:33 +02:00
András Veres-Szentkirályi dbc47c7297 filter enabled cipher suites by availability 2013-10-20 10:22:53 +02:00
András Veres-Szentkirályi d84ce6ddb9 Hardened TLS cipher suites and versions
As Georg Lukas wrote in his blog post about how Android handles TLS
handshake (http://op-co.de/blog/posts/android_ssl_downgrade/), an
explicit order of cipher suites and TLS versions must be supplied to
avoid having the weak (presumably broken) RC4 cipher at the top of the
preference list.

This commit adds the list included in the blog post to every TLS socket
creation, including IMAP, POP3 and SMTP, see Wireshark screenshots done
during testing at http://vsza.hu/k9mail-tls-hardening/
2013-10-15 10:16:42 +02:00
Joe Steele 5a46575dc2 Generally replace \n with \r\n when part of a message
This builds upon the efforts started 2 commits back where \r\n is used for
all message text and \n is only used when the text is inside an
EolConvertingEditText widget.
2013-10-11 11:39:46 -04:00
Joe Steele bfb0316583 Fix fatal IndexOutOfBoundsException
The problem:

Configure the account (just an example -- problems can occur in other
configurations as well):
Message Format: HTML
Reply quoting style: Prefix
Quote message when replying: yes
Reply after quoted text: yes

Reply to a message that has a large quantity (20+) of \r\n scattered in
the body of its HTML version (not an unusual scenario).

Add a reply.  Save the message as a draft.  Go back & open the draft
again.  A fatal IndexOutOfBoundsException occurs.

The cause:

When the draft was saved, the X-K9mail-Identity header was computed and
added to the message, then the text of the message was processed with
MimeUtility.fixDraftTextBody, replacing all occurrences of \r\n with \n in
the quoted message before being saved in LocalStore, thus invalidating the
X-K9mail-Identity header.

The fix:

Remove MimeUtility.fixDraftTextBody and implement
MessageCompose$EolConvertingEditText instead.  Any message text placed in
an EolConvertingEditText widget is assured to have \n line endings.  Any
message text extracted from an EolConvertingEditText widget is assured to
have \r\n line endings.  The X-K9mail-Identity header will always be
computed correctly.

Issues thought to be related:  4782, 5010, 5634, 5725

As noted in some of the referenced issues, errors didn't always result in
a fatal exception, but instead with mixed up text.

Ref:  commit f9a35aeaee
2013-10-11 11:39:15 -04:00
Joe Steele 33678ea022 Eliminate unnecessary override in LocalAttachmentBody.
The overriding code is the same as the overridden code.
2013-10-11 11:24:56 -04:00
Joe Steele 1afff1e38f Delete related files when deleting a database.
The journal file was not being deleted when an account was deleted.
Over time, one can end up with a collection of dead journal files.
2013-10-11 11:24:51 -04:00
Danny Baumann 8777441d92 Use address object as contact picture instead of email address only.
Fixes assignment problems for emails sent by some issue tracking
systems, which send out mails with a fixed mail address on behalf of
different people.
2013-09-27 08:53:14 +02:00
cketti 2b7f5e7b70 Merge branch 'pick_attachment_fix'
Update LocalStore code to handle the newly introduced temporary files
for attachments

Conflicts:
	res/values/strings.xml
	src/com/fsck/k9/activity/MessageCompose.java
2013-09-25 05:22:00 +02:00
cketti 62aa1b87d0 Fetch attachments while MessageCompose activity is running
Android allows other apps to access protected content of an app without requesting the
necessary permission when the app returns an Intent with FLAG_GRANT_READ_URI_PERMISSION.
This regularly happens as a result of ACTION_GET_CONTENT, i.e. what we use to pick content
to be attached to a message. Accessing that content only works while the receiving activity
is running. Afterwards accessing the content throws a SecurityException because of the
missing permission.
This commit changes K-9 Mail's behavior to copy the content to a temporary file in K-9's
cache directory while the activity is still running.

Fixes issue 4847, 5821

This also fixes bugs related to the fact that K-9 Mail didn't save a copy of attached content
in the message database.

Fixes issue 1187, 3330, 4930
2013-09-25 03:12:34 +02:00
Koji Arai 0be299fbd0 Avoid StringIndexOutOfBoundsException when the header is broken like "=?us-ascii?q?abc?= =?" 2013-09-20 13:15:57 +09:00
Joe Steele 1ff128890d Fix MimeMessage(InputStream in, boolean recurse)
No actual impact, because all existing calls were
with recurse = true.
2013-09-07 12:52:14 -04:00
cketti aaa0de4dbc Don't append "null" to the "MAIL FROM" line in case the server doesn't support 8BITMIME 2013-09-06 20:28:46 +02:00
cketti a7c9804995 Make sure the InputStream is always closed
Moving the code to create the OutputStream wrappers into the try-block ensures
that the InputStream is closed in case something goes wrong.
2013-09-06 19:42:10 +02:00
cketti 6cd52c123c Remove unnecessary type cast 2013-09-06 19:38:15 +02:00
Joe Steele 1ca1ef5c84 Content-type case-conversion changes.
Don't convert the content-type to lower case in
MimeMessage.getContentType.  The content-type may have optional parameters
that are case sensitive (boundary, name).

In removing the lower-case conversion from getContentType, a review was
made for inappropriate case-sensitive comparisons which use data obtained
with getContentType.  The only ones found were in isMimeType in both
Message and MimeBodyPart.

Case-sensitive instances of isMimeType were made case-insensitive.  Also,
isMimeType was moved from Message to MimeMessage for symmetry with
MimeBodyPart (MimeMessage & MimeBodyPart are similar and contain a good
bit of duplication such as this).

The unit test required fixing now that the case of the boundary text is
preserved.

References:

Commits 2c5186 and dc4002 added the toLowerCase to getContentType in
MimeMessage & MimeBodyPart (Issue 94).

Later, commit 50cd60 removed the toLowerCase addition from MimeBodyPart
(Issue 1289).
2013-09-03 19:54:18 -04:00
Joe Steele 63f68328ff Standardize line breaks within headers.
Fix the unit test to match.

All line endings in the unit test are now the same.

(Just for consistency.  Not a big deal, since such problems are fixed when
the messages are run through EOLConvertingOutputStream.)
2013-09-03 19:54:17 -04:00
Joe Steele 45e3d8459e Recursively convert attachments of type message/rfc822 to 7bit if necessary.
The preceding commit resulted in attachments of type message/rfc822 being
sent with 8bit encoding even when the SMTP server did not support
8BITMIME.  This commit assures that messages will be converted to 7bit
when necessary.

A new interface CompositeBody was created that extends Body, and classes
Message and Multipart were changed from implementing Body to
CompositeBody.  Additional classes BinaryTempFileMessageBody and
LocalAttachmentMessageBody were created (by extending BinaryTempFileBody
and LocalAttachmentBody, respectively), and they too implement
CompositeBody.

A CompositeBody is a Body containing a composite-type that can contain
subparts that may require recursive processing when converting from 8bit
to 7bit.  The Part to which a CompositeBody belongs is only permitted to
use 8bit or 7bit encoding for the CompositeBody.

Previously, a Message was created so that it was 7bit clean by default
(even though that meant base64 encoding all attachments, including
messages).  Then, if the SMTP server supported 8BITMIME,
Message.setEncoding("8bit") was called so that bodies of type TextBody
would been transmitted using 8bit encoding rather than quoted-printable.

Now, messages are created with 8bit encoding by default.  Then, if the
SMTP server does not support 8BITMIME, Message.setUsing7bitTransport is
called to recursively convert the message and its subparts to 7bit.  The
method setUsing7bitTransport was added to the interfaces Part and
CompositeBody.

setEncoding no longer iterates over parts in Multipart.  That task belongs
to setUsing7bitTransport, which may in turn call setEncoding on the parts.

MimeUtility.getEncodingforType was created as a helper function for
choosing a default encoding that should be used for a given MIME type when
an attachment is added to a message (either while composing or when
retrieving from LocalStore).

setEncoding was implemented in MimeBodyPart to assure that the encoding
set in the Part's headers was the same as set for the Part's Body.  (The
method already existed in MimeMessage, which has similarities with
MimeBodyPart.)

MimeMessage.parse(InputStream in, boolean recurse) was implemented so that
the parser could be told to recursively process nested messages read from
the InputStream, thus giving access to all subparts at any level that may
need to be converted from 8bit to 7bit.
2013-09-03 19:53:13 -04:00
Joe Steele 77407eb5b7 Don't base64 encode attachments of type message/rfc822.
The problem:  Receive a message with an attachment of type message/rfc822
and forward it.  When the message is sent, K-9 Mail uses base64 encoding
for the attachment.  (Alternatively, you could compose a new message and
add such an attachment from a file using a filing-picking app, but that is
not 100% effective because the app may not choose the correct
message/rfc822 MIME type for the attachment.)

Such encoding is prohibited per RFC 2046 (5.2.1) and RFC 2045 (6.4).  Only
8bit or 7bit encoding is permitted for attachments of type message/rfc822.

Thunderbird refuses to decode such attachments.  All that is shown is the
base64 encoded body.

This commit implements LocalAttachmentBody.setEncoding.  If an attachment
to a newly composed message is itself a message, then setEncoding("8bit")
is called, otherwise setEncoding("base64")  is called for the attachment.
Similar behavior occurs when an attachment is retrieved from LocalStore.

The setEncoding method was added to the Body interface, since all
implementations of Body now declare the method.

The problem here differs from that in the preceding commit:  Here, the
encoding problem occurs on sending, not on receipt.  Here, the entire
message (headers and body) is base64 encoded, not just the body.  Here,
the headers correctly identify the encoding used;  it's just that the RFC
does not permit such encoding of attached messages.  The problem here
could in fact occur in combination with the preceding problem.
2013-09-03 19:52:11 -04:00
Joe Steele 1d1db50a9f Don't always base64 encode in BinaryTempFileBody.writeTo
Issue 5734 exemplifies the problem:  receive a message with an attachment
of type message/rfc822 that doesn't use base64 encoding for the body of
the attached message.  K-9 Mail incorrectly stores the attached message
locally with its original headers but using base64 encoding for the body.
A discrepancy thus exists between what the headers say about the encoding
of the body versus the actual encoding used.  This is obvious when
attempting to view the attachment (either by using a compatible message
viewer available on the device or by saving the attachment to a file and
viewing the file contents).

The process: When a message with an attached sub-message is received,
Message.parse puts the attachment in a new MimeMessage with the
attachment's body in a BinaryTempFileBody.  LocalFolder.saveAttachment
then calls Message.writeTo (which later calls BinaryTempFileBody.writeTo)
to place the entire attachment (headers and body) in a new file that will
become a LocalAttachmentBody.  Until now,  BinaryTempFileBody.writeTo
could only save the message body using base64 encoding.

This commit implements BinaryTempFileBody.setEncoding and assures that the
body is written out with the same encoding that was found in its  headers.
2013-09-03 19:51:26 -04:00
Joe Steele de23a0e3e1 SMTP 8BITMIME compliance
Currently, K-9 Mail detects if an SMTP server supports 8BITMIME (RFC
6152), and if so, TextBody parts are sent with content-transfer-ecoding =
8bit.  Otherwise, they are sent using quoted-printable.

This adds the required "BODY=8BITMIME" parameter to the MAIL command when
sending messages to servers that support 8BITMIME.
2013-09-03 16:47:57 -04:00
cketti 8fb2c5a490 Merge branch 'pr/349'
Remove remote/local store references when deleting accounts
2013-08-30 03:03:30 +02:00
cketti 9b370d0620 Add code to remove references to LocalStore when deleting accounts 2013-08-30 03:02:50 +02:00
cketti 01336944ec Refactor code to remove remote store references when deleting accounts 2013-08-30 02:28:57 +02:00
Joe Steele ac42bce799 Clean-up related to certificate chains
Per comments in pull request #365
2013-08-27 18:48:07 -04:00
Joe Steele 604aa87ccf Fix erroneous SSL certificate warnings
If you attempted to use SSL to connect to a server that speaks
STARTTLS, you should get an SSL protocol error.  Instead, you
were likely to get an "Unrecognized Certificate" error that shows
you an unrelated certificate chain and asks you to accept it or
reject it.  Neither action would work because the actual problem
had nothing to do with certificates.  The unrelated certificate
chain that popped up had been statically stored when validating
a prior connection to a different server.

With this patch, certificate chains are no longer stored statically
when validating server connections.

Issue 5886 is an example of a user experiencing this problem.
2013-08-25 15:43:36 -04:00
Jesse Vincent 7aeaa46fe6 Koji Arai pointed out that I screwed up the constants for folder open modes 2013-08-25 11:36:25 -04:00
cketti ce56475a4f Avoid race condition that can lead to a NullPointerException 2013-08-21 12:33:27 +02:00
cketti f11f0fcc9b Skip incorrectly formatted/parsed LSUB/LIST replies
Example: * LIST (\HasChildren) "." [Folder]
"[Folder]" will be parsed as list containing the element "Folder".
2013-08-20 18:39:20 +02:00
Joe Steele b7c67019af SMTP dot stuffing should be done *after* line wrapping. 2013-08-13 19:27:40 -04:00
Joe Steele 564195bad9 Close stream when done (StrictMode error) 2013-08-09 19:17:07 -04:00
Joe Steele fcabb7395d bug fix: use "1" not "true" for SQL boolean
Ref:  Pull request #355 from @edemaine
2013-08-02 13:01:22 -04:00
Jesse Vincent bbad298bf6 Convert OpenMode from an Enum to static ints for perf improvement.
(Based on profiling of long folder list opens)

This should be backported to 4.4
2013-07-30 21:32:35 -04:00
Jesse Vincent e9a0cbb269 Ask SQLite for a simpler calculation when counting unread messages. The docs imply this is a decent perf win. I'm seeing at least a bit of one 2013-07-30 21:32:35 -04:00
Wang Li 50a8d89468 remoteStore assocciated with account indirectly should also be reseted when acccount was removed. Otherwise, there might be residual data that will affec account being added later. 2013-07-23 14:09:53 +08:00
Jesse Vincent 1fa6d5f960 Add an index for a heavy query 2013-07-18 20:09:39 -04:00
cketti dda8395c7a Make sure we always have the ID of a folder when fetching the unread/flagged count 2013-07-16 16:54:26 +02:00
Jesse Vincent 876d478ad7 Opening a folder (especially in threaded mode) does a very heavy SQL query. This method doesn't use or modify the folder object. Don't cargo-cult the Open here.
(cketti may well tell me I'm wrong about this. Happy to listen & fix)
2013-07-11 18:44:44 -04:00
cketti 306b258f25 Merge pull request #323 from jca02266/adapt_broken_server
Adapt broken pop3 server
2013-07-07 19:43:55 -07:00
cketti 5d07b6691b Don't load unread/flagged count when creating LocalFolder instances
This avoids a potentially expensive LEFT JOIN when querying the database.
2013-07-07 03:31:14 +02:00
cketti 422ba279b5 Refactor code to create LocalFolder instances from a database query. 2013-07-07 03:31:14 +02:00
cketti ed6eb7bade Fix code to reset the database when upgrading fails
Previously the app crashed when upgrading the database failed. Now we
reset the database version and run the upgrade code again (recreating
all tables).
2013-07-03 05:25:39 +02:00
Koji Arai b88fbb7fd8 adapt to a corner case for broken server. 2013-06-29 22:28:27 +09:00
Koji Arai 35c79a5656 adapt broken pop3 server 2013-06-26 23:57:55 +09:00
Joe Steele 769ad843dd Revert "Fix for StrictMode error."
This reverts commit bbdec62e37.

Aside from being the incorrect solution for fixing the problem
described in pull request 211, the patch generates 'Dead code'
warnings inside the if(){} statements on lines 46 and 47.

The correct fix for the problem was already implemented in commit
5678786c97.

Although the logcat in the pull request was generated after the fix,
line numbers in the log indicate that it was based on an outdated
version of MimeUtility.java from before the fix.
2013-06-03 11:59:58 -04:00
cketti 46ed21d9f5 Merge branch 'smtp_port_fix' 2013-04-26 02:30:27 +02:00
cketti b91b86c657 Change SMTP default port to 587 2013-04-26 02:29:36 +02:00
dzan 6c1b3cbc68 Change smtp+tls default port to 587. 2013-04-20 18:02:30 +02:00
cketti 4890cd9ea0 Better cope with messages having identical message IDs 2013-04-17 02:09:43 +02:00
cketti 9ea46cf03b Merge branch 'Issue_4019_pinch_zoom'
Conflicts:
	src/com/fsck/k9/view/MessageWebView.java
2013-03-19 21:09:23 +01:00
cketti a619a9df93 Hide delete policy "Mark as read on server" for POP3 accounts 2013-03-13 06:34:14 +01:00
cketti 0319ee4a5d Move KOI8-U -> KOI8-R fixup to charset fall-back table 2013-03-10 09:55:25 +01:00
cketti 4b42d0e062 Code cleanup 2013-03-10 09:39:03 +01:00
eagan c5802ed8ef Java5 style 'for' 2013-03-10 09:25:28 +01:00
eagan a5f4ddae91 Enhancement for charset fallback.
Fallback rule is defined in CHARSET_FALLBACK_MAP.
Multi-level fallback is supported.
2013-03-10 09:25:13 +01:00
cketti 337487e2ae Merge pull request #211 from colund/master
Close stream in case of an exception
2013-03-10 00:11:53 -08:00
cketti 658b5dbff6 Fix and vastly simplify the query for the threaded message list
This requires another database schema change. With this change messages
at the root of a thread reference themselves in the 'threads' table,
i.e. 'root' contains the value of 'id' for these messages. It makes
selecting all messages in a thread much simpler.
2013-03-08 01:15:26 +01:00
Joe Steele 6a844a2553 Remove unused code in HtmlConverter.
HtmlConverter.getHtmlHeader() and getHtmlFooter() are
no longer used.  Remove them and other related code.
2013-03-01 14:30:42 -05:00
Joe Steele e2c5229e85 Change when <html> tags are applied to messages.
Previously, <html>, <head>, & <body> tags were
attached to messages before they were stored locally.
But now that the <head> element also needs to include
a <meta> element (for proper MessageWebView display),
it seems unecesary to store all these tags with each
message.

Now the tags are no longer stored with the messages.  Instead,
MessageWebView applies the tags before displaying the message.

This also eliminates the need to upgrade an older
message database where all the old messages would have
otherwise needed to be wrapped with the new tags.
2013-03-01 12:59:59 -05:00
cketti 50fcff76fd Add a caching layer to EmailProvider
Database updates can be surprisingly slow. This lead to slow updates of
the user interface which in turn made working with K-9 Mail not as fun
as it should be. This commit hopefully changes that.
2013-02-19 04:45:14 +01:00
Danny Baumann 275700e482 Fix a number of build warnings.
This gets rid of about 50 warnings. Deprecation warnings still remain,
though.
2013-02-04 11:18:49 +01:00
Danny Baumann 5d1e42c453 Improve certificate failure notifications.
The commit that introduced those notifications also introduced a rather
... interesting design pattern: The CertificateValidationException
notified the user of its pure existance - it's no longer a 'message'
only, but defines policy. As this is more than unusual, replace this
pattern by the MessagingController treating
CertificateValidationException specially when accessing remote folders.

Also make clear which account failed when constructing the notification.
2013-02-02 13:04:41 +01:00
Jesse Vincent ae6c7e37ca Merge remote-tracking branch 'zjw/issue_2389_cert_error_notice2'
* zjw/issue_2389_cert_error_notice2:
  Fix Issue 2389:  Self signed certificate expiry causes silent send/receive failure.
2013-01-15 14:44:21 -05:00
cketti 543fefebaa Merge branch 'threading_schema_change' 2013-01-12 23:03:38 +01:00
cketti 65a5a73737 Rewrite/optimize some SQL statements 2013-01-12 04:21:53 +01:00
cketti 0a5a330820 Fix getFoldersAndUids() for threads 2013-01-12 04:02:21 +01:00
cketti 421558c148 Optimize setting flags for whole threads 2013-01-12 02:28:12 +01:00
cketti 1df88ea153 Use separate table to store the thread structure 2013-01-11 03:40:35 +01:00
Joe Steele 29f15d715d Fix Issue 2389: Self signed certificate expiry causes silent send/receive failure.
With this fix, a CertPathValidatorException or CertificateException will
create a "Certificate error:  Check your server settings" notification
in the status bar.  When the user clicks on the notification, they are
taken to the appropriate server settings screen where they can review their
settings and can accept a different server certificate.
2013-01-10 16:49:55 -05:00
Andrew Chen 5ac3d1d5c2 Use existing LocalMessage for the JB notifications.
LocalMessage already has a content preview in it; reuse that.
Remove unneeded MimeMessage#getPreview() method now that we don't need
to generate a preview anymore.
2013-01-07 23:20:35 -08:00
Andrew Chen 068f87380c More javadoc 2013-01-07 22:15:49 -08:00
Andrew Chen d4eae05630 Javadoc 2013-01-07 22:11:04 -08:00
Danny Baumann 45328189f3 Strip off signatures from preview. 2013-01-07 10:39:08 +01:00
Danny Baumann 6f7ec3e401 Incorporate review comments. 2013-01-05 13:21:17 +01:00
cketti bcd52a6707 Fetch in-reply-to header when downloading envelope on IMAP accounts 2012-12-13 17:03:35 +01:00
cketti 44348e08a4 Create non-existent IMAP folders when copying/moving messages 2012-12-10 21:42:57 +01:00
cketti 7db33ce8c6 Ignore null-rows in results of SQL queries 2012-12-10 17:34:48 +01:00
cketti 8d652b2f61 Fix some compiler warnings 2012-12-07 15:50:55 +01:00
cketti 7b49f67c51 Remove unused methods 2012-12-07 15:47:15 +01:00
cketti bc4fb530d0 Remove all code using the old unread/flagged count database columns 2012-12-07 15:45:36 +01:00
cketti 04ec2abd2c Use local unread/flagged count in folder list and message list 2012-12-07 15:04:53 +01:00
cketti d5bb462917 Don't create Message objects when changing flags of selected messages
This changes the interface to MessagingController and the way flags are
updated in the database. Now messages aren't changed one by one but in
batches of 500. This should give better performance, but breaks the
unread and flagged count. I'm not very sad about this, because now we
can move towards only displaying the number of unread/flagged messages
in the local database.
2012-12-06 06:26:38 +01:00
cketti c0b5255e24 Make most important flags separate database columns
This is the first step to speed up searches and flag updates.
2012-12-04 05:13:58 +01:00