In preparation for the upcoming SSPI implementation of GSSAPI
authentication, moved the definition of KERB_WRAP_NO_ENCRYPT from
socks_sspi.c to curl_sspi.h allowing it to be shared amongst other
SSPI based code.
... as mxr.mozilla.org is due to be retired.
The new host doesn't support If-Modified-Since nor ETags, meaning that
the script will now defer to download and do a post-transfer checksum
check to see if a new output is to be generated. The new output format
will hold the SHA1 checksum of the source file for that purpose.
We call this version 1.22
Reported-by: Ed Morley
Bug: http://curl.haxx.se/bug/view.cgi?id=1409
Bringing back the old functionality that was mistakenly removed when the
connection cache was remade. When creating a new connection, all the
existing ones are checked and those that are known to be dead get
disconnected for real and removed from the connection cache. It helps
the cache from holding on to very many stale connections and aids in
keeping down the number of system sockets in wait states.
Help-by: Jonatan Vela <jonatan.vela@ergon.ch>
Bug: http://curl.haxx.se/mail/lib-2014-06/0189.html
Curl_poll and Curl_wait_ms require the fix applied to Curl_socket_check
in commits b61e8b8 and c771968:
When poll or select are interrupted and coincides with the timeout
elapsing, the functions return -1 indicating an error instead of 0 for
the timeout.
Given the SSPI package info query indicates a token size of 4096 bytes,
updated to use a dynamic buffer for the response message generation
rather than a fixed buffer of 1024 bytes.
Updated to use a dynamic buffer for the SPN generation via the recently
introduced Curl_sasl_build_spn() function rather than a fixed buffer of
1024 characters, which should have been more than enough, but by using
the new function removes the need for another variable sname to do the
wide character conversion in Unicode builds.
Updated Curl_sasl_create_digest_md5_message() to use a dynamic buffer
for the SPN generation via the recently introduced Curl_sasl_build_spn()
function rather than a fixed buffer of 128 characters.
Curl_sasl_create_digest_md5_message() would simply cast the SPN variable
to a TCHAR when calling InitializeSecurityContext(). This meant that,
under Unicode builds, it would not be valid wide character string.
Updated to use the recently introduced Curl_sasl_build_spn() function
which performs the correct conversion for us.
Various parts of the libcurl source code build a SPN for inclusion in
authentication data. This information is either used by our own native
generation routines or passed to authentication functions in third-party
libraries such as SSPI. However, some of these instances use fixed
buffers rather than dynamically allocated ones and not all of those that
should, convert to wide character strings in Unicode builds.
Implemented a common function that generates a SPN and performs the
wide character conversion where necessary.
Following the recent changes and in attempt to align the SSPI based
authentication code performed the following:
* Use NULL and SECBUFFVERSION rather than hard coded constants.
* Avoid comparison of zero in if statements.
* Standardised the buf and desc setup code.
Given the SSPI package info query indicates a token size of 2888 bytes,
and as with the Winbind code and commit 9008f3d56, use a dynamic buffer
for the Type-1 and Type-3 message generation rather than a fixed buffer
of 1024 bytes.
Just as with the SSPI implementations of Digest and Negotiate added a
package info query so that libcurl can a) return a more appropriate
error code when the NTLM package is not supported and b) it can be of
use later to allocate a dynamic buffer for the Type-1 and Type-3
output tokens rather than use a fixed buffer of 1024 bytes.
OPENSSL_config() is "strongly recommended" to use but unfortunately that
function makes an exit() call on wrongly formatted config files which
makes it hard to use in some situations. OPENSSL_config() itself calls
CONF_modules_load_file() and we use that instead and we ignore its
return code!
Reported-by: Jan Ehrhardt
Bug: http://curl.haxx.se/bug/view.cgi?id=1401
If the server rejects our authentication attempt and curl hasn't
called CompleteAuthToken() then the status variable will be
SEC_I_CONTINUE_NEEDED and not SEC_E_OK.
As such the existing detection mechanism for determining whether or not
the authentication process has finished is not sufficient.
However, the WWW-Authenticate: Negotiate header line will not contain
any data when the server has exhausted the negotiation, so we can use
that coupled with the already allocated context pointer.
To prevent infinite loop in readwrite_data() function when stream is
reset before any response body comes, reset closed flag to false once
it is evaluated to true.
"Expect: 100-continue", which was once deprecated in HTTP/2, is now
resurrected in HTTP/2 draft 14. This change adds its support to
HTTP/2 code. This change also includes stricter header field
checking.
Previously it only returned a CURLcode for errors, which is when it
returns a different size than what was passed in to it.
The http2 code only checked the curlcode and thus failed.
Under these circumstances, the connection hasn't been fully established
and smtp_connect hasn't been called, yet smtp_done still calls the state
machine which dereferences the NULL conn pointer in struct pingpong.
This now provides a weak random function since PolarSSL doesn't have a
quick and easy way to provide a good one. It does however provide the
framework to make one so it _can_ and _should_ be done...
To force each backend implementation to really attempt to provide proper
random. If a proper random function is missing, then we can explicitly
make use of the default one we use when TLS support is missing.
This commit makes sure it works for darwinssl, gnutls, nss and openssl.
warning C4267: '=' : conversion from 'size_t' to 'long', possible loss
of data
The member connection_id of struct connectdata is a long (always a
32-bit signed integer on Visual C++) and the member next_connection_id
of struct conncache is a size_t, so one of them should be changed to
match the other.
This patch the size_t in struct conncache to long (the less invasive
change as that variable is only ever used in a single code line).
Bug: http://curl.haxx.se/bug/view.cgi?id=1399
1 - fixes the warnings when built without http2 support
2 - adds CURLE_HTTP2, a new error code for errors detected by nghttp2
basically when they are about http2 specific things.
CyaSSL 3.0.0 returns a unique error code if no CA cert is available,
so translate that into CURLE_SSL_CACERT_BADFILE when peer verification
is requested.
- Replace CURLAUTH_GSSNEGOTIATE with CURLAUTH_NEGOTIATE
- CURL_VERSION_GSSNEGOTIATE is deprecated which
is served by CURL_VERSION_SSPI, CURL_VERSION_GSSAPI and
CURUL_VERSION_SPNEGO now.
- Remove display of feature 'GSS-Negotiate'
This reverts commit cb3e6dfa35 and instead fixes the problem
differently.
The reverted commit addressed a test failure in test 1021 by simplifying
and generalizing the code flow in a way that damaged the
performance. Now we modify the flow so that Curl_proxyCONNECT() again
does as much as possible in one go, yet still do test 1021 with and
without valgrind. It failed due to mistakes in the multi state machine.
Bug: http://curl.haxx.se/bug/view.cgi?id=1397
Reported-by: Paul Saab
It's wrong to assume that we can send a single SPNEGO packet which will
complete the authentication. It's a *negotiation* — the clue is in the
name. So make sure we handle responses from the server.
Curl_input_negotiate() will already handle bailing out if it thinks the
state is GSS_S_COMPLETE (or SEC_E_OK on Windows) and the server keeps
talking to us, so we should avoid endless loops that way.
This is the correct way to do SPNEGO. Just ask for it
Now I correctly see it trying NTLMSSP authentication when a Kerberos ticket
isn't available. Of course, we bail out when the server responds with the
challenge packet, since we don't expect that. But I'll fix that bug next...
This is just fundamentally broken. SPNEGO (RFC4178) is a protocol which
allows client and server to negotiate the underlying mechanism which will
actually be used to authenticate. This is *often* Kerberos, and can also
be NTLM and other things. And to complicate matters, there are various
different OIDs which can be used to specify the Kerberos mechanism too.
A SPNEGO exchange will identify *which* GSSAPI mechanism is being used,
and will exchange GSSAPI tokens which are appropriate for that mechanism.
But this SPNEGO implementation just strips the incoming SPNEGO packet
and extracts the token, if any. And completely discards the information
about *which* mechanism is being used. Then we *assume* it was Kerberos,
and feed the token into gss_init_sec_context() with the default
mechanism (GSS_S_NO_OID for the mech_type argument).
Furthermore... broken as this code is, it was never even *used* for input
tokens anyway, because higher layers of curl would just bail out if the
server actually said anything *back* to us in the negotiation. We assume
that we send a single token to the server, and it accepts it. If the server
wants to continue the exchange (as is required for NTLM and for SPNEGO
to do anything useful), then curl was broken anyway.
So the only bit which actually did anything was the bit in
Curl_output_negotiate(), which always generates an *initial* SPNEGO
token saying "Hey, I support only the Kerberos mechanism and this is its
token".
You could have done that by manually just prefixing the Kerberos token
with the appropriate bytes, if you weren't going to do any proper SPNEGO
handling. There's no need for the FBOpenSSL library at all.
The sane way to do SPNEGO is just to *ask* the GSSAPI library to do
SPNEGO. That's what the 'mech_type' argument to gss_init_sec_context()
is for. And then it should all Just Work™.
That 'sane way' will be added in a subsequent patch, as will bug fixes
for our failure to handle any exchange other than a single outbound
token to the server which results in immediate success.
Before GnuTLS 3.3.6, the gnutls_x509_crt_check_hostname() function
didn't actually check IP addresses in SubjectAltName, even though it was
explicitly documented as doing so. So do it ourselves...
The old way using getpwuid could cause problems in programs that enable
reading from netrc files simultaneously in multiple threads.
Reported-by: David Woodhouse
The AES-GCM ciphers were added to GnuTLS as late as ver. 3.0.1 but
the code path in which they're referenced here is only ever used for
somewhat older GnuTLS versions. This caused undeclared identifier errors
when compiling against those.
This seems to have become necessary for SRP support to work starting
with GnuTLS ver. 2.99.0. Since support for SRP was added to GnuTLS
before the function that takes this priority string, there should be no
issue with backward compatibility.
When an error has been detected, skip the final forced call to the
progress callback by making sure to pass the current return code
variable in the Curl_done() call in the CURLM_STATE_DONE state.
This avoids the "extra" callback that could occur even if you returned
error from the progress callback.
Bug: http://curl.haxx.se/mail/lib-2014-06/0062.html
Reported by: Jonathan Cardoso Machado
The static connection counter caused a race condition. Moving the
connection id counter into conncache solves it, as well as simplifying
the related logic.
They were added because of an older code path that used allocations and
should not have been left in the code. With this change the logic goes
back to how it was.
Curl_rand() will return a dummy and repatable random value for this
case. Makes it possible to write test cases that verify output.
Also, fake timestamp with CURL_FORCETIME set.
Only when built debug enabled of course.
Curl_ssl_random() was not used anymore so it has been
removed. Curl_rand() is enough.
create_digest_md5_message: generate base64 instead of hex string
curl_sasl: also fix memory leaks in some OOM situations
httpproxycode is not reset in Curl_initinfo, so a 407 is not reset even
if curl_easy_reset is called between transfers.
Bug: http://curl.haxx.se/bug/view.cgi?id=1380
The method change is forbidden by the obsolete RFC2616, but libcurl did
it anyway for compatibility reasons. The new RFC7231 allows this
behaviour so there's no need for the scary "Violate RFC 2616/10.3.x"
notice. Also update the comments accordingly.
The SASL/Digest previously used the current time's seconds +
microseconds to add randomness but it is much better to instead get more
data from Curl_rand().
It will also allow us to easier "fake" that for debug builds on demand
in a future.
Rather than use a short 8-byte hex string, extended the cnonce to be
32-bytes long, like Windows SSPI does.
Used a combination of random data as well as the current date and
time for the generation.
"Any two of the parameters, readfds, writefds, or exceptfds, can be
given as null. At least one must be non-null, and any non-null
descriptor set must contain at least one handle to a socket."
http://msdn.microsoft.com/en-ca/library/windows/desktop/ms740141(v=vs.85).aspx
When using select(), cURL doesn't adhere to this (WinSock-specific)
rule, and can ask to monitor empty fd_sets, which leads to select()
returning WSAEINVAL (i.e. EINVAL) and connections failing in mysterious
ways as a result (at least when using the curl_multi_socket_action()
interface).
Bug: http://curl.haxx.se/mail/lib-2014-05/0278.html
OpenSSL passes out and outlen variable uninitialized to
select_next_proto_cb callback function. If the callback function
returns SSL_TLSEXT_ERR_OK, the caller assumes the callback filled
values in out and outlen and processes as such. Previously, if there
is no overlap in protocol lists, curl code does not fill any values in
these variables and returns SSL_TLSEXT_ERR_OK, which means we are
triggering undefined behavior. valgrind warns this.
This patch fixes this issue by fallback to HTTP/1.1 if there is no
overlap.
Make all code use connclose() and connkeep() when changing the "close
state" for a connection. These two macros take a string argument with an
explanation, and debug builds of curl will include that in the debug
output. Helps tracking connection re-use/close issues.
Commit 517b06d657 (in 7.36.0) that brought the CREDSPERREQUEST flag
only set it for HTTPS, making HTTP less good at doing connection re-use
than it should be. Now set it for HTTP as well.
Simple test case
"curl -v -u foo:bar localhost --next -u bar:foo localhos"
Bug: http://curl.haxx.se/mail/lib-2014-05/0127.html
Reported-by: Kamil Dudka
The variable wasn't assigned at all until step3 which would lead to a
failed connect never assigning the variable and thus returning a bad
value.
Reported-by: Larry Lin
Bug: http://curl.haxx.se/mail/lib-2014-04/0203.html
In commit 0b3750b5c2 (released in 7.36.0) we fixed a timeout issue
but instead broke the timings.
To fix this, I introduce a new timestamp to use for the timeouts and
restored the previous timestamp and timestamp position so that the old
timer functionality is restored.
In addition to that, that change also broke connection timeouts for when
more than one connect was used (as it would then count the total time
from the first connect and not for the most recent one). Now
Curl_timeleft() has been modified so that it checks against different
start times depending on which timeout it checks.
Test 1303 is updated accordingly.
Bug: http://curl.haxx.se/mail/lib-2014-05/0147.html
Reported-by: Ryan Braud
Whilst the qop directive isn't required to be present in a client's
response, as servers should assume a qop of "auth" if it isn't
specified, some may return authentication failure if it is missing.
To cater for the automatic generation of the new Visual Studio project
files, moved the lib file list into a separated variable so that lib
and lib/vtls can be referenced independently.
-p takes a list of Mozilla trust purposes and levels for certificates to
include in output. Takes the form of a comma separated list of
purposes, a colon, and a comma separated list of levels.
Now nghttp2_submit_request returns assigned stream ID, we don't have
to check stream ID using before_stream_send_callback. The
adjust_priority_callback was removed.
Depending on compiler line 3505 could generate the following warning or
error:
* warning: ISO C90 forbids mixed declarations and code
* A declaration cannot appear after an executable statement in a block
* error C2275: 'size_t' : illegal use of this type as an expression
As there's a default connection timeout and this wrongly used the
connection timeout during a transfer after the connection is completed,
this function would trigger timeouts during transfers erroneously.
Bug: http://curl.haxx.se/bug/view.cgi?id=1352
Figured-out-by: Radu Simionescu
If the precision is indeed shorter than the string, don't strlen() to
find the end because that's not how the precision operator works.
I also added a unit test for curl_msnprintf to make sure this works and
that the fix doesn't a few other basic use cases. I found a POSIX
compliance problem that I marked TODO in the unit test, and I figure we
need to add more tests in the future.
Reported-by: Török Edwin
Commit 07b66cbfa4 unfortunately broke native NTLM message support in
compilers, such as VC6, VC7 and others, that don't support long long
type declarations. This commit fixes VC6 and VC7 as they support the
__int64 extension, however, we should consider an additional fix for
other compilers that don't support this.
set.infilesize in this case was modified in several places, which could
lead to repeated requests using the same handle to get unintendent/wrong
consequences based on what the previous request did!
This makes the findprotocol() function work as intended so that libcurl
can properly be restricted to not support HTTP while still supporting
HTTPS - since the HTTPS handler previously set both the HTTP and HTTPS
bits in the protocol field.
This fixes --proto and --proto-redir for most SSL protocols.
This is done by adding a few new convenience defines that groups HTTP
and HTTPS, FTP and FTPS etc that should then be used when the code wants
to check for both protocols at once. PROTO_FAMILY_[protocol] style.
Bug: https://github.com/bagder/curl/pull/97
Reported-by: drizzt
As this makes curl_global_init_mem() behave the same way as
curl_global_init() already does in that aspect - the same number of
curl_global_cleanup() calls is then required to again decrease the
counter and then eventually do the cleanup.
Bug: http://curl.haxx.se/bug/view.cgi?id=1362
Reported-by: Tristan
ufds might not be allocated in case nfds overflows to zero while
extra_nfds is still non-zero. udfs is then accessed within the
extra_nfds-based for loop.
Should a command return untagged responses that contained no data then
the imap_matchresp() function would not detect them as valid responses,
as it wasn't taking the CRLF characters into account at the end of each
line.
Given that we presently support "auth" and not "auth-int" or "auth-conf"
for native challenge-response messages, added client side validation of
the quality-of-protection options from the server's challenge message.
To avoid urldata.h being included from the header file or that the
source file has the correct include order as highlighted by one of
the auto builds recently.
When CURL_DISABLE_CRYPTO_AUTH is defined the DIGEST-MD5 code should not
be included, regardless of whether USE__WINDOWS_SSPI is defined or not.
This is indicated by the definition of USE_HTTP_NEGOTIATE and USE_NTLM
in curl_setup.h.
Updated the docs to clarify and the code accordingly, with test 1528 to
verify:
When CURLHEADER_SEPARATE is set and libcurl is asked to send a request
to a proxy but it isn't CONNECT, then _both_ header lists
(CURLOPT_HTTPHEADER and CURLOPT_PROXYHEADER) will be used since the
single request is then made for both the proxy and the server.
When doing passive FTP, the multi state function needs to extract and
use the happy eyeballs sockets to wait for to check for completion!
Bug: http://curl.haxx.se/mail/lib-2014-02/0135.html (ruined)
Reported-by: Alan
In addition to commit fe260b75e7 fixed the same issue for RFC-821 based
SMTP servers and allow the credientials to be given to curl even though
they are not used with the server.
Specifying user credentials when the SMTP server doesn't support
authentication would cause curl to display "No known authentication
mechanisms supported!" and return CURLE_LOGIN_DENIED.
Reported-by: Tom Sparrow
Bug: http://curl.haxx.se/mail/lib-2014-03/0173.html
There are server certificates used with IP address in the CN field, but
we MUST not allow wild cart certs for hostnames given as IP addresses
only. Therefore we must make Curl_cert_hostcheck() fail such attempts.
Bug: http://curl.haxx.se/docs/adv_20140326B.html
Reported-by: Richard Moore
In addition to FTP, other connection based protocols such as IMAP, POP3,
SMTP, SCP, SFTP and LDAP require a new connection when different log-in
credentials are specified. Fixed the detection logic to include these
other protocols.
Bug: http://curl.haxx.se/docs/adv_20140326A.html
The debug messages printed inside PolarSSL always seems to end with a
newline. So 'infof()' should not add one. Besides the trace 'line'
should be 'const'.
Because of the socket is unblocking, PolarSSL does need call to getsock to
get the action to perform in multi environment.
In some cases, it might happen we have not received yet all data to perform
the handshake. ssh_handshake returns POLARSSL_ERR_NET_WANT_READ, the state
is updated but because of the getsock has not the proper #define macro to,
the library never prevents to select socket for input thus the socket will
never be awaken when last data is available. Thus it leads to timeout.
API has changed since version 1.3. A compatibility header has been created
to ensure forward compatibility for code using old API:
* x509 certificate structure has been renamed to from x509_cert to
x509_crt
* new dedicated setter for RSA certificates ssl_set_own_cert_rsa,
ssl_set_own_cert is for generic keys
* ssl_default_ciphersuites has been replaced by function
ssl_list_ciphersuites()
This patch drops the use of the compatibly header.
Rename x509_cert to x509_crt and add "compat-1.2.h"
include.
This would still need some more thorough conversion
in order to drop "compat-1.2.h" include.
Port number zero is perfectly allowed to connect to. I moved to storing
the remote port number in an int so that -1 means undefined and 0-65535
can be used for legitimate port numbers.
Setting the TIMER_STARTSINGLE timestamp first in CONNECT has the
drawback that for actions that go back to the CONNECT state, the time
stamp is reset and for the multi_socket API there's no corresponding
Curl_expire() then so the timeout logic gets wrong!
Reported-by: Brad Spencer
Bug: http://curl.haxx.se/mail/lib-2014-02/0036.html
Remove slash/backslash problem, now only slashes are used,
Wmake automaticaly translate slash/backslash to proper version or tools are not sensitive for it.
Enable spaces in path.
Use internal rm command for all host platforms
Add error message if old Open Watcom version is used. Some old versions exhibit build problems for Curl latest version. Now only versions 1.8, 1.9 and 2.O beta are supported
For HTTP/2, we may read up everything including responde body with
header fields in Curl_http_readwrite_headers. If no content-length is
provided, curl waits for the connection close, which we emulate it
using conn->proto.httpc.closed = TRUE. The thing is if we read
everything, then http2_recv won't be called and we cannot signal the
HTTP/2 stream has closed. As a workaround, we return nonzero from
data_pending to call http2_recv.
Original commit message was:
Don't omit CN verification in SChannel when an IP address is used.
Side-effect of this change:
SChannel and CryptoAPI do not support the iPAddress subjectAltName
according to RFC 2818. If present, SChannel will first compare the
IP address to the dNSName subjectAltNames and then fallback to the
most specific Common Name in the Subject field of the certificate.
This means that after this change curl will not connect to SSL/TLS
hosts as long as the IP address is not specified in the SAN or CN
of the server certificate or the verifyhost option is disabled.
This patch enables HTTP POST/PUT in HTTP2.
We disabled Expect header field and chunked transfer encoding
since HTTP2 forbids them.
In HTTP1, Curl sends small upload data with request headers, but
HTTP2 requires upload data must be in DATA frame separately.
So we added some conditionals to achieve this.
Perform more work in between sleeps. This is work around the
fact that axtls does not expose any knowledge about when work needs
to be performed. Depending on connection and how often perform is
being called this can save ~25% of time on SSL handshakes (measured
on 20ms latency connection calling perform roughly every 10ms).
When allowing NTLM, the re-use connection logic was too focused on
finding an existing NTLM connection to use and didn't properly allow
re-use of other ones. This made the logic not re-use perfectly re-usable
connections.
Added test case 1418 and 1419 to verify.
Regression brought in 8ae35102c (curl 7.35.0)
Reported-by: Jeff King
Bug: http://thread.gmane.org/gmane.comp.version-control.git/242213
For a function that returns a decoded version of a string, it seems
really strange to allow a NULL pointer to get passed in which then
prevents the decoded data from being returned!
This functionality was not documented anywhere either.
If anyone would use it that way, that memory would've been leaked.
Bug: https://github.com/bagder/curl/pull/90
Reported-by: Arvid Norberg
Make sure that the special NTLM magic we do is for HTTP+NTLM only since
that's where the authenticated connection is a weird non-standard
paradigm.
Regression brought in 8ae35102c (curl 7.35.0)
Bug: http://curl.haxx.se/mail/lib-2014-02/0100.html
Reported-by: Dan Fandrich
The code didn't properly check the return codes to detect overflows so
it could trigger incorrectly. Like on mingw32.
Regression introduced in 345891edba (curl 7.35.0)
Bug: http://curl.haxx.se/mail/lib-2014-02/0097.html
Reported-by: LM
when using --http2 one can now selectively disable NPN or ALPN with
--no-alpn and --no-npn. for now honored with NSS only.
TODO: honor this option with GnuTLS and OpenSSL
SSL_ENABLE_ALPN can be used for preprocessor ALPN feature detection,
but not SSL_NEXT_PROTO_SELECTED, since it is an enum value and not a
preprocessor macro.
Not comma, which is an inconsistency and a mistake probably inherited
from the examples section of RFC1867.
This bug has been present since the day curl started to support
multipart formposts, back in the 90s.
Reported-by: Rob Davies
Bug: http://curl.haxx.se/bug/view.cgi?id=1333
When using the multi socket interface, libcurl calls the
curl_multi_timer_callback asking to be woken up after
CURL_TIMEOUT_EXPECT_100 milliseconds.
After the timeout has expired, calling curl_multi_socket_action with
CURL_SOCKET_TIMEOUT as sockfd leads libcurl to check expired
timeouts. When handling the 100-continue one, the following check in
Curl_readwrite() fails if exactly CURL_TIMEOUT_EXPECT_100 milliseconds
passed since the timeout has been set!
It seems logical to consider that having waited for exactly
CURL_TIMEOUT_EXPECT_100 ms is enough.
Bug: http://curl.haxx.se/bug/view.cgi?id=1334
A server might respond with a content-encoding header and a response
that was encoded accordingly in HTTP-draft-09/2.0 mode, even if the
client did not send an accept-encoding header earlier. The server might
not send a content-encoding header if the identity encoding was used to
encode the response.
See:
http://tools.ietf.org/html/draft-ietf-httpbis-http2-09#section-9.3
This patch chooses different approach to integrate HTTP2 into HTTP curl
stack. The idea is that we insert HTTP2 layer between HTTP code and
socket(TLS) layer. When HTTP2 is initialized (either in NPN or Upgrade),
we replace the Curl_recv/Curl_send callbacks with HTTP2's, but keep the
original callbacks in http_conn struct. When sending serialized data by
nghttp2, we use original Curl_send callback. Likewise, when reading data
from network, we use original Curl_recv callback. In this way we can
treat both TLS and non-TLS connections.
With this patch, one can transfer contents from https://twitter.com and
from nghttp2 test server in plain HTTP as well.
The code still has rough edges. The notable one is I could not figure
out how to call nghttp2_session_send() when underlying socket is
writable.
Check the NPN result before preparing an HTTP request and switch into
HTTP/2.0 mode if necessary. This is a work in progress, the actual code
to prepare and send the request using nghttp2 is still missing from
Curl_http2_send_request().
the number of elements in the 'nghttp2_session_callbacks' structure is
now reduced by 2 in version 0.3.0 (I'm not sure when the change
happened, but checking for ver 0.3.0 work for me).
Something is wrong in 'userp' for the HTTP2 recv_callback(). The
session is created using bogus user-data; '&conn' and not 'conn'.
I noticed this since the socket-value in Curl_read_plain() was set to a
impossible high value.
hostcache_timestamp_remove() should remove old *unused* entries from the
host cache, but it never checked whether the entry was actually in
use. This complements commit 030a2b8cb.
Bug: http://curl.haxx.se/bug/view.cgi?id=1327
tftp_done() can get called with its TFTP state pointer still being NULL
on an early time-out, which caused a segfault when dereferenced.
Reported-by: Glenn Sheridan
Bug: http://curl.haxx.se/mail/lib-2014-01/0246.html
Make it possible to call
curl_easy_getinfo(curl, CURLINFO_CONTENT_LENGTH_DOWNLOAD, &filesize)
and related functions on remote sftp:// files, without downloading them.
Reported-by: Yingwei Liu
Bug: http://curl.haxx.se/mail/lib-2014-01/0139.html
This prevents sending a `Content-Length: -1` header, e.g this ocurred
with the following combination:
* standard HTTP POST (no chunked encoding),
* user-defined read function set,
* `CURLOPT_POSTFIELDSIZE(_LARGE)` NOT set.
With this fix it now behaves like HTTP PUT.
Make GnuTLS old and new consistent, specify the desired protocol, cipher
and certificate type in always in both modes. Disable insecure ciphers
as reported by howsmyssl.com. Honor not only --sslv3, but also the
--tlsv1[.N] switches.
Related Bug: http://curl.haxx.se/bug/view.cgi?id=1323
conversion from 'curl_off_t' to 'size_t', possible loss of data
Where curl_off_t is a 64-bit word and size_t is 32-bit - for example
with 32-bit Windows builds.
1 - allow >31 bit max-age values
2 - don't overflow on extremely large max-age values when we add the
value to the current time
3 - make sure max-age takes precedence over expires as dictated by
RFC6265
Bug: http://curl.haxx.se/mail/lib-2014-01/0130.html
Reported-by: Chen Prog
Starting with Visual Studio 2013 (VC12) and Windows 8.1 the
GetVersionInfoEx() function has been marked as deprecated and it's
return value atered. Updated connect.c and curl_sspi.c to use
VerifyVersionInfo() where possible, which has been available since
Windows 2000.
A transfer timeout could result in an error message such as "Operation
timed out after 3000 milliseconds with 19 bytes of -1 received". This
patch removes the non-sensical "of -1" when the size of the transfer
is unknown, mirroring the logic in lib/transfer.c
By default even recent versions of OpenSSL support and accept both
"export strength" ciphers, small-bitsize ciphers as well as downright
deprecated ones.
This change sets a default cipher set that avoids the worst ciphers, and
subsequently makes https://www.howsmyssl.com/a/check no longer grade
curl/OpenSSL connects as 'Bad'.
Bug: http://curl.haxx.se/bug/view.cgi?id=1323
Reported-by: Jeff Hodges
With the recently added timeout "reminder" functionality, there's no
reason left for us to execute timeout code before the time is
ripe. Simplifies the handling too.
This will make the *TIMEOUT and *CONNECTTIMEOUT options more accurate
again, which probably is most important when the *_MS versions are used.
In multi_socket, make sure to update 'now' after having handled activity
on a socket.
BACKGROUND:
We have learned that on some systems timeout timers are inaccurate and
might occasionally fire off too early. To make the multi_socket API work
with this, we made libcurl execute timeout actions a bit early too if
they are within our MULTI_TIMEOUT_INACCURACY. (added in commit
2c72732ebf, present since 7.21.0)
Switching everything to the multi API made this inaccuracy problem
slightly more notable as now everyone can be affected.
Recently (commit 21091549c0) we tweaked that inaccuracy value to make
timeouts more accurate and made it platform specific. We also figured
out that we have code at places that check for fixed timeout values so
they MUST NOT run too early as then they will not trigger at all (see
commit be28223f35 and a691e04470) - so there are definitately problems
with running timeouts before they're supposed to run. (We've handled
that so far by adding the inaccuracy margin to those specific timeouts.)
The libcurl multi_socket API tells the application with a callback that
a timeout expires in N milliseconds (and it explicitly will not tell it
again for the same timeout), and the application is then supposed to
call libcurl when that timeout expires. When libcurl subsequently gets
called with curl_multi_socket_action(...CURL_SOCKET_TIMEOUT...), it
knows that the application thinks the timeout expired - and alas, if it
is within the inaccuracy level libcurl will run code handling that
handle.
If the application says CURL_SOCKET_TIMEOUT to libcurl and _isn't_
within the inaccuracy level, libcurl will not consider the timeout
expired and it will not tell the application again since the timeout
value is still the same.
NOW:
This change introduces a modified behavior here. If the application says
CURL_SOCKET_TIMEOUT and libcurl finds no timeout code to run, it will
inform the application about the timeout value - *again* even if it is
the same timeout that it already told about before (although libcurl
will of course tell it the updated time so that it'll still get the
correct remaining time). This way, we will not risk that the application
believes it has done its job and libcurl thinks the time hasn't come yet
to run any code and both just sit waiting. This also allows us to
decrease the MULTI_TIMEOUT_INACCURACY margin, but that will be handled
in a separate commit.
A repeated timeout update to the application risk that the timeout will
then fire again immediately and we have what basically is a busy-loop
until the time is fine even for libcurl. If that becomes a problem, we
need to address it.
The net effect of this bug as it appeared to users, would be that
libcurl would timeout in the connect phase.
When disabling IPv6 use but still using getaddrinfo, libcurl would
wrongly not init the "hints" struct field in init_thread_sync() which
would subsequently lead to a getaddrinfo() invoke with a zeroed hints
with ai_socktype set to 0 instead of SOCK_STREAM. This would lead to
different behaviors on different platforms but basically incorrect
output.
This code was introduced in 483ff1ca75, released in curl 7.20.0.
This bug became a problem now due to the happy eyeballs code and how
libcurl now traverses the getaddrinfo() results differently.
Bug: http://curl.haxx.se/mail/lib-2014-01/0061.html
Reported-by: Fabian Frank
Debugged-by: Fabian Frank
Removed some of the infof() calls that were added with the recent
pipeline improvements but they're not useful to the vast majority of
readers and the pipelining seems to fundamentaly work - the debugging
outputs can easily be added there if debugging these functions is needed
again.
When the requested authentication bitmask includes NTLM, we cannot
re-use a connection for another username/password as we then risk
re-using NTLM (connection-based auth).
This has the unfortunate downside that if you include NTLM as a possible
auth, you cannot re-use connections for other usernames/passwords even
if NTLM doesn't end up the auth type used.
Reported-by: Paras S
Patched-by: Paras S
Bug: http://curl.haxx.se/mail/lib-2014-01/0046.html
When the progress callback returned 1 at a very early state, the code
would not make CURLE_ABORTED_BY_CALLBACK get returned but the process
would still be interrupted. In the HTTP case, this would then cause a
CURLE_GOT_NOTHING to erroneously get returned instead.
Reported-by: Petr Novak
Bug: http://curl.haxx.se/bug/view.cgi?id=1318
This is a debug function only and serves no purpose in production code,
it only slows things down. I left the code #ifdef'ed for possible future
pipeline debugging.
Also, this was a global function without proper namespace usage.
Reported-by: He Qin
Bug: http://curl.haxx.se/bug/view.cgi?id=1320
If OpenSSL is built to support SSLv2 this brings back the ability to
explicitly select that as a protocol level.
Reported-by: Steve Holme
Bug: http://curl.haxx.se/mail/lib-2014-01/0013.html
Some feedback provided by byte_bucket on IRC pointed out that commit
db11750cfa wasn’t really correct because it allows for “upgrading” to a
newer protocol when it should be only allowing for SSLv3.
This change fixes that.
When SSLv3 connection is forced, don't allow SSL negotiations for newer
versions. Feedback provided by byte_bucket in #curl. This behavior is
also consistent with the other force flags like --tlsv1.1 which doesn't
allow for TLSv1.2 negotiation, etc
Feedback-by: byte_bucket
Bug: http://curl.haxx.se/bug/view.cgi?id=1319
Since ad34a2d5c8 (present in 7.34.0 release) forcing
SSLv3 will always return the error "curl: (35) Unsupported SSL protocol
version" Can be replicated with `curl -I -3 https://www.google.com/`.
This fix simply allows for v3 to be forced.
Following commit 0aafd77fa4, replaced the internal usage of
FORMAT_OFF_T and FORMAT_OFF_TU with the external versions that we
expect API programmers to use.
This negates the need for separate definitions which were subtly
different under different platforms/compilers.
Added support to the built-in printf() replacement functions, for these
non-ANSI extensions when compiling under Visual Studio, Borland, Watcom
and MinGW.
This fixes problems when generating libcurl source code that contains
curl_off_t variables.
Fixes a bug when all addresses in the first family fail immediately, due
to "Network unreachable" for example, curl would hang and never try the
next address family.
Iterate through all address families when to trying establish the first
connection attempt.
Bug: http://curl.haxx.se/bug/view.cgi?id=1315
Reported-by: Michal Górny and Anthony G. Basile
Introduced in commit 2a4ee0d221 sending of data via the FILE
protocol would always return CURLE_WRITE_ERROR regardless of whether
CURL_WRITEFUNC_PAUSE was returned from the callback function or not.
Make sure that we detect such attempts and return a proper error code
instead of silently handling this in problematic ways.
Updated the documentation to mention this limitation.
Bug: http://curl.haxx.se/bug/view.cgi?id=1286
Previously this memdebug free() replacement didn't properly work with a
NULL argument which has made us write code that avoids calling
free(NULL) - which causes some extra nuisance and unnecessary code.
Starting now, we should allow free(NULL) even when built with the
memdebug system enabled.
free(NULL) is permitted by POSIX
free() itself allows a NULL input but our memory debug system requires
Curl_safefree() to be used instead when a "legitimate" NULL may be freed. Like
in the code here.
Pointed-out-by: Steve Holme
If a user indicated they preferred to authenticate using a SASL
mechanism, but SASL authentication wasn't supported by the server, curl
would always fall back to clear text when CAPABILITY wasn't supported,
even though the user didn't want to use this.
If a user indicated they preferred to authenticate using APOP or a SASL
mechanism, but neither were supported by the server, curl would always
fall back to clear text when CAPA wasn't supported, even though the
user didn't want to use this.
This also fixes the auto build failure caused by commit 6f2d5f0562.
This commit replaces that of 9f260b5d66 because according to RFC-2449,
section 6, there is no APOP capability "...even though APOP is an
optional command in [POP3]. Clients discover server support of APOP by
the presence in the greeting banner of an initial challenge enclosed in
angle brackets."
The FILE:// code doesn't support this option - and it doesn't make sense
to support it as long as it works as it does since then it'd only block
even longer.
But: setting CURLOPT_MAX_RECV_SPEED_LARGE would make the transfer first
get done and then libcurl would wait until the average speed would get
low enough. This happened because the transfer happens completely in the
DO state for FILE:// but then it would still unconditionally continue in
to the PERFORM state where the speed check is made.
Starting now, the code will skip from DO_DONE to DONE immediately if no
socket is set to be recv()ed or send()ed to.
Bug: http://curl.haxx.se/bug/view.cgi?id=1312
Reported-by: Mohammad AlSaleh
The comment in the code mentions the zero terminating after having
copied data, but it mistakingly zero terminated the source data and not
the destination! This caused the test 864 problem discussed on the list:
http://curl.haxx.se/mail/lib-2013-12/0113.html
Signed-off-by: Daniel Stenberg <daniel@haxx.se>
Although highlighted by a bug in commit 1cfb436a2f, APOP
authentication could be chosen if the server was to reply with an empty
or missing timestamp in the server greeting and APOP was given in the
capability list by the server.
Added a loop to pop3_statemach_act() in which Curl_pp_readresp() is
called until the cache is drained. Without this multiple responses
received in a single packet could result in a hang or delay.
Similar to the processing of untagged CAPABILITY responses in IMAP and
multi-line EHLO responses in SMTP, moved the processing of multi-line
CAPA responses to pop3_state_capa_resp().
In an effort to reduce what pop3_endofresp() does and bring the POP3
source back inline with the IMAP and SMTP protocols, moved the APOP
detection into pop3_state_servergreet_resp().
Added support for downgrading the SASL authentication mechanism when the
decoding of CRAM-MD5, DIGEST-MD5 and NTLM messages fails. This enhances
the previously added support for graceful cancellation by allowing the
client to retry a lesser SASL mechanism such as LOGIN or PLAIN, or even
APOP / clear text (in the case of POP3 and IMAP) when supported by the
server.
In preparation for the upcoming SASL downgrade feature renamed the
imap__perform_authenticate(), pop3__perform_authenticate() and
smtp__perform_authenticate() functions.
Security flaw CVE-2013-6422
This is conceptually the same problem and fix that 3c3622b6 brought to the
OpenSSL backend and that resulted in CVE-2013-4545.
This version of the problem was independently introduced to the GnuTLS
backend with commit 59cf93cc, present in the code since the libcurl
7.21.4 release.
Advisory: http://curl.haxx.se/docs/adv_20131217.html
Bug: http://curl.haxx.se/mail/lib-2013-11/0214.html
Reported-by: Marc Deslauriers
Since all systems have inaccuracy in the timeout handling it is
imperative that we add an inaccuracy margin to the general timeout and
connecttimeout handling with the multi interface. This way, when the
timeout fires we should be fairly sure that it has passed the timeout
value and will be suitably detected.
For cases where the timeout fire before the actual timeout, we would
otherwise consume the timeout action and still not run the timeout code
since the condition wasn't met.
Reported-by: He Qin
Bug: http://curl.haxx.se/bug/view.cgi?id=1298
To avoid the regression when users pass in passwords containing semi-
colons, we now drop the ability to set the login options with the same
options. Support for login options in CURLOPT_USERPWD was added in
7.31.0.
Test case 83 was modified to verify that colons and semi-colons can be
used as part of the password when using -u (CURLOPT_USERPWD).
Bug: http://curl.haxx.se/bug/view.cgi?id=1311
Reported-by: Petr Bahula
Assisted-by: Steve Holme
Signed-off-by: Daniel Stenberg <daniel@haxx.se>
It is not 100% clear whether * should include clear text LOGIN or not
from RFC-5092, however, including it is then consistent with current
POP3 behaviour where clear text, APOP or SASL may be chosen.
If a specific SASL authentication mechanism was requested by the user
as part of the login options but wasn't supported by the server then
curl would fallback to clear text, when it shouldn't, rather than
reporting "No known authentication mechanisms supported" as the POP3
and SMTP protocols do.
In C, signed integer overflow is undefined behavior. Thus, the compiler
is allowed to assume that it will not occur. In the check for an
overflow, the developer assumes that the signed integer of type time_t
will wrap around if it overflows. However, this behavior is undefined in
the C standard. Thus, when the compiler sees this, it simplifies t +
delta < t to delta < 0. Since delta > 0 and delta < 0 can't both be
true, the entire if statement is optimized out under certain
optimization levels. Thus, the parsedate function would return
PARSEDATE_OK with an undefined value in the time, instead of return -1 =
PARSEDATE_FAIL.
The comment here says that SecKeychainSearch causes a deprecation
warning when used with a minimum Mac OS X SDK version of 10.7.0, which
is correct. However, the #if guard did not match. It was intended to
only use the code if 10.6.0 support was enabled, but it had 10.7.0
instead. This caused a warning if the minimum was exactly 10.7.0.
The URI that is passed in as part of the Authorization: header needs to
be cut off at '?' if CURLAUTH_DIGEST_IE is set. Previously the code only
did when calculating the MD5sum.
Bug: http://curl.haxx.se/bug/view.cgi?id=1308
Patched-by: Sergey Tatarincev
POP3_TYPE_ANY, or ~0, is written to pop3c->preftype in lib/pop3c.c, an
unsigned int variable. The result of ~0 is -1, which caused a warning
due to writing a negative number to an unsigned variable. To fix this,
make the expression ~0U so that its value is considered the unsigned
number UINT_MAX which is what SASL_AUTH_ANY does in curl_sasl.h.
1) Renamed curl_tlsinfo to curl_tlssessioninfo as discussed on the
mailing list.
2) Renamed curl_ssl_backend to curl_sslbackend so it doesn't follow our
function naming convention.
3) Updated sessioninfo.c example accordingly.
This fixes a NULL dereference in the case where the client asks for
CURLINFO_TLS_SESSION data after the (TLS) session has already been
destroyed (i.e. curl_easy_perform has already completed for this
handle). Instead of crashing, we now return a CURLSSLBACKEND_NONE
error.
This is an extension to the fix in 7d80ed64e4. We may
call Curl_disconnect() while cleaning up the multi handle,
which could lead to openssl sending packets, which could get
a SIGPIPE.
Signed-off-by: Jeff King <peff@peff.net>
Commit 7d80ed64e4 introduced some helpers to handle
sigpipe in easy.c. However, that fix was incomplete, and we
need to add more callers in other files. The first step is
making the helpers globally accessible.
Since the functions are small and should generally end up
inlined anyway, we simply define them in the header as
static functions.
Signed-off-by: Jeff King <peff@peff.net>
This fixes a rare Happy Eyeballs bug where if the first IP family runs
out of addresses before the second-family-timer fires, and the second
IP family's first connect fails immediately, no further IPs of the
second family are attempted.
When adding entries to the DNS cache with CURLOPT_RESOLVE, they are
marked 'inuse' forever to prevent them from ever being removed in normal
operations. Still, the code that pruned out-of-date DNS entries didn't
care for the 'inuse' struct field and pruned it anyway!
Reported-by: Romulo A. Ceccon
Bug: http://curl.haxx.se/bug/view.cgi?id=1303
Added new API for returning a SSL backend type and pointer, in order to
allow access to the TLS internals, that may then be used to obtain X509
certificate information for example.
Otherwise a NOOP operation would be performed which a) only returns a
single line response and not a multiline response where -I needs to be
used, and b) provides an inconsistent user experience compared to that
of the POP3 and IMAP protocols.
singleipconnect() could return the file descriptor of an open socket
even though the function returned a CURLE_COULDNT_CONNECT error code
from commit ed1662c374 and 02fbc26d59.
This could cause tests 19, 704 and 1233 to fail on FreeBSD, AIX and
Solaris.
Incorrectly processed multiline server greeting responses as "wanted"
continue responses in smtp_endofresp(), from commit f16c0de4e9,
which in turn broke the SMTP server detection in the test suite,
because the EHLO command would not be sent.
Added a loop to smtp_statemach_act() in which Curl_pp_readresp() is
called until the cache is drained. Without this multiple responses
received in a single packet could result in a hang or delay.
Similar to the processing of untagged CAPABILITY responses in IMAP moved
the processing of multiline EHLO responses to smtp_state_ehlo_resp() and
introduced an internal response code of one to differentiate a multiline
continuation from the end of command. This also allows for the separate
processing of multiline responses from commands such as VRFY and EXPN.
singleipconnect() did not return the open socket descriptor on some
errors, thereby sometimes causing a socket leak. This patch ensures
the socket is always returned.
Even though this is only a formality (since not many people build on
Mavericks while targeting Leopard), since we still support Leopard
at the earliest, we might as well be pedantic.
It turns out that some of the constants necessary to make this feature
work are missing from Snow Leopard's Security framework even though
they are defined in the headers.
Bug: http://curl.haxx.se/mail/lib-2013-11/0076.html
Reported by: myriachan
Rather than set the authentication options as part of the login details
specified in the URL, or via the older CURLOPT_USERPWD option, added a
new libcurl option to allow the login options to be set separately.
This patch fixes and issue introduced in commit 7d7df83198, if the
tunnel state was TUNNEL_CONNECT, waitconnect_getsock() would return a
bitmask indicating a readable socket but never stored the socket in the
return array.
Our own printf() replacement clearly can't properly handle %.*s with a
string that isn't zero terminated. Instead of fixing the printf code or
even figuring out what the proper posix behavior is, I reverted this
piece of the code back to the previous version where it does malloc +
memcpy instead.
Regression added in e839446c2a, released in curl 7.32.0.
Reported-by: Felix Yan
Bug: http://curl.haxx.se/bug/view.cgi?id=1295
This patch adds a 200ms delay between the first and second address
family socket connection attempts.
It also iterates over IP addresses in the order returned by the
system, meaning most dual-stack systems will try IPv6 first.
Additionally, it refactors the connect code, removing most code that
handled synchronous connects. Since all sockets are now non-blocking,
the logic can be made simpler.
nss.c:702: warning: pointer targets in passing argument 3 of
'Curl_extract_certinfo' differ in signedness
nss.c:702: warning: pointer targets in passing argument 4 of
'Curl_extract_certinfo' differ in signedness
Made sure the cast was correctly "unsigned char *" to "char *" and not
"unsigned char *" to "unsigned char *".
nss.c:700: warning: pointer targets in passing argument 3 of
'Curl_extract_certinfo' differ in signedness
nss.c:700: warning: pointer targets in passing argument 4 of
'Curl_extract_certinfo' differ in signedness
Introduced in commit 7d7df83198 curl would loop displaying "Whut?"
if it was trying to connect to an address and port that didn't have
anything listening on it.
Renamed copy_header_value() to Curl_copy_header_value() as this
function is now non static.
Simplified proxy flag in Curl_http_input_auth() when calling
sub-functions.
Removed unnecessary white space removal when using negotiate as it had
been missed in commit cdccb42267.
...following recent changes to Curl_base64_decode() rather than trying
to parse a header line for the authentication mechanisms which is CRLF
terminated and inline zero terminate it.
...following recent changes to Curl_base64_decode() rather than trying
to parse a header line for the authentication mechanisms which is CRLF
terminated and inline zero terminate it.
The code rejected 0 as a valid timeout while in fact the function could
indeed legitimately return that and it should be respected.
Reported-by: Bjorn Stenberg
A base64 string should be a multiple of 4 characters in length, not
contain any more than 2 padding characters and only contain padding
characters at the end of string. For example: Y3VybA==
Strings such as the following are considered invalid:
Y= - Invalid length
Y== - Invalid length
Y=== - More than two padding characters
Y=x= - Padding character contained within string
This patch fixes a bug in Happy Eyeballs where curl would wait for a
connect response from socket1 before checking socket2.
Also, it updates error messages for failed connections, showing the ip
addresses that failed rather than just the host name repeatedly.
Bug: http://curl.haxx.se/mail/lib-2013-10/0236.html
Reported-by: Paul Marks
This patch invokes two socket connect()s nearly simultaneously, and
the socket that is first connected "wins" and is subsequently used for
the connection. The other is terminated.
There is a very slight IPv4 preference, in that if both sockets connect
simultaneously IPv4 is checked first and thus will win.
Should a client application fail to decode an authentication message
received from a server, or not support any of the parameters given by
the server in the message, then the authentication phrase should be
cancelled gracefully by the client rather than simply terminating the
connection.
The authentication phrase should be cancelled by simply sending a '*'
to the server, in response to erroneous data being received, as per
RFC-3501, RFC-4954 and RFC-5034.
This patch adds the necessary state machine constants and appropriate
response handlers in order to add this functionality for the CRAM-MD5,
DIGEST-MD5 and NTLM authentication mechanisms.
This is a regression since the switch to always-multi internally
c43127414d.
Test 1316 was modified since we now clearly call the Curl_client_write()
function when doing the LIST transfer part and then the
handler->protocol says FTP and ftpc.transfertype is 'A' which implies
text converting even though that the response is initially a HTTP
CONNECT response in this case.
This workaround had been previously been implemented for IMAP and POP3
but not SMTP. Some of the recent test case additions implemented this
behaviour to emulate a bad server and the SMTP code didn't cope with it.
... if not already initialized. This fixes a regression introduced by
commit 4ad8e142da, which caused test619
to intermittently fail on certain machines (namely Fedora build hosts).
Changed the failure code when TLS v1.1 and v1.2 is requested but not
supported by older OpenSSL versions, following review from libcurl
peers, and reduced the number of required preprocessor if statements.
...with the use of CURL_SSLVERSION_TLSv1_1 and CURL_SSLVERSION_TLSv1_2
being conditional on OpenSSL v1.0.1 as the appropriate flags are not
supported under earlier versions.
Commit ad34a2d5c8 relies on definitions that are only present in
OpenSSL v1.0.1 and up. This quick fix allows the builds that use
older versions of OpenSSL to continue building.
According to the documentation for libssh2_userauth_list(), a NULL
return value is not necessarily an error. You must call
libssh2_userauth_authenticated() to determine if the SSH_USERAUTH_NONE
request was successful.
This fixes a segv when using sftp on a server that allows logins with an
empty password. When NULL was interpreted as an error, it would
free the session but not flag an error since the libssh2 errno would be
clear. This resulted in dereferencing a NULL session pointer.
Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
CURL_SSLVERSION_TLSv1_0, CURL_SSLVERSION_TLSv1_1,
CURL_SSLVERSION_TLSv1_2 enum values are added to force exact TLS version
(CURL_SSLVERSION_TLSv1 means TLS 1.x).
axTLS:
axTLS only supports TLS 1.0 and 1.1 but it cannot be set that only one
of these should be used, so we don't allow the new enum values.
darwinssl:
Added support for the new enum values.
SChannel:
Added support for the new enum values.
CyaSSL:
Added support for the new enum values.
Bug: The original CURL_SSLVERSION_TLSv1 value enables only TLS 1.0 (it
did the same before this commit), because CyaSSL cannot be configured to
use TLS 1.0-1.2.
GSKit:
GSKit doesn't seem to support TLS 1.1 and TLS 1.2, so we do not allow
those values.
Bugfix: There was a typo that caused wrong SSL versions to be passed to
GSKit.
NSS:
TLS minor version cannot be set, so we don't allow the new enum values.
QsoSSL:
TLS minor version cannot be set, so we don't allow the new enum values.
OpenSSL:
Added support for the new enum values.
Bugfix: The original CURL_SSLVERSION_TLSv1 value enabled only TLS 1.0,
now it enables 1.0-1.2.
Command-line tool:
Added command line options for the new values.
Setting only CURLOPT_SSL_VERIFYHOST without CURLOPT_SSL_VERIFYPEER set
should still verify that the host name fields in the server certificate
is fine or return failure.
Bug: http://curl.haxx.se/mail/lib-2013-10/0002.html
Reported-by: Ishan SinghLevett
If no WINVER and/or _WIN32_IWNNT define was set, the Windows platform
SDK often defaults to high value, e.g. 0x601 (whoch may probably depend
on the Windows version being used, in my case Windows 7).
If WINVER >= 0x600 then winsock2.h includes some defines for WSAPoll(),
e.g. POLLIN, POLLPRI, POLLOUT etc. These defines clash with cURL's
lib/select.h.
Make sure HAVE_STRUCT_POLLFD is defined then.
Bug: http://curl.haxx.se/bug/view.cgi?id=1282
Reported-by: "kdekker"
Patch-by: Marcel Raad
Moved the standard SASL mechanism strings into curl_sasl.h rather than
hard coding the same values over and over again in the protocols that
use SASL authentication.
For more information about the mechanism strings see:
http://www.iana.org/assignments/sasl-mechanisms
In ossl_connect_step2() when the "Unknown SSL protocol error" occurs, it
would output the local port number instead of the remote one which
showed when doing SSL over a proxy (but with the correct remote host
name). As libcurl only speaks SSL to the remote we know it is the remote
port.
Bug: http://curl.haxx.se/bug/view.cgi?id=1281
Reported-by: Gordon Marler
Added the ability to use an XOAUTH2 bearer token [RFC6750] with POP3 for
authentication using RFC6749 "OAuth 2.0 Authorization Framework".
The bearer token is expected to be valid for the user specified in
conn->user. If CURLOPT_XOAUTH2_BEARER is defined and the connection has
an advertised auth mechanism of "XOAUTH2", the user and access token are
formatted as a base64 encoded string and sent to the server as
"AUTH XOAUTH2 <bearer token>".
iOS 7 finally added the option to enable 1/n-1 when using TLS 1.0
and a CBC cipher, so we now always turn that on unless the user
manually turns it off using CURLSSLOPT_ALLOW_BEAST.
It appears Apple also added some new PSK ciphers, but no interface to
use them yet, so we at least support printing them if we find them.
Implement: Expired Cookies These following situation, curl removes
cookie(s) from struct CookieInfo if the cookie expired.
- Curl_cookie_add()
- Curl_cookie_getlist()
- cookie_output()
Solaris with the SunStudio Compiler is reportedly missing this define,
but as we're using it without any good reason on all the places it was
used I've now instead switched to just use sensible buffer sizes that
fit a 32 bit decimal number. Which also happens to be smaller than the
common NI_MAXSERV value which is 32 on most machines.
Bug: http://curl.haxx.se/bug/view.cgi?id=1277
Reported-by: D.Flinkmann
Otherwise, the FTP protocol would unnecessarily hang 60 seconds if
aborted in the CURLOPT_HEADERFUNCTION callback.
Reported by: Tomas Mlcoch
Bug: https://bugzilla.redhat.com/1005686
Make sure that the custom struct fields are only used by code that
doesn't use a struct defintion from the outside.
Attempts to fix the problem introduced in 3dc6fc42bf
Otherwise the connection can get stuck during various phases, waiting
for new data on the socket using select() etc., but it will never be
received as the data has already been read into SSL library.
The transfer size would be calculated incorrectly if the email contained
within the FETCH response, had been partially received by the pingpong
layer. As such the following, example output, would be seen if the
amount remaining was smaller than the amount received:
* Excess found in a non pipelined read: excess = 1394, size = 262,
maxdownload = 262, bytecount = 1374
* transfer closed with -1112 bytes remaining to read
Bug: http://curl.haxx.se/mail/lib-2013-08/0170.html
Reported-by: John Dunn
When building the code using LLVM Clang without NGHTTP2, I was getting
this warning:
../lib/http.h:155:1: warning: empty struct is a GNU extension [-Wgnu]
Placing a dummy variable into the data structure silenced the warning.
Recent OpenSSL uses user interface abstraction to negotiate access to
private keys in the cryprographical engines. An OpenSSL application is
expected to implement the user interface. Otherwise a default one
provided by OpenSSL (interactive standard I/O) will be used and the
aplication will have no way how to pass a password to the engine.
Longer-desc: http://curl.haxx.se/mail/lib-2013-08/0265.html
Using the first little merge of nghttp2 into libcurl, I stumbeled on the
missing 'snprintf' in MSVCRT. Isn't this how we do it for other libcurl
files? I.e. use 'curl_msnprintf' and not 'snprintf' directly:
When an error occurs parsing an LDAP URL, The ludp->lud_attrs[i] entries
could be freed even though they sometimes point to data within an
allocated area.
This change introduces a lud_attrs_dup[] array for the duplicated string
pointers, and it removes the unused lud_exts array.
Bug: http://curl.haxx.se/mail/lib-2013-08/0209.html
XOAUTH2 would be selected in preference to LOGIN and PLAIN if the IMAP
or SMTP server advertised support for it even though a user's password
was supplied but bearer token wasn't.
Modified the selection logic so that XOAUTH2 will only be selected if
the server supports it and A) The curl user/libcurl programmer has
specifically asked for XOAUTH via the ;AUTH=XOAUTH login option or 2)
The bearer token is specified. Obviously if XOAUTH is asked for via
the login option but no token is specified the user will receive a
authentication failure which makes more sense than no known
authentication mechanisms supported!
'struct monitor', introduced in 6cf8413e, already exists in an IRIX
header file (sys/mon.h) which gets included via various standard headers
by lib/easy.c
cc-1101 cc: ERROR File = ../../curl/lib/easy.c, Line = 458
"monitor" has already been declared in the current scope.
Reported-by: Tor Arntsen
When waiting for a 100-continue response from the server, the
Curl_readwrite() will refuse to run if called until the timeout has been
reached.
We timeout code in multi_socket() allows code to run slightly before the
actual timeout time, so for test 154 it could lead to the function being
executed but refused in Curl_readwrite() and then the application would
just sit idling forever.
This was detected with runtests.pl -e on test 154.
I brought back security.h in commit bb55293313. As we actually
already found out back in 2005 in commit 62970da675, the file name
security.h causes problems so I renamed it curl_sec.h instead.
Added the ability to use an XOAUTH2 bearer token [RFC6750] with SMTP for
authentication using RFC6749 "OAuth 2.0 Authorization Framework".
The bearer token is expected to be valid for the user specified in
conn->user. If CURLOPT_XOAUTH2_BEARER is defined and the connection has
an advertised auth mechanism of "XOAUTH2", the user and access token are
formatted as a base64 encoded string and sent to the server as
"AUTH XOAUTH2 <bearer token>".
Added the ability to use an XOAUTH2 bearer token [RFC6750] with IMAP for
authentication using RFC6749 "OAuth 2.0 Authorization Framework".
The bearer token is expected to be valid for the user specified in
conn->user. If CURLOPT_XOAUTH2_BEARER is defined and the connection has
an advertised auth mechanism of "XOAUTH2", the user and access token are
formatted as a base64 encoded string and sent to the server as
"A001 AUTHENTICATE XOAUTH2 <bearer token>".
Added the ability to specify an XOAUTH2 bearer token [RFC6750] via the
option CURLOPT_XOAUTH2_BEARER for authentication using RFC6749 "OAuth
2.0 Authorization Framework".
Added the ability to generated a base64 encoded XOAUTH2 token
containing: "user=<username>^Aauth=Bearer <bearer token>^A^A"
as per RFC6749 "OAuth 2.0 Authorization Framework".
We've announced this pending removal for a long time and we've
repeatedly asked if anyone would care or if anyone objects. Nobody has
objected. It has probably not even been working for a good while since
nobody has tested/used this code recently.
The stuff in krb4.h that was generic enough to be used by other sources
is now present in security.h
Make sure we always return CURLM_CALL_MULTI_PERFORM when we reach
CURLM_STATE_DONE since the state is transient and it can very well
continue executing as there is nothing to wait for.
Bug: http://curl.haxx.se/mail/lib-2013-08/0211.html
Reported-by: Yi Huang
... this also makes sure that the progess callback gets called more
often during TFTP transfers.
Added test 1238 to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=1269
Reported-by: Jo3
I build curl.exe (using MingW) with '-DCURLDEBUG' and by importing from
libcurl.dll. Which means the new curl_easy_perform_ev() must be
exported from libcurl.dll.
Doing curl_multi_add_handle() on an easy handle that is already added to
a multi handle now returns this error code. It previously returned
CURLM_BAD_EASY_HANDLE for this condition.
The closure_handle is "owned" by the multi handle and it is
unconditional so the setting up of it should be in the Curl_multi_handle
function rather than curl_multi_add_handle.
This function is meant to work *exactly* as curl_easy_perform() but will
use the event-based libcurl API internally instead of
curl_multi_perform(). To avoid relying on an actual event-based library
and to not use non-portable functions (like epoll or similar), there's a
rather inefficient emulation layer implemented on top of Curl_poll()
instead.
There's currently some convenience logging done in curl_easy_perform_ev
which helps when tracking down problems. They may be suitable to remove
or change once things seem to be fine enough.
curl has a new --test-event option when built with debug enabled that
then uses curl_easy_perform_ev() instead of curl_easy_perform(). If
built without debug, using --test-event will only output a warning
message.
NOTE: curl_easy_perform_ev() is not part if the public API on purpose.
It is only present in debug builds of libcurl and MUST NOT be considered
stable even then. Use it for libcurl-testing purposes only.
runtests.pl now features an -e command line option that makes it use
--test-event for all curl command line tests. The man page is updated.
libcurl quietly truncates usernames, passwords, and options from
before an '@' sign in a URL to 255 (= MAX_CURL_PASSWORD_LENGTH - 1)
characters to fit in fixed-size buffers on the stack. Allocate a
buffer large enough to fit the parsed fields on the fly instead to
support longer passwords.
After this change, there are no more uses of MAX_CURL_OPTIONS_LENGTH
left, so stop defining that constant while at it. The hardcoded max
username and password length constants, on the other hand, are still
used in HTTP proxy credential handling (which this patch doesn't
touch).
Reported-by: Colby Ranger
Instead of nesting "if(success)" blocks and leaving the reader in
suspense about what happens in the !success case, deal with failure
cases early, usually with a simple goto to clean up and return from
the function.
No functional change intended. The main effect is to decrease the
indentation of this function slightly.
libcurl truncates usernames, passwords, and options set with
curl_easy_setopt to 255 (= MAX_CURL_PASSWORD_LENGTH - 1) characters.
This doesn't affect the return value from curl_easy_setopt(), so from
the caller's point of view, there is no sign anything strange has
happened, except that authentication fails.
For example:
# Prepare a long (300-char) password.
s=0123456789; s=$s$s$s$s$s$s$s$s$s$s; s=$s$s$s;
# Start a server.
nc -l -p 8888 | tee out & pid=$!
# Tell curl to pass the password to the server.
curl --user me:$s http://localhost:8888 & sleep 1; kill $pid
# Extract the password.
userpass=$(
awk '/Authorization: Basic/ {print $3}' <out |
tr -d '\r' |
base64 -d
)
password=${userpass#me:}
echo ${#password}
Expected result: 300
Actual result: 255
The fix is simple: allocate appropriately sized buffers on the heap
instead of trying to squeeze the provided values into fixed-size
on-stack buffers.
Bug: http://bugs.debian.org/719856
Reported-by: Colby Ranger
libcurl truncates usernames and passwords it reads from .netrc to
LOGINSIZE and PASSWORDSIZE (64) characters without any indication to
the user, to ensure the values returned from Curl_parsenetrc fit in a
caller-provided buffer.
Fix the interface by passing back dynamically allocated buffers
allocated to fit the user's input. The parser still relies on a
256-character buffer to read each line, though.
So now you can include an ~246-character password in your .netrc,
instead of the previous limit of 63 characters.
Reported-by: Colby Ranger
Instead of remembering before each "return" statement which temporary
allocations, if any, need to be freed, take care to set pointers to
NULL when no longer needed and use a goto to a common block to exit
the function and free all temporaries.
No functional change intended. Currently the only temporary buffer in
this function is "proxy" which is already correctly freed when
appropriate, but there will be more soon.
Use appropriately sized buffers on the heap instead of fixed-size
buffers on the stack, to allow for longer usernames and passwords.
Callers never pass anything longer than MAX_CURL_USER_LENGTH (resp.
MAX_CURL_PASSWORD_LENGTH), so no functional change inteded yet.
This makes the socket callback get called with the proper bitmask as
otherwise the application could be left hanging waiting for reading on
an upload connection!
Bug: http://curl.haxx.se/mail/lib-2013-08/0043.html
Reported-by: Bill Doyle
With everything being struct SessionHandle pointers now, this rename
makes multi.c use the library-wide practise of calling that pointer
'data' instead of the previously used 'easy'.
Moved Curl_easy_addmulti() from easy.c to multi.c, renamed it to
easy_addmulti and made it static.
Removed Curl_easy_initHandleData() and uses of it since it was emptied
in commit cdda92ab67b47d74a.
All protocol handler structs are now opaque (void *) in the
SessionHandle struct and moved in the request-specific sub-struct
'SingleRequest'. The intension is to keep the protocol specific
knowledge in their own dedicated source files [protocol].c etc.
There's some "leakage" where this policy is violated, to be addressed at
a later point in time.
1 - always allocate the struct in protocol->setup_connection. Some
protocol handlers had to get this function added.
2 - always free at the end of a request. This is also an attempt to keep
less memory in the handle after it is completed.
CURLOPT_DNS_USE_GLOBAL_CACHE broke in commit c43127414d (been
broken since the libcurl 7.29.0 release). While this option has been
documented as deprecated for almost a decade and nobody even reported
this bug, it should remain functional.
Added test case 1512 to verify
The previous naming scheme ftp_state_post_XXXX() wasn't really helpful
as it wasn't always immediately after 'xxxx' and it wasn't easy to
understand what it does based on such a name.
This new one is instead ftp_state_yyyy() where yyyy describes what it
does or sends.
This is a regression as this logic used to work. It isn't clear when it
broke, but I'm assuming in 7.28.0 when we went all-multi internally.
This likely never worked with the multi interface. As the failed
connection is detected once the multi state has reached DO_MORE, the
Curl_do_more() function was now expanded somewhat so that the
ftp_do_more() function can request to go "back" to the previous state
when it makes another attempt - using PASV.
Added test case 1233 to verify this fix. It has the little issue that it
assumes no service is listening/accepting connections on port 1...
Reported-by: byte_bucket in the #curl IRC channel
For some reason, OS X 10.5's GCC suddenly stopped working correctly with
macros that change MD5_Init etc. in the code to CC_MD5_Init etc., so I
worked around this by removing use of the macros and inserting static
functions that just call CommonCrypto's implementations of the functions
instead.
This changes the previous check for untrusted certs to a check for
certs explicitely marked as trusted.
The change is backward-compatible (tested with certdata.txt v1.80).
The internal function that's used to detect known file extensions for
the default Content-Type got the the wrong pointer passed in when
CURLFORM_BUFFER + CURLFORM_BUFFERPTR were used. This had the effect that
strlen() would be used which could lead to an out-of-bounds read (and
thus segfault). In most cases it would only lead to it not finding or
using the correct default content-type.
It also showed that test 554 and test 587 were testing for the
previous/wrong behavior and now they're updated as well.
Bug: http://curl.haxx.se/bug/view.cgi?id=1262
Reported-by: Konstantin Isakov
Christian Heimes brought to our attention that the certdata.txt
format has recently changed [1], causing ca-bundle.crt created
with mk-ca-bundle.[pl|vbs] to include untrusted certs.
[1] http://lists.debian.org/debian-release/2012/11/msg00411.html
The motivation for having a separate struct that keep track of an easy
handle when using the multi handle was removed when we switched to
always using the multi interface internally. Now they were just two
separate struct that was always allocated for each easy handle.
This first step just moves the Curl_one_easy struct members into the
SessionHandle struct and hides this somehow (== keeps the source code
changes to a minimum) by defining Curl_one_easy to SessionHandle
The biggest changes in this commit are:
1 - the linked list of easy handles had to be changed somewhat due
to the new struct layout. This made the main linked list pointer
get renamed to 'easyp' and there's also a new pointer to the last
node, called easylp. It is no longer circular but ends with ->next
pointing to NULL. New nodes are still added last.
2 - easy->state is now called easy->mstate to avoid name collision
(This doesn't need to appear in the release notes.) I noticed a few places
where infof() was called, and there should've been an LF at the end of the
string, but there wasn't.
It turns out Snow Leopard not only has SecItemCopyMatching() defined in
a header not included by the omnibus header, but it won't work for our
purposes, because searching for SecIdentityRef objects wasn't added
to that API until Lion. So we now use the old SecKeychainSearch API
instead if the user is building under, or running under, Snow Leopard.
Bug: http://sourceforge.net/p/curl/bugs/1255/
Reported by: Edward Rudd
Previously we used __MAC_10_X and __IPHONE_X to mark digest-generating
code that was specific to OS X and iOS. Now we use
__MAC_OS_X_VERSION_MAX_ALLOWED and __IPHONE_OS_VERSION_MAX_ALLOWED
instead of those macros.
Bug: http://sourceforge.net/p/curl/bugs/1255/
Reported by: Edward Rudd
For the standard VMS text file formats, VMS needs to read the file to
get the actual file size.
For the standard VMS binary file formats, VMS needs a special format of
fopen() call so that it stops reading at the logical end of file instead
of at the end of the blocks allocated to the file.
I structured the patch this way as I was not sure about changing the
structures or parameters to the routines, but would prefer to only call
the stat() function once and pass the information to where the fopen()
call is made.
Bug: https://sourceforge.net/p/curl/bugs/758/
The code for CURLFORM_FILECONTENT had its check for duplicate options
wrong so that it would reject CURLFORM_PTRNAME if used in combination
with it (but not CURLFORM_COPYNAME)! The flags field used for this
purpose cannot be interpreted that broadly.
Bug: http://curl.haxx.se/mail/lib-2013-07/0258.html
Reported-by: Byrial Jensen
Linking on Solaris 10 x86 with Sun Studio 12 failed when we upgraded
automake for the release builds.
Bug: http://curl.haxx.se/bug/view.cgi?id=1217
Reported-by: Dagobert Michelsen
Commit 6d30f8ebed didn't work properly. First, it used the wrong
array index, but this fix also:
1 - only does the copying if indeed there was any activity
2 - makes sure to properly translate between internal and external
bitfields, which are not guaranteed to match
Reported-by: Evgeny Turnaev
Instead of going 50,100,150 etc millisecond delay time when nothing has
been found to do or wait for, we now start lower and double each loop as
in 4,8,16,32 etc.
This lowers the minimum wait without sacrifizing the longer wait too
much with unnecessary CPU cycles burnt.
Bug: http://curl.haxx.se/mail/lib-2013-07/0103.html
Reported-by: Andreas Malzahn
In the case of an active connection when ftp_do_more() detects that the
server has connected back, it must make sure to mark it as complete so
that the multi_runsingle() function will detect this and move on to the
next state.
Bug: http://curl.haxx.se/mail/lib-2013-07/0115.html
Reported-by: Clemens Gruber
CURLOPT_XFERINFOFUNCTION is now the preferred progress callback function
and CURLOPT_PROGRESSFUNCTION is considered deprecated.
This new callback uses pure 'curl_off_t' arguments to pass on full
resolution sizes. It otherwise retains the same characteristics: the
same call rate, the same meanings for the arguments and the return code
is used the same way.
The progressfunc.c example is updated to show how to use the new
callback for newer libcurls while supporting the older one if built with
an older libcurl or even built with a newer libcurl while running with
an older.
This reverts commit 7ed25cc, reinstating commit 8ec2cb5.
As of 18-jul-2013 we still do have code in libcurl that makes use of these
memory functions. Commit 8ec2cb5 comment still applies and is yet valid.
These memory functions are solely used in Windows builds, so all related
code is protected with '#ifdef WIN32' preprocessor conditional compilation
directives.
Specifically, wcsdup() _wcsdup() are used when building a Windows target with
UNICODE and USE_WINDOWS_SSPI preprocessor symbols defined. This is the case
when building a Windows UNICODE target with Windows native SSL/TLS support
enabled.
Realizing that wcsdup() _wcsdup() are used is a bit tricky given that usage
of these is hidden behind _tcsdup() which is MS way of dealing with code
that must tolerate UNICODE and non-UNICODE compilation. Additionally, MS
header files and those compatible from other compilers use this preprocessor
conditional compilation directive in order to select at compilation time
whether 'wide' or 'ansi' MS API functions are used.
Without this code, Windows build targets with Windows native SSL/TLS support
enabled and MemoryTracking support enabled misbehave in tracking memory usage,
regardless of being a UNICODE enabled build or not.
Fixed issue with static build for MSVC2010.
After some investigation I've discovered known issue
http://public.kitware.com/Bug/view.php?id=11240 When .rc file is linked
to static lib it fails with following linker error
LINK : warning LNK4068: /MACHINE not specified; defaulting to X86
file.obj : fatal error LNK1112: module machine type 'x64' conflicts with
target machine type 'X86'
Fix add target property /MACHINE: for MSVC generation.
Also removed old workarounds - it caused errors during msvc build.
Bug: http://curl.haxx.se/mail/lib-2013-07/0046.html
I just noticed that OS X no longer supports SSLv2. Other TLS engines return
an error if the requested protocol isn't supported by the underlying
engine, so we do that now for SSLv2 if the framework returns an error
when trying to turn on SSLv2 support. (Note: As always, SSLv2 support is
only enabled in curl when starting the app with the -2 argument; it's off
by default. SSLv2 is really old and insecure.)
When doing multi-part formposts, libcurl used a pseudo-random value that
was seeded with time(). This turns out to be bad for users who formpost
data that is provided with users who then can guess how the boundary
string will look like and then they can forge a different formpost part
and trick the receiver.
My advice to such implementors is (still even after this change) to not
rely on the boundary strings being cryptographically strong. Fix your
code and logic to not depend on them that much!
I moved the Curl_rand() function into the sslgen.c source file now to be
able to take advantage of the SSL library's random function if it
provides one. If not, try to use the RANDOM_FILE for seeding and as a
last resort keep the old logic, just modified to also add microseconds
which makes it harder to properly guess the exact seed.
The formboundary() function in formdata.c is now using 64 bit entropy
for the boundary and therefore the string of dashes was reduced by 4
letters and there are 16 hex digits following it. The total length is
thus still the same.
Bug: http://curl.haxx.se/bug/view.cgi?id=1251
Reported-by: "Floris"
When using %x, the number must be treated as unsigned as otherwise it
would get sign-extended on for example 64bit machines and do wrong
output. This problem showed when doing printf("%08x", 0xffeeddcc) on a
64bit host.
Follow-up fix from 7d80ed64e4.
The SessionHandle may not be around to use when we restore the sigpipe
sighandler so we store the no_signal boolean in the local struct to know
if/how to restore.
When the c-ares based resolver backend failed to resolve a name, it
tried to show the name that failed from existing structs. This caused
the wrong output and shown hostname when for example --interface
[hostname] was used and that name resolving failed.
Now we use the hostname used in the actual resolve attempt in the error
message as well.
Bug: http://curl.haxx.se/bug/view.cgi?id=1191
Reported-by: Kim Vandry
When we recently started to treat a zero return code from SSL_read() as
an error we also got false positives - which primarily looks to be
because the OpenSSL documentation is wrong and a zero return code is not
at all an error case in many situations.
Now ossl_recv() will check with ERR_get_error() to see if there is a
stored error and only then consider it to be a true error if SSL_read()
returned zero.
Bug: http://curl.haxx.se/bug/view.cgi?id=1249
Reported-by: Nach M. S.
Patch-by: Nach M. S.
Something (a recent security update maybe?) changed in Lion, and now it
has changed SSLCopyPeerTrust such that it may return noErr but also give
us a null trust, which caught us off guard and caused an eventual crash.
... and restore the ordinary handling again when it returns. This is
done for curl_easy_perform() and curl_easy_cleanup() only for now - and
only when built to use OpenSSL as backend as this is the known culprit
for the spurious SIGPIPEs people have received.
Bug: http://curl.haxx.se/bug/view.cgi?id=1180
Reported by: Lluís Batlle i Rossell
This doesn't need to be in the release notes. I cleaned up a lot of the #if
lines in the code to use MAC_OS_X_VERSION_MIN_REQUIRED and
MAC_OS_X_VERSION_MAX_ALLOWED instead of checking for whether things like
__MAC_10_6 or whatever were defined, because for some SDKs Apple has released
they were defined out of place.
RFC3986 details how a path part passed in as part of a URI should be
"cleaned" from dot sequences before getting used. The described
algorithm is now implemented in lib/dotdot.c with the accompanied test
case in test 1395.
Bug: http://curl.haxx.se/bug/view.cgi?id=1200
Reported-by: Alex Vinnik
Security problem: CVE-2013-2174
If a program would give a string like "%FF" to curl_easy_unescape() but
ask for it to decode only the first byte, it would still parse and
decode the full hex sequence. The function then not only read beyond the
allowed buffer but it would also deduct the *unsigned* counter variable
for how many more bytes there's left to read in the buffer by two,
making the counter wrap. Continuing this, the function would go on
reading beyond the buffer and soon writing beyond the allocated target
buffer...
Bug: http://curl.haxx.se/docs/adv_20130622.html
Reported-by: Timo Sirainen
As a remedy to the problem when a socket gets closed and a new one is
opened with the same file descriptor number and as a result
multi.c:singlesocket() doesn't detect the difference, the new function
Curl_multi_closed() gets told when a socket is closed so that it can be
removed from the socket hash. When the old one has been removed, a new
socket should be detected fine by the singlesocket() on next invoke.
Bug: http://curl.haxx.se/bug/view.cgi?id=1248
Reported-by: Erik Johansson
When performing COOKIELIST operations the cookie lock needs to be taken
for the cases where the cookies are shared among multiple handles!
Verified by Benjamin Gilbert's updated test 506
Bug: http://curl.haxx.se/bug/view.cgi?id=1215
Reported-by: Benjamin Gilbert
When curl_multi_wait() finds no file descriptor to wait for, it returns
instantly and this must be handled gracefully within curl_easy_perform()
or cause a busy-loop. Starting now, repeated fast returns without any
file descriptors is detected and a gradually increasing sleep will be
used (up to a max of 1000 milliseconds) before continuing the loop.
Bug: http://curl.haxx.se/bug/view.cgi?id=1238
Reported-by: Miguel Angel
The initial fix to only compare full path names were done in commit
04f52e9b4d but found out to be incomplete. This takes should make the
change more complete and there's now two additional tests to verify
(test 31 and 62).
By always returning the md5 for an empty body when auth-int is asked
for, libcurl now at least sometimes does the right thing.
Bug: http://curl.haxx.se/bug/view.cgi?id=1235
Patched-by: Nach M. S.
Allow less room for "triggered too early" mistakes by applications /
timers on non-windows platforms. Starting now, we assume that a timeout
call is never made earlier than 3 milliseconds before the actual
timeout. This greatly improves timeout accuracy on Linux.
Bug: http://curl.haxx.se/bug/view.cgi?id=1228
Reported-by: Hang Su
In the pkcs12 code, we get a list of x509 records returned from
PKCS12_parse but when iterating over the list and passing each to
SSL_CTX_add_extra_chain_cert() we didn't also properly remove them from
the "stack", which made them get freed twice (both in sk_X509_pop_free()
and then later in SSL_CTX_free).
This isn't really documented anywhere...
Bug: http://curl.haxx.se/bug/view.cgi?id=1236
Reported-by: Nikaiw
commit 29bf0598aa introduced a problem when the "internal" timeout is
prefered to the given if shorter, as it didn't consider the case where
-1 was returned. Now the internal timeout is only considered if not -1.
Reported-by: Tor Arntsen
Bug: http://curl.haxx.se/mail/lib-2013-06/0015.html
If the multi handle's pending timeout is less than what is passed into
this function, it will now opt to use the shorter time anyway since it
is a very good hint that the handle wants to process something in a
shorter time than what otherwise would happen.
curl_multi_wait.3 was updated accordingly to clarify
This is the reason for bug #1224
Bug: http://curl.haxx.se/bug/view.cgi?id=1224
Reported-by: Andrii Moiseiev
When sending the HTTP Authorization: header for digest, the user name
needs to be escaped if it contains a double-quote or backslash.
Test 1229 was added to verify
Reported and fixed by: Nach M. S
Bug: http://curl.haxx.se/bug/view.cgi?id=1230
We found that in specific cases if the connection is abruptly closed,
the underlying socket is listed in a close_wait state. We continue to
call the curl_multi_perform, curl_mutli_fdset etc. None of these APIs
report the socket closed / connection finished. Since we have cases
where the multi connection is only used once, this can pose a problem
for us. I've read that if another connection was to come in, curl would
see the socket as bad and attempt to close it at that time -
unfortunately, this does not work for us.
I found that in specific situations, if SSL_write returns 0, curl did
not recognize the socket as closed (or errored out) and did not report
it to the application. I believe we need to change the code slightly, to
check if ssl_write returns 0. If so, treat it as an error - the same as
a negative return code.
For OpenSSL - the ssl_write documentation is here:
http://www.openssl.org/docs/ssl/SSL_write.html
1 - don't skip host names with a colon in them in an attempt to bail out
on HTTP headers in the cookie file parser. It was only a shortcut anyway
and trying to parse a file with HTTP headers will still be handled, only
slightly slower.
2 - don't skip domain names based on number of dots. The original
netscape cookie spec had this oddity mentioned and while our code
decreased the check to only check for two, the existing cookie spec has
no such dot counting required.
Bug: http://curl.haxx.se/bug/view.cgi?id=1221
Reported-by: Stefan Neis
I found a bug which cURL sends cookies to the path not to aim at.
For example:
- cURL sends a request to http://example.fake/hoge/
- server returns cookie which with path=/hoge;
the point is there is NOT the '/' end of path string.
- cURL sends a request to http://example.fake/hogege/ with the cookie.
The reason for this old "feature" is because that behavior is what is
described in the original netscape cookie spec:
http://curl.haxx.se/rfc/cookie_spec.html
The current cookie spec (RFC6265) clarifies the situation:
http://tools.ietf.org/html/rfc6265#section-5.2.4
This reverts commit 8ec2cb5544.
We don't have any code anywhere in libcurl (or the curl tool) that use
wcsdup so there's no such memory use to track. It seems to cause mild
problems with the Borland compiler though that we may avoid by reverting
this change again.
Bug: http://curl.haxx.se/mail/lib-2013-05/0070.html
If the mail sent during the transfer contains a terminating <CRLF> then
we should not send the first <CRLF> of the EOB as specified in RFC-5321.
Additionally don't send the <CRLF> if there is "no mail data" as the
DATA command already includes it.
The code within #ifdef HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID wrongly had two
closing braces when it should only have one, so builds without that
define would fail.
Bug: http://curl.haxx.se/mail/lib-2013-05/0000.html
The curl command line utility would display the the completed progress
bar with a percentage of zero as the progress routines didn't know the
size of the transfer.
Removed the hard returns from imap and pop3 by using the same style for
sending the authentication string as smtp. Moved the "Other mechanisms
not supported" check in smtp to match that of imap and pop3 to provide
consistency between the three email protocols.
Users using the Secure Transport (darwinssl) back-end can now use a
certificate and private key to authenticate with a site using TLS. Because
Apple's security system is based around the keychain and does not have any
non-public function to create a SecIdentityRef data structure from data
loaded outside of the Keychain, the certificate and private key have to be
loaded into the Keychain first (using the certtool command line tool or
the Security framework's C API) before we can find it and use it.
In addition to checking for the SASL-IR capability the user can override
the sending of the client's initial response in the AUTHENTICATION
command with the use of CURLOPT_SASL_IR should the server erroneously
not report SASL-IR when it does support it.
Updated the default behaviour of sending the client's initial response in the AUTH
command to not send it and added support for CURLOPT_SASL_IR to allow the user to
specify including the response.
Related Bug: http://curl.haxx.se/mail/lib-2012-03/0114.html
Reported-by: Gokhan Sengun
By introducing an internal alternative to curl_multi_init() that accepts
parameters to set the hash sizes, easy handles will now use tiny socket
and connection hash tables since it will only ever add a single easy
handle to that multi handle.
This decreased the number mallocs in test 40 (which is a rather simple
and typical easy interface use case) from 1142 to 138. The maximum
amount of memory allocated used went down from 118969 to 78805.
When connecting back to an FTP server after having sent PASV/EPSV,
libcurl sometimes didn't use the proxy properly even though the proxy
was used for the initial connect.
The function wrongly checked for the CURLOPT_PROXY variable to be set,
which made it act wrongly if the proxy information was set with an
environment variable.
Added test case 711 to verify (based on 707 which uses --socks5). Also
added test712 to verify another variation of setting the proxy: with
--proxy socks5://
Bug: http://curl.haxx.se/bug/view.cgi?id=1218
Reported-by: Zekun Ni
... in order to prevent an artificial timeout event based on stale
speed-check data from a previous network transfer. This commit fixes
a regression caused by 9dd85bced5.
Bug: https://bugzilla.redhat.com/906031
Fixed an issue in parse_proxy(), introduced in commit 11332577b3,
where an empty username or password (For example: http://:@example.com)
would cause a crash.
There is no need to perform separate clearing of data if a NULL option
pointer is passed in. Instead this operation can be performed by simply
not calling parse_login_details() and letting the rest of the code do
the work.
setstropt_userpwd() was calling setstropt() in commit fddb7b44a7 to
set each of the login details which would duplicate the strings and
subsequently cause a memory leak.
In addition to parsing the optional login options from the URL, added
support for parsing them from CURLOPT_USERPWD, to allow the following
supported command line:
--user username:password;options
Added bounds checking when searching for the separator characters within
the login string as this string may not be NULL terminated (For example
it is the login part of a URL). We do this in preference to allocating a
new string to copy the login details into which could then be passed to
parse_login_details() for performance reasons.
As well as parsing the username and password from the URL, added support
for parsing the optional options part from the login details, to allow
the following supported URL format:
schema://username:password;options@example.com/path?q=foobar
This will only be used by IMAP, POP3 and SMTP at present but any
protocol that may be given login options in the URL will be able to
add support for them.
...instead of the 220 we otherwise expect.
Made the ftpserver.pl support sending a custom "welcome" and then
created test 1219 to verify this fix with such a 230 welcome.
Bug: http://curl.haxx.se/mail/lib-2013-02/0102.html
Reported by: Anders Havn
Accessing a file with an absolute path in the root dir but with no
directory specified was not handled correctly. This fix comes with four
new test cases that verify it.
Bug: http://curl.haxx.se/mail/lib-2013-04/0142.html
Reported by: Sam Deane
Cookies set for 'example.com' could accidentaly also be sent by libcurl
to the 'bexample.com' (ie with a prefix to the first domain name).
This is a security vulnerabilty, CVE-2013-1944.
Bug: http://curl.haxx.se/docs/adv_20130412.html
The previously applied patch didnt work on Windows; we cant rely
on shell commands like 'echo' since they act diffently on each
platform and each shell.
In order to keep this script platform-independent the code must
only use pure Perl.
When doing PWD, there's a 257 response which apparently some servers
prefix with a comment before the path instead of after it as is
otherwise the norm.
Failing to parse this, several otherwise legitimate use cases break.
Bug: http://curl.haxx.se/mail/lib-2013-04/0113.html
The OpenSSL pipe wrote to the final CA bundle file, but the encoded PEM
output wrote to a temporary file. Consequently, the OpenSSL output was
lost when the temp file was renamed to the final file at script finish
(overwriting the final file written earlier by openssl).
Patch posted to the list by Richard Michael (rmichael edgeofthenet org).
I noticed that aria2's SecureTransport code disables insecure ciphers such
as NULL, anonymous, IDEA, and weak-key ciphers used by SSLv3 and later.
That's a good idea, and now we do the same thing in order to prevent curl
from accessing a "secure" site that only negotiates insecure ciphersuites.
Previously it only compared credentials if the requested needle
connection wasn't using a proxy. This caused NTLM authentication
failures when using proxies as the authentication code wasn't send on
the connection where the challenge arrived.
Added test 1215 to verify: NTLM server authentication through a proxy
(This is a modified copy of test 67)
Since qsort implementations vary with regards to handling the order
of similiar elements, this change makes the internal sort function
more deterministic by comparing path length first, then domain length
and finally the cookie name. Spotted with testcase 62 on Windows.
When doing PORT and upload (STOR), this function needs to extract the
file descriptor for both connections so that it will respond immediately
when the server eventually connects back.
This flaw caused active connections to become unnecessary slow but they
would still often work due to the normal polling on a timeout. The bug
also would not occur if the server connected back very fast, like when
testing on local networks.
Bug: http://curl.haxx.se/bug/view.cgi?id=1183
Reported by: Daniel Theron
I am using curl_easy_setopt(CURLOPT_INTERFACE, "if!something") to force
transfers to use a particular interface but the transfer fails with
CURLE_INTERFACE_FAILED, "Failed binding local connection end" if the
interface I specify has no IPv6 address. The cause is as follows:
The remote hostname resolves successfully and has an IPv6 address and an
IPv4 address.
cURL attempts to connect to the IPv6 address first.
bindlocal (in lib/connect.c) fails because Curl_if2ip cannot find an
IPv6 address on the interface.
This is a fatal error in singleipconnect()
This change will make cURL try the next IP address in the list.
Also included are two changes related to IPv6 address scope:
- Filter the choice of address in Curl_if2ip to only consider addresses
with the same scope ID as the connection address (mismatched scope for
local and remote address does not result in a working connection).
- bindlocal was ignoring the scope ID of addresses returned by
Curl_if2ip . Now it uses them.
Bug: http://curl.haxx.se/bug/view.cgi?id=1189
At some point recently we lost the default value for the easy handle's
connection cache, and this change puts it back to 5 - which is the
former default value and it is documented in the curl_easy_setopt.3 man
page.
The Microsoft knowledge-base article
http://support.microsoft.com/kb/823764 describes how to use SNDBUF to
overcome a performance shortcoming in winsock, but it doesn't apply to
Windows Vista and later versions. If the described SNDBUF magic is
applied when running on those more recent Windows versions, it seems to
instead have the reversed effect in many cases and thus make libcurl
perform less good on those systems.
This fix thus adds a run-time version-check that does the SNDBUF magic
conditionally depending if it is deemed necessary or not.
Bug: http://curl.haxx.se/bug/view.cgi?id=1188
Reported by: Andrew Kurushin
Tested by: Christian Hägele
The last remaining code piece that still used FTPSENDF now uses PPSENDF.
In the problematic case, a PREQUOTE series was done on a re-used
connection when Curl_pp_init() hadn't been called so it had messed up
pointers. The init call is done properly from Curl_pp_sendf() so this
change fixes this particular crash.
Bug: http://curl.haxx.se/mail/lib-2013-03/0319.html
Reported by: Sam Deane
As of 25-mar-2013 wcsdup() _wcsdup() and _tcsdup() are only used in
WIN32 specific code, so tracking of these has not been extended for
other build targets. Without this fix, memory tracking system on
WIN32 builds, when using these functions, would provide misleading
results.
In order to properly extend this support for all targets curl.h
would have to define curl_wcsdup_callback prototype and consequently
wchar_t should be visible before that in curl.h. IOW curl_wchar_t
defined in curlbuild.h and this pulling whatever system header is
required to get wchar_t definition.
Additionally a new curl_global_init_mem() function that also receives
user defined wcsdup() callback would be required.
Proxy servers tend to add their own headers at the beginning of
responses. The size of these headers was not taken into account by
CURLINFO_HEADER_SIZE before this change.
Bug: http://curl.haxx.se/bug/view.cgi?id=1204
After having done a POST over a CONNECT request, the 'rewindaftersend'
boolean could be holding the previous value which could lead to badness.
This should be tested for in a new test case!
Bug: https://groups.google.com/d/msg/msysgit/B31LNftR4BI/KhRTz0iuGmUJ
Fixed incorrect initial response generation for the NTLM and LOGIN SASL
authentication mechanisms when the SASL-IR was detected.
Introduced in commit: 6da7dc026c.
curl has been accepting URLs using slightly wrong syntax for a long
time, such as when completely missing as slash "http://example.org" or
missing a slash when a query part is given
"http://example.org?q=foobar".
curl would translate these into a legitimate HTTP request to servers,
although as was shown in bug #1206 it was not adjusted properly in the
cases where a HTTP proxy was used.
Test 1213 and 1214 were added to the test suite to verify this fix.
The test HTTP server was adjusted to allow us to specify test number in
the host name only without using any slashes in a given URL.
Bug: http://curl.haxx.se/bug/view.cgi?id=1206
Reported by: ScottJi
Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.
Two new options for limiting the number of connections:
CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.
CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.
Several new options for pipelining:
CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).
CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.
CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.
CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.
CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.
See the curl_multi_setopt() man page for details.
Following commit e450f66a02 and the changes in the multi interface
being used internally, from 7.29.0, the transfer cancellation in
pop3_dophase_done() is no longer required.
When Curl_do() returns failure, the connection pointer could be NULL so
the code path following needs to that that into account.
Bug: http://curl.haxx.se/mail/lib-2013-03/0062.html
Reported by: Eric Hu
Moved the blocking state machine to the disconnect functions so that the
logout / quit functions are only responsible for sending the actual
command needed to logout or quit.
Additionally removed the hard return on failure.
Added an exception, for the STORE command, to the untagged response
processor in imap_endofresp() as servers will back respones containing
the FETCH keyword instead.
The list of unsafe functions currently consists of sprintf, vsprintf,
strcat, strncat and gets.
Subsequently, some existing code needed updating to avoid warnings on
this.
As the UID has to be specified by the user for the FETCH command to work
correctly, added a check to imap_fetch(), although strictly speaking it
is protected by the call from imap_perform().
The option needs to be set on the SSL socket. Setting it on the model
takes no effect. Note that the non-blocking mode is still not enabled
for the handshake because the code is not yet ready for that.
Commit 26eaa83830 introduces the use of S_ISDIR() yet some compilers,
such as MSVC don't support it, so we must define a substitute using
file flags and mask.
Commit f4cc54cb47 (shipped as part of the 7.29.0 release) was a
bug fix that introduced a regression in that while trying to avoid
allowing directory names, it also forbade "special" files like character
devices and more. like "/dev/null" as was used by Oliver who reported
this regression.
Reported by: Oliver Gondža
Bug: http://curl.haxx.se/mail/archive-2013-02/0040.html
If the server hung up the connection without sending a closure alert,
then we'd keep probing the socket for data even though it's dead. Now
we're ready for this situation.
Bug: http://curl.haxx.se/mail/lib-2013-03/0014.html
Reported by: Aki Koskinen
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Added imap_custom(), which initiates the custom command processing,
and an associated response handler imap_state_custom_resp(), which
handles any responses by sending them to the client as body data.
All untagged responses with the same name as the first word of the
custom request string are accepted, with the exception of SELECT and
EXAMINE which have responses that cannot be easily identified. An
extra check has been provided for them so that any untagged responses
are accepted for them.
Added imap_parse_custom_request() for parsing the CURLOPT_CUSTOMREQUEST
parameter which URL decodes the value and separates the request from
any parameters - This makes it easier to filter untagged responses
by the request command.
For consistency changed the logic of the imap_state_append_resp()
function to test for an unsucessful continuation response rather than a
succesful one.
The APPEND operation needs to be performed in several steps:
1) We send "<tag> APPEND <mailbox> <flags> {<size>}\r\n"
2) Server responds with continuation respose "+ ...\r\n"
3) We start the transfer and send <size> bytes of data
4) Only now we end the request command line by sending "\r\n"
5) Server responds with "<tag> OK ...\r\n"
This commit performs steps 4 and 5, in the DONE phase, as more
processing is required after the transfer.
Some state changes would be performed after a failure test that
performed a hard return, whilst others would be performed within a test
for success. Updated the code, for consistency, so all instances are
performed within a success test.
Not processing the final FETCH responses was not optimal, not only
because the response code would be ignored but it would also leave data
unread on the socket which would prohibit connection reuse.
A typical FETCH response can be broken down into four parts:
1) "* <uid> FETCH (<what> {<size>}\r\n", using continuation syntax
2) <size> bytes of the actual message
3) ")\r\n", finishing the untagged response
4) "<tag> OK ...", finishing the command
Part 1 is read in imap_fetch_resp(), part 2 is consumed in the PERFORM
phase by the transfer subsystem, parts 3 and 4 are currently ignored.
Added a loop to imap_statemach_act() in which Curl_pp_readresp() is
called until the cache is drained. Without this multiple responses
received in a single packet could result in a hang or delay.
RFC 3501 states that "the client MUST be prepared to accept any response
at all times" yet we assume anything received with "* " at the beginning
is the untagged response we want.
Introduced a helper function that checks whether the input looks like a
response to specified command, so that we may filter the ones we are
interested in according to the current state.
Introduced similar handling to the FETCH responses, where even the
untagged data responses are handled by the response handler of the
individual state.
Removed this pointer to a downloaded bytes counter because it was set in
smtp_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
Removed this pointer to a downloaded bytes counter because it was set in
pop3_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
Removed this pointer to a downloaded bytes counter because it was set in
imap_init() to point to the same variable the transfer functions keep
the count in (k->bytecount), effectively making the code in transfer.c
"*k->bytecountp = k->bytecount" a no-op.
From a maintenance point of view the code reads better to view tagged
responses, then untagged followed by continuation responses.
Additionally, this matches the order of responses in POP3.
Updated the mailbox variable to correctly reflect it's purpose. The
name mailbox was a leftover from when IMAP and POP3 support was
initially added to curl.
Updated the FETCH command to send the UID and SECTION parsed from the
URL. By default the BODY specifier doesn't include a section, BODY[] is
now sent whereas BODY[TEXT] was previously sent. In my opinion
retrieving just the message text is rarely useful when dealing with
emails, as the headers are required for example, so that functionality
is not retained. In can however be simulated by adding SECTION=TEXT to
the URL.
Also updated test801 and test1321 due to the BODY change.
Removed user and passwd from the SMTP struct as these cannot be set on
a per-request basis and are leftover from legacy FTP code.
Changed some comments still using FTP terminology.
Removed user and passwd from the POP3 struct as these cannot be set on
a per-request basis and are leftover from legacy FTP code.
Changed some comments still using FTP terminology.
Moved the mailbox and custom request variables from the per-connection
struct pop3_conn to the new per-request struct and fixed references
accordingly.
Created a new IMAP structure and changed the type of the imap proto
variable in connectdata from FTP* to the new IMAP*.
Moved the mailbox variable from the per-connection struct imap_conn to
the new per-request struct and fixed references accordingly.
Moved the clean-up of the mailbox variable from imap_disconnect() to
imap_done() as this variable is allocated in the do phase, yet would
have only been freed only once if multiple selects where preformed
on a single connection.
Always interprets the pointer passed with the CURLOPT_WRITEDATA or
CURLOPT_READDATA options of curl_easy_setopt() as a void pointer in
order to avoid problems in environments where FILE and void pointers
have non-trivial conversion.
Use Curl_pp_moredata() in Curl_pp_multi_statemach() to check if there is
more data to be received, rather than the socket state, as a task could
hang waiting for more data from the socket itself.
A simple function to test whether the PP is not sending and there are
still more data in its receiver cache. This will be later utilized to:
1) Change Curl_pp_multi_statemach() and Curl_pp_easy_statemach() to
not test socket state and just call user's statemach_act() function
when there are more data to process, because otherwise the task would
just hang, waiting for more data from the socket.
2) Allow PP users to read multiple responses by looping as long as there
are more data available and current phase is not finished.
(Currently needed for correct processing of IMAP SELECT responses.)
The attempt to use gai_strerror() or alternative function didn't work as
the 'sock_error' field didn't contain the proper error code. But since
this hasn't been reported and thus isn't really a big deal I decided to
just scrap the whole attempt to output the detailed resolver error and
instead remain with just stating that the resolving of the name failed.
It seems older gcc installations (at least) will cause warnings if we
name a variable 'wait'. Now changed to 'block' instead.
Reported by: Jiří Hruška
Bug: http://curl.haxx.se/mail/lib-2013-02/0247.html
Apple made a number of changes to Xcode 4. The SDKs were moved, the entire
Developer folder was moved, and PowerPC support was removed. The script
will now adapt to those changes and should be future-proofed against
additional changes in case Apple moves the Developer folder ever again.
Also, the minimum OS X version compiler option was removed, so that the
framework can be built against the latest SDK but still run in older cats.
... since they're not used by the easy interface really, I wanted to
remove the association. Also, I unified the pingpong statemachine driver
into a single function with a 'wait' argument: Curl_pp_statemach.
A call to Curl_ssl_connect() was accidentally left in when the SSL/TLS
connection layer was reworked in 7.29. Not only would this cause the
connection to block but had the additional overhead of calling the
non-blocking connect a little bit later.
This function was only used twice, both in places where performance
isn't crucial (socks + if2ip). Removing the use of this function removes
the need to have our private version for systems without it == reduced
amount of code.
Also, in the SOCKS case it is clearly better to fail gracefully rather
than to truncate the results.
This work was triggered by a bug report on the strcal prototype in
strequal.h.
strlcat was added in commit db70cd28 in February 2001!
Bug: http://curl.haxx.se/bug/view.cgi?id=1192
Reported by: Jeremy Huddleston
Fixed a null pointer reference when an empty challenge is passed to the
Curl_sasl_create_digest_md5_message() function.
Bug: http://sourceforge.net/p/curl/bugs/1193/
Reported by: Saran Neti
An ambiguity in the SSLWrite() documentation lead to a bad inference in the
code where we assumed SSLWrite() returned the amount of bytes written to
the socket, when that is not actually true; it returns the amount of data
that is buffered for writing to the socket if it returns errSSLWouldBlock.
Now darwinssl_send() returns CURLE_AGAIN if data is buffered but not written.
Reference URL: http://curl.haxx.se/mail/lib-2013-02/0145.html
Reworked the pp->endofresp() function so that the conndata, line and
line length are passed down to it just as with Curl_client_write()
rather than each implementation of the function having to query
these values.
Additionally changed the int return type to bool as this is more
representative of the function's usage.
Corrected the order of the upgrade_tls() functions and moved the handler
upgrade and getsock() functions out from the middle of the state related
functions.
Corrected the order of the pop3_state_capa() / imap_state_capability()
and the pop3_state_capa_resp() / imap_state_capability_resp() functions
to match the execution order.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added honoring of the tls_supported flag when starting a TLS upgrade
rather than unconditionally attempting it. If the use_ssl flag is set
to CURLUSESSL_TRY and the server doesn't support TLS upgrades then the
connection will continue to authenticate. If this flag is set to
CURLUSESSL_ALL then the connection will complete with a failure as it
did previously.
Added sending of initial CAPABILITY command before STARTTLS is sent.
This allows for the detection of the capability before trying to
upgrade the connection.
Follow up fix to commit 62bd217464 to cater for servers that don't
respond with a 250 in their EHLO responses. Additionally updated the
SMTP tests to respond with a 250 response code as per RFC5321.
Fixed the SASL capability detection to include the space character
before the authentication mechanism list. Otherwise a capability such
as SASLSOMETHING would be interpreted as enabling SASL and potentially
trying to identify SOMETHING as a mechanism.
Previously if a problem was found with one of the server's certificates,
we'd log an OSStatus for the end user to look up. Now we explain what
was wrong with the site's certificate chain. Also un-did part of the
previous commit where the code wouldn't catch errSSLServerAuthCompleted
if built under Leopard.
Fixed a small issue where smtp_endofresp() would look for capabilities
in the description part of a failure response. In theory a server
shouldn't respond with SIZE or AUTH in an EHLO command's failure
response but if it did then capabilities would be unnecessarily set
before eventually failing.
Renamed the authstate1 and authstate2 variables in imap_authenticate()
as the old name was a left over from when there was only one state
variable which was named due to a clash with the state() function.
Additionally this provides consistency with the smtp module.
Running tests\libtest\libntlmconnect.exe reveals a 1 byte (!) leak in
./lib/curl_ntlm_msgs.c:
perl ..\memanalyze.pl c:memdebug.curl
Leak detected: memory still allocated: 1 bytes
At 9771e8, there's 1 bytes.
allocated by curl_ntlm_msgs.c:399
Snippet from curl_ntlm_msgs.c:
/* setup ntlm identity's domain and length */
dup_domain.tchar_ptr = malloc(sizeof(TCHAR) * (domlen + 1));
(my domlen == 0).
'dup_domain.tbyte_ptr' looks to be freed in Curl_ntlm_sspi_cleanup() via
'ntlm->identity.Domain'. But I see no freeing of 'dup_domain.tchar_ptr'.
This bug report properly identified that when doing SMTP and aborting
the transfer with a callback, it must be considered aborted prematurely
by the code to avoid QUIT etc to be attempted as that would cause a
hang.
The new test case 1507 verifies this behavior.
Reported by: Patricia Muscalu
Bug: http://curl.haxx.se/bug/view.cgi?id=1184
It turns out that Leopard (OS X 10.5) doesn't have constants for the ECDH
ciphers in its headers, so the cases for them have been taken out of the
build when building under Leopard. Also added a standard function for
getting a string description of a SecCertificateRef.
Changed the SMTP_AUTH_PASSWD state constant to SMTP_AUTH_LOGIN_PASSWD to
better describe the state as the second part of an AUTH LOGIN command,
as well as for consistency with the imap and pop3 modules.
Introduced detection of the SASL-IR capability, in order to add support
for sending the initial response with the AUTHENTICATE command, as per
RFC4959.
Updated the automatic response tag generation to follow the examples
given in RC3501, which list a 4 character string such as A001, A002,
etc.
As a unique identifier should be generated for each command the string
generation is based on the connection id and the incrementing command
id.
When negotiating SASL DIGEST-MD5 authentication, the function
Curl_sasl_create_digest_md5_message() uses the data provided from the
server without doing the proper length checks and that data is then
appended to a local fixed-size buffer on the stack.
This vulnerability can be exploited by someone who is in control of a
server that a libcurl based program is accessing with POP3, SMTP or
IMAP. For applications that accept user provided URLs, it is also
thinkable that a malicious user would feed an application with a URL to
a server hosting code targetting this flaw.
Bug: http://curl.haxx.se/docs/adv_20130206.html
config_h.com is a new file that generates a config.h file based on the
curl_config.h.in file and a quick scan of the configure script. This is
actually a generic procedure that is shared with other VMS packages.
The existing pre-built config-vms.h had over 100 entries that were not
correct and in some cases conflicted with the build options available in
the build_vms.com.
generate_config_vms_h_curl.com is a helper procedure to the
config_h.com. It covers the cases that the generic config_h.com is not
able to figure out, and accepts input from the build_vms.com procedure.
build_curlbuild_h.com is a new file to generate the curlbuild.h file
that Curl is now using when it is using a curl_config.h file.
post-config-vms.h is a new file that is needed to provide VMS specific
definitions, and most of them need to be set before the system header
files are included.
The VMS build procedure is fixed:
1. Fixed to link in the correct HP ssl library.
2. Fixed to detect if HP Kerberos is installed.
3. Fixed to detect if HP LDAP is installed.
4. Fixed to detect if gnv$libzshr is installed.
5. Simplified the input parameter parsing to not use a loop.
6. Warn that 64 bit pointer option support is not complete
in comments.
7. Default to IEEE floating if platform supports it so
resulting libcurl will be compatible with other
open source projects on VMS.
8. Default to LARGEFILE if platform supports it.
9. Default to enable SSL, LDAP, Kerberos, libz
if the libraries are present.
10. Build with exact case global symbols for libcurl.
11. Generate linker option file needed.
12. Compiler list option only commonly needed items.
13. fulllist option for those who really want it.
14. Create debug symbol file on Alpha, IA64.
By doing this unconditionally, we infer a simpler and more defined
behavior. This also has the upside that test 1021 no longer fails for me
even if I run with valgrind.
Also fixed some wrong comments.
According KB975858 this flag may cause problems on Windows 7 and
Windows Server 2008 R2 systems. Extended error information is not
currently used by libcurl and therefore not a requirement.
The flag may improve the SSL-connection shutdown in case of an
error. This means it might be a good improvement in the future.
Fixes bug/issue #1187 - thanks for the report
When a connection is no longer used, it is kept in the cache. If the
cache is full, the oldest idle connection is closed. If no connection is
idle, the current one is closed instead.
Tidied up code from commit 6b6bdc83bdUpdated where a few instances of
the pop3c struct variable used the longer conndata struct rather than
matching what other code in pop3_authenticate() used.
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
imap: Fixed no known authentication mechanism when fallback is required
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
Related bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to APOP or Clear Text
authentication.
Bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Remove timeout argument that's never used.
Make the actual connection get detected on a single spot to reduce code
duplication.
Store the IPv6 state already when the connection is attempted.
There was a bug where, if SSLWrite() returned errSSLWouldBlock but did
succeed in transmitting at least something, then we'd incorrectly
resend the packet. Now we never take errSSLWouldBlock as a sign that
nothing was transferred to/from the server.
Bug: http://curl.haxx.se/mail/lib-2013-01/0295.html
Reported by: Bruno de Carvalho
Minor code tidy up to add comments similar to those used in the pop3
and imap end of resp functions, in order to assist anyone reading the
code and highlight the similarities between each of these protocols.
smtp_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to smtps and start the EHLO command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
pop3_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to pop3s and start the CAPA command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
imap_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to imaps and start the CAPABILITY command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
- document the double-quote and backslash need be escaped if quoting.
- libcurl formdata escape double-quote in filename by backslash.
- curl formparse can parse filename both contains '"' and ',' or ';'.
- curl now can uploading file with ',' or ';' in filename.
Bug: http://curl.haxx.se/bug/view.cgi?id=1171
Fixed an issue where Curl_ssl_connect_nonblocking() wouldn't complete
correctly and the ssldone flag wouldn't be set to true for pop3s based
connections.
Bug introduced in commit: 4ffb8a6398.
Remove internal separated behavior of the easy vs multi intercace.
curl_easy_perform() is now using the multi interface itself.
Several minor multi interface quirks and bugs have been fixed in the
process.
Much help with debugging this has been provided by: Yang Tse
Fixes initial proxy response being processed by the tunneled protocol
handler instead of the HTTP wrapper handler. This issue would trigger
upon delayed CONNECT response from the proxy.
Additionally fixes a multi interface code-path in which connections
would not time out properly.
This does not fix known bug #39.
URL: http://curl.haxx.se/mail/lib-2013-01/0191.html
This commit fixes a regression introduced in 052a08ff.
NSS caches certs/keys returned by the SSL_GetClientAuthDataHook callback
and if we connect second time to the same server, the cached cert/key
pair is used. If we use multiple client certificates for different
paths on the same server, we need to clear the session cache to force
NSS to call the hook again. The commit 052a08ff prevented the session
cache from being cleared if a client certificate from file was used.
The condition is now fixed to cover both cases: consssl->client_nickname
is not NULL if a client certificate from the NSS database is used and
connssl->obj_clicert is not NULL if a client certificate from file is
used.
Review by: Kai Engert
This commit renames lib/setup.h to lib/curl_setup.h and
renames lib/setup_once.h to lib/curl_setup_once.h.
Removes the need and usage of a header inclusion guard foreign
to libcurl. [1]
Removes the need and presence of an alarming notice we carried
in old setup_once.h [2]
----------------------------------------
1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard
up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H,
this single inclusion guard is enough to ensure that inclusion of
lib/setup_once.h done from lib/setup.h is only done once.
Additionally lib/setup.h has always used __SETUP_ONCE_H macro to
protect inclusion of setup_once.h even after commit ec691ca3, this
was to avoid a circular header inclusion triggered when building a
c-ares enabled version with c-ares sources available which also has
a setup_once.h header. Commit ec691ca3 exposes the real nature of
__SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard
foreign to libcurl belonging to c-ares's setup_once.h
The renaming this commit does, fixes the circular header inclusion,
and as such removes the need and usage of a header inclusion guard
foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl.
2 - Due to the circular interdependency of old lib/setup_once.h and the
c-ares setup_once.h header, old file lib/setup_once.h has carried
back from 2006 up to now days an alarming and prominent notice about
the need of keeping libcurl's and c-ares's setup_once.h in sync.
Given that this commit fixes the circular interdependency, the need
and presence of mentioned notice is removed.
All mentioned interdependencies come back from now old days when
the c-ares project lived inside a curl subdirectory. This commit
removes last traces of such fact.
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:
f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files
This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:
c087374... curl_setup.h: remove redundant include guard
This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:
13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1
Start of related discussion thread:
http://curl.haxx.se/mail/lib-2013-01/0012.html
Asking for confirmation on pushing this revertion commit:
http://curl.haxx.se/mail/lib-2013-01/0048.html
Confirmation summary:
http://curl.haxx.se/mail/lib-2013-01/0079.html
NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.
lib/curl_imap.h
lib/curl_smtp.h
1. When the downloaded data file from Mozilla is current, but the output
bundle does not exist: continue processing to create the bundle. The
goal is to have the output file - not just download the latest input.
2. added -f option to force re-processing the file. Useful for
debugging/testing the process.
3. added support for output to '-' (stdout), allowing the output to be
piped.
4. All progress and error messages go to STDERR rather than STDOUT (3)
5. The script opened and closed the output file many times
unnecessarily. It now opens it once, does the output and closes it.
6. Backup of the input files happens after successful processing, not
before.
7. The output is written to a temporary file, and renamed to the
requested name after backup - this greatly reduces the window where the
file can be seen partially written.
8. all die calls have a \n at the end to suppress perl's traceback - the
traceback isn't useful to end users.
Patch: http://curl.haxx.se/mail/lib-2013-01/0045.html
lib/objnames.inc provides definition of curl_10char_object_name() shell
function. The intended purpose of this function is to transliterate a
(*.c) source file name that may be longer than 10 characters, or not,
into a string with at most 10 characters which may be used as an OS/400
object name.
Test case 1221 does unit testng of this function and also verifies
that it is possible to generate distinct short object names for all
curl and libcurl *.c source file names.
lib/objnames-test.sh is the shell script used for test case 1221.
tests/runtests.pl modified to accept shell script test cases.
More details inside lib/objnames.inc and lib/objnames-test.sh
* Changing the order of the state machine to represent the order in
which commands are sent to the server.
* Reworking the imap_endofresp() function as the FETCH response doesn't
include the command id and shouldn't be part of the length comparison
that takes into account the id string.
Fixed a problem with the state machine when attempting to log in with
invalid credentials. The server would report login failure but libcurl
would not read the response due to inappropriate IMAP_STOP states being
set after the login was sent.
Applied some of the comment and layout changes that had already been
applied to the pop3 and smtp code over the last 6 to 9 months.
This is in preparation of adding SASL based authentication.
... on Snow Leopard and Lion
Snow Leopard introduced the SSLSetSessionOption() function, but it
doesn't disable peer verification as expected on Snow Leopard or
Lion (it works as expected in Mountain Lion). So we now use sysctl()
to detect whether or not the user is using Snow Leopard or Lion,
and if that's the case, then we now use the deprecated
SSLSetEnableCertVerify() function instead to disable peer verification.
... it also clobbered the 'result' return value so that it wouldn't
return the error back to the parent function properly, which broke test
809 when run with 'multi-always'.
When prefixing a path with /~/ it is supposed to be used relative to the
user's home directory but it didn't work. Now we cut off the entire
three byte sequenct "/~/" which seems to be how OpenSSH does it.
Bug: http://curl.haxx.se/bug/view.cgi?id=1173
Reported by: Balaji Parasuram
Issue: When building a 32bit target with large file support HP-UX
<sys/socket.h> header file may simultaneously provide two different
sets of declarations for sendfile and sendpath functions, one with
static and another with external linkage. Given that we do not use
mentioned functions we really don't care which linkage is the
appropriate one, but on the other hand, the double declaration emmits
warnings when using the HP-UX compiler and errors when using modern
gcc versions resulting in fatal compilation errors.
Mentioned issue is now fixed as long as we don't use sendfile nor
sendpath functions.
A bundle is a list of all persistent connections to the same host.
The connection cache consists of a hash of bundles, with the
hostname as the key.
The benefits may not be obvious, but they are two:
1) Faster search for connections to reuse, since the hash
lookup only finds connections to the host in question.
2) It lays out the groundworks for an upcoming patch,
which will introduce multiple HTTP pipelines.
This patch also removes the awkward list of "closure handles",
which were needed to send QUIT commands to the FTP server
when closing a connection.
Now we allocate a separate closure handle and use that
one to close all connections.
This has been tested in a live system for a few weeks, and of
course passes the test suite.
BLANK_AT_MAKETIME may be used in our Makefile.am files to blank
LIBS variable used in generated makefile at makefile processing
time. Doing this functionally prevents LIBS from being used for
all link targets in given makefile.
This handling already works with the easy-interface code. When a request
is sent on a re-used connection that gets closed by the server at the
same time as the request is sent, the situation may occur so that we can
send the request and we discover the broken connection as a RECV_ERROR
in the PERFORM state and then the request needs to be retried on a fresh
connection. Test 64 broke with 'multi-always-internally'.
Although it is not explicitly stated in the documentation, NSS uses
*pRetCert and *pRetKey even if the client authentication hook returns
a failure. Namely, if we destroy *pRetCert without clearing *pRetCert
afterwards, NSS destroys the certificate once again, which causes a
double free.
Reported by: Bob Relyea
.. that are sent when auth-negotiating before a chunked
upload or when setting the 'Transfer-Encoding: chunked'
header and intentionally sending no content.
Adjust test565 and test1333 accordingly.
DNS cache entries populated with CURLOPT_RESOLVE were not properly freed
again when done using the multi interface.
Test case 1502 added to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=3575448
Reported by: Alex Gruz
If we use memory functions (malloc, free, strdup etc) in C sources in
libcurl and we fail to include curl_memory.h or memdebug.h we either
fail to properly support user-provided memory callbacks or the memory
leak system of the test suite fails.
After Ajit's report of a failure in the first category in http_proxy.c,
I spotted a few in the second category as well. These problems are now
tested for by test 1132 which runs a perl program that scans for and
attempts to check that we use the correct include files if a memory
related function is used in the source code.
Reported by: Ajit Dhumale
Bug: http://curl.haxx.se/mail/lib-2012-11/0125.html
When using only 1 second precision, curl doesn't create new cnonce
values quickly enough for all uses.
For example, issuing the following command multiple times to a recent
Tomcat causes authentication failures:
curl --digest -utest:test http://tomcat.test.com:8080/manager/list
This is because curl uses the same cnonce for several seconds, but
doesn't increment the nonce counter. Tomcat correctly interprets
this as a replay attack and rejects the request.
When microsecond-precision is available, this commit causes curl to
change cnonce values much more frequently.
With microsecond resolution, increasing the nounce length used in the
headers to 32 was made to further reduce the risk of duplication.
axTLS:
This will make the axTLS backend perform the RFC2818 checks, honoring
the VERIFYHOST setting similar to the OpenSSL backend.
Generic for OpenSSL and axTLS:
Move the hostcheck and cert_hostcheck functions from the lib/ssluse.c
files to make them genericly available for both the OpenSSL, axTLS and
other SSL backends. They are now in the new lib/hostcheck.c file.
CyaSSL:
CyaSSL now also has the RFC2818 checks enabled by default. There is a
limitation that the verifyhost can not be enabled exclusively on the
Subject CN field comparison. This SSL backend will thus behave like the
NSS and the GnuTLS (meaning: RFC2818 ok, or bust). In other words:
setting verifyhost to 0 or 1 will disable the Subject Alt Names checks
too.
Schannel:
Updated the schannel information messages: Split the IP address usage
message from the verifyhost setting and changed the message about
disabling SNI (Server Name Indication, used in HTTP virtual hosting)
into a message stating that the Subject Alternative Names checks are
being disabled when verifyhost is set to 0 or 1. As a side effect of
switching off the RFC2818 related servername checks with
SCH_CRED_NO_SERVERNAME_CHECK
(http://msdn.microsoft.com/en-us/library/aa923430.aspx) the SNI feature
is being disabled. This effect is not documented in MSDN, but Wireshark
output clearly shows the effect (details on the libcurl maillist).
PolarSSL:
Fix the prototype change in PolarSSL of ssl_set_session() and the move
of the peer_cert from the ssl_context to the ssl_session. Found this
change in the PolarSSL SVN between r1316 and r1317 where the
POLARSSL_VERSION_NUMBER was at 0x01010100. But to accommodate the Ubuntu
PolarSSL version 1.1.4 the check is to discriminate between lower then
PolarSSL version 1.2.0 and 1.2.0 and higher. Note: The PolarSSL SVN
trunk jumped from version 1.1.1 to 1.2.0.
Generic:
All the SSL backends are fixed and checked to work with the
ssl.verifyhost as a boolean, which is an internal API change.
The text "additional stuff not fine" text was added for debug purposes a
while ago, but it isn't really helping anyone and for some reason some
Linux distributions provide their libcurls built with debug info still
present and thus (far too many) users get to read this info.
The logic previously checked for a started NTLM negotiation only for
host and not also with proxy, leading to problems doing POSTs over a
proxy NTLM that are larger than 2000 bytes. Now it includes proxy in the
check.
Bug: http://curl.haxx.se/bug/view.cgi?id=3582321
Reported by: John Suprock
The existing logic only cut off the fragment from the separate 'path'
buffer which is used when sending HTTP to hosts. The buffer that held
the full URL used for proxies were not dealt with. It is now.
Test case 5 was updated to use a fragment on a URL over a proxy.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579813
As a handle can be re-used after having done HTTP auth in a previous
request, it must make sure to clear out the HTTP types that aren't
wanted in this new request.
This reverts commit ce8311c7e4.
The commit made test 2024 work but caused a regression with repeated
Digest authentication. We need to fix this differently.
After a research team wrote a document[1] that found several live source
codes out there in the wild that misused the CURLOPT_SSL_VERIFYHOST
option thinking it was a boolean, this change now bans 1 as a value and
will make libcurl return error for it.
1 was never a sensible value to use in production but was introduced
back in the days to help debugging. It was always documented clearly
this way.
1 was never supported by all SSL backends in libcurl, so this cleanup
makes the treatment of it unified.
The report's list of mistakes for this option were all PHP code and
while there's a binding layer between libcurl and PHP, the PHP team has
decided that they have an as thin layer as possible on top of libcurl so
they will not alter or specifically filter a 'TRUE' value for this
particular option. I sympathize with that position.
[1] = http://daniel.haxx.se/blog/2012/10/25/libcurl-claimed-to-be-dangerous/
Since automake 1.12.4, the warnings are issued on running automake:
warning: 'INCLUDES' is the old name for 'AM_CPPFLAGS' (or '*_CPPFLAGS')
Avoid INCLUDES and roll these flags into AM_CPPFLAGS.
Compile tested on:
Ubuntu 10.04 (automake 1:1.11.1-1)
Ubuntu 12.04 (automake 1:1.11.3-1ubuntu2)
Arch Linux (automake 1.12.4)
As pointed out in Bug report #3579064, curl_multi_perform() would
wrongly use a blocking mechanism internally for some commands which
could lead to for example a very long block if the LIST response never
showed.
The solution was to make sure to properly continue to use the multi
interface non-blocking state machine.
The new test 1501 verifies the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579064
Reported by: Guido Berhoerster
When given a string as 'srp' it didn't work, but required 'SRP'.
Starting now, the check disregards casing.
Bug: http://curl.haxx.se/bug/view.cgi?id=3578418
Reported by: Jeff Connelly
Previously the Metalink code used Apple's CommonCrypto library only if
curl was built using the --with-darwinssl option. Now we use CommonCrypto
on all Apple operating systems including Tiger or later, or iOS 5 or
later, so you don't need to build --with-darwinssl anymore. Also rolled
out this change to libcurl's md5 code.
The iOS build was broken by a reference to a function that only existed
under OS X; fixed. Also fixed a hard-to-reproduce problem where, if the
server disconnected before libcurl got the chance to hang up first and
SecureTransport was in use, then we'd raise an error instead of failing
gracefully.
This is a minor change in behavior after having been pointed out by Mark
Tully and discussed on the list. Initially this case would internally
call poll() with no sockets and a timeout which would equal a sleep for
that specified time.
Bug: http://curl.haxx.se/mail/lib-2012-10/0076.html
Reported by: Mark Tully
Since there are servers that seem to return very big encrypted
data packages, we need to be able to handle those without having
an internal size limit. To avoid the buffer growing to fast to
early the initial size was decreased and the minimum free space
in the buffer was decreased as well.
During the periods of rate limitation, the speedcheck function wasn't
called and thus the values weren't updated accordingly and it would then
easily trigger wrongly once data got transferred again.
Also, the progress callback's return code was not acknowledged in this
state so it could make an "abort" return code to get ignored and not
have the documented effect of aborting an ongoing transfer.
Bug: http://curl.haxx.se/mail/lib-2012-09/0081.html
Reported by: Jie He
The Curl_reconnect_request() function could end up returning a pointer
to a free()d struct when Curl_done() failed inside. Clearing the pointer
unconditionally after Curl_done() avoids this risk.
Reported by: Ho-chi Chen
Bug: http://curl.haxx.se/mail/lib-2012-09/0188.html
Selected socks proxy in Google's Chrome browser. Resulting in the
following environment variables:
NO_PROXY=localhost,127.0.0.0/8
ALL_PROXY=socks://localhost:1080/
all_proxy=socks://localhost:1080/
no_proxy=localhost,127.0.0.0/8
... and libcurl didn't treat 'socks://' as socks but instead picked HTTP
proxy.
Reported by: Scott Bailey
Bug: http://curl.haxx.se/bug/view.cgi?id=3566860
Each certificate section of the input certdata.txt file has a trust
section following it with details.
This script failed to detect the start of the trust for at least one
cert[*], which made the script continue pass that section into the next
one where it found an 'untrusted' marker and as a result that certficate
was not included in the output.
[*] = "Hellenic Academic and Research Institutions RootCA 2011"
Bug: http://curl.haxx.se/mail/lib-2012-09/0019.html
SMTP client will send SIZE parameter in MAIL FROM command only if server
supports it. Without this patch server might say "504 Command parameter
not implemented" and reject the message.
Bug: http://curl.haxx.se/bug/view.cgi?id=3564114
/*
* Name: curl_multi_wait()
*
* Desc: Poll on all fds within a CURLM set as well as any
* additional fds passed to the function.
*
* Returns: CURLMcode type, general multi error code.
*/
CURL_EXTERN CURLMcode curl_multi_wait(CURLM *multi_handle,
struct curl_waitfd extra_fds[],
unsigned int extra_nfds,
int timeout_ms);
In Mountain Lion, Apple added TLS 1.1 and 1.2, and deprecated a number
of SecureTransport functions, some of which we were using. We now check
to see if the replacement functions are present, and if so, we use them
instead. The old functions are still present for users of older
cats. Also fixed a build warning that started to appear under Mountain
Lion
Commit b91d29a28e170c16d65d956db79f2cd3a82372d2 introduces a bug and breaks Curl_closesocket function. sock_accepted flag for the second socket should be tagged as TRUE before the sockopt callback is called because in case the callback returns an error, Curl_closesocket function is going to call the - fclosesocket - callback for the accept()ed socket
For active FTP connections, applications may need setting the sockopt after accept() call returns successful. This fix gives a call to the callback registered with CURL_SOCKOPTFUNCTION option. Also a new sock type - CURLSOCKTYPE_ACCEPT - is added. This type is to be passed to application callbacks with - purpose - parameter. Applications may use this parameter to distinguish between socket types.
Commit e351972bc8 brought in the ssh agent support but some uses of
the libssh2 agent API was done unconditionally which wasn't good enough
since that API hasn't always been present.
By reading the ->head pointer and using that instead of the ->size
number to figure out if there's a list remaining we avoid the (false
positive) clang-analyzer warning that we might dereference of a null
pointer.
I suspect this is a regression introduced in commit 207cf150, included
since 7.24.0.
Avoid showing '(nil)' as hostname in verbose output by making sure the
hostname fixup function is called early enough to set the pointers that
are used for this. The name data is set again for each request even for
re-used connections to handle multiple hostnames over the same
connection (like with proxy) or that the casing etc of the host name is
changed between requests (which has proven to be important at least once
in the past).
Test1011 was modified to use a redirect with a re-used a connection
since it then showed the bug and now lo longer does. There's currently
no easy way to have the test suite detect 'nil' texts in verbose ouputs
so no tests will detect if this problem gets reintroduced.
Bug: http://curl.haxx.se/mail/lib-2012-07/0111.html
Reported by: Gisle Vanem
We found a problem with ftp transfer using libcurl (7.23 and 7.25)
inside an application which is receiving unix signals (SIGUSR1,
SIGUSR2...) almost continuously. (Linux 2.4, PowerPC, HAVE_POLL_FINE
defined).
Curl_socket_check() uses poll() to wait for the socket, and retries it
when a signal is received (EINTR). However, if a signal is received and
it also happens that the timeout has been reached, Curl_socket_check()
returns -1 instead of 0 (indicating an error instead of a timeout).
In our case, the result is an aborted connection even before the ftp
banner is received from the server, and a return value of
CURLE_OUT_OF_MEMORY from curl_easy_perform() (Curl_pp_multi_statemach(),
in pingpong.c, actually returns OOM if Curl_socket_check() fails :-)
Funny to debug on a system on which OOM is a possible cause).
Bug: http://curl.haxx.se/mail/lib-2012-07/0122.html
Due to WSAPoll bugs, libcurl does not work as intended. When the cURL
library is used to setup a connection to an incorrect port, normally the
result is CURLE_COULDNT_CONNECT, /* 7 */, but due to the bug in WSAPoll,
the result now is CURLE_OPERATION_TIMEDOUT, /* 28 - the timeout time was
reached */.
On August 1, Jan Koen Annot opened a case for this to Microsoft Premier
Online (https://premier.microsoft.com/). The support engineer handling
the case wrote that the case description is quite clear. He will try to
reproduce the issue and then proceed with troubleshooting it.
Reported by: Jan Koen Annot
Bug: http://curl.haxx.se/mail/lib-2012-07/0310.html
When figuring out if the data stream needs to be rewound when the
request is to be resent, we must not access the HTTP struct unless the
protocol used is indeed HTTP...
Bug: http://curl.haxx.se/bug/view.cgi?id=3544688
Previously the curl_multi interface would freeze if darwinssl was
enabled and at least one of the handles tried to connect to a Web site
using HTTPS. Removed the "wouldblock" state darwinssl was using because
I figured out a solution for our "would block but in which direction?"
dilemma.
In many states the easy_conn pointer is referenced and just assumed to
be working. This is an added extra check since analyzing indicates
there's a risk we can end up in these states with a NULL pointer there.
A HEAD response has no body length and gets the headers like the
corresponding GET would so it should not get closed after the response
based on the same rules. This mistake caused connections that did HEAD
to get closed too often without a valid reason.
Bug: http://curl.haxx.se/bug/view.cgi?id=3542731
Reported by: Eelco Dolstra
The function https_getsock was only implemented properly when USE_SSLEAY
or USE_GNUTLS is defined, but it is also necessary for USE_SCHANNEL.
The problem occurs when Curl_read_plain or Curl_write_plain returns
CURLE_AGAIN. In that case CURL_OK is returned to the multi-interface an
the used socket is set to state CURL_POLL_REMOVE and the easy-state is
set to CURLM_STATE_PROTOCONNECT. This is fine, because later the socket
should be set to CURL_POLL_IN or CURL_POLL_OUT via multi_getsock. That's
where https_getsock is called and doesn't return any sockets.
The code was printing a warning when SNI was set up successfully. Oops.
Printing the cipher number in verbose mode was something only TLS/SSL
programmers might understand, so I had it print the name of the cipher,
just like in the OpenSSL code. That'll be at least a little bit easier
to understand. The SecureTransport API doesn't have a method of getting
a string from a cipher like OpenSSL does, so I had to generate the
strings manually.
When doing CONNECT requests, libcurl must make sure the connection is
alive as much as possible. NTLM requires it and it is generally good for
other cases as well.
NTLM over CONNECT requests has been broken since this regression I
introduced in my CONNECT cleanup commits that started with 41b0237834,
included since 7.25.0.
Bug: http://curl.haxx.se/bug/view.cgi?id=3538625
Reported by: Marcel Raad
Allow NTLM authentication when building using SecureTransport (Darwin) for SSL.
This uses CommonCrypto, a cryptography library that ships with all versions of
iOS and Mac OS X. It's like OpenSSL's libcrypto, except that it's missing a few
less-common cyphers and doesn't have a big number data structure.
Before commit 2dded8fedb (dec 2010) there was logic that used
RAND_screen() at times and now I remove the leftover #ifdef check for
it.
The seeding code that uses Curl_FormBoundary() in ossl_seed() is dubious
to keep since it hardly increases randomness but I fear I'll break
something if I remove it now...
- Renamed st_ function prefix to darwinssl_
- Renamed Curl_st_ function prefix to Curl_darwinssl_
- Moved the duplicated ssl_connect_done out of the #ifdef in lib/urldata.h
- Fixed a teensy little bug that made non-blocking connection attempts block
- Made it so that it builds cleanly against the iOS 5.1 SDK
Removed two, not intended to exist, RESOURCE declarations.
Bug: http://curl.haxx.se/bug/view.cgi?id=3535977
And sorted configuration hunks to reflect same internal order
as the one shown in the usage message.
Increase decrypted and encrypted cache buffers using limitted
doubling strategy. More information on the mailinglist:
http://curl.haxx.se/mail/lib-2012-06/0255.html
It updates the two remaining reallocations that have already been there
and fixes the other one to use the same "do we need to increase the
buffer"-condition as the other two. CURL_SCHANNEL_BUFFER_STEP_SIZE was
renamed to CURL_SCHANNEL_BUFFER_FREE_SIZE since that is actually what it
is now. Since we don't know how much more data we are going to read
during the handshake, CURL_SCHANNEL_BUFFER_FREE_SIZE is used as the
minimum free space required in the buffer for the next operation.
CURL_SCHANNEL_BUFFER_STEP_SIZE was used for that before, too, but since
we don't have a step size now, the define was renamed.
Process extra data buffer before returning from schannel_connect_step2.
Without this change I've seen WinCE hang when schannel_connect_step2
returns and calls Curl_socket_ready.
If the encrypted handshake does not fit in the intial buffer (seen with
large certificate chain), increasing the encrypted data buffer is necessary.
Fixed warning in curl_schannel.c line 1215.
Implemented timeout loop in schannel_send while sending data. This
is as close as I think we can get to write buffering; I put a big
comment in to explain my thinking.
With some committer adjustments
Make the Schannel implementation use libcurl's default buffer size
for the initial received encrypted and decrypted data cache buffers.
The implementation still needs to handle more data since more data
might have already been received or decrypted during the handshake
or a read operation which needs to be cached for the next read.
curl_schannel.c - implemented graceful SSL shutdown. If we fail to
shutdown the connection gracefully, I've seen schannel try to use a
session ID for future connects and the server aborts the connection
during the handshake.
curl_schannel.c - auto certificate validation doesn't seem to work
right on CE. I added a method to perform the certificate validation
which uses CertGetCertificateChain and manually handles the result.
Coverity actually pointed out flawed logic in the previous call to
Curl_strntoupper() where the code used sizeof() of a pointer to pass in
a size argument. That code still worked since it only needed to
uppercase 4 letters. Still, the entire malloc/uppercase/free sequence
was pointless since the code has already matched the string once in the
condition that starts the block of code.
As spotted by Coverity, va_end() was not used previously. To make it
used I took away a bunch of return statements and made them into
assignments instead.
SSPI related code now compiles with ANSI and WCHAR versions of security
methods (WinCE requires WCHAR versions of methods).
Pulled UTF8 to WCHAR conversion methods out of idn_win32.c into their own file.
curl_sasl.c - include curl_memory.h to use correct memory functions.
getenv.c and telnet.c - WinCE compatibility fix
With some committer adjustments
Building with CyaSSL failed compilation. Reason being that OCSP_REQUEST and
OCSP_RESPONSE are enum values in CyaSSL and defines in <wincrypt.h> included
via <winldap.h> in ldap.c.
http://curl.haxx.se/mail/lib-2012-06/0196.html
Version number is removed in order to make this info consistent with
how we do it with other MS and Linux system libraries for which we don't
provide this info.
Identifier changed from 'WinSSPI' to 'schannel' given that this is the
actual provider of the SSL/TLS support. libcurl can still be built with
SSPI and without SCHANNEL support.
Removed obsolete minor status variable and parameter of status function
which was never used or set at all. Also Curl_sspi_strerror does support
only one status and there is no need for a second sub status.
Added Windows SSPI version information to the curl version string when
SCHANNEL SSL is not enabled, as the version of the library should also
be included when SSPI is used to generate security contexts.
Removed SSPI from the feature list as the features are GSS-Negotiate,
NTLM and SSL depending on the usage of the SSPI library.
Removed duplicate blank lines.
Removed spaces between the not and test in various if statements.
Removed explicit test of NULL in an if statement.
Placed function returns on same line as function declarations.
Replaced the use of curl_maprintf() with aprintf() as it is the
preprocessor job to do this substitution if ENABLE_CURLX_PRINTF
is set.
curl_sspi.c: Fixed mingw32-gcc compiler warnings
curl_sspi.c: Fixed length of error code hex output
The hex value was printed as signed 64-bit value on 64-bit systems:
SEC_E_WRONG_PRINCIPAL (0xFFFFFFFF80090322)
It is now correctly printed as the following:
SEC_E_WRONG_PRINCIPAL (0x80090322)
curl_sspi.c: Fallback to security function table version number
Instead of reporting an unknown version, the interface version is used.
curl_sspi.c: Removed SSPI/ version prefix from Curl_sspi_version
curl_schannel: Replaced static buffer sizes with defined names
curl_schannel.c: First brace when declaring functions on column 0
curl_schannel.c: Put the pointer sign directly at variable name
curl_schannel.c: Use structs directly instead of typedef'ed structs
curl_schannel.c: Removed space before opening brace
curl_schannel.c: Fixed lines being longer than 80 chars
Moved the error constant switch to curl_sspi.c and added two new helper
functions to curl_sspi.[ch] which either return the constant or a fully
translated message representing the SSPI security status.
Updated socks_sspi.c and curl_schannel.c to use the new functions.
Windows 2000 Professional: Schannel returns SEC_E_OK instead
of SEC_I_CONTEXT_EXPIRED. If the length of the output buffer
is zero and the first byte of the encrypted packet is 0x15,
the application can safely assume that the message was a
close_notify message and change the return value to
SEC_I_CONTEXT_EXPIRED.
Connection shutdown does not mean that there is no data to read
Correctly handle incomplete message and ask curl to re-read
Fixed buffer for decrypted being to small
Re-structured read condition to be more effective
Removed obsolete verbose messages
Changed memory reduction method to keep a minimum buffer of size 4096
Fixed warning: dereferencing pointer does break strict-aliasing rules
by using a union instead of separate pointer variables.
Internal union sockaddr_u could probably be moved to generic header.
Thanks to Paul Howarth for the hint about using unions for this.
Important for winbuild: Separate declaration of sockaddr_u pointer.
The pointer variable *sock cannot be declared and initialized right
after the union declaration. Therefore it has to be a separate statement.
Since Curl_pgrsDone() itself calls Curl_pgrsUpdate() which may return an
abort instruction or similar we need to return that info back and
subsequently properly handle return codes from Curl_pgrsDone() where
used.
(Spotted by a Coverity scan)
Previously it would use a 256 byte buffer and thus cut off very long
subject names. The limit is now upped to the receive buffer size, 16K.
Bug: http://curl.haxx.se/bug/view.cgi?id=3533045
Reported by: Anthony G. Basile
Re-factored the smtp_state_*_resp() functions to 1) Match the constants
that were refactored in commit 00fddba672, 2) To be more readable and
3) To match their counterparties in pop3.c.
Corrected lines longer than 78 characters.
Removed unnecessary braces in smtp_state_helo_resp().
Introduced some comments in data sending functions.
Tidied up comments to match changes made in pop3.c.
Corrected lines longer than 78 characters.
Changed POP3_AUTH_FINAL to POP3_AUTH to match SMTP code now that the
AUTH command is no longer sent on its own.
Introduced some comments in data sending functions.
Another attempt at trying to rational code and comment style.
Added a service type parameter to Curl_sasl_create_digest_md5_message()
to allow the function to be used by different services rather than being
hard coded to "smtp".
Not all SASL enabled POP3 servers support the AUTH command on its own
when trying to detect the supported mechanisms. As such changed the
mechanism detection to use the CAPA command instead.
Because pop3_endofresp() is called for each line of data yet is not
passed the line and line length, so we have to use the data pointed to
by pp->linestart_resp which contains the whole packet, the mechanisms
were being detected in one call yet the function would be called for
each line of data.
Using curl with verbose mode enabled would show that one line of data
would be received in response to the AUTH command, before the AUTH
<mechanism> command was sent to the server and then the next few lines
of the original AUTH command would be displayed before the response from
the AUTH <mechanism> command. This would then cause problems when
parsing the CRAM-MD5 challenge data as extra data was contained in the
buffer.
Changed the parsing so that each line is checked for the mechanisms
and the function returns FALSE until the whole of the AUTH response has
been processed.
Previously it wasn't possible to connect to POP3 and not specify the
user name as a CURLE_ACCESS_DENIED error would be returned. This error
occurred because USER would be sent to the server with a blank user name
if no mailbox user was specified as the server would reply with -ERR.
This wasn't a problem prior to the 7.26.0 release but with the
introduction of custom commands the user and/or application developer
might want to issue a CAPA command without having to log in as a
specific mailbox user.
Additionally this fix won't send the newly introduced AUTH command if no
user name is specified.
Rather than encoding the password message itself the
smtp_state_authpasswd_resp() function now delegates the work to the same
function that smtp_state_authlogin_resp() and smtp_authenticate() use
when constructing the encoded user name.
In preparation for moving to the SASL module re-factored the
smtp_auth_login_user() function to smtp_auth_login() so that it can be
used for both user names and passwords as sending both of these under
the login authentication mechanism is the same.
The POP3 protocol doesn't really have the concept of error codes and
uses +, +OK and -ERR in response to commands to indicate continue,
success and error.
The AUTH command is one of those commands that requires multiple pieces
of data to be sent to the server where the server will respond with + as
part of the handshaking. This meant changing the values before
continuing with the next stage of adding authentication support.
Changed the order of the state machine to match the order of actual
events.
Reworked some comments and function parameter positioning that I missed
the other day.
Added support for detecting the supported SASL authentication mechanisms
via the AUTH command. There are two ways of detecting them, either by
using the AUTH command, that will return -ERR if not supported or by
using the CAPA command which will return SASL and the list of mechanisms
if supported, not include SASL if SASL authentication is not supported
or -ERR if the CAPA command is not supported. As such it seems simpler
to use the AUTH command and fallback to normal clear text authentication
if the the command is not supported.
Additionally updated the test cases to return -ERR when the AUTH command
is encountered. Additional test cases will be added when support for the
individual authentication mechanisms is added.
Moved EOB definition into header file.
Switched the logic around in pop3_endofresp() to allow for the
introduction of auth-mechanism detection.
Repositioned second and third function variables where they will fit
within the 78 character line limit.
Tidied up some comments.
Move the SMTP_AUTH constants into a separate header file in
preparation for adding SASL based authentication to POP3 as the two
protocols will need to share them.
Due to the result code being reset to CURLE_OK when smtp_dophase_done()
was called, postdata would incorrectly be sent to the server when the
MAIL FROM or RCPT command was rejected.
As such, libcurl would return the wrong result code from performing the
operation and additionally set CURLINFO_RESPONSE_CODE to be that
returned by the postdata command.
Bug: http://curl.haxx.se/mail/lib-2012-05/0108.html
Reported by: Gokhan Sengun
In nettle/md5.h, md5_init and md5_update are defined as macros to
nettle_md5_init and nettle_md5_update respectively. This causes
error when using MD5_params.md5_init and md5_update. This patch
renames these members as md5_init_func and md5_update_func to
avoid name conflict. For completeness, MD5_params.md5_final was
also renamed as md5_final_func.
The changes in curl_ntlm_core.c is conversion error and fixed by
casting to proper type.
A dot character at the beginning of a line would not be escaped to a
double dot as required by RFC-2821, instead it would be deleted by the
mail server. Please see section 4.5.2 of the RFC for more information.
Note: This fix also simplifies the detection of repeated CRLF.CRLF
combinations, such as CRLF.CRLF.CRLF, a little rather than having to
advance the eob counter to 2.
Roman Mamedov spotted (in
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=670126) that curl would
not complain when given a URL with an IPv6 numerical address without
brackets. It would simply cut off the last ":[hex]" part and thus not
work correctly.
That's a URL using an illegal syntax and now libcurl will instead return
a clear error code and error message detailing the error.
The above mentioned bug report claims this to be a regression but
libcurl does not guarantee functionality when given URLs that aren't
following the URL spec (RFC3986 mostly). I consider the fact that it
used to handle this differently a mere coincidence.
When doing a chunked-encoded POST with -d (CURLOPT_POSTFIELDS) and the
size of the POST was zero length, it made libcurl first send a zero
chunk and then the terminating one. This could confuse a receiver and it
should rather just send the terminating chunk as it does with this fix.
Test case 1333 is added to verify.
Bug: http://curl.haxx.se/mail/archive-2012-04/0060.html
Reported by: Arnaud Compan
Commit 9109cdec11 brought this regression (shipped since 7.24.0).
The singleipconnect() function must not return an error if Curl_socket()
returns an error. It should then simply return OK and pass a SOCKET_BAD
back simply because that is how the user of this function expects it to
work and something else is not fine.
Reported by: Blaise Potard
Bug: http://curl.haxx.se/bug/view.cgi?id=3516508
Include stdbool.h only when it is available and configure is capable of
detecting a proper 'bool' data type when the header is included.
Compilation fix for old or unpatched versions of XL C compiler.
Report: http://curl.haxx.se/mail/archive-2012-04/0022.html
NSS_InitContext() was introduced in NSS 3.12.5 and helps to prevent
collisions on NSS initialization/shutdown with other libraries.
Bug: https://bugzilla.redhat.com/738456
configure script now provides conditional definitions for Makefile.am
that result in CURL_HIDDEN_SYMBOLS being defined by resulting makefiles
when appropriate.
Additionally, configure script option for symbol hiding control is now
named --enable-symbol-hiding --disable-symbol-hiding. While still valid,
old option name --enable-hidden-symbols --disable-hidden-symbols will
be deprecated in some future release.
BUILDING_LIBCURL and CURL_STATICLIB are no longer defined in curl_config.h,
configure will generate appropriate conditionals so that mentioned symbols
get defined and used in Makefiles at compilation time
Configuration files such as curl_config.h and all config-*.h no longer exist
nor are generated/copied into 'src' directory, now these only exist in 'lib'
directory from where curl tool sources uses them.
Additionally old src/setup.h has been refactored into src/tool_setup.h which
now pulls lib/setup.h
The possibility of a makefile needing an include path adjustment exists.
Curl_socket returns CURLE_COULDNT_CONNECT when the opensocket callback
returns CURL_SOCKET_BAD. Previous return value CURLE_FAILED_INIT
conveys incorrect information to the user.
Reworked the command sending from two specific LIST and RETR command
functions into a single command based function as well as the two
associated response handlers into a generic command handler.
If an empty string is passed to CURLOPT_SSH_PUBLIC_KEYFILE, libcurl will
pass no public key to libssh2 which then tries to compute it from the
private key. This is known to work when libssh2 1.4.0+ is linked against
OpenSSL.
This change replaces RFC 2818 based hostname check in OpenSSL build with
RFC 6125 [1] based one.
The hostname check in RFC 2818 is ambiguous and each project implements
it in the their own way and they are slightly different. I check curl,
gnutls, Firefox and Chrome and they are all different.
I don't think there is a bug in current implementation of hostname
check. But it is not as strict as the modern browsers do. Currently,
curl allows multiple wildcard character '*' and it matches '.'. (as
described in the comment in ssluse.c).
Firefox implementation is also based on RFC 2818 but it only allows at
most one wildcard character and it must be in the left-most label in the
pattern and the wildcard must not be followed by any character in the
label.[2] Chromium implementation is based on RFC 6125 as my patch does.
Firefox and Chromium both require wildcard in the left-most label in the
presented identifier.
This patch is more strict than the current implementation, so there may
be some cases where old curl works but new one does not. But at the same
time I think it is good practice to follow the modern browsers do and
follow the newer RFC.
[1] http://tools.ietf.org/html/rfc6125#section-6.4.3
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=159483
With FOLLOWLOCATION enabled. When a 3xx page is downloaded and the
download size was known (like with a Content-Length header), but the
subsequent URL (transfered after the 3xx page) was chunked encoded, then
the previous "known download size" would linger and cause the progress
meter to get incorrect information, ie the former value would remain
being sent in. This could easily result in downloads that were WAY
larger than "expected" and would cause >100% outputs with the curl
command line tool.
Test case 599 was created and it was used to repeat the bug and then
verify the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3510057
Reported by: Michael Wallner
It is now possible to calculate the md5 sum as the stream of buffers
becomes known where as previously it was only possible to calculate the
md5 sum of a pre-prepared buffer.
This feature allows the user to specify and use additional POP3
commands such as UIDL and DELE via libcurl's CURLOPT_CUSTOMREQUEST or
curl's -X command line option.
Simplified the code to remove the need for a separate "LIST <msg id>"
command handler and state machine and instead use the LIST command
handler for both operations.
Moved the server greeting response handling code from the statemach_act
functions to separate response functions. This makes the code simpler
to follow and provides consistency with the other responses that are
handled here.
The commit e650dbde86 that stripped off [brackets] from ipv6-only host
headers for the sake of cookie parsing wrongly incremented the host
pointer which would cause a bad free() call later on.
The refactoring of HTTP CONNECT handling in commit 41b0237834 that
made it protocol independent broke it for the multi interface. This fix
now introduce a better state handling and moved some logic to the
http_proxy.c source file.
Reported by: Yang Tse
Bug: http://curl.haxx.se/mail/lib-2012-03/0162.html
Changed the returned curl error codes for EHLO and HELO responses from
CURLE_LOGIN_DENIED to CURLE_REMOTE_ACCESS_DENIED as a negative response
from these commands represents no service as opposed to a login error.
An alternative would be:
1. specify HTTPS_CA_DIR and/or HTTPS_CA_FILE
2. ensure that Net::SSL is being used, and IO::Socket::SSL is NOT being
used
This question and answer explain:
http://stackoverflow.com/questions/74358/
Curl_protocol_connect() now does the tunneling through the HTTP proxy if
requested instead of letting each protocol specific connection function
do it.
Commit 466150bc64 fixed the Host: header with CONNECT, but I then
forgot the preceeding request-line. Now this too uses [brackets]
properly if a ipv6 numerical address was given.
Bug: http://curl.haxx.se/bug/view.cgi?id=3493129
Reported by: "Blacat"
Set the conn->data->info.httpcode variable in smtp_statemach_act() to
allow Curl_getinfo() to return the SMTP response code via the
CURLINFO_RESPONSE_CODE action.
Curl_pop3_write() would drop the final CRLF of a message as it was
considered part of the EOB as opposed to part of the message. Whilst
the EOB sequence needs to be searched for by the function only the
final 3 characters should be removed as per RFC-1939 section 3.
Reported by: Rich Gray
Bug: http://curl.haxx.se/mail/lib-2012-02/0051.html
Curl_smtp_escape_eob() would leave off final CRLFs from emails ending
in multiple blank lines additionally leaving the smtpc->eob variable
with the character count in, which would cause problems for additional
emails when sent through multiple calls to curl_easy_perform() after a
CURLOPT_CONNECT_ONLY.
Fixed the use of angled brackets "<>" in the optional AUTH parameter as
per RFC-2554 section 5. The address should not include them but an
empty address should be replaced by them.
Added a new CURLOPT_MAIL_AUTH option that allows the calling program to
set the optional AUTH parameter in the MAIL FROM command.
When this option is specified and an authentication mechanism is used
to communicate with the mail server then the AUTH parameter will be
included in the MAIL FROM command. This is particularly useful when the
calling program is acting as a relay in a trusted environment and
performing server to server communication, as it allows the relaying
server to specify the address of the mailbox that was used to
authenticate and send the original email.
Modify configure.ac to test for new CyaSSL Init function and remove
default install path to system. Change to CyaSSL OpenSSL header and
proper Init in code as well.
Note that this no longer detects or works with CyaSSL before v2
Fixed incorrect behavior in smtp_done() which would cause the end of
block data to be sent to the SMTP server if libcurl was operating in
connect only mode. This would cause the server to return an error as
data would not be expected which in turn caused libcurl to return
CURLE_RECV_ERROR.
... by making sure that the string is always freed after the invoke as
parse_proxy will always copy the data and this way there's a single
free() instead of multiple ones.
The proxy parser function strips off trailing slashes off the proxy name
which could lead to a mistaken zero length proxy name which would be
treated as no proxy at all by subsequent functions!
This is now detected and an error is returned. Verified by the new test
1329.
Reported by: Chandrakant Bagul
Bug: http://curl.haxx.se/mail/lib-2012-02/0000.html
Allow an appliction to set libcurl specific SSL options. The first and
only options supported right now is CURLSSLOPT_ALLOW_BEAST.
It will make libcurl to disable any work-arounds the underlying SSL
library may have to address a known security flaw in the SSL3 and TLS1.0
protocol versions.
This is a reaction to us unconditionally removing that behavior after
this security advisory:
http://curl.haxx.se/docs/adv_20120124B.html
... it did however cause a lot of programs to fail because of old
servers not liking this work-around. Now programs can opt to decrease
the security in order to interoperate with old servers better.
This adds three new options to control the behavior of TCP keepalives:
- CURLOPT_TCP_KEEPALIVE: enable/disable probes
- CURLOPT_TCP_KEEPIDLE: idle time before sending first probe
- CURLOPT_TCP_KEEPINTVL: delay between successive probes
While not all operating systems support the TCP_KEEPIDLE and
TCP_KEEPINTVL knobs, the library will still allow these options to be
set by clients, silently ignoring the values.
When CURLOPT_REFERER has been used, curl_easy_reset() did not properly
clear it.
Verified with the new test 598
Bug: http://curl.haxx.se/bug/view.cgi?id=3481551
Reported by: Michael Day
When the target host was given as a IPv6 numerical address, it was not
properly put within square brackets for the Host: header in the CONNECT
request. The "normal" request did fine.
Reported by: "zooloo"
Bug: http://curl.haxx.se/bug/view.cgi?id=3482093
When connecting to a domain with multiple IP addresses, allow different,
decreasing connection timeout values. This should guarantee some
connections attempts with sufficiently long timeouts, while still
providing fallback.
With advice from Nikos Mavrogiannopoulos, changed the priority string to
add "actual priorities" and favour ARCFOUR. This makes libcurl work
better when enforcing SSLv3 with GnuTLS. Both in the sense that the
libmicrohttpd test is now working again but also that it mitigates a
weakness in the older SSL/TLS protocols.
Bug: http://curl.haxx.se/mail/lib-2012-01/0225.html
Reported by: Christian Grothoff
Protocols (IMAP, POP3 and SMTP) that use the path part of a URL in a
decoded manner now use the new Curl_urldecode() function to reject URLs
with embedded control codes (anything that is or decodes to a byte value
less than 32).
URLs containing such codes could easily otherwise be used to do harm and
allow users to do unintended actions with otherwise innocent tools and
applications. Like for example using a URL like
pop3://pop3.example.com/1%0d%0aDELE%201 when the app wants a URL to get
a mail and instead this would delete one.
This flaw is considered a security vulnerability: CVE-2012-0036
Security advisory at: http://curl.haxx.se/docs/adv_20120124.html
Reported by: Dan Fandrich
OpenSSL added a work-around for a SSL 3.0/TLS 1.0 CBC vulnerability
(http://www.openssl.org/~bodo/tls-cbc.txt). In 0.9.6e they added a bit
to SSL_OP_ALL that _disables_ that work-around despite the fact that
SSL_OP_ALL is documented to do "rather harmless" workarounds.
The libcurl code uses the SSL_OP_ALL define and thus logically always
disables the OpenSSL fix.
In order to keep the secure work-around workding, the
SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS bit must not be set and this change
makes sure of this.
Reported by: product-security at Apple
Using a URL with embedded user name and password didn't work if the host
was given as a numerical IPv6 string, like ftp://user:password@[::1]/
Reported by: Brandon Wang
Bug: http://curl.haxx.se/mail/archive-2012-01/0047.html
SSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG option enabling allowed successfull
interoperability with web server Netscape Enterprise Server 2.0.1 released
back in 1996 more than 15 years ago.
Due to CVE-2010-4180, option SSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG has
become ineffective as of OpenSSL 0.9.8q and 1.0.0c. In order to mitigate
CVE-2010-4180 when using previous OpenSSL versions we no longer enable
this option regardless of OpenSSL version and SSL_OP_ALL definition.
Some functions using getaddrinfo and gethostbyname were still
mistakingly being used/linked even if c-ares was selected as resolver
backend.
Reported by: Arthur Murray
Bug: http://curl.haxx.se/mail/lib-2012-01/0160.html
Previously the code would create a dummy socket while resolving just to
have curl_multi_fdset() return something but the non-win32 version
doesn't do it this way and the creation and use of a socket that isn't
made with the common create-socket callback can be confusing to apps
using the multi_socket API etc.
This change removes the dummy socket and thus will cause
curl_multi_fdset() to return with maxfd == -1 more often.
Fixed a problem in POP3 and IMAP where a connection would fail when
CURLUSESSL_TRY was specified for a server that didn't support
SSL/TLS connections rather than continuing.
The STARTTLS response code in SMTP, POP3 and IMAP would return
CURLE_LOGIN_DENIED rather than CURLE_USE_SSL_FAILED when SSL/TLS
was not available on the server.
Reported by: Gokhan Sengun
Bug: http://curl.haxx.se/mail/lib-2012-01/0018.html
Unfortunately we have no test cases for this and I have no SSPI build or
server to verify this with. The change seems simple enough though.
Bug: http://curl.haxx.se/bug/view.cgi?id=3466497
Reported by: Patrice Guerin
When the buffer gets realloced to hold the file name in the
SSH_SFTP_READDIR_LINK state, the counter was not bumped accordingly.
Reported by: Armel Asselin
Patch by: Armel Asselin
Bug: http://curl.haxx.se/mail/lib-2011-12/0249.html
When a HTTP connection is re-used for a subsequent request without
proxy, it would always re-use the Host: header of the first request. As
host names are case insensitive it would make curl send another host
name case that what the particular request used.
Now it will instead always use the most recent host name to always use
the desired casing.
Added test case 1318 to verify.
Bug: http://curl.haxx.se/mail/lib-2011-12/0314.html
Reported by: Alex Vinnik
The load host names to DNS cache function was moved to hostip.c and it
now makes sure to not add host names that already are present in the
cache. It would previously lead to memory leaks when for example using
the --resolve and multiple URLs on the command line.
The commit 9dd85bc unintentionally changed the way we compute the time
spent waiting for 100-continue. In particular, when using a SSL client
certificate, the time spent by SSL handshake was included and could
cause the CURL_TIMEOUT_EXPECT_100 timeout to be mistakenly fired up.
Bug: https://bugzilla.redhat.com/767490
Reported by: Mamoru Tasaka
ftp_do_more() returns after accepting the server connect however it
needs to fall through and set "*complete" to TRUE before exit from the
function.
Bug: http://curl.haxx.se/mail/lib-2011-12/0250.html
Reported by: Gokhan Sengun
In the recent do_more fix the new logic was mistakenly checking the
pointer instead of what it points to.
Reported by: Gokhan Sengun
Bug: http://curl.haxx.se/mail/lib-2011-12/0250.html
When sending quote command to a SFTP server and 'mkdir' was used, it
would send fixed permissions and not use the CURLOPT_NEW_DIRECTORY_PERMS
as it should.
Reported by: Armel
Patch by: Armel
Bug: http://curl.haxx.se/mail/lib-2011-12/0249.html
CURLOPT_RESOLVE populates the DNS cache with entries that are marked as
eternally in use. Those entries need to be taken care of when the cache
is killed off.
Bug: http://curl.haxx.se/bug/view.cgi?id=3463121
Reported by: "tw84452852"
First off the timeout for accepting a server connect back must of course
respect a global timeout. Then the timeleft function is only used by ftp
code so it was moved to ftp.c and made static.
"wait_data_conn" was added to the connectionbits in commit c834213ad5 for
handling active FTP connections but as it is purely FTP specific and now
only ever accessed by ftp.c I moved it into the FTP connection struct.
Backpedaled out the funny double-change of state in the multi state
machine by adding a new argument to the do_more() function to signal
completion. This way it can remain in the DO_MORE state properly until
done. Long term, the entire DO_MORE logic should be moved into the FTP
code and be hidden from the multi code as the logic is only used for
FTP.
1- Two new error codes are introduced.
CURLE_FTP_ACCEPT_FAILED to be set whenever ACCEPTing fails because of
FTP server connected.
CURLE_FTP_ACCEPT_TIMEOUT to be set whenever ACCEPTing timeouts.
Neither of these errors are considered fatal and control connection
remains OK because it could just be a firewall blocking server to
connect to the client.
2- One new setopt option was introduced.
CURLOPT_ACCEPTTIMEOUT_MS
It sets the maximum amount of time FTP client is going to wait for a
server to connect. Internal default accept timeout is 60 seconds.
It makes it easier to introduce debug outputs in this function, and
everything in the function is using the value anyway so it might even be
more efficient.
Regression introduced in 7.23.0 with commit 9dd85bce. The function in
which the PRETRANSFER time stamp was recorded was moved in time causing
it be stored very quickly after the start timestamp. On most systems
shorter than 1 millisecond and thus it wouldn't even show with -w
"%{time_pretransfer}" using the command line tool.
Bug: http://curl.haxx.se/mail/archive-2011-12/0022.html
Reported by: Toni Moreno
Allow, at configure time, the production of versioned symbols. The
symbols will look like "CURL_<FLAVOUR>_<VERSION> <SYMBOL>", where
<FLAVOUR> represents the SSL flavour (e.g. OPENSSL, GNUTLS, NSS, ...),
<VERSION> is the major SONAME version and <SYMBOL> is the actual symbol
name. If no SSL library is enabled the symbols will be just
"CURL_<VERSION> <SYMBOL>".
