to allow code to act differently on the situation.
Also added some more info message for the connection re-use function to
make it clearer when connections are not re-used.
Previously when we do pause because of out of buffer, we just throw
away unread data in connection buffer. This just broke protocol
framing, and I saw occasional FRAME_SIZE_ERROR. This commit fix this
issue by remembering how much data read, and in the next iteration, we
process remaining data.
This commit fixes the bug that streams get stuck if stream gets some
DATA, and stream->closed becomes true at the same time. Previously,
in this condition, after we processed DATA, we are going to try to
read data from underlying transport, but there is no data, and gets
EAGAIN. There was no code path to evaludate stream->closed.
... from the connection struct. The stream one being the 'struct HTTP'
which is kept in the SessionHandle struct (easy handle).
lookup streams for incoming frames in the stream hash, hashing is based
on the stream id and we get the SessionHandle for the incoming stream
that way.
Previously we counted all connections to a specific host name and that
would be used for the CURLMOPT_MAX_HOST_CONNECTIONS check for example,
while servers on different port numbers are normally considered
different "origins" on the web and should thus be considered different
hosts.
All the existing Curl_bundle* functions were only ever used from within
the conncache.c file, so I moved them over and made them static (and
removed the Curl_ prefix).
This avoids unnecessary dynamic allocs and as this also removed the last
users of *hash_alloc() and *hash_destroy(), those two functions are now
removed.
The OpenSSL trace callback is wonderfully undocumented but given a
journey in the source code, it seems the cases were ssl_ver is zero
doesn't follow the same pattern and thus turned out confusing and
misleading. For now, we skip doing any CURLINFO_TEXT logging on those
but keep sending them as CURLINFO_SSL_DATA_OUT/IN.
Also, I added direction to the text info and I edited some functions
slightly.
Bug: https://github.com/bagder/curl/issues/219
Reported-by: Jay Satiro, Ashish Shukla
- update default versions of dependencies (except for rare/old platforms)
- update urls
- sync examples makefiles with main ones
- remove line ending space
Make the HTTP headers separated by default for improved security and
reduced risk for information leakage.
Bug: http://curl.haxx.se/docs/adv_20150429.html
Reported-by: Yehezkel Horowitz, Oren Souroujon
When doing HTTP requests Negotiate authenticated, the entire connnection
may become authenticated and not just the specific HTTP request which is
otherwise how HTTP works, as Negotiate can basically use NTLM under the
hood. curl was not adhering to this fact but would assume that such
requests would also be authenticated per request.
CVE-2015-3148
Bug: http://curl.haxx.se/docs/adv_20150422B.html
Reported-by: Isaac Boukris
If a URL is given with a zero-length host name, like in "http://:80" or
just ":80", `fix_hostname()` will index the host name pointer with a -1
offset (as it blindly assumes a non-zero length) and both read and
assign that address.
CVE-2015-3144
Bug: http://curl.haxx.se/docs/adv_20150422D.html
Reported-by: Hanno Böck
The internal libcurl function called sanitize_cookie_path() that cleans
up the path element as given to it from a remote site or when read from
a file, did not properly validate the input. If given a path that
consisted of a single double-quote, libcurl would index a newly
allocated memory area with index -1 and assign a zero to it, thus
destroying heap memory it wasn't supposed to.
CVE-2015-3145
Bug: http://curl.haxx.se/docs/adv_20150422C.html
Reported-by: Hanno Böck
At some point, Firefox has changed and generates different directory
names for the default profile that made this script fail to find them.
Bug: https://github.com/bagder/curl/issues/207
Reported-by: sneakyimp
Add 'gdi32' and 'crypt32' Windows implibs to avoid failure
while building libcurl.dll using the mingw compiler.
The same logic is used in 'src/makefile.m32' when
building curl.exe.
The factor of 8 is a bytes-to-bits conversion factor, but pkt_size and
rate_bps are both in bytes. When using the rate limiting option, curl
waits 8 times too long, and then transfers very quickly until the
average rate reaches the limit. The average rate follows the limit over
time, but the actual traffic is bursty.
Thanks-to: Benjamin Gilbert
The key length in bits will always fit in an unsigned long so the
loss-of-data warning assigning the result of x64 pointer arithmetic to
an unsigned long is unnecessary.
Prior to this change libcurl could show multiple 'CyaSSL: Connecting to'
messages since cyassl_connect_step2 is called multiple times, typically.
The message is superfluous even once since libcurl already informs the
user elsewhere in code that it is connecting.
- cache entries must be also refreshed when they are in use
- have the cache count as inuse reference too, freeing timestamp == 0 special
value
- use timestamp == 0 for CURLOPT_RESOLVE entries which don't get refreshed
- remove CURLOPT_RESOLVE special inuse reference (timestamp == 0 will prevent refresh)
- fix Curl_hostcache_clean - CURLOPT_RESOLVE entries don't have a special
reference anymore, and it would also release non CURLOPT_RESOLVE references
- fix locking in Curl_hostcache_clean
- fix unit1305.c: hash now keeps a reference, need to set inuse = 1
This change is to allow the user's CTX callback to change the minimum
protocol version in the CTX without us later overriding it, as we did
prior to this change.
SSL_CTX_load_verify_locations can return negative values on fail,
therefore to check for failure we check if load is != 1 (success)
instead of if load is == 0 (failure), the latter being incorrect given
that behavior.
Previously in Curl_http2_switched, we called nghttp2_session_mem_recv to
parse incoming data which were already received while curl was handling
upgrade. But we didn't call nghttp2_session_send, and it led to make
curl not send any response to the received frames. Most likely, we
received SETTINGS from server at this point, so we missed opportunity to
send SETTINGS + ACK. This commit adds missing nghttp2_session_send call
in Curl_http2_switched to fix this issue.
Bug: https://github.com/bagder/curl/issues/192
Reported-by: Stefan Eissing
"name =value" is fine and the space should just be skipped.
Updated test 31 to also test for this.
Bug: https://github.com/bagder/curl/issues/195
Reported-by: cromestant
Help-by: Frank Gevaerts
(Curl_cyassl_init)
- Return 1 on success, 0 in failure.
Prior to this change the fail path returned an incorrect value and the
evaluation to determine whether CyaSSL_Init had succeeded was incorrect.
Ironically that combined with the way curl_global_init tests SSL library
initialization (!Curl_ssl_init()) meant that CyaSSL having been
successfully initialized would be seen as that even though the code path
and return value in Curl_cyassl_init were wrong.
If the handle removed from the multi handle happens to be the one
"owning" the pipeline other transfers will be waiting indefinitely. Now
we move such handles back to connect to have them race (again) for
getting the connection and thus avoid hanging.
Bug: http://curl.haxx.se/bug/view.cgi?id=1465
Reported-by: Jiri Dvorak
... even if they don't have an associated connection anymore. It could
leave the waiting transfers pending with no active one on the
connection.
Bug: http://curl.haxx.se/bug/view.cgi?id=1465
Reported-by: Jiri Dvorak
(cyassl_connect_step1)
- Use TLS 1.0-1.2 by default when available.
CyaSSL/wolfSSL >= v3.3.0 supports setting a minimum protocol downgrade
version.
cyassl/cyassl@322f79f
This header file must be included after all header files except
memdebug.h, as it does similar memory function redefinitions and can be
similarly affected by conflicting definitions in system or dependent
library headers.
CID 1202732 warns on the previous use, although I cannot fine any
problems with it. I'm doing this change only to make the code use a more
familiar approach to accomplish the same thing.
We prematurely changed protocol handler to HTTP/2 which made things very
slow (and wrong).
Reported-by: Stefan Eissing
Bug: https://github.com/bagder/curl/issues/169
Since we just started make use of free(NULL) in order to simplify code,
this change takes it a step further and:
- converts lots of Curl_safefree() calls to good old free()
- makes Curl_safefree() not check the pointer before free()
The (new) rule of thumb is: if you really want a function call that
frees a pointer and then assigns it to NULL, then use Curl_safefree().
But we will prefer just using free() from now on.
The following functions return immediately if a null pointer was passed.
* Curl_cookie_cleanup
* curl_formfree
It is therefore not needed that a function caller repeats a corresponding check.
This issue was fixed by using the software Coccinelle 1.0.0-rc24.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
The function "free" is documented in the way that no action shall occur for
a passed null pointer. It is therefore not needed that a function caller
repeats a corresponding check.
http://stackoverflow.com/questions/18775608/free-a-null-pointer-anyway-or-check-first
This issue was fixed by using the software Coccinelle 1.0.0-rc24.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
Bug: https://github.com/bagder/curl/pull/168
(trynextip)
- Don't try the "other" protocol family unless IPv6 is available. In an
IPv4-only build the other family can only be IPv6 which is unavailable.
This change essentially stops IPv4-only builds from attempting the
"happy eyeballs" secondary parallel connection that is supposed to be
used by the "other" address family.
Prior to this change in IPv4-only builds that secondary parallel
connection attempt could be erroneously used by the same family (IPv4)
which caused a bug where every address after the first for a host could
be tried twice, often in parallel. This change fixes that bug. An
example of the bug is shown below.
Assume MTEST resolves to 3 addresses 127.0.0.2, 127.0.0.3 and 127.0.0.4:
* STATE: INIT => CONNECT handle 0x64f4b0; line 1046 (connection #-5000)
* Rebuilt URL to: http://MTEST/
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x64f4b0; line 1083
(connection #0)
* Trying 127.0.0.2...
* STATE: WAITRESOLVE => WAITCONNECT handle 0x64f4b0; line 1163
(connection #0)
* Trying 127.0.0.3...
* connect to 127.0.0.2 port 80 failed: Connection refused
* Trying 127.0.0.3...
* connect to 127.0.0.3 port 80 failed: Connection refused
* Trying 127.0.0.4...
* connect to 127.0.0.3 port 80 failed: Connection refused
* Trying 127.0.0.4...
* connect to 127.0.0.4 port 80 failed: Connection refused
* connect to 127.0.0.4 port 80 failed: Connection refused
* Failed to connect to MTEST port 80: Connection refused
* Closing connection 0
* The cache now contains 0 members
* Expire cleared
curl: (7) Failed to connect to MTEST port 80: Connection refused
The bug was born in commit bagder/curl@2d435c7.
In function Curl_closesocket() in connect.c the call to
Curl_multi_closed() was wrongly omitted if a socket close function
(CURLOPT_CLOSESOCKETFUNCTION) is registered.
That would lead to not removing the socket from the internal hash table
and not calling the multi socket callback appropriately.
Bug: http://curl.haxx.se/bug/view.cgi?id=1493
A signal handler for SIGALRM is installed in Curl_resolv_timeout. It is
configured to interrupt system calls and uses siglongjmp to return into
the function if alarm() goes off.
The signal handler is installed before curl_jmpenv is initialized.
This means that an already installed alarm timer could trigger the
newly installed signal handler, leading to undefined behavior when it
accesses the uninitialized curl_jmpenv.
Even if there is no previously installed alarm available, the code in
Curl_resolv_timeout itself installs an alarm before the environment is
fully set up. If the process is sent into suspend right after that, the
signal handler could be called too early as in previous scenario.
To fix this, the signal handler should only be installed and the alarm
timer only be set after sigsetjmp has been called.
... by using the regular Curl_http_done() method which checks for
that. This makes test 1801 fail consistently with error 56 (which seems
fine) to that test is also updated here.
Reported-by: Ben Darnell
Bug: https://github.com/bagder/curl/issues/166
This makes curl pick better (stronger) ciphers by default. The strongest
available ciphers are fine according to the HTTP/2 spec so an OpenSSL
built curl is no longer rejected by string HTTP/2 servers.
Bug: http://curl.haxx.se/bug/view.cgi?id=1487
...after the method line:
"Since the Host field-value is critical information for handling a
request, a user agent SHOULD generate Host as the first header field
following the request-line." / RFC 7230 section 5.4
Additionally, this will also make libcurl ignore multiple specified
custom Host: headers and only use the first one. Test 1121 has been
updated accordingly
Bug: http://curl.haxx.se/bug/view.cgi?id=1491
Reported-by: Rainer Canavan
When checking for a connection to re-use, a proxy-using request must
check for and use a proxy connection and not one based on the host
name!
Added test 1421 to verify
Bug: http://curl.haxx.se/bug/view.cgi?id=1492
Instead of priting cipher and MAC algorithms names separately, print the
whole cipher suite string which also includes the key exchange algorithm,
along with the negotiated TLS version.
The code used some happy eyeballs logic even _after_ CONNECT has been
sent to a proxy, while the happy eyeball phase is already (should be)
over by then.
This is solved by splitting the multi state into two separate states
introducing the new SENDPROTOCONNECT state.
Bug: http://curl.haxx.se/mail/lib-2015-01/0170.html
Reported-by: Peter Laser
Since 1342a96ecf, a timeout detected in the multi state machine didn't
necesarily clear everything up, like formpost data.
Bug: https://github.com/bagder/curl/issues/147
Reported-by: Michel Promonet
Patched-by: Michel Promonet
SSLeay was the name of the library that was subsequently turned into
OpenSSL many moons ago (1999). curl does not work with the old SSLeay
library since years. This is now reflected by only using USE_OPENSSL in
code that depends on OpenSSL.
Previously, we just ignored error code passed to
on_stream_close_callback and just return 0 (success) after stream
closure even if stream was reset with error. This patch records error
code in on_stream_close_callback, and return -1 and use CURLE_HTTP2
error code on abnormal stream closure.
The vtls layer now checks the return value, so it is no longer necessary
to abort if a random number cannot be provided by NSS. This also fixes
the following Coverity report:
Error: FORWARD_NULL (CWE-476):
lib/vtls/nss.c:1918: var_compare_op: Comparing "data" to null implies that "data" might be null.
lib/vtls/nss.c:1923: var_deref_model: Passing null pointer "data" to "Curl_failf", which dereferences it.
lib/sendf.c:154:3: deref_parm: Directly dereferencing parameter "data".
obj_count can be 1 if the custom read function is set or the stdin
handle is a reference to a pipe. Since the pipe should be handled
using the PeekNamedPipe-check below, the custom read function should
only be used if it is actually enabled.
According to [1]: "Returning 0 will signal end-of-file to the library
and cause it to stop the current transfer."
This change makes the Windows telnet code handle this case accordingly.
[1] http://curl.haxx.se/libcurl/c/CURLOPT_READFUNCTION.html
SSL_CTX_load_verify_locations by default (and if given non-Null
parameters) searches the CAfile first and falls back to CApath. This
allows for CAfile to be a basis (e.g. installed by the package manager)
and CApath to be a user configured directory.
This wasn't reflected by the previous configure constraint which this
patch fixes.
Bug: https://github.com/bagder/curl/pull/139
Correctly check for memcmp() return value (it returns 0 if the strings match).
This is not really important, since curl is going to use http/1.1 anyway, but
it's still a bug I guess.
For consistency with other conditionally compiled code in openssl.c,
use OPENSSL_IS_BORINGSSL rather than HAVE_BORINGSSL and try to use
HAVE_BORINGSSL outside of openssl.c when the OpenSSL header files are
not included.
Previously we don't ignore PUSH_PROMISE header fields in on_header
callback. It makes header values mixed with following HEADERS,
resulting protocol error.
Prior to this change the options for exclusive SSL protocol versions did
not actually set the protocol exclusive.
http://curl.haxx.se/mail/lib-2015-01/0002.html
Reported-by: Dan Fandrich
The struct went private in 1.0.2 so we cannot read the version number
from there anymore. Use SSL_version() instead!
Reported-by: Gisle Vanem
Bug: http://curl.haxx.se/mail/lib-2015-02/0034.html
Modified the Curl_ossl_cert_status_request() function to return FALSE
when built with BoringSSL or when OpenSSL is missing the necessary TLS
extensions.
Commit 7a8b2885e2 made some functions static and removed the public
Curl_ prefix. Unfortunately, it also removed the sasl_ prefix, which
is the naming convention we use in this source file.
curl_sasl.c:1221: error C2065: 'mechtable' : undeclared identifier
This error could also happen for non-SSPI builds when cryptography is
disabled (CURL_DISABLE_CRYPTO_AUTH is defined).
There is an issue with conflicting "struct timeval" definitions with
certain AmigaOS releases and C libraries, depending on what gets
included when. It's a minor difference - the OS one is unsigned,
whereas the common structure has signed elements. If the OS one ends up
getting defined, this causes a timing calculation error in curl.
It's easy enough to resolve this at the curl end, by casting the
potentially errorneous calculation to a signed long.
... of the other cert verification checks so that you can set verifyhost
and verifypeer to FALSE and still check the public key.
Bug: http://curl.haxx.se/bug/view.cgi?id=1471
Reported-by: Kyle J. McKay
Use a dynamicly allocated buffer for the temporary SPN variable similar
to how the SASL GSS-API code does, rather than using a fixed buffer of
2048 characters.
Carrying on from commit 037cd0d991, removed the following unimplemented
instances of curlssl_close_all():
Curl_axtls_close_all()
Curl_darwinssl_close_all()
Curl_cyassl_close_all()
Curl_gskit_close_all()
Curl_gtls_close_all()
Curl_nss_close_all()
Curl_polarssl_close_all()
Fixed the following warning and error from commit 3af90a6e19 when SSL
is not being used:
url.c:2004: warning C4013: 'Curl_ssl_cert_status_request' undefined;
assuming extern returning int
error LNK2019: unresolved external symbol Curl_ssl_cert_status_request
referenced in function Curl_setopt
Use the SECURITY_STATUS typedef rather than a unsigned long for the
QuerySecurityPackageInfo() return and rename the variable as per other
areas of SSPI code.
Also known as "status_request" or OCSP stapling, defined in RFC6066 section 8.
This requires GnuTLS 3.1.3 or higher to build, however it's recommended to use
at least GnuTLS 3.3.11 since previous versions had a bug that caused the OCSP
response verfication to fail even on valid responses.
This option can be used to enable/disable certificate status verification using
the "Certificate Status Request" TLS extension defined in RFC6066 section 8.
This also adds the CURLE_SSL_INVALIDCERTSTATUS error, to be used when the
certificate status verification fails, and the Curl_ssl_cert_status_request()
function, used to check whether the SSL backend supports the status_request
extension.
If the session is still used by active SSL/TLS connections, it
cannot be closed yet. Thus we mark the session as not being cached
any longer so that the reference counting mechanism in
Curl_schannel_shutdown is used to close and free the session.
Reported-by: Jean-Francois Durand
... and make sure we can connect the data connection to a host name that
is longer than 48 bytes.
Also simplifies the code somewhat by re-using the original host name
more, as it is likely still in the DNS cache.
Original-Patch-by: Vojtěch Král
Bug: http://curl.haxx.se/bug/view.cgi?id=1468
...to avoid a session ID getting cached without certificate checking and
then after a subsequent _enabling_ of the check libcurl could still
re-use the session done without cert checks.
Bug: http://curl.haxx.se/docs/adv_20150108A.html
Reported-by: Marc Hesse
As we get the length for the DN and attribute variables, and we know
the length for the line terminator, pass the length values rather than
zero as this will save Curl_client_write() from having to perform an
additional strlen() call.
curl_ntlm_core.c:146: warning: passing 'DES_cblock' (aka 'unsigned char
[8]') to parameter of type 'char *' converts
between pointers to integer types with different
sign
Rather than duplicate the code in setup_des_key() for OpenSSL and in
extend_key_56_to_64() for non-OpenSSL based crypto engines, as it is
the same, use extend_key_56_to_64() for all engines.
smb.c:780: warning: passing 'char *' to parameter of type 'unsigned
char *' converts between pointers to integer types with
different sign
smb.c:781: warning: passing 'char *' to parameter of type 'unsigned
char *' converts between pointers to integer types with
different sign
smb.c:804: warning: passing 'char *' to parameter of type 'unsigned
char *' converts between pointers to integer types with
different sign
Prefer void for unused parameters, rather than assigning an argument to
itself as a) unintelligent compilers won't optimize it out, b) it can't
be used for const parameters, c) it will cause compilation warnings for
clang with -Wself-assign and d) is inconsistent with other areas of the
curl source code.
Moved our Initialize Security Context return attribute definitions to
the SSPI module, as a) these can be used by other SSPI based providers
and b) the ISC required attributes are defined there.
curl_schannel.h:123: warning: right-hand operand of comma expression
has no effect
Some instances of the curlssl_close_all() function were declared with a
void return type whilst others as int. The schannel version returned
CURLE_NOT_BUILT_IN and others simply returned zero, but in all cases the
return code was ignored by the calling function Curl_ssl_close_all().
For the time being and to keep the internal API consistent, changed all
declarations to use a void return type.
To reduce code we might want to consider removing the unimplemented
versions and use a void #define like schannel does.
For consistency, as we seem to have a bit of a mixed bag, changed all
instances of ipv4 and ipv6 in comments and documentations to use the
correct case.
Otherwise Curl_ssl_init_certinfo() can fail and set the num_of_certs
member variable to the requested count, which could then be used
incorrectly as libcurl closes down.
The return type for this function was 0 on success and 1 on error. This
was then examined by the calling functions and, in most cases, used to
return CURLE_OUT_OF_MEMORY.
Instead use CURLcode for the return type and return the out of memory
error directly, propagating it up the call stack.
The return type of this function is a boolean value, and even uses a
bool internally, so use bool in the function declaration as well as
the variables that store the return value, to avoid any confusion.
curl_ntlm_core.c:301: warning: pointer targets in passing argument 2 of
'CryptImportKey' differ in signedness
curl_ntlm_core.c:310: warning: passing argument 6 of 'CryptEncrypt' from
incompatible pointer type
curl_ntlm_core.c:540: warning: passing argument 4 of 'CryptGetHashParam'
from incompatible pointer type
... as it never copies the trailing zero anyway and always just the four
bytes so let's not mislead anyone into thinking it is actually treated
as a string.
Coverity CID: 1260214
lib/setup-vms.h : VAX HP OpenSSL port is ancient, needs help.
More defines to set symbols to uppercase.
src/tool_main.c : Fix parameter to vms_special_exit() call.
packages/vms/ :
backup_gnv_curl_src.com : Fix the error message to have the correct package.
build_curl-config_script.com : Rewrite to be more accurate.
build_libcurl_pc.com : Use tool_version.h now.
build_vms.com : Fix to handle lib/vtls directory.
curl_gnv_build_steps.txt : Updated build procedure documentation.
generate_config_vms_h_curl.com :
* VAX does not support 64 bit ints, so no NTLM support for now.
* VAX HP SSL port is ancient, needs some help.
* Disable NGHTTP2 for now, not ported to VMS.
* Disable UNIX_SOCKETS, not available on VMS yet.
* HP GSSAPI port does not have gss_nt_service_name.
gnv_link_curl.com : Update for new curl structure.
pcsi_product_gnv_curl.com : Set up to optionally do a complete build.
Removed 'next' variable in Curl_convert_form(). Rather than setting it
from 'form->next' and using that to set 'form' after the conversion
just use 'form = form->next' instead.
There was a confusion between these: this commit tries to disambiguate them.
- Scope can be computed from the address itself.
- Scope id is scope dependent: it is currently defined as 1-based local
interface index for link-local scoped addresses, and as a site index(?) for
(obsolete) site-local addresses. Linux only supports it for link-local
addresses.
The URL parser properly parses a scope id as an interface index, but stores it
in a field named "scope": confusion. The field has been renamed into "scope_id".
Curl_if2ip() used the scope id as it was a scope. This caused failures
to bind to an interface.
Scope is now computed from the addresses and Curl_if2ip() matches them.
If redundantly specified in the URL, scope id is check for mismatch with
the interface index.
This commit should fix SF bug #1451.
- do not grow memory by doubling its size
- do not leak previously allocated memory if reallocation fails
- replace while-loop with a single check to make sure
that the requested amount of data fits into the buffer
Bug: http://curl.haxx.se/bug/view.cgi?id=1450
Reported-by: Warren Menzer
There is no need to set the 'state' and 'result' member variables to
SMB_REQUESTING (0) and CURLE_OK (0) after the allocation via calloc()
as calloc() initialises the contents to zero.
I don't think both of my fix ups from yesterday were needed to fix the
compilation warning, so remove the one that I think is unnecessary and
let the next Android autobuild prove/disprove it.
