1
0
mirror of https://github.com/moparisthebest/curl synced 2024-12-22 16:18:48 -05:00
Commit Graph

8962 Commits

Author SHA1 Message Date
Anders Bakken
406506ca92 curl_multi_remove_handle: fix a double-free
In short the easy handle needs to be disconnected from its connection at
this point since the connection still is serving other easy handles.

In our app we can reliably reproduce a crash in our http2 stress test
that is fixed by this change. I can't easily reproduce the same test in
a small example.

This is the gdb/asan output:

==11785==ERROR: AddressSanitizer: heap-use-after-free on address 0xe9f4fb80 at pc 0x09f41f19 bp 0xf27be688 sp 0xf27be67c
READ of size 4 at 0xe9f4fb80 thread T13 (RESOURCE_HTTP)
    #0 0x9f41f18 in curl_multi_remove_handle /path/to/source/3rdparty/curl/lib/multi.c:666

0xe9f4fb80 is located 0 bytes inside of 1128-byte region [0xe9f4fb80,0xe9f4ffe8)
freed by thread T13 (RESOURCE_HTTP) here:
    #0 0xf7b1b5c2 in __interceptor_free /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_malloc_linux.cc:45
    #1 0x9f7862d in conn_free /path/to/source/3rdparty/curl/lib/url.c:2808
    #2 0x9f78c6a in Curl_disconnect /path/to/source/3rdparty/curl/lib/url.c:2876
    #3 0x9f41b09 in multi_done /path/to/source/3rdparty/curl/lib/multi.c:615
    #4 0x9f48017 in multi_runsingle /path/to/source/3rdparty/curl/lib/multi.c:1896
    #5 0x9f490f1 in curl_multi_perform /path/to/source/3rdparty/curl/lib/multi.c:2123
    #6 0x9c4443c in perform /path/to/source/src/net/resourcemanager/ResourceManagerCurlThread.cpp:854
    #7 0x9c445e0 in ...
    #8 0x9c4cf1d in ...
    #9 0xa2be6b5 in ...
    #10 0xf7aa5780 in asan_thread_start /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_interceptors.cc:226
    #11 0xf4d3a16d in __clone (/lib/i386-linux-gnu/libc.so.6+0xe716d)

previously allocated by thread T13 (RESOURCE_HTTP) here:
    #0 0xf7b1ba27 in __interceptor_calloc /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_malloc_linux.cc:70
    #1 0x9f7dfa6 in allocate_conn /path/to/source/3rdparty/curl/lib/url.c:3904
    #2 0x9f88ca0 in create_conn /path/to/source/3rdparty/curl/lib/url.c:5797
    #3 0x9f8c928 in Curl_connect /path/to/source/3rdparty/curl/lib/url.c:6438
    #4 0x9f45a8c in multi_runsingle /path/to/source/3rdparty/curl/lib/multi.c:1411
    #5 0x9f490f1 in curl_multi_perform /path/to/source/3rdparty/curl/lib/multi.c:2123
    #6 0x9c4443c in perform /path/to/source/src/net/resourcemanager/ResourceManagerCurlThread.cpp:854
    #7 0x9c445e0 in ...
    #8 0x9c4cf1d in ...
    #9 0xa2be6b5 in ...
    #10 0xf7aa5780 in asan_thread_start /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_interceptors.cc:226
    #11 0xf4d3a16d in __clone (/lib/i386-linux-gnu/libc.so.6+0xe716d)

SUMMARY: AddressSanitizer: heap-use-after-free /path/to/source/3rdparty/curl/lib/multi.c:666 in curl_multi_remove_handle
Shadow bytes around the buggy address:
  0x3d3e9f20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9f30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9f40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9f50: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
  0x3d3e9f60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x3d3e9f70:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9f80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9f90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9fa0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9fb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x3d3e9fc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==11785==ABORTING

Thread 14 "RESOURCE_HTTP" received signal SIGABRT, Aborted.
[Switching to Thread 0xf27bfb40 (LWP 12324)]
0xf7fd8be9 in __kernel_vsyscall ()
 (gdb) bt
 #0  0xf7fd8be9 in __kernel_vsyscall ()
 #1  0xf4c7ee89 in __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:54
 #2  0xf4c803e7 in __GI_abort () at abort.c:89
 #3  0xf7b2ef2e in __sanitizer::Abort () at /opt/toolchain/src/gcc-6.2.0/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cc:122
 #4  0xf7b262fa in __sanitizer::Die () at /opt/toolchain/src/gcc-6.2.0/libsanitizer/sanitizer_common/sanitizer_common.cc:145
 #5  0xf7b21ab3 in __asan::ScopedInErrorReport::~ScopedInErrorReport (this=0xf27be171, __in_chrg=<optimized out>) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_report.cc:689
 #6  0xf7b214a5 in __asan::ReportGenericError (pc=166993689, bp=4068206216, sp=4068206204, addr=3925146496, is_write=false, access_size=4, exp=0, fatal=true) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_report.cc:1074
 #7  0xf7b21fce in __asan::__asan_report_load4 (addr=3925146496) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_rtl.cc:129
 #8  0x09f41f19 in curl_multi_remove_handle (multi=0xf3406080, data=0xde582400) at /path/to/source3rdparty/curl/lib/multi.c:666
 #9  0x09f6b277 in Curl_close (data=0xde582400) at /path/to/source3rdparty/curl/lib/url.c:415
 #10 0x09f3354e in curl_easy_cleanup (data=0xde582400) at /path/to/source3rdparty/curl/lib/easy.c:860
 #11 0x09c6de3f in ...
 #12 0x09c378c5 in ...
 #13 0x09c48133 in ...
 #14 0x09c4d092 in ...
 #15 0x0a2be6b6 in ...
 #16 0xf7aa5781 in asan_thread_start (arg=0xf2d22938) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_interceptors.cc:226
 #17 0xf5de52b5 in start_thread (arg=0xf27bfb40) at pthread_create.c:333
 #18 0xf4d3a16e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114

Fixes #1083
2016-10-22 15:50:22 +02:00
Daniel Stenberg
5c3d8d20a6 curl_multi_add_handle: set timeouts in closure handles
The closure handle only ever has default timeouts set. To improve the
state somewhat we clone the timeouts from each added handle so that the
closure handle always has the same timeouts as the most recently added
easy handle.

Fixes #739
2016-10-19 14:43:49 +02:00
Daniel Stenberg
0f1996321f s/cURL/curl
The tool was never called cURL, only the project. But even so, we have
more and more over time switched to just use lower case.
2016-10-18 13:59:54 +02:00
Daniel Stenberg
88753c1e89 polarssl: indented code, removed unused variables 2016-10-18 13:27:51 +02:00
Daniel Stenberg
8748d4787f polarssl: reduce #ifdef madness with a macro 2016-10-18 13:17:05 +02:00
Daniel Stenberg
1e3161a20d polarssl: fix unaligned SSL session-id lock 2016-10-18 13:14:11 +02:00
Daniel Stenberg
ace237ae4e Curl_polarsslthreadlock_thread_setup: clear array at init
... since if it fails to init the entire array and then tries to clean
it up, it would attempt to work on an uninitialized pointer.
2016-10-18 13:05:43 +02:00
Daniel Stenberg
9a509680f6 gopher: properly return error for poll failures 2016-10-18 11:14:48 +02:00
Daniel Stenberg
8a6e89a9eb select: switch to macros in uppercase
Curl_select_ready() was the former API that was replaced with
Curl_select_check() a while back and the former arg setup was provided
with a define (in order to leave existing code unmodified).

Now we instead offer SOCKET_READABLE and SOCKET_WRITABLE for the most
common shortcuts where only one socket is checked. They're also more
visibly macros.
2016-10-18 11:05:45 +02:00
Daniel Stenberg
a05906190c select: use more proper macro-looking names
... so that it becomes more obvious in the code what is what. Also added
a typecast for one of the calculations.
2016-10-18 09:46:10 +02:00
Daniel Stenberg
9aa2afc3a5 Curl_socket_check: add extra check to avoid integer overflow 2016-10-18 09:45:34 +02:00
Jay Satiro
aec0c993cb url: skip to-be-closed connections when pipelining (follow-up)
- Change back behavior so that pipelining is considered possible for
connections that have not yet reached the protocol level.

This is a follow-up to e5f0b1a which had changed the behavior of
checking if pipelining is possible to ignore connections that had
'bits.close' set. Connections that have not yet reached the protocol
level also have that bit set, and we need to consider pipelining
possible on those connections.
2016-10-17 02:44:39 -04:00
Marcel Raad
422db18f15
win: fix Universal Windows Platform build
This fixes a merge error in commit 7f3df80 caused by commit 332e8d6.

Additionally, this changes Curl_verify_windows_version for Windows App
builds to assume to always be running on the target Windows version.
There seems to be no way to determine the Windows version from a
UWP app. Neither GetVersion(Ex), nor VerifyVersionInfo, nor the
Version Helper functions are supported.

Bug: https://github.com/curl/curl/pull/820#issuecomment-250889878
Reported-by: Paul Joyce

Closes https://github.com/curl/curl/pull/1048
2016-10-16 12:09:12 +02:00
Rider Linden
e5f0b1ab43 url: skip to-be-closed connections when pipelining
No longer attempt to use "doomed" to-be-closed connections when
pipelining. Prior to this change connections marked for deletion (e.g.
timeout) would be erroneously used, resulting in sporadic crashes.

As originally reported and fixed by Carlo Wood (origin unknown).

Bug: https://github.com/curl/curl/issues/627
Reported-by: Rider Linden

Closes https://github.com/curl/curl/pull/1075
Participation-by: nopjmp@users.noreply.github.com
2016-10-14 03:16:46 -04:00
Daniel Stenberg
1671d84b38 vtls: only re-use session-ids using the same scheme
To make it harder to do cross-protocol mistakes
2016-10-13 11:24:16 +02:00
Remo E
d522ff4690 cmake: add nghttp2 support
Closes #922
2016-10-10 19:47:31 +02:00
Andreas Streichardt
eb5199317e resolve: add error message when resolving using SIGALRM
Closes #1066
2016-10-10 14:14:20 +02:00
Tobias Stoeckmann
8272049062 smb: properly check incoming packet boundaries
Not all reply messages were properly checked for their lengths, which
made it possible to access uninitialized memory (but this does not lead
to out of boundary accesses).

Closes #1052
2016-10-09 00:14:26 +02:00
Daniel Stenberg
71588c9aef mprintf: return error on too many arguments
128 arguments should be enough for everyone
2016-10-08 20:47:04 +02:00
Daniel Stenberg
8238ba9c5f ftp: fix Curl_ftpsendf()
... it no longer takes printf() arguments since it was only really taken
advantage by one user and it was not written and used in a safe
way. Thus the 'f' is removed from the function name and the proto is
changed.

Although the current code wouldn't end up in badness, it was a risk that
future changes could end up springf()ing too large data or passing in a
format string inadvertently.
2016-10-08 15:13:46 +02:00
Daniel Stenberg
9885c9508e formpost: avoid silent snprintf() truncation
The previous use of snprintf() could make libcurl silently truncate some
input data and not report that back on overly large input, which could
make data get sent over the network in a bad format.

Example:

 $ curl --form 'a=b' -H "Content-Type: $(perl -e 'print "A"x4100')"
2016-10-08 13:00:45 +02:00
Daniel Stenberg
8fd2a754f0 memdup: use 'void *' as return and source type 2016-10-04 23:31:25 +02:00
Daniel Stenberg
bdf162af25 formpost: trying to attach a directory no longer crashes
The error path would previously add a freed entry to the linked list.

Reported-by: Toby Peterson

Fixes #1053
2016-10-04 09:13:02 +02:00
Sergei Kuzmin
54e48b14e1 cookies: same domain handling changed to match browser behavior
Cokie with the same domain but different tailmatching property are now
considered different and do not replace each other.  If header contains
following lines then two cookies will be set: Set-Cookie: foo=bar;
domain=.foo.com; expires=Thu Mar 3 GMT 8:56:27 2033 Set-Cookie: foo=baz;
domain=foo.com; expires=Thu Mar 3 GMT 8:56:27 2033

This matches Chrome, Opera, Safari, and Firefox behavior. When sending
stored tokens to foo.com Chrome, Opera, Firefox store send them in the
stored order, while Safari pre-sort the cookies.

Closes #1050
2016-10-03 16:49:35 +02:00
Michael Kaufmann
e9e5366193 New libcurl option to keep sending on error
Add the new option CURLOPT_KEEP_SENDING_ON_ERROR to control whether
sending the request body shall be completed when the server responds
early with an error status code.

This is suitable for manual NTLM authentication.

Reviewed-by: Jay Satiro

Closes https://github.com/curl/curl/pull/904
2016-09-22 22:22:31 +02:00
Kamil Dudka
d1f1c857ad nss: add chacha20-poly1305 cipher suites if supported by NSS 2016-09-22 10:20:12 +02:00
Kamil Dudka
049aa92546 nss: add cipher suites using SHA384 if supported by NSS 2016-09-22 10:17:26 +02:00
Kamil Dudka
7591220270 nss: fix typo in ecdhe_rsa_null cipher suite string
As it seems to be a rarely used cipher suite (for securely established
but _unencrypted_ connections), I believe it is fine not to provide an
alias for the misspelled variant.
2016-09-22 10:17:26 +02:00
Jay Satiro
22cfeac730 easy: Reset all statistical session info in curl_easy_reset
Bug: https://github.com/curl/curl/issues/1017
Reported-by: Jeroen Ooms
2016-09-20 01:14:01 -04:00
Daniel Gustafsson
79607eec51 darwinssl: Fix typo in comment
Closes https://github.com/curl/curl/pull/1028
2016-09-19 17:06:00 -04:00
Bernard Spil
01c8db47aa libressl: fix version output
LibreSSL defines `OPENSSL_VERSION_NUMBER` as `0x20000000L` for all
versions returning `LibreSSL/2.0.0` for any LibreSSL version.

This change provides a local OpenSSL_version_num function replacement
returning LIBRESSL_VERSION_NUMBER instead.

Closes #1029
2016-09-19 22:47:35 +02:00
Daniel Stenberg
a2b360f890 openssl: don't call ERR_remote_thread_state on >= 1.1.0
Follow-up fix to d9321562
2016-09-19 14:32:59 +02:00
Daniel Stenberg
de71e68000 openssl: don’t call CRYTPO_cleanup_all_ex_data
The OpenSSL function CRYTPO_cleanup_all_ex_data() cannot be called
multiple times without crashing - and other libs might call it! We
basically cannot call it without risking a crash. The function is a
no-op since OpenSSL 1.1.0.

Not calling this function only risks a small memory leak with OpenSSL <
1.1.0.

Bug: https://curl.haxx.se/mail/lib-2016-09/0045.html
Reported-by: Todd Short
2016-09-19 11:33:14 +02:00
Nick Zitzmann
227ee9d84d darwinssl: disable RC4 cipher-suite support
RC4 was a nice alternative to CBC back in the days of BEAST, but it's insecure and obsolete now.
2016-09-18 15:01:03 -05:00
Daniel Stenberg
f423a00971 http2: debug ouput sent HTTP/2 request headers 2016-09-16 09:00:20 +02:00
Daniel Stenberg
d4c5a91722 http: accept "Transfer-Encoding: chunked" for HTTP/2 as well
... but don't send the actual header over the wire as it isn't accepted.
Chunked uploading is still triggered using this method.

Fixes #1013
Fixes #662
2016-09-16 09:00:20 +02:00
Daniel Stenberg
d93215621f openssl: fix per-thread memory leak usiong 1.0.1 or 1.0.2
OpenSSL 1.0.1 and 1.0.2 build an error queue that is stored per-thread
so we need to clean it when easy handles are freed, in case the thread
will be killed in which the easy handle was used. All OpenSSL code in
libcurl should extract the error in association with the error already
so clearing this queue here should be harmless at worst.

Fixes #964
2016-09-14 14:35:44 +02:00
Daniel Stenberg
01cf1308ee curl_easy_unescape: deny negative string lengths as input
CVE-2016-7167

Bug: https://curl.haxx.se/docs/adv_20160914.html
2016-09-14 07:49:43 +02:00
Daniel Stenberg
826a9ced2b curl_easy_escape: deny negative string lengths as input
CVE-2016-7167

Bug: https://curl.haxx.se/docs/adv_20160914.html
2016-09-14 07:49:43 +02:00
Tony Kelman
665694979b mbedtls: switch off NTLM in build if md4 isn't available
NTLM support with mbedTLS was added in 497e7c9 but requires that mbedTLS
is built with the MD4 functions available, which it isn't in default
builds. This now adapts if the funtion isn't there and builds libcurl
without NTLM support if so.

Fixes #1004
2016-09-12 23:08:10 +02:00
Daniel Stenberg
a8e751a51a http: refuse to pass on response body with NO_NODY was set
... like when a HTTP/0.9 response comes back without any headers at all
and just a body this now prevents that body from being sent to the
callback etc.

Adapted test 1144 to verify.

Fixes #973

Assisted-by: Ray Satiro
2016-09-11 12:02:07 +02:00
Jakub Zakrzewski
6140dfcf3e CMake: Try to (un-)hide private library symbols
Detect support for compiler symbol visibility flags and apply those
according to CURL_HIDDEN_SYMBOLS option.
It should work true to the autotools build except it tries to unhide
symbols on Windows when requested and prints warning if it fails.

Ref: https://github.com/curl/curl/issues/981#issuecomment-242665951
Reported-by: Daniel Stenberg
2016-09-10 00:35:38 +02:00
Daniel Stenberg
83ef21e5e9 openssl: fix bad memory free (regression)
... by partially reverting f975f06033. The allocation could be made by
OpenSSL so the free must be made with OPENSSL_free() to avoid problems.

Reported-by: Harold Stuart
Fixes #1005
2016-09-09 23:35:10 +02:00
Daniel Stenberg
85033bcfcc http2: support > 64bit sized uploads
... by making sure we don't count down the "upload left" counter when the
uploaded size is unknown and then it can be allowed to continue forever.

Fixes #996
2016-09-09 15:41:39 +02:00
Jay Satiro
af2d679e14 errors: new alias CURLE_WEIRD_SERVER_REPLY (8)
Since we're using CURLE_FTP_WEIRD_SERVER_REPLY in imap, pop3 and smtp as
more of a generic "failed to parse" introduce an alias without FTP in
the name.

Closes https://github.com/curl/curl/pull/975
2016-09-07 21:24:27 -04:00
Daniel Stenberg
5871dfcc16 checksrc: detect strtok() use
... as that function slipped through once before.
2016-09-07 10:41:57 +02:00
Viktor Szakats
8fa20da8e1 mk-ca-bundle.pl: use SHA256 instead of SHA1
This hash is used to verify the original downloaded certificate bundle
and also included in the generated bundle's comment header. Also
rename related internal symbols to algorithm-agnostic names.
2016-09-07 10:41:11 +02:00
Gaurav Malhotra
8e176a7c79 openssl: fix CURLINFO_SSL_VERIFYRESULT
CURLINFO_SSL_VERIFYRESULT does not get the certificate verification
result when SSL_connect fails because of a certificate verification
error.

This fix saves the result of SSL_get_verify_result so that it is
returned by CURLINFO_SSL_VERIFYRESULT.

Closes https://github.com/curl/curl/pull/995
2016-09-06 16:17:37 -04:00
Daniel Gustafsson
022dbdb8ac darwinssl: test for errSecSuccess in PKCS12 import rather than noErr (#993)
While noErr and errSecSuccess are defined as the same value, the API
documentation states that SecPKCS12Import() returns errSecSuccess if
there were no errors in importing. Ensure that a future change of the
defined value doesn't break (however unlikely) and be consistent with
the API docs.
2016-09-06 10:37:31 +02:00
Marcel Raad
4af389353a openssl: Fix compilation with OPENSSL_API_COMPAT=0x10100000L
With OPENSSL_API_COMPAT=0x10100000L (OpenSSL 1.1 API), the cleanup
functions are unavailable (they're no-ops anyway in OpenSSL 1.1). The
replacements for SSL_load_error_strings, SSLeay_add_ssl_algorithms, and
OpenSSL_add_all_algorithms are called automatically [1][2]. SSLeay() is
now called OpenSSL_version_num().

[1]: https://www.openssl.org/docs/man1.1.0/ssl/OPENSSL_init_ssl.html
[2]: https://www.openssl.org/docs/man1.1.0/crypto/OPENSSL_init_crypto.html

Closes #992
2016-09-05 22:51:12 +02:00