This change fixes a regression where redirect body would needlessly be
decompressed even though it was to be ignored anyway. As it happens this
causes secondary issues since there appears to be a bug in apache2 that
it in certain conditions generates a corrupt zlib response. The
regression was created by commit:
dbcced8e32
Discovered-by: Harry Sintonen
Closes#2798
RNG structure must be freed by call to FreeRng after its use in
Curl_cyassl_random. This call fixes Valgrind failures when running the
test suite with wolfSSL.
Closes#2784
This fixes a memory leak when CURLOPT_LOGIN_OPTIONS is used, together with
connection reuse.
I found this with oss-fuzz on GDAL and curl master:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9582
I couldn't reproduce with the oss-fuzz original test case, but looking
at curl source code pointed to this well reproducable leak.
Closes#2790
In the current version, VERSION_GREATER_THAN_EQUAL 6.3 will return false
when run on windows 10.0. This patch addresses that error.
Closes https://github.com/curl/curl/pull/2792
So far, the code tries to pick an authentication method only if
user/password credentials are available, which is not the case for
Bearer authentictation...
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes#2754
The Bearer authentication was added to cURL 7.61.0, but there is a
problem: if CURLAUTH_ANY is selected, and the server supports multiple
authentication methods including the Bearer method, we strongly prefer
that latter method (only CURLAUTH_NEGOTIATE beats it), and if the Bearer
authentication fails, we will never even try to attempt any other
method.
This is particularly unfortunate when we already know that we do not
have any Bearer token to work with.
Such a scenario happens e.g. when using Git to push to Visual Studio
Team Services (which supports Basic and Bearer authentication among
other methods) and specifying the Personal Access Token directly in the
URL (this aproach is frequently taken by automated builds).
Let's make sure that we have a Bearer token to work with before we
select the Bearer authentication among the available authentication
methods.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes#2754
Follow-up to 1b76c38904. The VTLS backends that close down the TLS
layer for a connection still needs a Curl_easy handle for the session_id
cache etc.
Fixes#2764Closes#2771
... the protocol is doing read/write a lot, so it needs to write often
even when downloading. A more proper fix could check for eactly when it
wants to write and only ask for it then.
Without this fix, an SMB download could easily get stuck when the event-driven
API was used.
Closes#2768
Some servers issue raw deflate data that may be followed by an undocumented
trailer. This commit makes curl tolerate such a trailer of up to 4 bytes
before considering the data is in error.
Reported-by: clbr on github
Fixes#2719
It was previously erroneously skipped in some situations.
libtest/libntlmconnect.c wrongly depended on wrong behavior (that it
would get a zero timeout) when no handles are "running" in a multi
handle. That behavior is no longer present with this fix. Now libcurl
will always return a -1 timeout when all handles are completed.
Closes#2733
Commit 38203f1585 changed engine detection to be version-based,
with a baseline of openssl 1.0.1. This does in fact break builds
with openssl 1.0.0, which has engine support - the configure script
detects that ENGINE_cleanup() is available - but <openssl/engine.h>
doesn't get included to declare it.
According to upstream documentation, engine support was added to
mainstream openssl builds as of version 0.9.7:
https://github.com/openssl/openssl/blob/master/README.ENGINE
This commit drops the version test down to 1.0.0 as version 1.0.0d
is the oldest version I have to test with.
Closes#2732
MinGW warns:
/lib/vtls/schannel.c:219:64: warning: signed and unsigned type in
conditional expression [-Wsign-compare]
Fix this by casting the ptrdiff_t to size_t as we know it's positive.
Closes https://github.com/curl/curl/pull/2721
... not the read buffer size, as that can be set smaller and thus cause
a buffer overflow! CVE-2018-0500
Reported-by: Peter Wu
Bug: https://curl.haxx.se/docs/adv_2018-70a2.html
telnet.c(1401,28): warning: cast from function call of type 'int' to
non-matching type 'HANDLE' (aka 'void *') [-Wbad-function-cast]
Fixes#2696Closes#2700
The code treated the set version as the *exact* version to require in
the TLS handshake, which is not what other TLS backends do and probably
not what most people expect either.
Reported-by: Andreas Olsson
Assisted-by: Gaurav Malhotra
Fixes#2691Closes#2694
... and trim the threaded Curl_resolver_getsock() to return zero
millisecond wait times during the first three milliseconds so that
localhost or names in the OS resolver cache gets detected and used
faster.
Closes#2685
By masking sure to use the *current* easy handle with extracted
connections from the cache, and make sure to NULLify the ->data pointer
when the connection is put into the cache to make this mistake easier to
detect in the future.
Reported-by: Will Dietz
Fixes#2669Closes#2672
When the application just started the transfer and then stops it while
the name resolve in the background thread hasn't completed, we need to
wait for the resolve to complete and then cleanup data accordingly.
Enabled test 1553 again and added test 1590 to also check when the host
name resolves successfully.
Detected by OSS-fuzz.
Closes#1968
certdata.txt should be deleted also when the process is interrupted by
"same certificate downloaded, exiting"
The certdata.txt is currently kept on disk even if you give the -u
option
Closes#2655
with clang-6.0:
```
vtls/schannel_verify.c: In function 'add_certs_to_store':
vtls/schannel_verify.c:212:30: warning: passing argument 11 of 'CryptQueryObject' from incompatible pointer type [-Wincompatible-pointer-types]
&cert_context)) {
^
In file included from /usr/share/mingw-w64/include/schannel.h:10:0,
from /usr/share/mingw-w64/include/schnlsp.h:9,
from vtls/schannel.h:29,
from vtls/schannel_verify.c:40:
/usr/share/mingw-w64/include/wincrypt.h:4437:26: note: expected 'const void **' but argument is of type 'CERT_CONTEXT ** {aka struct _CERT_CONTEXT **}'
WINIMPM WINBOOL WINAPI CryptQueryObject (DWORD dwObjectType, const void *pvObject, DWORD dwExpectedContentTypeFlags, DWORD dwExpectedFormatTypeFlags, DWORD dwFlags,
^~~~~~~~~~~~~~~~
```
Ref: https://msdn.microsoft.com/library/windows/desktop/aa380264
Closes https://github.com/curl/curl/pull/2648
Given the contstraints of SChannel, I'm exposing these as the algorithms
themselves instead; while replicating the ciphersuite as specified by
OpenSSL would have been preferable, I found no way in the SChannel API
to do so.
To use this from the commandline, you need to pass the names of contants
defining the desired algorithms. For example, curl --ciphers
"CALG_SHA1:CALG_RSA_SIGN:CALG_RSA_KEYX:CALG_AES_128:CALG_DH_EPHEM"
https://github.com The specific names come from wincrypt.h
Closes#2630
- Get rid of variable that was generating false positive warning
(unitialized)
- Fix issues in tests
- Reduce scope of several variables all over
etc
Closes#2631
Previously it was checked for in configure/cmake, but that would then
leave other build systems built without engine support.
While engine support probably existed prior to 1.0.1, I decided to play
safe. If someone experience a problem with this, we can widen the
version check.
Fixes#2641Closes#2644
URL: https://curl.haxx.se/mail/lib-2018-06/0000.html
This is step one. It adds #error statements that require source edits to
make curl build again if asked to use axTLS. At a later stage we might
remove the axTLS specific code completely.
Closes#2628
... it might call infof() with a NULL first argument that isn't harmful
but makes it not do anything. The infof() line is not very useful
anymore, it has served it purpose. Good riddance!
Fixes#2627
If configure detects fnmatch to be available, use that instead of our
custom one for FTP wildcard pattern matching. For standard compliance,
to reduce our footprint and to use already well tested and well
exercised code.
A POSIX fnmatch behaves slightly different than the internal function
for a few test patterns currently and the macOS one yet slightly
different. Test case 1307 is adjusted for these differences.
Closes#2626
On our x86 Android toolchain, getpwuid_r is implemented but the header
is missing:
netrc.c:81:7: error: implicit declaration of function 'getpwuid_r' [-Werror=implicit-function-declaration]
Unfortunately, the function is used in curl_ntlm_wb.c, too, so I moved
the prototype to curl_setup.h.
Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
Closes#2609
Adds CURLOPT_TLS13_CIPHERS and CURLOPT_PROXY_TLS13_CIPHERS.
curl: added --tls13-ciphers and --proxy-tls13-ciphers
Fixes#2435
Reported-by: zzq1015 on github
Closes#2607
A non-escaped bracket ([) is for a character group - as documented. It
will *not* match an individual bracket anymore. Test case 1307 updated
accordingly to match.
Problem detected by OSS-Fuzz, although this fix is probably not a final
fix for the notorious timeout issues.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8525Closes#2614
The latest psl is cached in the multi or share handle. It is refreshed
before use after 72 hours.
New share lock CURL_LOCK_DATA_PSL controls the psl cache sharing.
If the latest psl is not available, the builtin psl is used.
Reported-by: Yaakov Selkowitz
Fixes#2553Closes#2601
This avoids appending error data to already existing good data.
Test 92 is updated to match this change.
New test 1156 checks all combinations of --range/--resume, --fail,
Content-Range header and http status code 200/416.
Fixes#1163
Reported-By: Ithubg on github
Closes#2578
OpenSSL has supported --cacert for ages, always accepting LF-only line
endings ("Unix line endings") as well as CR/LF line endings ("Windows
line endings").
When we introduced support for --cacert also with Secure Channel (or in
cURL speak: "WinSSL"), we did not take care to support CR/LF line
endings, too, even if we are much more likely to receive input in that
form when using Windows.
Let's fix that.
Happily, CryptQueryObject(), the function we use to parse the ca-bundle,
accepts CR/LF input already, and the trailing LF before the END
CERTIFICATE marker catches naturally any CR/LF line ending, too. So all
we need to care about is the BEGIN CERTIFICATE marker. We do not
actually need to verify here that the line ending is CR/LF. Just
checking for a CR or an LF is really plenty enough.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes https://github.com/curl/curl/pull/2592
The previous limit of 5 can still end up in situation that takes a very
long time and consumes a lot of CPU.
If there is still a rare use case for this, a user can provide their own
fnmatch callback for a version that allows a larger set of wildcards.
This commit was triggered by yet another OSS-Fuzz timeout due to this.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8369Closes#2587
Provide a set of new timers that return the time intervals using integer
number of microseconds instead of floats.
The new info names are as following:
CURLINFO_APPCONNECT_TIME_T
CURLINFO_CONNECT_TIME_T
CURLINFO_NAMELOOKUP_TIME_T
CURLINFO_PRETRANSFER_TIME_T
CURLINFO_REDIRECT_TIME_T
CURLINFO_STARTTRANSFER_TIME_T
CURLINFO_TOTAL_TIME_T
Closes#2495
Original MinGW targets Windows 2000 by default, which lacks some APIs and
definitions for this feature. Disable it if these APIs are not available.
Closes https://github.com/curl/curl/pull/2522
Response data for a handle with a large buffer might be cached and then
used with the "closure" handle when it has a smaller buffer and then the
larger cache will be copied and overflow the new smaller heap based
buffer.
Reported-by: Dario Weisser
CVE: CVE-2018-1000300
Bug: https://curl.haxx.se/docs/adv_2018-82c2.html
RFC 6265 section 4.2.1 does not set restrictions on cookie names.
This is a follow-up to commit 7f7fcd0.
Also explicitly check proper syntax of cookie name/value pair.
New test 1155 checks that cookie names are not reserved words.
Reported-By: anshnd at github
Fixes#2564Closes#2566
To make builds with VS2015 work. Recent changes in VS2015 _IOB_ENTRIES
handling is causing problems. This fix changes the OpenSSL backend code
to use BIO functions instead of FILE I/O functions to circumvent those
problems.
Closes#2512
... instead of previous separate struct fields, to make it easier to
extend and change individual backends without having to modify them all.
closes#2547
Curl_setup_transfer() can be called to setup a new individual transfer
over a multiplexed connection so it shouldn't unset writesockfd.
Bug: #2520Closes#2549
ssh-libssh.c:2429:21: warning: result of '1 << 31' requires 33 bits to
represent, but 'int' only has 32 bits [-Wshift-overflow=]
'len' will never be that big anyway so I converted the run-time check to
a regular assert.
Commit 3c630f9b0a partially reverted the
changes from commit dd7521bcc1 because of
the problem that strcpy_url() was modified unilaterally without also
modifying strlen_url(). As a consequence strcpy_url() was again
depending on ASCII encoding.
This change fixes strlen_url() and strcpy_url() in parallel to use a
common host-encoding independent criterion for deciding whether an URL
character must be %-escaped.
Closes#2535
This extends the INDENTATION case to also handle 'else' statements
and require proper indentation on the following line. Also fixes the
offending cases found in the codebase.
Closes#2532
This function can get called on a connection that isn't setup enough to
have the 'recv_underlying' function pointer initialized so it would try
to call the NULL pointer.
Reported-by: Dario Weisser
Follow-up to db1b2c7fe9 (never shipped in a release)
Closes#2536
Follow-up to 1514c44655: replace another strstr() call done on a
buffer that might not be zero terminated - with a memchr() call, even if
we know the substring will be found.
Assisted-by: Max Dymond
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8021Closes#2534
With commit 4272a0b0fc curl-speficic
character classification macros and functions were introduced in
curl_ctype.[ch] to avoid dependencies on the locale. This broke curl on
non-ASCII, e.g. EBCDIC platforms. This change restores the previous set
of character classification macros when CURL_DOES_CONVERSIONS is
defined.
Closes#2494
Fuzzing has proven we can reach code in on_frame_recv with status_code
not having been set, so let's detect that in run-time (instead of with
assert) and error error accordingly.
(This should no longer happen with the latest nghttp2)
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7903Closes#2514
This reverts commit 8fb78f9ddc.
Unfortunately this fix introduces memory leaks I've not been able to fix
in several days. Reverting this for now to get the leaks fixed.
When receiving REFUSED_STREAM, mark the connection for close and retry
streams accordingly on another/fresh connection.
Reported-by: Terry Wu
Fixes#2416Fixes#1618Closes#2510
It's not strictly clear if the API contract allows us to call strstr()
on a string that isn't zero terminated even when we know it will find
the substring, and clang's ASAN check dislikes us for it.
Also added a check of the return code in case it fails, even if I can't
think of a situation how that can trigger.
Detected by OSS-Fuzz
Closes#2513
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7760
Curl_cert_hostcheck operates with the host character set, therefore the
ASCII subjectAltName string retrieved with OpenSSL must be converted to
the host encoding before comparison.
Closes#2493
This triggered an assert if called more than once in debug mode (and a
memory leak if not debug build). With the right sequence of HTTP/2
headers incoming it can happen.
Detected by OSS-Fuzz
Closes#2507
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7764
- Move verify_certificate functionality in schannel.c into a new
file called schannel_verify.c. Additionally, some structure defintions
from schannel.c have been moved to schannel.h to allow them to be
used in schannel_verify.c.
- Make verify_certificate functionality for Schannel available on
all versions of Windows instead of just Windows CE. verify_certificate
will be invoked on Windows CE or when the user specifies
CURLOPT_CAINFO and CURLOPT_SSL_VERIFYPEER.
- In verify_certificate, create a custom certificate chain engine that
exclusively trusts the certificate store backed by the CURLOPT_CAINFO
file.
- doc updates of --cacert/CAINFO support for schannel
- Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
when available. This implements a TODO in schannel.c to improve
handling of multiple SANs in a certificate. In particular, all SANs
will now be searched instead of just the first name.
- Update tool_operate.c to not search for the curl-ca-bundle.crt file
when using Schannel to maintain backward compatibility. Previously,
any curl-ca-bundle.crt file found in that search would have been
ignored by Schannel. But, with CAINFO support, the file found by
that search would have been used as the certificate store and
could cause issues for any users that have curl-ca-bundle.crt in
the search path.
- Update url.c to not set the build time CURL_CA_BUNDLE if the selected
SSL backend is Schannel. We allow setting CA location for schannel
only when explicitly specified by the user via CURLOPT_CAINFO /
--cacert.
- Add new test cases 3000 and 3001. These test cases check that the first
and last SAN, respectively, matches the connection hostname. New test
certificates have been added for these cases. For 3000, the certificate
prefix is Server-localhost-firstSAN and for 3001, the certificate
prefix is Server-localhost-secondSAN.
- Remove TODO 15.2 (Add support for custom server certificate
validation), this commit addresses it.
Closes https://github.com/curl/curl/pull/1325
- Fix warning 'integer from pointer without a cast' on 3rd arg in
CertOpenStore. The arg type HCRYPTPROV may be a pointer or integer
type of the same size.
Follow-up to e35b025.
Caught by Marc's CI builds.
Users can now specify a client certificate in system certificates store
explicitly using expression like `--cert "CurrentUser\MY\<thumbprint>"`
Closes#2376
If you pass empty user/pass asking curl to use Windows Credential
Storage (as stated in the docs) and it has valid credentials for the
domain, e.g.
curl -v -u : --ntlm example.com
currently authentication fails.
This change fixes it by providing proper SPN string to the SSPI API
calls.
Fixes https://github.com/curl/curl/issues/1622
Closes https://github.com/curl/curl/pull/1660
The ifdefs have become quite long. Also, the condition for the
definition of CURLOPT_SERVICE_NAME and for setting it from
CURLOPT_SERVICE_NAME have diverged. We will soon also need the two
options for NTLM, at least when using SSPI, for
https://github.com/curl/curl/pull/1660.
Just make the definitions unconditional to make that easier.
Closes https://github.com/curl/curl/pull/2479
When a zeroed out allocation is required, use calloc() rather than
malloc() followed by an explicit memset(). The result will be the
same, but using calloc() everywhere increases consistency in the
codebase and avoids the risk of subtle bugs when code is injected
between malloc and memset by accident.
Closes https://github.com/curl/curl/pull/2497
In debug mode, MingGW-w64's GCC 7.3 issues null-dereference warnings
when dereferencing pointers after DEBUGASSERT-ing that they are not
NULL.
Fix this by removing the DEBUGASSERTs.
Suggested-by: Daniel Stenberg
Ref: https://github.com/curl/curl/pull/2463
unit1309 and vtls/gtls: error: arithmetic on a null pointer treated as a
cast from integer to pointer is a GNU extension
Reported-by: Rikard Falkeborn
Fixes#2466Closes#2468
In the situation of a client connecting to an FTP server using an IPv6
tunnel proxy, the connection info will indicate that the connection is
IPv6. However, because the server behing the proxy is IPv4, it is
permissable to attempt PSV mode. In the case of the FTP server being
IPv4 only, EPSV will always fail, and with the current logic curl will
be unable to connect to the server, as the IPv6 fwdproxy causes curl to
think that EPSV is impossible.
Closes#2432
curl 7.57.0 and up interpret this according to Appendix E.3.2 of RFC
8089 but then returns an error saying this is unimplemented. This is
actually a regression in behavior on both Windows and Unix.
Before curl 7.57.0 this URL was treated as a path of "//foo/bar" and
then passed to the relevant OS API. This means that the behavior of this
case is actually OS dependent.
The Unix path resolution rules say that the OS must handle swallowing
the extra "/" and so this path is the same as "/foo/bar"
The Windows path resolution rules say that this is a UNC path and
automatically handles the SMB access for the program. So curl on Windows
was already doing Appendix E.3.2 without any special code in curl.
Regression
Closes#2438
This reverts commit dc85437736.
libcurl (with the OpenSSL backend) performs server certificate verification
even if verifypeer == 0 and the verification result is available using
CURLINFO_SSL_VERIFYRESULT. The commit that is being reverted caused the
CURLINFO_SSL_VERIFYRESULT to not have useful information for the
verifypeer == 0 use case (it would always have
X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY).
Closes#2451
This fixes a segfault occurring when a name of the (invalid) form "domain..tld"
is processed.
test46 updated to cover this case.
Follow-up to commit c990ead.
Ref: https://github.com/curl/curl/pull/2440
In order to make curl_multi_timeout() return suitable "sleep" times even
when there's no socket to wait for while the name is being resolved in a
helper thread.
It will increases the timeouts as time passes.
Closes#2419
If a connection has received a GOAWAY frame while not being used, the
function now reads frames off the connection before trying to reuse it
to avoid reusing connections the server has told us not to use.
Reported-by: Alex Baines
Fixes#1967Closes#2402
Currently CMake cannot detect Brotli support. This adds detection of the
libraries and associated header files. It also adds this to the
generated config.
Closes#2392
This patch adds CURLOPT_DNS_SHUFFLE_ADDRESSES to explicitly request
shuffling of IP addresses returned for a hostname when there is more
than one. This is useful when the application knows that a round robin
approach is appropriate and is willing to accept the consequences of
potentially discarding some preference order returned by the system's
implementation.
Closes#1694
When a transfer is requested to get done and it is put in the pending
queue when limited by number of connections, total or per-host, libcurl
would previously very aggressively retry *ALL* pending transfers to get
them transferring. That was very time consuming.
By reducing the aggressiveness in how pending are being retried, we
waste MUCH less time on putting transfers back into pending again.
Some test cases got a factor 30(!) speed improvement with this change.
Reported-by: Cyril B
Fixes#2369Closes#2383
Especially unpausing a transfer might have to move the socket back to the
"currently used sockets" hash to get monitored. Otherwise it would never get
any more data and get stuck. Easily triggered with pausing using the
multi_socket API.
Reported-by: Philip Prindeville
Bug: https://curl.haxx.se/mail/lib-2018-03/0048.htmlFixes#2393Closes#2391
Due to very frequent updates of the rate limit "window", it could
attempt to rate limit within the same milliseconds and that then made
the calculations wrong, leading to it not behaving correctly on very
fast transfers.
This new logic updates the rate limit "window" to be no shorter than the
last three seconds and only updating the timestamps for this when
switching between the states TOOFAST/PERFORM.
Reported-by: 刘佩东
Fixes#2386Closes#2388
Refuse to operate when given path components featuring byte values lower
than 32.
Previously, inserting a %00 sequence early in the directory part when
using the 'singlecwd' ftp method could make curl write a zero byte
outside of the allocated buffer.
Test case 340 verifies.
CVE-2018-1000120
Reported-by: Duy Phan Thanh
Bug: https://curl.haxx.se/docs/adv_2018-9cd6.html
gss_seal/gss_unseal have been deprecated in favor of
gss_wrap/gss_unwrap with GSS-API v2 from January 1997 [1]. The first
version of "The Kerberos Version 5 GSS-API Mechanism" [2] from June
1996 already says "GSS_Wrap() (formerly GSS_Seal())" and
"GSS_Unwrap() (formerly GSS_Unseal())".
Use the nondeprecated functions to avoid deprecation warnings.
[1] https://tools.ietf.org/html/rfc2078
[2] https://tools.ietf.org/html/rfc1964
Closes https://github.com/curl/curl/pull/2356
On MinGW and Cygwin, GCC and clang have been complaining about macro
redefinitions since 4272a0b0fc. Fix this
by undefining the macros before redefining them as suggested in
https://github.com/curl/curl/pull/2269.
Suggested-by: Daniel Stenberg
When targeting x64, MinGW-w64 complains about conversions between
32-bit long and 64-bit pointers. Fix this by reusing the
GNUTLS_POINTER_TO_SOCKET_CAST / GNUTLS_SOCKET_TO_POINTER_CAST logic
from gtls.c, moving it to warnless.h as CURLX_POINTER_TO_INTEGER_CAST /
CURLX_INTEGER_TO_POINTER_CAST.
Closes https://github.com/curl/curl/pull/2341
- Add new option CURLOPT_RESOLVER_START_FUNCTION to set a callback that
will be called every time before a new resolve request is started
(ie before a host is resolved) with a pointer to backend-specific
resolver data. Currently this is only useful for ares.
- Add new option CURLOPT_RESOLVER_START_DATA to set a user pointer to
pass to the resolver start callback.
Closes https://github.com/curl/curl/pull/2311
- In keeping with the naming of our other connect timeout options rename
CURLOPT_HAPPY_EYEBALLS_TIMEOUT to CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS.
This change adds the _MS suffix since the option expects milliseconds.
This is more intuitive for our users since other connect timeout options
that expect milliseconds use _MS such as CURLOPT_TIMEOUT_MS,
CURLOPT_CONNECTTIMEOUT_MS, CURLOPT_ACCEPTTIMEOUT_MS.
The tool option already uses an -ms suffix, --happy-eyeballs-timeout-ms.
Follow-up to 2427d94 which added the lib and tool option yesterday.
Ref: https://github.com/curl/curl/pull/2260
- Add new option CURLOPT_HAPPY_EYEBALLS_TIMEOUT to set libcurl's happy
eyeball timeout value.
- Add new optval macro CURL_HET_DEFAULT to represent the default happy
eyeballs timeout value (currently 200 ms).
- Add new tool option --happy-eyeballs-timeout-ms to expose
CURLOPT_HAPPY_EYEBALLS_TIMEOUT. The -ms suffix is used because the
other -timeout options in the tool expect seconds not milliseconds.
Closes https://github.com/curl/curl/pull/2260
This enables users to preresolve but still take advantage of happy
eyeballs and trying multiple addresses if some are not connecting.
Ref: https://github.com/curl/curl/pull/2260
Previously, it would only check for max length if the existing alloc
buffer was to small to fit it, which often would make the header still
get used.
Reported-by: Guido Berhoerster
Bug: https://curl.haxx.se/mail/lib-2018-02/0056.htmlCloses#2315
The list of state names (used in debug builds) was out of sync in
relation to the list of states (used in all builds).
I now added an assert to make sure the sizes of the two lists match, to
aid in detecting this mistake better in the future.
Regression since c92d2e14cf, shipped in 7.58.0.
Reported-by: Somnath Kundu
Fixes#2312Closes#2313
RFC 5321 4.1.1.4 specifies the CRLF terminating the DATA command
should be taken into account when chasing the <CRLF>.<CRLF> end marker.
Thus a leading dot character in data is also subject to escaping.
Tests 911 and test server are adapted to this situation.
New tests 951 and 952 check proper handling of initial dot in data.
Closes#2304
Some servers return a "content-encoding" header with a non-standard
"none" value.
Add "none" as an alias to "identity" as a work-around, to avoid
unrecognised content encoding type errors.
Signed-off-by: Mohammad AlSaleh <CE.Mohammad.AlSaleh@gmail.com>
Closes https://github.com/curl/curl/pull/2298
Windows 10.0.17061 SDK introduces support for Unix Domain Sockets.
Added the necessary include file to curl_addrinfo.c.
Note: The SDK (which is considered beta) has to be installed, VS 2017
project file has to be re-targeted for Windows 10.0.17061 and #define
enabled in config-win32.h.
When peer verification is disabled, calling
SSL_CTX_load_verify_locations is not necessary. Only call it when
verification is enabled to save resources and increase performance.
Closes#2290
Reduce code duplication by making Curl_mime_contenttype available and
used by the formdata function. This also makes the formdata function
recognize a set of more file extensions by default.
PR #2280 brought this to my attention.
Closes#2282
Whenever an expected pattern syntax rule cannot be matched, the
character starting the rule loses its special meaning and the parsing
is resumed:
- backslash at the end of pattern string matches itself.
- Error in [:keyword:] results in set containing :\[dekorwy.
Unit test 1307 updated for this new situation.
Closes#2273
... since the libc provided one are locale dependent in a way we don't
want. Also, the "native" isalnum() (for example) works differently on
different platforms which caused test 1307 failures on macos only.
Closes#2269
Make curl_getdate() handle dates before 1970 as well (returning negative
values).
Make test 517 test dates for 64 bit time_t.
This fixes bug (3) mentioned in #2238Closes#2250
... unless CURLOPT_UNRESTRICTED_AUTH is set to allow them. This matches how
curl already handles Authorization headers created internally.
Note: this changes behavior slightly, for the sake of reducing mistakes.
Added test 317 and 318 to verify.
Reported-by: Craig de Stigter
Bug: https://curl.haxx.se/docs/adv_2018-b3bf.html
In case an identity didn't match[0], the state machine would fail in
state SSH_AUTH_AGENT instead of progressing to the next identity in
ssh-agent. As a result, ssh-agent authentication only worked if the
identity required happened to be the first added to ssh-agent.
This was introduced as part of commit c4eb10e2f0, which
stated that the "else" statement was required to prevent getting stuck
in state SSH_AUTH_AGENT. Given the state machine's logic and libssh2's
interface I couldn't see how this could happen or reproduce it and I
also couldn't find a more detailed description of the problem which
would explain a test case to reproduce the problem this was supposed to
fix.
[0] libssh2_agent_userauth returning LIBSSH2_ERROR_AUTHENTICATION_FAILED
Closes#2248
Follow-up to 84fcaa2e7. libressl does not have the API even if it says it is
late OpenSSL version...
Fixes#2246Closes#2247
Reported-by: jungle-boogie on github
1. don't use "ULL" suffix since unsupported in older MSVC
2. use curl_off_t instead of custom long long ifdefs
3. make get_posix_time() not do unaligned data access
Fixes#2211Closes#2240
Reported-by: Chester Liu
A mime tree attached to an easy handle using CURLOPT_MIMEPOST is
strongly bound to the handle: there is a pointer to the easy handle in
each item of the mime tree and following the parent pointer list
of mime items ends in a dummy part stored within the handle.
Because of this binding, a mime tree cannot be shared between different
easy handles, thus it needs to be cloned upon easy handle duplication.
There is no way for the caller to get the duplicated mime tree
handle: it is then set to be automatically destroyed upon freeing the
new easy handle.
New test 654 checks proper mime structure duplication/release.
Add a warning note in curl_mime_data_cb() documentation about sharing
user data between duplicated handles.
Closes#2235
Prior to this change the stored byte count of each trailer was
miscalculated and 1 less than required. It appears any trailer
after the first that was passed to Curl_client_write would be truncated
or corrupted as well as the size. Potentially the size of some
subsequent trailer could be erroneously extracted from the contents of
that trailer, and since that size is used by client write an
out-of-bounds read could occur and cause a crash or be otherwise
processed by client write.
The bug appears to have been born in 0761a51 (precedes 7.49.0).
Closes https://github.com/curl/curl/pull/2231
Decoding loop implementation did not concern the case when all
received data is consumed by Brotli decoder and the size of decoded
data internally hold by Brotli decoder is greater than CURL_MAX_WRITE_SIZE.
For content with unencoded length greater than CURL_MAX_WRITE_SIZE this
can result in the loss of data at the end of content.
Closes#2194
Move curl_mime_initpart() and curl_mime_cleanpart() calls to lower-level
functions dealing with UserDefined structure contents.
This avoids memory leakages on curl-generated part mime headers.
New test 2073 checks this using the cli tool --next option: it
triggers a valgrind error if bug is present.
Bug: https://curl.haxx.se/mail/lib-2017-12/0060.html
Reported-by: Martin Galvan
- When zlib version is < 1.2.0.4, process gzip trailer before considering
extra data as an error.
- Inflate with Z_BLOCK instead of Z_SYNC_FLUSH to maximize correct data
and minimize corrupt data output.
- Do not try to restart deflate decompression in raw mode if output has
started or if the leading data is not available anymore.
- New test 232 checks inflating raw-deflated content.
Closes#2068
scan-build would warn on a potential access of an uninitialized
buffer. I deem it a false positive and had to add this somewhat ugly
work-around to silence it.
Fixed undefined symbol of getenv() which does not exist when compiling
for Windows 10 App (CURL_WINDOWS_APP). Replaced getenv() with
curl_getenv() which is aware of getenv() absence when CURL_WINDOWS_APP
is defined.
Closes#2171
Prune the DNS cache immediately after the dns entry is unlocked in
multi_done. Timed out entries will then get discarded in a more orderly
fashion.
Test506 is updated
Reported-by: Oleg Pudeyev
Fixes#2169Closes#2170
Prior to this change SSLKEYLOGFILE used line buffering on WIN32 just
like it does for other platforms. However, the Windows CRT does not
actually support line buffering (_IOLBF) and will use full buffering
(_IOFBF) instead. We can't use full buffering because multiple processes
may be writing to the file and that could lead to corruption, and since
full buffering is the only buffering available this commit disables
buffering for Windows SSLKEYLOGFILE entirely (_IONBF).
Ref: https://github.com/curl/curl/pull/1346#issuecomment-350530901
These are OS/2-specific things added to the code in the year 2000. They
were always ugly. If there's any user left, they still don't need it
done this way.
Closes#2166
- Allow proxy_ssl to be checked for pending data even when connssl does
not yet have an SSL handle.
This change is for posterity. Currently there doesn't seem to be a code
path that will cause a pending data check when proxyssl could have
pending data and the connssl handle doesn't yet exist [1].
[1]: Recall that an https proxy connection starts out in connssl but if
the destination is also https then the proxy SSL backend data is moved
from connssl to proxyssl, which means connssl handle is temporarily
empty until an SSL handle for the destination can be created.
Ref: https://github.com/curl/curl/commit/f4a6238#commitcomment-24396542
Closes https://github.com/curl/curl/pull/1916
Connections that are used for HTTP/1.1 Pipelining or HTTP/2 multiplexing
only get additional transfers added to them if the existing connection
is held by the same multi or easy handle. libcurl does not support doing
HTTP/2 streams in different threads using a shared connection.
Closes#2152
If the lock is released before the dealings with the bundle is over, it may
have changed by another thread in the mean time.
Fixes#2132Fixes#2151Closes#2139
For pop3/imap/smtp, added test 891 to somewhat verify the pop3
case.
For this, I enhanced the pingpong test server to be able to send back
responses with LF-only instead of always using CRLF.
Closes#2150
Figured out while reviewing code in the libssh backend. The pointer was
checked for NULL after having been dereferenced, so we know it would
always equal true or it would've crashed.
Pointed-out-by: Nikos Mavrogiannopoulos
Bug #2143Closes#2148
The previous code was incorrectly following the libssh2 error detection
for libssh2_sftp_statvfs, which is not correct for libssh's sftp_statvfs.
Fixes#2142
Signed-off-by: Nikos Mavrogiannopoulos <nmav@gnutls.org>
The SFTP back-end supports asynchronous reading only, limited
to 32-bit file length. Writing is synchronous with no other
limitations.
This also brings keyboard-interactive authentication.
Signed-off-by: Nikos Mavrogiannopoulos <nmav@gnutls.org>
That also updates tests to expect the right error code
libssh2 back-end returns CURLE_SSH error if the remote file
is not found. Expect instead CURLE_REMOTE_FILE_NOT_FOUND
which is sent by the libssh backend.
Signed-off-by: Nikos Mavrogiannopoulos <nmav@redhat.com>
libssh is an alternative library to libssh2.
https://www.libssh.org/
That patch set also introduces support for ECDSA
ed25519 keys, as well as gssapi authentication.
Signed-off-by: Nikos Mavrogiannopoulos <nmav@redhat.com>
Absent any 'symbol map' or script to limit what gets exported, static
linking of libraries previously resulted in a libcurl with curl's and
those other symbols being (re-)exported.
This did not happen if 'versioned symbols' were enabled (which is not
the default) because then a version script is employed.
This limits exports to everything starting in 'curl_*'., which is
what "libcurl.vers" exports.
This avoids strange side-effects such as with mixing methods
from system libraries and those erroneously offered by libcurl.
Closes#2127
Originally, my idea was to allocate the two structures (or more
precisely, the connectdata structure and the four SSL backend-specific
strucutres required for ssl[0..1] and proxy_ssl[0..1]) in one go, so
that they all could be free()d together.
However, getting the alignment right is tricky. Too tricky.
So let's just bite the bullet and allocate the SSL backend-specific
data separately.
As a consequence, we now have to be very careful to release the memory
allocated for the SSL backend-specific data whenever we release any
connectdata.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes#2119
commit d3ab7c5a21 broke the boringssl build since it doesn't have
RSA_flags(), so we disable that code block for boringssl builds.
Reported-by: W. Mark Kubacki
Fixes#2117
This bit is no longer used. It is not clear what it meant for users to
"init the TLS" in a world with different TLS backends and since the
introduction of multissl, libcurl didn't properly work if inited without
this bit set.
Not a single user responded to the call for users of it:
https://curl.haxx.se/mail/lib-2017-11/0072.html
Reported-by: Evgeny Grin
Assisted-by: Jay Satiro
Fixes#2089Fixes#2083Closes#2107
- Align the array of ssl_backend_data on a max 32 byte boundary.
8 is likely to be ok but I went with 32 for posterity should one of
the ssl_backend_data structs change to contain a larger sized variable
in the future.
Prior to this change (since dev 70f1db3, release 7.56) the connectdata
structure was undersized by 4 bytes in 32-bit builds with ssl enabled
because long long * was mistakenly used for alignment instead of
long long, with the intention being an 8 byte boundary. Also long long
may not be an available type.
The undersized connectdata could lead to oob read/write past the end in
what was expected to be the last 4 bytes of the connection's secondary
socket https proxy ssl_backend_data struct (the secondary socket in a
connection is used by ftp, others?).
Closes https://github.com/curl/curl/issues/2093
CVE-2017-8818
Bug: https://curl.haxx.se/docs/adv_2017-af0a.html
With this check present, scan-build warns that we might dereference this
point in other places where it isn't first checked for NULL. Thus, if it
*can* be NULL we have a problem on a few places. However, this pointer
should not be possible to be NULL here so I remove the check and thus
also three different scan-build warnings.
Closes#2111
* LOTS of comment updates
* explicit error for SMB shares (e.g. "file:////share/path/file")
* more strict handling of authority (i.e. "//localhost/")
* now accepts dodgy old "C:|" drive letters
* more precise handling of drive letters in and out of Windows
(especially recognising both "file:c:/" and "file:/c:/")
Closes#2110
The new API added in Linux 4.11 only requires setting a socket option
before connecting, without the whole sento() machinery.
Notably, this makes it possible to use TFO with SSL connections on Linux
as well, without the need to mess around with OpenSSL (or whatever other
SSL library) internals.
Closes#2056
Host names like "127.0.0.1 moo" would otherwise be accepted by some
getaddrinfo() implementations.
Updated test 1034 and 1035 accordingly.
Fixes#2073Closes#2092
... so that IPv6 addresses can be passed like they can for connect-to
and how they're used in URLs.
Added test 1324 to verify
Reported-by: Alex Malinovich
Fixes#2087Closes#2091
There is a conflict on symbol 'free_func' between openssl/crypto.h and
zlib.h on AIX. This is an attempt to resolve it.
Bug: https://curl.haxx.se/mail/lib-2017-11/0032.html
Reported-By: Michael Felt
The --interface command (CURLOPT_INTERFACE option) already uses
SO_BINDTODEVICE on Linux, but it tries to parse it as an interface or IP
address first, which fails in case the user passes a VRF.
Try to use the socket option immediately and parse it as a fallback
instead. Update the documentation to mention this feature, and that it
requires the binary to be ran by root or with CAP_NET_RAW capabilities
for this to work.
Closes#2024
... previously it would store it already in the happy eyeballs stage
which could lead to the IPv6 bit being set for an IPv4 connection,
leading to curl not wanting to do EPSV=>PASV for FTP transfers.
Closes#2053
- Don't call zlib's inflate() when avail_in stream bytes is 0.
This is a follow up to the parent commit 19e66e5. Prior to that change
libcurl's inflate_stream could call zlib's inflate even when no bytes
were available, causing inflate to return Z_BUF_ERROR, and then
inflate_stream would treat that as a hard error and return
CURLE_BAD_CONTENT_ENCODING.
According to the zlib FAQ, Z_BUF_ERROR is not fatal.
This bug would happen randomly since packet sizes are arbitrary. A test
of 10,000 transfers had 55 fail (ie 0.55%).
Ref: https://zlib.net/zlib_faq.html#faq05
Closes https://github.com/curl/curl/pull/2060
This uses the brotli external library (https://github.com/google/brotli).
Brotli becomes a feature: additional curl_version_info() bit and
structure fields are provided for it and CURLVERSION_NOW bumped.
Tests 314 and 315 check Brotli content unencoding with correct and
erroneous data.
Some tests are updated to accomodate with the now configuration dependent
parameters of the Accept-Encoding header.
This is implemented as an output streaming stack of unencoders, the last
calling the client write procedure.
New test 230 checks this feature.
Bug: https://github.com/curl/curl/pull/2002
Reported-By: Daniel Bankhead
Since CURLSSH_AUTH_ANY (aka CURLSSH_AUTH_DEFAULT) is ~0 an arg value
check on this option is incorrect; we have to accept any value.
Prior to this change since f121575 (7.56.1+) CURLOPT_SSH_AUTH_TYPES
erroneously rejected CURLSSH_AUTH_ANY with CURLE_BAD_FUNCTION_ARGUMENT.
Bug: https://github.com/curl/curl/commit/f121575#commitcomment-25347120
... which is valid according to documentation. Regression since
f121575c0b.
Verified now in test 501.
Reported-by: cbartl on github
Fixes#2038Closes#2039
.. also add same arg value check to CURLOPT_POSTFIELDSIZE_LARGE.
Prior to this change since f121575 (7.56.1+) CURLOPT_POSTFIELDSIZE
erroneously rejected -1 value with CURLE_BAD_FUNCTION_ARGUMENT.
Bug: https://curl.haxx.se/mail/lib-2017-11/0000.html
Reported-by: Andrew Lambert
The config files define curl and libcurl targets as imported targets
CURL::curl and CURL::libcurl. For backward compatibility with CMake-
provided find-module the CURL_INCLUDE_DIRS and CURL_LIBRARIES are
also set.
Closes#1879
returning 'time_t' is problematic when that type is unsigned and we
return values less than zero to signal "already expired", used in
several places in the code.
Closes#2021
If WINAPI_FAMILY is defined, it should be safe to try to include
winapifamily.h to check what the define evaluates to.
This should fix detection of CURL_WINDOWS_APP if building with
_WIN32_WINNT set to 0x0600.
Closes#2025
- When uploading via chunked-encoding don't compare file size to bytes
sent to determine whether the upload has finished.
Chunked-encoding adds its own overhead which why the bytes sent is not
equal to the file size. Prior to this change if a file was uploaded in
chunked-encoding and its size was known it was possible that the upload
could end prematurely without sending the final few chunks. That would
result in a server hang waiting for the remaining data, likely followed
by a disconnect.
The scope of this bug is limited to some arbitrary file sizes which have
not been determined. One size that triggers the bug is 475020.
Bug: https://github.com/curl/curl/issues/2001
Reported-by: moohoorama@users.noreply.github.com
Closes https://github.com/curl/curl/pull/2010
... by using curl_off_t for the typedef if time_t is larger than 4
bytes.
Reported-by: Gisle Vanem
Bug: b9d25f9a6b (co)
mmitcomment-25205058
Closes#2019
... since the 'tv' stood for timeval and this function does not return a
timeval struct anymore.
Also, cleaned up the Curl_timediff*() functions to avoid typecasts and
clean up the descriptive comments.
Closes#2011
... to cater for systems with unsigned time_t variables.
- Renamed the functions to curlx_timediff and Curl_timediff_us.
- Added overflow protection for both of them in either direction for
both 32 bit and 64 bit time_ts
- Reprefixed the curlx_time functions to use Curl_*
Reported-by: Peter Piekarski
Fixes#2004Closes#2005
... that are multiplied by 1000 when stored.
For 32 bit long systems, the max value accepted (2147483 seconds) is >
596 hours which is unlikely to ever be set by a legitimate application -
and previously it didn't work either, it just caused undefined behavior.
Also updated the man pages for these timeout options to mention the
return code.
Closes#1938
Allow to ovverride certain build tools, making it possible to
use LLVM/Clang to build curl. The default behavior is unchanged.
To build with clang (as offered by MSYS2), these settings can
be used:
CURL_CC=clang
CURL_AR=llvm-ar
CURL_RANLIB=llvm-ranlib
Closes https://github.com/curl/curl/pull/1993
Use memset() to initialize a structure to avoid LLVM/Clang warning:
ldap.c:193:39: warning: missing field 'UserLength' initializer [-Wmissing-field-initializers]
Closes https://github.com/curl/curl/pull/1992
Now VERIFYHOST, VERIFYPEER and VERIFYSTATUS options change during active
connection updates the current connection's (i.e.'connectdata'
structure) appropriate ssl_config (and ssl_proxy_config) structures
variables, making these options effective for ongoing connection.
This functionality was available before and was broken by the
following change:
"proxy: Support HTTPS proxy and SOCKS+HTTP(s)"
CommitId: cb4e2be7c6.
Bug: https://github.com/curl/curl/issues/1941
Closes https://github.com/curl/curl/pull/1951
Those were temporary things we'd add and remove for our own convenience
long ago. The last few stayed around for too long as an oversight but
have since been removed. These days we have a running
BORINGSSL_API_VERSION counter which is bumped when we find it
convenient, but 2015-11-19 was quite some time ago, so just check
OPENSSL_IS_BORINGSSL.
Closes#1979
This reverts commit f3e03f6c0a.
Caused memory leaks in the fuzzer, needs to be done differently.
Disable test 1553 for now too, as it causes memory leaks without this
commit!
When imap_done() got called before a connection is setup, it would try
to "finish up" and dereffed a NULL pointer.
Test case 1553 managed to reproduce. I had to actually use a host name
to try to resolve to slow it down, as using the normal local server IP
will make libcurl get a connection in the first curl_multi_perform()
loop and then the bug doesn't trigger.
Fixes#1953
Assisted-by: Max Dymond
... fixes a memory leak with at least IMAP when remove_handle is never
called and the transfer is abruptly just abandoned early.
Test 1552 added to verify
Detected by OSS-fuzz
Assisted-by: Max Dymond
Closes#1954
The source code is now prepared to handle the case when both
Win32 Crypto and OpenSSL/NSS crypto backends are enabled
at the same time, making it now possible to enable `USE_WIN32_CRYPTO`
whenever the targeted Windows version supports it. Since this
matches the minimum Windows version supported by curl
(Windows 2000), enable it unconditionally for the Win32 platform.
This in turn enables SMB (and SMBS) protocol support whenever
Win32 Crypto is available, regardless of what other crypto backends
are enabled.
Ref: https://github.com/curl/curl/pull/1840#issuecomment-325682052
Closes https://github.com/curl/curl/pull/1943
- New `CURL_DLL_SUFFIX` envvar will add a suffix to the generated
libcurl dll name. Useful to add `-x64` to 64-bit builds so that
it can live in the same directory as the 32-bit one. By default
this is empty.
- New `CURL_DLL_A_SUFFIX` envvar to customize the suffix of the
generated import library (implib) for libcurl .dll. It defaults
to `dll`, and it's useful to modify that to `.dll` to have the
standard naming scheme for mingw-built .dlls, i.e. `libcurl.dll.a`.
Closes https://github.com/curl/curl/pull/1942
Compare these settings in Curl_ssl_config_matches():
- verifystatus (CURLOPT_SSL_VERIFYSTATUS)
- random_file (CURLOPT_RANDOM_FILE)
- egdsocket (CURLOPT_EGDSOCKET)
Also copy the setting "verifystatus" in Curl_clone_primary_ssl_config(),
and copy the setting "sessionid" unconditionally.
This means that reusing connections that are secured with a client
certificate is now possible, and the statement "TLS session resumption
is disabled when a client certificate is used" in the old advisory at
https://curl.haxx.se/docs/adv_20170419.html is obsolete.
Reviewed-by: Daniel Stenberg
Closes#1917
... a single double quote could leave the entry path buffer without a zero
terminating byte. CVE-2017-1000254
Test 1152 added to verify.
Reported-by: Max Dymond
Bug: https://curl.haxx.se/docs/adv_20171004.html
When curl and libcurl are built with some protocols disabled, they stop
setting and receiving some options that don't make sense with those
protocols. In particular, when HTTP is disabled many options aren't set
that are used only by HTTP. However, some options that appear to be
HTTP-only are actually used by other protocols as well (some despite
having HTTP in the name) and should be set, but weren't. This change now
causes some of these options to be set and used for more (or for all)
protocols. In particular, this fixes tests 646 through 649 in an
HTTP-disabled build, which use the MIME API in the mail protocols.
The timer should be started after conn->connecttime is set. Otherwise
the timer could expire without this condition being true:
/* should we try another protocol family? */
if(i == 0 && conn->tempaddr[1] == NULL &&
curlx_tvdiff(now, conn->connecttime) >= HAPPY_EYEBALLS_TIMEOUT) {
Ref: #1928
A connection can only be reused if the flags "conn_to_host" and
"conn_to_port" match. Therefore it is not necessary to copy these flags
in reuse_conn().
Closes#1918
.. and include the core NTLM header in all NTLM-related source files.
Follow up to 6f86022. Since then http_ntlm checks NTLM_NEEDS_NSS_INIT
but did not include vtls.h where it was defined.
Closes https://github.com/curl/curl/pull/1911
With the recently introduced MultiSSL support multiple SSL backends
can be compiled into cURL That means that now the order of the SSL
One option would be to use the same SSL backend as was configured
via `curl_global_sslset()`, however, NTLMv2 support would appear
to be available only with some SSL backends. For example, when
eb88d778e (ntlm: Use Windows Crypt API, 2014-12-02) introduced
support for NTLMv1 using Windows' Crypt API, it specifically did
*not* introduce NTLMv2 support using Crypt API at the same time.
So let's select one specific SSL backend for NTLM support when
compiled with multiple SSL backends, using a priority order such
that we support NTLMv2 even if only one compiled-in SSL backend can
be used for that.
Ref: https://github.com/curl/curl/pull/1848
In some cases the RSA key does not support verifying it because it's
located on a smart card, an engine wants to hide it, ...
Check the flags on the key before trying to verify it.
OpenSSL does the same thing internally; see ssl/ssl_rsa.c
Closes#1904
... instead of truncating them.
There's no fixed limit for acceptable cookie names in RFC 6265, but the
entire cookie is said to be less than 4096 bytes (section 6.1). This is
also what browsers seem to implement.
We now allow max 5000 bytes cookie header. Max 4095 bytes length per
cookie name and value. Name + value together may not exceed 4096 bytes.
Added test 1151 to verify
Bug: https://curl.haxx.se/mail/lib-2017-09/0062.html
Reported-by: Kevin Smith
Closes#1894
lib/vtls/openssl.c uses OpenSSL APIs from BUF_MEM and BIO APIs. Include
their headers directly rather than relying on other OpenSSL headers
including things.
Closes https://github.com/curl/curl/pull/1891
If the INTERLEAVEFUNCTION is defined, then use that plus the
INTERLEAVEDATA information when writing RTP. Otherwise, use
WRITEFUNCTION and WRITEDATA.
Fixes#1880Closes#1884
If the default write callback is used and no destination has been set, a
NULL pointer would be passed to fwrite()'s 4th argument.
OSS-fuzz bug https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3327
(not publicly open yet)
Detected by OSS-fuzz
Closes#1874
`conn->connect_state` is NULL when doing a regular non-CONNECT request
over the proxy and should therefor be considered complete at once.
Fixes#1853Closes#1862
Reported-by: Lawrence Wagerfield
Another mistake in my manual fixups of the largely mechanical
search-and-replace ("connssl->" -> "BACKEND->"), just like the previous
commit concerning HTTPS proxies (and hence not caught during my
earlier testing).
Fixes#1855Closes#1871
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In d65e6cc4f (vtls: prepare the SSL backends for encapsulated private
data, 2017-06-21), this developer prepared for a separation of the
private data of the SSL backends from the general connection data.
This conversion was partially automated (search-and-replace) and
partially manual (e.g. proxy_ssl's backend data).
Sadly, there was a crucial error in the manual part, where the wrong
handle was used: rather than connecting ssl[sockindex]' BIO to the
proxy_ssl[sockindex]', we reconnected proxy_ssl[sockindex]. The reason
was an incorrect location to paste "BACKEND->"... d'oh.
Reported by Jay Satiro in https://github.com/curl/curl/issues/1855.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Ever since 70f1db321 (vtls: encapsulate SSL backend-specific data,
2017-07-28), the code handling HTTPS proxies was broken because the
pointer to the SSL backend data was not swapped between
conn->ssl[sockindex] and conn->proxy_ssl[sockindex] as intended, but
instead set to NULL (causing segmentation faults).
[jes: provided the commit message, tested and verified the patch]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
... instead of the prefix-less version since WolfSSL 3.12 now uses an
enum with that name that causes build failures for us.
Fixes#1865Closes#1867
Reported-by: Gisle Vanem
- The part kind MIMEKIND_FILE and associated code are suppressed.
- Seek data origin offset not used anymore: suppressed.
- MIMEKIND_NAMEDFILE renamed MIMEKIND_FILE; associated fields/functions
renamed accordingly.
- Curl_getformdata() processes stdin via a callback.
Back in 2008, (and commit 3f3d6ebe66) we changed the logic in how we
determine the native type for `curl_off_t`. To really make sure we
didn't break ABI without bumping SONAME, we introduced logic that
attempted to detect that it would use a different size and thus not be
compatible. We also provided a manual switch that allowed users to tell
configure to bump SONAME by force.
Today, we know of no one who ever got a SONAME bump auto-detected and we
don't know of anyone who's using the manual bump feature. The auto-
detection is also no longer working since we introduced defining
curl_off_t in system.h (7.55.0).
Finally, this bumping logic is not present in the cmake build.
Closes#1861
This is an adaptation of 2 of Peter Wu's SSLKEYLOGFILE implementations.
The first one, written for old OpenSSL versions:
https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.c
The second one, written for BoringSSL and new OpenSSL versions:
https://github.com/curl/curl/pull/1346
Note the first one is GPL licensed but the author gave permission to
waive that license for libcurl.
As of right now this feature is disabled by default, and does not have
a configure option to enable it. To enable this feature define
ENABLE_SSLKEYLOGFILE when building libcurl and set environment
variable SSLKEYLOGFILE to a pathname that will receive the keys.
And in Wireshark change your preferences to point to that key file:
Edit > Preferences > Protocols > SSL > Master-Secret
Co-authored-by: Peter Wu
Ref: https://github.com/curl/curl/pull/1030
Ref: https://github.com/curl/curl/pull/1346
Closes https://github.com/curl/curl/pull/1866
curl_mime_encoder() is operational and documented.
curl tool -F option is extended with ";encoder=".
curl tool --libcurl option generates calls to curl_mime_encoder().
New encoder tests 648 & 649.
Test 1404 extended with an encoder specification.
Up2date versions of OpenSSL maintain the default reasonably secure
without breaking compatibility, so it is better not to override the
default by curl. Suggested at https://bugzilla.redhat.com/1483972Closes#1846
To support telling a string is nul-terminated, symbol CURL_ZERO_TERMINATED
has been introduced.
Documentation updated accordingly.
symbols in versions updated. Added form API symbols deprecation info.
This feature is badly supported in Windows: as a replacement, a caller has
to use curl_mime_data_cb() with fread, fseek and possibly fclose
callbacks to process opened files.
The cli tool and documentation are updated accordingly.
The feature is however kept internally for form API compatibility, with
the known caveats it always had.
As a side effect, stdin size is not determined by the cli tool even if
possible and this results in a chunked transfer encoding. Test 173 is
updated accordingly.
Some calls in different modules were setting the data handle to NULL, causing
segmentation faults when using builds that enable character code conversions.
destroy_async_data() assumes that if the flag "done" is not set yet, the
thread itself will clean up once the request is complete. But if an
error (generally OOM) occurs before the thread even has a chance to
start, it will never get a chance to clean up and memory will be leaked.
By clearing "done" only just before starting the thread, the correct
cleanup sequence will happen in all cases.
Previously, we used as default SSL backend whatever was first in the
`available_backends` array.
However, some users may want to override that default without patching
the source code.
Now they can: with the --with-default-ssl-backend=<backend> option of
the ./configure script.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When only one SSL backend is configured, it is totally unnecessary to
let multissl_init() configure the backend at runtime, we can select the
correct backend at build time already.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Let's add a compile time safe API to select an SSL backend. This
function needs to be called *before* curl_global_init(), and can be
called only once.
Side note: we do not explicitly test that it is called before
curl_global_init(), but we do verify that it is not called multiple times
(even implicitly).
If SSL is used before the function was called, it will use whatever the
CURL_SSL_BACKEND environment variable says (or default to the first
available SSL backend), and if a subsequent call to
curl_global_sslset() disagrees with the previous choice, it will fail
with CURLSSLSET_TOO_LATE.
The function also accepts an "avail" parameter to point to a (read-only)
NULL-terminated list of available backends. This comes in real handy if
an application wants to let the user choose between whatever SSL backends
the currently available libcurl has to offer: simply call
curl_global_sslset(-1, NULL, &avail);
which will return CURLSSLSET_UNKNOWN_BACKEND and populate the avail
variable to point to the relevant information to present to the user.
Just like with the HTTP/2 push functions, we have to add the function
declaration of curl_global_sslset() function to the header file
*multi.h* because VMS and OS/400 require a stable order of functions
declared in include/curl/*.h (where the header files are sorted
alphabetically). This looks a bit funny, but it cannot be helped.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
There is information about the compiled-in SSL backends that is really
no concern of any code other than the SSL backend itself, such as which
function (if any) implements SHA-256 summing.
And there is information that is really interesting to the user, such as
the name, or the curl_sslbackend value.
Let's factor out the latter into a publicly visible struct. This
information will be used in the upcoming API to set the SSL backend
globally.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When building software for the masses, it is sometimes not possible to
decide for all users which SSL backend is appropriate.
Git for Windows, for example, uses cURL to perform clones, fetches and
pushes via HTTPS, and some users strongly prefer OpenSSL, while other
users really need to use Secure Channel because it offers
enterprise-ready tools to manage credentials via Windows' Credential
Store.
The current Git for Windows versions use the ugly work-around of
building libcurl once with OpenSSL support and once with Secure Channel
support, and switching out the binaries in the installer depending on
the user's choice.
Needless to say, this is a super ugly workaround that actually only
works in some cases: Git for Windows also comes in a portable form, and
in a form intended for third-party applications requiring Git
functionality, in which cases this "swap out libcurl-4.dll" simply is
not an option.
Therefore, the Git for Windows project has a vested interest in teaching
cURL to make the SSL backend a *runtime* option.
This patch makes that possible.
By running ./configure with multiple --with-<backend> options, cURL will
be built with multiple backends.
For the moment, the backend can be configured using the environment
variable CURL_SSL_BACKEND (valid values are e.g. "openssl" and
"schannel").
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
So far, all of the SSL backends' private data has been declared as
part of the ssl_connect_data struct, in one big #if .. #elif .. #endif
block.
This can only work as long as the SSL backend is a compile-time option,
something we want to change in the next commits.
Therefore, let's encapsulate the exact data needed by each SSL backend
into a private struct, and let's avoid bleeding any SSL backend-specific
information into urldata.h. This is also necessary to allow multiple SSL
backends to be compiled in at the same time, as e.g. OpenSSL's and
CyaSSL's headers cannot be included in the same .c file.
To avoid too many malloc() calls, we simply append the private structs
to the connectdata struct in allocate_conn().
This requires us to take extra care of alignment issues: struct fields
often need to be aligned on certain boundaries e.g. 32-bit values need to
be stored at addresses that divide evenly by 4 (= 32 bit / 8
bit-per-byte).
We do that by assuming that no SSL backend's private data contains any
fields that need to be aligned on boundaries larger than `long long`
(typically 64-bit) would need. Under this assumption, we simply add a
dummy field of type `long long` to the `struct connectdata` struct. This
field will never be accessed but acts as a placeholder for the four
instances of ssl_backend_data instead. the size of each ssl_backend_data
struct is stored in the SSL backend-specific metadata, to allow
allocate_conn() to know how much extra space to allocate, and how to
initialize the ssl[sockindex]->backend and proxy_ssl[sockindex]->backend
pointers.
This would appear to be a little complicated at first, but is really
necessary to encapsulate the private data of each SSL backend correctly.
And we need to encapsulate thusly if we ever want to allow selecting
CyaSSL and OpenSSL at runtime, as their headers cannot be included within
the same .c file (there are just too many conflicting definitions and
declarations for that).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
At the moment, cURL's SSL backend needs to be configured at build time.
As such, it is totally okay for them to hard-code their backend-specific
data in the ssl_connect_data struct.
In preparation for making the SSL backend a runtime option, let's make
the access of said private data a bit more abstract so that it can be
adjusted later in an easy manner.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In 86b889485 (sasl_gssapi: Added GSS-API based Kerberos V5 variables,
2014-12-03), an SSPI-specific field was added to the kerberos5data
struct without moving the #include "curl_sspi.h" later in the same file.
This broke the build when SSPI was enabled, unless Secure Channel was
used as SSL backend, because it just so happens that Secure Channel also
requires "curl_sspi.h" to be #included.
In f4739f639 (urldata: include curl_sspi.h when Windows SSPI is enabled,
2017-02-21), this bug was fixed incorrectly: Instead of moving the
appropriate conditional #include, the Secure Channel-conditional part
was now also SSPI-conditional.
Fix this problem by moving the correct #include instead.
This is also required for an upcoming patch that moves all the Secure
Channel-specific stuff out of urldata.h and encapsulates it properly in
vtls/schannel.c instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since 5017d5ada (polarssl: now require 1.3.0+, 2014-03-17), we require
a newer PolarSSL version. No need to keep code trying to support any
older version.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In the ongoing endeavor to abstract out all SSL backend-specific
functionality, this is the next step: Instead of hard-coding how the
different SSL backends access their internal data in getinfo.c, let's
implement backend-specific functions to do that task.
This will also allow for switching SSL backends as a runtime option.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
There are convenient no-op versions of the init/cleanup functions now,
no need to define private ones for axTLS.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
These functions are all available via the Curl_ssl struct now, no need
to declare them separately anymore.
As the global declarations are removed, the corresponding function
definitions are marked as file-local. The only two exceptions here are
Curl_mbedtls_shutdown() and Curl_polarssl_shutdown(): only the
declarations were removed, there are no function definitions to mark
file-local.
Please note that Curl_nss_force_init() is *still* declared globally, as
the only SSL backend-specific function, because it was introduced
specifically for the use case where cURL was compiled with
`--without-ssl --with-nss`. For details, see f3b77e561 (http_ntlm: add
support for NSS, 2010-06-27).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The _shutdown() function calls the _session_free() function; While this
is not a problem now (because schannel.h declares both functions), a
patch looming in the immediate future with make all of these functions
file-local.
So let's just move the _session_free() function's definition before it
is called.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The connect_finish() function (like many other functions after it) calls
the Curl_axtls_close() function; While this is not a problem now
(because axtls.h declares the latter function), a patch looming in the
immediate future with make all of these functions file-local.
So let's just move the Curl_axtls_close() function's definition before
it is called.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The entire idea of introducing the Curl_ssl struct to describe SSL
backends is to prepare for choosing the SSL backend at runtime.
To that end, convert all the #ifdef have_curlssl_* style conditionals
to use bit flags instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The SHA-256 checksumming is also an SSL backend-specific function.
Let's include it in the struct declaring the functionality of SSL
backends.
In contrast to MD5, there is no fall-back code. To indicate this, the
respective entries are NULL for those backends that offer no support for
SHA-256 checksumming.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The MD5 summing is also an SSL backend-specific function. So let's
include it, offering the previous fall-back code as a separate function
now: Curl_none_md5sum(). To allow for that, the signature had to be
changed so that an error could be returned from the implementation
(Curl_none_md5sum() can run out of memory).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is the first step to unify the SSL backend handling. Now all the
SSL backend-specific functionality is accessed via a global instance of
the Curl_ssl struct.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The idea of introducing the Curl_ssl struct was to unify how the SSL
backends are declared and called. To this end, we now provide an
instance of the Curl_ssl struct for each and every SSL backend.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This new struct is similar in nature to Curl_handler: it will define the
functions and capabilities of all the SSL backends (where Curl_handler
defines the functions and capabilities of protocol handlers).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch makes the signature of the _sha256sum() functions consistent
among the SSL backends, in preparation for unifying the way all SSL
backends are accessed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch makes the signature of the _data_pending() functions
consistent among the SSL backends, in preparation for unifying the way
all SSL backends are accessed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch makes the signature of the _cleanup() functions consistent
among the SSL backends, in preparation for unifying the way all SSL
backends are accessed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
... as the previous fixed length 128 bytes buffer was sometimes too
small.
Fixes#1823Closes#1831
Reported-by: Benjamin Sergeant
Assisted-by: Bill Pyne, Ray Satiro, Nick Zitzmann
Recent changes that replaced CURL_SIZEOF_LONG in the source with
SIZEOF_LONG broke builds that use the premade configuration files and
don't have SIZEOF_LONG defined.
Bug: https://github.com/curl/curl/issues/1816
libidn was replaced with libidn2 last year in configure.
Caveat: libidn2 may depend on a list of further libs.
These can be manually specified via CURL_LDFLAG_EXTRAS.
Closes https://github.com/curl/curl/pull/1815
Recent changes that replaced CURL_SIZEOF_LONG in the source with
SIZEOF_LONG broke builds that use the premade configuration files and
don't have SIZEOF_LONG defined.
Closes https://github.com/curl/curl/pull/1814
Fixes
$ valgrind --leak-check=full ~/install-curl-git/bin/curl tftp://localhost/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaz
==9752== Memcheck, a memory error detector
==9752== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==9752== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==9752== Command: /home/even/install-curl-git/bin/curl tftp://localhost/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaz
==9752==
curl: (71) TFTP file name too long
==9752==
==9752== HEAP SUMMARY:
==9752== 505 bytes in 1 blocks are definitely lost in loss record 11 of 11
==9752== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9752== by 0x4E61CED: Curl_urldecode (in /home/even/install-curl-git/lib/libcurl.so.4.4.0)
==9752== by 0x4E75868: tftp_state_machine (in /home/even/install-curl-git/lib/libcurl.so.4.4.0)
==9752== by 0x4E761B6: tftp_do (in /home/even/install-curl-git/lib/libcurl.so.4.4.0)
==9752== by 0x4E711B6: multi_runsingle (in /home/even/install-curl-git/lib/libcurl.so.4.4.0)
==9752== by 0x4E71D00: curl_multi_perform (in /home/even/install-curl-git/lib/libcurl.so.4.4.0)
==9752== by 0x4E6950D: curl_easy_perform (in /home/even/install-curl-git/lib/libcurl.so.4.4.0)
==9752== by 0x40E0B7: operate_do (in /home/even/install-curl-git/bin/curl)
==9752== by 0x40E849: operate (in /home/even/install-curl-git/bin/curl)
==9752== by 0x402693: main (in /home/even/install-curl-git/bin/curl)
Fixes https://oss-fuzz.com/v2/testcase-detail/5232311106797568
Credit to OSS Fuzz
Closes#1808
Since curl 7.55.0, NetworkManager almost always failed its connectivity
check by timeout. I bisected this to 5113ad04 (http-proxy: do the HTTP
CONNECT process entirely non-blocking).
This patch replaces !Curl_connect_complete with Curl_connect_ongoing,
which returns false if the CONNECT state was left uninitialized and lets
the connection continue.
Closes#1803Fixes#1804
Also-fixed-by: Gergely Nagy
The required low-level logic was already available as part of
`libssh2` (via `LIBSSH2_FLAG_COMPRESS` `libssh2_session_flag()`[1]
option.)
This patch adds the new `libcurl` option `CURLOPT_SSH_COMPRESSION`
(boolean) and the new `curl` command-line option `--compressed-ssh`
to request this `libssh2` feature. To have compression enabled, it
is required that the SSH server supports a (zlib) compatible
compression method and that `libssh2` was built with `zlib` support
enabled.
[1] https://www.libssh2.org/libssh2_session_flag.html
Ref: https://github.com/curl/curl/issues/1732
Closes https://github.com/curl/curl/pull/1735
This change does two things:
1. It un-breaks the build in Xcode 9.0. (Xcode 9.0 is currently
failing trying to compile connectx() in lib/connect.c.)
2. It finally weak-links the connectx() function, and falls back on
connect() when run on older operating systems.
Update the progress timers `t_nslookup`, `t_connect`, `t_appconnect`,
`t_pretransfer`, and `t_starttransfer` to track the total times for
these activities when a redirect is followed. Previously, only the times
for the most recent request would be tracked.
Related changes:
- Rename `Curl_pgrsResetTimesSizes` to `Curl_pgrsResetTransferSizes`
now that the function only resets transfer sizes and no longer
modifies any of the progress timers.
- Add a bool to the `Progress` struct that is used to prevent
double-counting `t_starttransfer` times.
Added test case 1399.
Fixes#522 and Known Bug 1.8
Closes#1602
Reported-by: joshhe on github
Fixes the below leak:
$ valgrind --leak-check=full ~/install-curl-git/bin/curl --proxy "http://a:b@/x" http://127.0.0.1
curl: (5) Couldn't resolve proxy name
==5048==
==5048== HEAP SUMMARY:
==5048== in use at exit: 532 bytes in 12 blocks
==5048== total heap usage: 5,288 allocs, 5,276 frees, 445,271 bytes allocated
==5048==
==5048== 2 bytes in 1 blocks are definitely lost in loss record 1 of 12
==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5048== by 0x4E6CB79: parse_login_details (url.c:5614)
==5048== by 0x4E6BA82: parse_proxy (url.c:5091)
==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346)
==5048== by 0x4E6EA18: create_conn (url.c:6498)
==5048== by 0x4E6F9B4: Curl_connect (url.c:6967)
==5048== by 0x4E86D05: multi_runsingle (multi.c:1436)
==5048== by 0x4E88432: curl_multi_perform (multi.c:2160)
==5048== by 0x4E7C515: easy_transfer (easy.c:708)
==5048== by 0x4E7C74A: easy_perform (easy.c:794)
==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813)
==5048== by 0x414025: operate_do (tool_operate.c:1563)
==5048==
==5048== 2 bytes in 1 blocks are definitely lost in loss record 2 of 12
==5048== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5048== by 0x4E6CBB6: parse_login_details (url.c:5621)
==5048== by 0x4E6BA82: parse_proxy (url.c:5091)
==5048== by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346)
==5048== by 0x4E6EA18: create_conn (url.c:6498)
==5048== by 0x4E6F9B4: Curl_connect (url.c:6967)
==5048== by 0x4E86D05: multi_runsingle (multi.c:1436)
==5048== by 0x4E88432: curl_multi_perform (multi.c:2160)
==5048== by 0x4E7C515: easy_transfer (easy.c:708)
==5048== by 0x4E7C74A: easy_perform (easy.c:794)
==5048== by 0x4E7C7B1: curl_easy_perform (easy.c:813)
==5048== by 0x414025: operate_do (tool_operate.c:1563)
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2984
Credit to OSS Fuzz for discovery
Closes#1761
... and thereby avoid telling send() to send off more bytes than the
size of the buffer!
CVE-2017-1000100
Bug: https://curl.haxx.se/docs/adv_20170809B.html
Reported-by: Even Rouault
Credit to OSS-Fuzz for the discovery
First: this function is only used in debug-builds and not in
release/real builds. It is used to drive tests using the event-based
API.
A pointer to the local struct is passed to CURLMOPT_TIMERDATA, but the
CURLMOPT_TIMERFUNCTION calback can in fact be called even after this
funtion returns, namely when curl_multi_remove_handle() is called.
Reported-by: Brian Carpenter
When multiple rounds are needed to establish a security context
(usually ntlm), we overwrite old token with a new one without free.
Found by proposed gss tests using stub a gss implementation (by
valgrind error), though I have confirmed the leak with a real
gssapi implementation as well.
Closes https://github.com/curl/curl/pull/1733
clang complains:
vtls/darwinssl.c:40:8: error: extra tokens at end of #endif directive
[-Werror,-Wextra-tokens]
This breaks the darwinssl build on Travis. Fix it by making this token
a comment.
Closes https://github.com/curl/curl/pull/1734
The MSVC warning level defaults to 3 in CMake. Change it to 4, which is
consistent with the Visual Studio and NMake builds. Disable level 4
warning C4127 for the library and additionally C4306 for the test
servers to get a clean CURL_WERROR build as that warning is raised in
some macros in older Visual Studio versions.
Ref: https://github.com/curl/curl/pull/1667#issuecomment-314082794
Closes https://github.com/curl/curl/pull/1711
Use LongToHandle to convert from long to HANDLE in the Win32
implementation.
This should fix the following warning when compiling with
MSVC 11 (2012) in 64-bit mode:
lib\curl_threads.c(113): warning C4306:
'type cast' : conversion from 'long' to 'HANDLE' of greater size
Closes https://github.com/curl/curl/pull/1717
There are some bugs in how timers are managed for a single easy handle
that causes the wrong "next timeout" value to be reported to the
application when a new minimum needs to be recomputed and that new
minimum should be an existing timer that isn't currently set for the
easy handle. When the application drives a set of easy handles via the
`curl_multi_socket_action()` API (for example), it gets told to wait the
wrong amount of time before the next call, which causes requests to
linger for a long time (or, it is my guess, possibly forever).
Bug: https://curl.haxx.se/mail/lib-2017-07/0033.html
The headers of librtmp declare the socket as `int`, and on Windows, that
disagrees with curl_socket_t.
Bug: #1652
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
... to make all libcurl internals able to use the same data types for
the struct members. The timeval struct differs subtly on several
platforms so it makes it cumbersome to use everywhere.
Ref: #1652Closes#1693
... causing a SIGSEGV in showit() in case the handle used to initiate
the connection has already been freed.
This commit fixes a bug introduced in curl-7_19_5-204-g5f0cae803.
Reported-by: Rob Sanders
Bug: https://bugzilla.redhat.com/1436158
This is a follow-up to af02162 which removed (SET_)ERRNO macros. That
commit was an earlier draft that I committed by mistake, which was then
remedied by a5834e5 and e909de6, and now this commit. With this commit
there is now no difference between the current code and the changes that
were approved in the final draft.
Thanks-to: Max Dymond, Marcel Raad, Daniel Stenberg, Gisle Vanem
Ref: https://github.com/curl/curl/pull/1589
ldap_bind_s is marked as deprecated in w32api's winldap.h shipping with
the latest original MinGW, resulting in compiler warnings since commit
f0fe66f13c. Fix this for the non-SSPI
case by using ldap_simple_bind_s again instead of ldap_bind_s with
LDAP_AUTH_SIMPLE.
Closes https://github.com/curl/curl/pull/1664
Prior to this change (SET_)ERRNO mapped to GetLastError/SetLastError
for Win32 and regular errno otherwise.
I reviewed the code and found no justifiable reason for conflating errno
on WIN32 with GetLastError/SetLastError. All Win32 CRTs support errno,
and any Win32 multithreaded CRT supports thread-local errno.
Fixes https://github.com/curl/curl/issues/895
Closes https://github.com/curl/curl/pull/1589
GCC 4.6.3 on travis complains:
smb.c: In function ‘get_posix_time’:
smb.c:725:13: error: declaration of ‘time’ shadows a global declaration
[-Werror=shadow]
Fix this by renaming the variable.
On a 64 bit host, sparse says:
timeval.c:148:15: warning: constant 0x7fffffffffffffff is so big it is long
timeval.c:149:12: warning: constant 0x7fffffffffffffff is so big it is long
so let's use long long constant types in order to prevent undesired overflow
failures.
Bug: https://curl.haxx.se/mail/lib-2017-07/0003.htmlCloses#1636
Signed-off-by: Martin Kepplinger <martink@posteo.de>
... since CURLOPT_URL should follow the same rules as other options:
they remain set until changed or cleared.
Added test 1551 to verify.
Fixes#1631Closes#1632
Reported-by: Pavel Rochnyak
- Change gnutls pointer/int macros to pointer/curl_socket_t.
Prior to this change they used long type as well.
The size of the `long` data type can be shorter than that of pointer
types. This is the case most notably on Windows.
If C99 were acceptable, we could simply use `intptr_t` here. But we
want to retain C89 compatibility.
Simply use the trick of performing pointer arithmetic with the NULL
pointer: to convert an integer `i` to a pointer, simply take the
address of the `i`th element of a hypothetical character array
starting at address NULL. To convert back, simply cast the pointer
difference.
Thanks to Jay Satiro for the initial modification to use curl_socket_t
instead of int/long.
Closes#1617
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add a new type of callback to Curl_handler which performs checks on
the connection. Alter RTSP so that it uses this callback to do its
own check on connection health.
Prevent `Curl_pgrsTime` from modifying `t_starttransfer` when invoked
with `TIMER_STARTTRANSFER` more than once during a single request.
When a redirect occurs, this is considered a new request and
`t_starttransfer` can be updated to reflect the `t_starttransfer` time
of the redirect request.
Closes#1616
Bug: https://github.com/curl/curl/pull/1602#issuecomment-310267370
If libcurl was built with GSS-API support, it unconditionally advertised
GSS-API authentication while connecting to a SOCKS5 proxy. This caused
problems in environments with improperly configured Kerberos: a stock
libcurl failed to connect, despite libcurl built without GSS-API
connected fine using username and password.
This commit introduces the CURLOPT_SOCKS5_AUTH option to control the
allowed methods for SOCKS5 authentication at run time.
Note that a new option was preferred over reusing CURLOPT_PROXYAUTH
for compatibility reasons because the set of authentication methods
allowed by default was different for HTTP and SOCKS5 proxies.
Bug: https://curl.haxx.se/mail/lib-2017-01/0005.html
Closes https://github.com/curl/curl/pull/1454
... to enable sending "OPTIONS *" which wasn't possible previously.
This option currently only works for HTTP.
Added test cases 1298 + 1299 to verify
Fixes#1280Closes#1462
This change introduces new alternatives for the existing six
curl_easy_getinfo() options that return sizes or speeds as doubles. The
new versions are named like the old ones but with an appended '_T':
CURLINFO_CONTENT_LENGTH_DOWNLOAD_T
CURLINFO_CONTENT_LENGTH_UPLOAD_T
CURLINFO_SIZE_DOWNLOAD_T
CURLINFO_SIZE_UPLOAD_T
CURLINFO_SPEED_DOWNLOAD_T
CURLINFO_SPEED_UPLOAD_T
Closes#1511
The list was freed incorrectly since the llist refactor of
cbae73e1dd. Added test 1550 to verify that it works and avoid future
regressions.
Reported-by: Pascal Terjan
Fixes#1584Closes#1585
... the previous code would reset the header length wrongly (since
5113ad0424). This makes test 1060 reliable again.
Also: make sws send even smaller chunks of data to increase the
likeliness of this happening.
- No longer allow partial downloads of certdata.
Prior to this change partial downloads were (erroneously?) allowed since
only the server code was checked to be 200.
Bug: https://github.com/curl/curl/pull/1577
Reported-by: Matteo B.
... all other non-HTTP protocol schemes are now defaulting to "tunnel
trough" mode if a HTTP proxy is specified. In reality there are no HTTP
proxies out there that allow those other schemes.
Assisted-by: Ray Satiro, Michael Kaufmann
Closes#1505
When this define was set, libcurl would check the environment variable
named CURL_CA_BUNDLE at run-time and use that CA cert bundle. This
feature was only defined by the watcom and m32 makefiles and caused
inconsistent behaviours among libcurls built on different platforms.
The curl tool does already feature its own similar logic and the library
does not really need it, and it isn't documented libcurl behavior. So
this change removes it.
Ref: #1538
This gives us accurate precision and it allows us to avoid storing "no
time" for systems with too low timer resolution as we then bump the time
up to 1 microsecond. Should fix test 573 on windows.
Remove the now unused curlx_tvdiff_secs() function.
Maintains the external getinfo() API with using doubles.
Fixes#1531
With the introduction of expire IDs and the fact that existing timers
can be removed now and thus never expire, the concept with adding a
"latest" timer is not working anymore as it risks to not expire at all.
So, to be certain the timers actually are in line and will expire, the
plain Curl_expire() needs to be used. The _latest() function was added
as a sort of shortcut in the past that's quite simply not necessary
anymore.
Follow-up to 31b39c40cf
Reported-by: Paul Harris
Closes#1555
mk-lib1521.pl generates a test program (lib1521.c) that calls
curl_easy_setopt() for every known option with a few typical values to
make sure they work (ignoring the return codes).
Some small changes were necessary to avoid asserts and NULL accesses
when doing this.
The perl script needs to be manually rerun when we add new options.
Closes#1543
This fixes the following warning with CURL_DISABLE_CRYPTO_AUTH,
as seen in the autobuilds:
curl_sasl.c:417:9: warning: unused variable 'serverdata'
[-Wunused-variable]
cyassl/ssl.h needs the macros from cyassl/options.h, so define them
before including cyassl/ssl.h the first time, which happens in
urldata.h.
This broke the build on Ubuntu Xenial, which comes with WolfSSL 3.4.8
and therefore redefines the symbols from cyassl/options.h instead of
including the header.
Closes https://github.com/curl/curl/pull/1536
... with a strlen() if no size was set, and do this in the pretransfer
function so that the info is set early. Otherwise, the default strlen()
done on the POSTFIELDS data never sets state.infilesize.
Reported-by: Vincas Razma
Bug: #1294
* LDAP: using ldap_bind_s on Windows with methods(BASIC/DIGEST/NTLM/AUTONEG)
* ldap: updated per build options handling
* ldap: fixed logic for auth selection
Fixed a syntax error with setting cache variables (The type and
docstring were missing), resulting in build errors. Quoted the
CURL_CA_PATH and CURL_CA_BUNDLE otherwise the path was written without
quotes in C code, resulting in build errors.
Closes#1503
Signed-off-by: Akhil <akhil.kedia@samsung.com>
.... caused by a typo in the last commit (fixing issue #1504):
memdebug.c: In function ‘curl_fclose’:
memdebug.c:444:3: error: implicit declaration of function
‘DEBUGDEBUGASSERT’ [-Werror=implicit-function-declaration]
Some code (e.g. Curl_fillreadbuffer) assumes that this buffer is not
exceedingly tiny and will break if it is. This same check is already
done at run time in the CURLOPT_BUFFERSIZE option.
The function IsPipeliningPossible() would return TRUE if either
pipelining OR HTTP/2 were possible on a connection, which would lead to
it returning TRUE even for POSTs on HTTP/1 connections.
It now returns a bitmask so that the caller can differentiate which kind
the connection allows.
Fixes#1481Closes#1483
Reported-by: stootill at github
... since the total amount is low this is faster, easier and reduces
memory overhead.
Also, Curl_expire_done() can now mark an expire timeout as done so that
it never times out.
Closes#1472
A) reduces the timeout lists drastically
B) prevents a lot of superfluous loops for timers that expires "in vain"
when it has actually already been extended to fire later on
When the random seed is purposely made predictable for testing purposes
by using the CURL_ENTROPY environment variable, process that data in an
endian agnostic way so the the initial random seed is the same
regardless of endianness.
- Change Curl_rand to write to a char array instead of int array.
- Add Curl_rand_hex to write random hex characters to a buffer.
Fixes#1315Closes#1468
Co-authored-by: Daniel Stenberg
Reported-by: Michael Kaufmann
SEND_QUAL_ARG2 had to be set, but was never used. Use it in swrite to
avoid warnings about casting away low-level const.
Closes https://github.com/curl/curl/pull/1464
Fix the following warnings when building the tests by using the correct
types:
cast from 'const char *' to 'void *' drops const qualifier
[-Wcast-qual]
implicit conversion changes signedness [-Wsign-conversion]
Previous TODO wanting to write in chunks. We should support writing more
at once since some TELNET servers may respond immediately upon first
byte written such as WHOIS servers.
Closes#1389
Prior to this change it was possible for libcurl to be built with both
Windows' native IDN lib (normaliz) and libidn2 enabled. It appears that
doesn't offer any benefit --and could cause a bug-- since libcurl's IDN
handling is written to use either one but not both.
Bug: https://github.com/curl/curl/issues/1441#issuecomment-297689856
Reported-by: Gisle Vanem
This fixes the following clang warnings:
http2.c:184:27: error: no previous extern declaration for non-static
variable 'Curl_handler_http2' [-Werror,-Wmissing-variable-declarations]
http2.c:204:27: error: no previous extern declaration for non-static
variable 'Curl_handler_http2_ssl'
[-Werror,-Wmissing-variable-declarations]
clang complains:
curl_rtmp.c:61:27: error: no previous extern declaration for non-static variable 'Curl_handler_rtmp' [-Werror,-Wmissing-variable-declarations]
curl_rtmp.c:81:27: error: no previous extern declaration for non-static variable 'Curl_handler_rtmpt' [-Werror,-Wmissing-variable-declarations]
curl_rtmp.c:101:27: error: no previous extern declaration for non-static variable 'Curl_handler_rtmpe' [-Werror,-Wmissing-variable-declarations]
curl_rtmp.c:121:27: error: no previous extern declaration for non-static variable 'Curl_handler_rtmpte' [-Werror,-Wmissing-variable-declarations]
curl_rtmp.c:141:27: error: no previous extern declaration for non-static variable 'Curl_handler_rtmps' [-Werror,-Wmissing-variable-declarations]
curl_rtmp.c:161:27: error: no previous extern declaration for non-static variable 'Curl_handler_rtmpts' [-Werror,-Wmissing-variable-declarations]
Fix this by including the header file.
This fixes the following clang warnings:
macro is not used [-Wunused-macros]
will never be executed [-Wunreachable-code]
Closes https://github.com/curl/curl/pull/1448
get_protocol_family() is not defined static even though there is a
static local forward declaration. Let's simply make the definition match
it's declaration.
Bug: https://curl.haxx.se/mail/lib-2017-04/0127.html
The module contains a more comprehensive set of trust information than
supported by nss-pem, because libnssckbi.so also includes information
about distrusted certificates.
Reviewed-by: Kai Engert
Closes#1414
The data->req.uploadbuf struct member served no good purpose, instead we
use ->state.uploadbuffer directly. It makes it clearer in the code which
buffer that's being used.
Removed the 'SingleRequest *' argument from the readwrite_upload() proto
as it can be derived from the Curl_easy struct. Also made the code in
the readwrite_upload() function use the 'k->' shortcut to all references
to struct fields in 'data->req', which previously was made with a mix of
both.
- Track when the cached encrypted data contains only a partial record
that can't be decrypted without more data (SEC_E_INCOMPLETE_MESSAGE).
- Change Curl_schannel_data_pending to return false in such a case.
Other SSL libraries have pending data functions that behave similarly.
Ref: https://github.com/curl/curl/pull/1387
Closes https://github.com/curl/curl/pull/1392
`if(nfds || extra_nfds) {` is followed by `malloc(nfds * ...)`.
If `extra_fs` could be non-zero when `nfds` was zero, then we have
`malloc(0)` which is allowed to return `NULL`. But, malloc returning
NULL can be confusing. In this code, the next line would treat the NULL
as an allocation failure.
It turns out, if `nfds` is zero then `extra_nfds` must also be zero.
The final value of `nfds` includes `extra_nfds`. So the test for
`extra_nfds` is redundant. It can only confuse the reader.
Closes#1439
With -Og, GCC complains:
easy.c:628:7: error: ‘mcode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
../lib/strcase.h:35:29: error: ‘tok_buf’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
vauth/digest.c:208:9: note: ‘tok_buf’ was declared here
../lib/strcase.h:35:29: error: ‘tok_buf’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
vauth/digest.c:566:15: note: ‘tok_buf’ was declared here
Fix this by initializing the variables.
The 'list element' struct now has to be within the data that is being
added to the list. Removes 16.6% (tiny) mallocs from a simple HTTP
transfer. (96 => 80)
Also removed return codes since the llist functions can't fail now.
Test 1300 updated accordingly.
Closes#1435
In that case, use libcurl's internal MD4 routine. This fixes tests 1013
and 1014 which were failing due to configure assuming NTLM and SMB were
always available whenever mbed TLS was in use (which is now true).
This fixes 3 warnings issued by MinGW:
1. PR_ImportTCPSocket actually has a paramter of type PROsfd instead of
PRInt32, which is 64 bits on Windows. Fixed this by including the
corresponding header file instead of redeclaring the function, which is
supported even though it is in the private include folder. [1]
2. In 64-bit mode, size_t is 64 bits while CK_ULONG is 32 bits, so an explicit
narrowing cast is needed.
3. Curl_timeleft returns time_t instead of long since commit
21aa32d30d.
[1] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSPR/Reference/PR_ImportTCPSocket
Closes https://github.com/curl/curl/pull/1393
ERR_error_string with NULL parameter is not thread-safe. The library
writes the string into some static buffer. Two threads doing this at
once may clobber each other and run into problems. Switch to
ERR_error_string_n which avoids this problem and is explicitly
bounds-checked.
Also clean up some remnants of OpenSSL 0.9.5 around here. A number of
comments (fixed buffer size, explaining that ERR_error_string_n was
added in a particular version) date to when ossl_strerror tried to
support pre-ERR_error_string_n OpenSSLs.
Closes#1424
ssl_session_init was only introduced in version 1.3.8, the penultimate
version. The function only contains a memset, so replace it with that.
Suggested-by: Jay Satiro
Fixes https://github.com/curl/curl/issues/1401
The POSIX standard location is <poll.h>. Using <sys/poll.h> results in
warning spam when using the musl standard library.
Closes https://github.com/curl/curl/pull/1406
... because they may include an intermediate certificate for a client
certificate and the intermediate certificate needs to be presented to
the server, no matter if we verify the peer or not.
Reported-by: thraidh
Closes#851
When UNICODE is not defined, the Curl_convert_UTF8_to_tchar macro maps
directly to its argument. As it is declared as a pointer to const and
InitializeSecurityContext expects a pointer to non-const, both MSVC and MinGW
issue a warning about implicitly casting away the const. Fix this by declaring
the variables as pointers to non-const.
Closes https://github.com/curl/curl/pull/1394
Previously, periods of fast speed between periods of slow speed would
not count and could still erroneously trigger a timeout.
Reported-by: Paul Harris
Fixes#1345Closes#1390
Multi handles repeatedly invert the queue of pending easy handles when
used with CURLMOPT_MAX_TOTAL_CONNECTIONS. This is caused by a multistep
process involving Curl_splaygetbest and violates the FIFO property of
the multi handle.
This patch fixes this issue by redefining the "best" node in the
context of timeouts as the "smallest not larger than now", and
implementing the necessary data structure modifications to do this
effectively, namely:
- splay nodes with the same key are now stored in a doubly-linked
circular list instead of a non-circular one to enable O(1)
insertion to the tail of the list
- Curl_splayinsert inserts nodes with the same key to the tail of
the same list
- in case of multiple nodes with the same key, the one on the head of
the list gets selected
- Don't free postponed data on a connection that will be reused since
doing so can cause data loss when pipelining.
Only Windows builds are affected by this.
Closes https://github.com/curl/curl/issues/1380
Safe to silence warning adding time delta of poll, which can trigger on
Windows since sizeof time_t > sizeof long.
warning C4244: '+=' : conversion from 'time_t' to 'long', possible loss
of data
system.h is aimed to replace curlbuild.h at a later point in time when
we feel confident system.h works sufficiently well.
curl/system.h is currently used in parallel with curl/curlbuild.h
curl/system.h determines a data sizes, data types and include file
status based on available preprocessor defines instead of getting
generated at build-time. This, in order to avoid relying on a build-time
generated file that makes it complicated to do 32 and 64 bit bields from
the same installed set of headers.
Test 1541 verifies that system.h comes to the same conclusion that
curlbuild.h offers.
Closes#1373
telnet.c(1427,21): warning: comparison of constant 268435456 with
expression of type 'CURLcode' is always false
telnet.c(1433,21): warning: comparison of constant 268435457 with
expression of type 'CURLcode' is always false
Reviewed-by: Jay Satiro
Reported-by: Gisle Vanem
Bug: https://github.com/curl/curl/issues/1225#issuecomment-290340890Closes#1374
'left' is used as time_t but declared as long.
MinGW complains:
error: conversion to 'long int' from 'time_t {aka long long int}' may alter
its value [-Werror=conversion]
Changed the declaration to time_t.
At least under Windows, there is no SIZEOF_LONG, so it evaluates to 0 even
though sizeof(int) == sizeof(long). This should probably have been
CURL_SIZEOF_LONG, but the type of timeout_ms changed from long to time_t
anyway.
This triggered MSVC warning C4668 about implicitly replacing undefined
macros with '0'.
Closes https://github.com/curl/curl/pull/1362
If we use FTPS over CONNECT, the TLS handshake for the FTPS control
connection needs to be initiated in the SENDPROTOCONNECT state, not
the WAITPROXYCONNECT state. Otherwise, if the TLS handshake completed
without blocking, the information about the completed TLS handshake
would be saved to a wrong flag. Consequently, the TLS handshake would
be initiated in the SENDPROTOCONNECT state once again on the same
connection, resulting in a failure of the TLS handshake. I was able to
observe the failure with the NSS backend if curl ran through valgrind.
Note that this commit partially reverts curl-7_21_6-52-ge34131d.
When receiving chunked encoded data with trailers, and the write
callback returns PAUSE, there might be both body and header to store to
resend on unpause. Previously libcurl returned error for that case.
Added test case 1540 to verify.
Reported-by: Stephen Toub
Fixes#1354Closes#1357
When using basic-auth, connections and proxy connections
can be re-used with different Authorization headers since
it does not authenticate the connection (like NTLM does).
For instance, the below command should re-use the proxy
connection, but it currently doesn't:
curl -v -U alice:a -x http://localhost:8181http://localhost/
--next -U bob:b -x http://localhost:8181http://localhost/
This is a regression since refactoring of ConnectionExists()
as part of: cb4e2be7c6
Fix the above by removing the username and password compare
when re-using proxy connection at proxy_info_matches().
However, this fix brings back another bug would make curl
to re-print the old proxy-authorization header of previous
proxy basic-auth connection because it wasn't cleared.
For instance, in the below command the second request should
fail if the proxy requires authentication, but would succeed
after the above fix (and before aforementioned commit):
curl -v -U alice:a -x http://localhost:8181http://localhost/
--next -x http://localhost:8181http://localhost/
Fix this by clearing conn->allocptr.proxyuserpwd after use
unconditionally, same as we do for conn->allocptr.userpwd.
Also fix test 540 to not expect digest auth header to be
resent when connection is reused.
Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Closes https://github.com/curl/curl/pull/1350
- If SSL_get_error is called but no extended error detail is available
then show that SSL_ERROR_* as a string.
Prior to this change there was some inconsistency in that case: the
SSL_ERROR_* code may or may not have been shown, or may have been shown
as unknown even if it was known.
Ref: https://github.com/curl/curl/issues/1300
Closes https://github.com/curl/curl/pull/1348
- Add new option CURLOPT_SUPPRESS_CONNECT_HEADERS to allow suppressing
proxy CONNECT response headers from the user callback functions
CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION.
- Add new tool option --suppress-connect-headers to expose
CURLOPT_SUPPRESS_CONNECT_HEADERS and allow suppressing proxy CONNECT
response headers from --dump-header and --include.
Assisted-by: Jay Satiro
Assisted-by: CarloCannas@users.noreply.github.com
Closes https://github.com/curl/curl/pull/783
A client MUST ignore any Content-Length or Transfer-Encoding header
fields received in a successful response to CONNECT.
"Successful" described as: 2xx (Successful). RFC 7231 4.3.6
Prior to this change such a case would cause an error.
In some ways this bug appears to be a regression since c50b878. Prior to
that libcurl may have appeared to function correctly in such cases by
acting on those headers instead of causing an error. But that behavior
was also incorrect.
Bug: https://github.com/curl/curl/issues/1317
Reported-by: mkzero@users.noreply.github.com
This flag is meant for the current request based on authentication
state, once the request is done we can clear the flag.
Also change auth.multi to auth.multipass for better readability.
Fixes https://github.com/curl/curl/issues/1095
Closes https://github.com/curl/curl/pull/1326
Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reported-by: Michael Kaufmann
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as
the --tls-max option of the curl tool.
Closes https://github.com/curl/curl/pull/1166
This fixes assertion error which occurs when redirect is done with 0
length body via HTTP/2, and the easy handle is reused, but new
connection is established due to hostname change:
curl: http2.c:1572: ssize_t http2_recv(struct connectdata *,
int, char *, size_t, CURLcode *):
Assertion `httpc->drain_total >= data->state.drain' failed.
To fix this bug, ensure that http2_handle_stream is called.
Fixes#1286Closes#1302
... because it causes confusion with users. Example URLs:
"http://[127.0.0.1]:11211:80" which a lot of languages' URL parsers will
parse and claim uses port number 80, while libcurl would use port number
11211.
"http://user@example.com:80@localhost" which by the WHATWG URL spec will
be treated to contain user name 'user@example.com' but according to
RFC3986 is user name 'user' for the host 'example.com' and then port 80
is followed by "@localhost"
Both these formats are now rejected, and verified so in test 1260.
Reported-by: Orange Tsai
Mark intended fallthroughs with /* FALLTHROUGH */ so that gcc will know
it's expected and won't warn on [-Wimplicit-fallthrough=].
Closes https://github.com/curl/curl/pull/1297
In DarwinSSL the SSLSetPeerDomainName function is used to enable both
sending SNI and verifying the host. When host verification is disabled
the function cannot be called, therefore SNI is disabled as well.
Closes https://github.com/curl/curl/pull/1240
If size_t is 32 bits, MSVC warns:
warning C4310: cast truncates constant value
The warning is harmless as CURL_MASK_SCOFFT gets
truncated to the maximum value of size_t.
If the compile-time CURL_CA_BUNDLE location is defined use it as the
default value for the proxy CA bundle location, which is the same as
what we already do for the regular CA bundle location.
Ref: https://github.com/curl/curl/pull/1257
- Change CURLOPT_PROXY_CAPATH to return CURLE_NOT_BUILT_IN if the option
is not supported, which is the same as what we already do for
CURLOPT_CAPATH.
- Change the curl tool to handle CURLOPT_PROXY_CAPATH error
CURLE_NOT_BUILT_IN as a warning instead of as an error, which is the
same as what we already do for CURLOPT_CAPATH.
- Fix CAPATH docs to show that CURLE_NOT_BUILT_IN is returned when the
respective CAPATH option is not supported by the SSL library.
Ref: https://github.com/curl/curl/pull/1257
The CURLOPT_SSL_VERIFYSTATUS option was not properly handled by libcurl
and thus even if the status couldn't be verified, the connection would
be allowed and the user would not be told about the failed verification.
Regression since cb4e2be7c6
CVE-2017-2629
Bug: https://curl.haxx.se/docs/adv_20170222.html
Reported-by: Marcus Hoffmann
- on the first invocation: keep security context returned by
InitializeSecurityContext()
- on subsequent invocations: use MakeSignature() instead of
InitializeSecurityContext() to generate HTTP digest response
Bug: https://github.com/curl/curl/issues/870
Reported-by: Andreas Roth
Closes https://github.com/curl/curl/pull/1251
Properly resolve, convert and log the proxy host names.
Support the "--connect-to" feature for SOCKS proxies and for passive FTP
data transfers.
Follow-up to cb4e2be
Reported-by: Jay Satiro
Fixes https://github.com/curl/curl/issues/1248
- While negotiating auth during PUT/POST if a user-specified
Content-Length header is set send 'Content-Length: 0'.
This is what we do already in HTTPREQ_POST_FORM and what we did in the
HTTPREQ_POST case (regression since afd288b).
Prior to this change no Content-Length header would be sent in such a
case.
Bug: https://curl.haxx.se/mail/lib-2017-02/0006.html
Reported-by: Dominik Hölzl
Closes https://github.com/curl/curl/pull/1242
Builds with axTLS 2.1.2. This then also breaks compatibility with axTLS
< 2.1.0 (the older API)
... and fix the session_id mixup brought in 04b4ee549Fixes#1220
If the NSS code was in the middle of a non-blocking handshake and it
was asked to finish the handshake in blocking mode, it unexpectedly
continued in the non-blocking mode, which caused a FTPS connection
over CONNECT to fail with "(81) Socket not ready for send/recv".
Bug: https://bugzilla.redhat.com/1420327
When removing an easy handler from a multi before it completed its
transfer, and it had pushed streams, it would segfault due to the pushed
counted not being cleared.
Fixed-by: zelinchen@users.noreply.github.comFixes#1249
Using sftp to delete a file with CURLOPT_NOBODY set with a reused
connection would fail as curl expected to get some data. Thus it would
retry the command again which fails as the file has already been
deleted.
Fixes#1243
The information extracted from the server certificates in step 3 is only
used when in verbose mode, and there is no error handling or validation
performed as that has already been done. Only run the certificate
information extraction when in verbose mode and libcurl was built with
verbose strings.
Closes https://github.com/curl/curl/pull/1246
- Remove the SNI disabled when host verification disabled message
since that is incorrect.
- Show a message for legacy versions of Windows <= XP that connections
may fail since those versions of WinSSL lack SNI, algorithms, etc.
Bug: https://github.com/curl/curl/pull/1240
SSL_CTX_add_extra_chain_cert takes ownership of the given certificate
while, despite the similar name, SSL_CTX_add_client_CA does not. Thus
it's best to call SSL_CTX_add_client_CA before
SSL_CTX_add_extra_chain_cert, while the code still has ownership of the
argument.
Closes https://github.com/curl/curl/pull/1236
This repairs cookies for localhost.
Non-PSL builds will now only accept "localhost" without dots, while PSL
builds okeys everything not listed as PSL.
Added test 1258 to verify.
This was a regression brought in a76825a5ef
Replace use of fixed macro BUFSIZE to define the size of the receive
buffer. Reappropriate CURLOPT_BUFFERSIZE to include enlarging receive
buffer size. Upon setting, resize buffer if larger than the current
default size up to a MAX_BUFSIZE (512KB). This can benefit protocols
like SFTP.
Closes#1222
Regression since 1d4202ad, which moved the buffer into a more narrow
scope, but the data in that buffer was used outside of that more narrow
scope.
Reported-by: Dan Fandrich
Bug: https://curl.haxx.se/mail/lib-2017-01/0093.html
curl_addrinfo.c:519:20: error: conversion to ‘curl_socklen_t {aka
unsigned int}’ from ‘long unsigned int’ may alter its value
[-Werror=conversion]
Follow-up to 1d786faee1
In addition to unix domain sockets, Linux also supports an
abstract namespace which is independent of the filesystem.
In order to support it, add new CURLOPT_ABSTRACT_UNIX_SOCKET
option which uses the same storage as CURLOPT_UNIX_SOCKET_PATH
internally, along with a flag to specify abstract socket.
On non-supporting platforms, the abstract address will be
interpreted as an empty string and fail gracefully.
Also add new --abstract-unix-socket tool parameter.
Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reported-by: Chungtsun Li (typeless)
Reviewed-by: Daniel Stenberg
Reviewed-by: Peter Wu
Closes#1197Fixes#1061
It made the german ß get converted to ss, IDNA2003 style, and we can't
have that for the .de TLD - a primary reason for our switch to IDNA2008.
Test 165 verifies.
Under condition using http_proxy env var, noproxy list was the
combination of --noproxy option and NO_PROXY env var previously. Since
this commit, --noproxy option overrides NO_PROXY environment variable
even if use http_proxy env var.
Closes#1140
If defined CURL_DISABLE_HTTP, detect_proxy() returned NULL. If not
defined CURL_DISABLE_HTTP, detect_proxy() checked noproxy list.
Thus refactor to set proxy to NULL instead of calling detect_proxy() if
define CURL_DISABLE_HTTP, and refactor to call detect_proxy() if not
define CURL_DISABLE_HTTP and the host is not in the noproxy list.
The combination of --noproxy option and http_proxy env var works well
both for proxied hosts and non-proxied hosts.
However, when combining NO_PROXY env var with --proxy option,
non-proxied hosts are not reachable while proxied host is OK.
This patch allows us to access non-proxied hosts even if using NO_PROXY
env var with --proxy option.
Check for presence of gnutls_alpn_* and gnutls_ocsp_* functions during
configure instead of relying on the version number. GnuTLS has options
to turn these features off and we ca just work with with such builds
like we work with older versions.
Signed-off-by: Marcus Hoffmann <m.hoffmann@cartelsol.com>
Closes#1204
Follow-up to 3463408.
Prior to 3463408 file:// hostnames were silently stripped.
Prior to this commit it did not work when a schemeless url was used with
file as the default protocol.
Ref: https://curl.haxx.se/mail/lib-2016-11/0081.html
Closes https://github.com/curl/curl/pull/1124
Also fix for drive letters:
- Support --proto-default file c:/foo/bar.txt
- Support file://c:/foo/bar.txt
- Fail when a file:// drive letter is detected and not MSDOS/Windows.
Bug: https://github.com/curl/curl/issues/1187
Reported-by: Anatol Belski
Assisted-by: Anatol Belski
Both IMAP and POP3 response characters are used internally, but when
appended to the STARTTLS denial message likely could confuse the user.
Closes https://github.com/curl/curl/pull/1203
Fixed an old leftover use of the USE_SSLEAY define which would make a
socket get removed from the applications sockets to monitor when the
multi_socket API was used, leading to timeouts.
Bug: #1174
Visual C++ complained:
warning C4267: '=': conversion from 'size_t' to 'long', possible loss of data
warning C4701: potentially uninitialized local variable 'path' used
Fixes a few issues in manual wildcard cert name validation in
schannel support code for Win32 CE:
- when comparing the wildcard name to the hostname, the wildcard
character was removed from the cert name and the hostname
was checked to see if it ended with the modified cert name.
This allowed cert names like *.com to match the connection
hostname. This violates recommendations from RFC 6125.
- when the wildcard name in the certificate is longer than the
connection hostname, a buffer overread of the connection
hostname buffer would occur during the comparison of the
certificate name and the connection hostname.
It doesn't benefit us much as the connection could get closed at
any time, and also by checking we lose the ability to determine
if the socket was closed by reading zero bytes.
Reported-by: Michael Kaufmann
Closes https://github.com/curl/curl/pull/1134
CURLOPT_SOCKS_PROXY -> CURLOPT_PRE_PROXY
Added the corresponding --preroxy command line option. Sets a SOCKS
proxy to connect to _before_ connecting to a HTTP(S) proxy.
This was added as part of the SOCKS+HTTPS proxy merge but there's no
need to support this as we prefer to have the protocol specified as a
prefix instead.
ERR_PACK is an internal detail of OpenSSL. Also, when using it, a
function name must be specified which is overly specific: the test will
break whenever OpenSSL internally change things so that a different
function creates the error.
Closes#1157
Since it now reads responses one byte a time, a loop could be removed
and it is no longer limited to get the whole response within 16K, it is
now instead only limited to 16K maximum header line lengths.
... so that it doesn't read data that is actually coming from the
remote. 2xx responses have no body from the proxy, that data is from the
peer.
Fixes#1132
A server MUST NOT send any Transfer-Encoding or Content-Length header
fields in a 2xx (Successful) response to CONNECT. (RFC 7231 section
4.3.6)
Also fixes the three test cases that did this.
If a port number in a "connect-to" entry does not match, skip this
entry instead of connecting to port 0.
If a port number in a "connect-to" entry matches, use this entry
and look no further.
Reported-by: Jay Satiro
Assisted-by: Jay Satiro, Daniel Stenberg
Closes#1148
Adds access to the effectively used protocol/scheme to both libcurl and
curl, both in string and numeric (CURLPROTO_*) form.
Note that the string form will be uppercase, as it is just the internal
string.
As these strings are declared internally as const, and all other strings
returned by curl_easy_getinfo() are de-facto const as well, string
handling in getinfo.c got const-ified.
Closes#1137
vtls/gtls.c: In function ‘Curl_gtls_data_pending’:
vtls/gtls.c:1429:3: error: this ‘if’ clause does not guard... [-Werror=misleading-indentation]
if(conn->proxy_ssl[connindex].session &&
^~
vtls/gtls.c:1433:5: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’
return res;
* HTTPS proxies:
An HTTPS proxy receives all transactions over an SSL/TLS connection.
Once a secure connection with the proxy is established, the user agent
uses the proxy as usual, including sending CONNECT requests to instruct
the proxy to establish a [usually secure] TCP tunnel with an origin
server. HTTPS proxies protect nearly all aspects of user-proxy
communications as opposed to HTTP proxies that receive all requests
(including CONNECT requests) in vulnerable clear text.
With HTTPS proxies, it is possible to have two concurrent _nested_
SSL/TLS sessions: the "outer" one between the user agent and the proxy
and the "inner" one between the user agent and the origin server
(through the proxy). This change adds supports for such nested sessions
as well.
A secure connection with a proxy requires its own set of the usual SSL
options (their actual descriptions differ and need polishing, see TODO):
--proxy-cacert FILE CA certificate to verify peer against
--proxy-capath DIR CA directory to verify peer against
--proxy-cert CERT[:PASSWD] Client certificate file and password
--proxy-cert-type TYPE Certificate file type (DER/PEM/ENG)
--proxy-ciphers LIST SSL ciphers to use
--proxy-crlfile FILE Get a CRL list in PEM format from the file
--proxy-insecure Allow connections to proxies with bad certs
--proxy-key KEY Private key file name
--proxy-key-type TYPE Private key file type (DER/PEM/ENG)
--proxy-pass PASS Pass phrase for the private key
--proxy-ssl-allow-beast Allow security flaw to improve interop
--proxy-sslv2 Use SSLv2
--proxy-sslv3 Use SSLv3
--proxy-tlsv1 Use TLSv1
--proxy-tlsuser USER TLS username
--proxy-tlspassword STRING TLS password
--proxy-tlsauthtype STRING TLS authentication type (default SRP)
All --proxy-foo options are independent from their --foo counterparts,
except --proxy-crlfile which defaults to --crlfile and --proxy-capath
which defaults to --capath.
Curl now also supports %{proxy_ssl_verify_result} --write-out variable,
similar to the existing %{ssl_verify_result} variable.
Supported backends: OpenSSL, GnuTLS, and NSS.
* A SOCKS proxy + HTTP/HTTPS proxy combination:
If both --socks* and --proxy options are given, Curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS
proxy.
TODO: Update documentation for the new APIs and --proxy-* options.
Look for "Added in 7.XXX" marks.
- Fix connection reuse for when the proposed new conn 'needle' has a
specified local port but does not have a specified device interface.
Bug: https://curl.haxx.se/mail/lib-2016-11/0137.html
Reported-by: bjt3[at]hotmail.com
Visual C++ now complains about implicitly casting time_t (64-bit) to
long (32-bit). Fix this by changing some variables from long to time_t,
or explicitly casting to long where the public interface would be
affected.
Closes#1131
- In Curl_http2_switched don't call memcpy when src is NULL.
Curl_http2_switched can be called like:
Curl_http2_switched(conn, NULL, 0);
.. and prior to this change memcpy was then called like:
memcpy(dest, NULL, 0)
.. causing address sanitizer to warn:
http2.c:2057:3: runtime error: null pointer passed as argument 2, which
is declared to never be null
Now Curl_rand() is made to fail if it cannot get the necessary random
level.
Changed the proto of Curl_rand() slightly to provide a number of ints at
once.
Moved out from vtls, since it isn't a TLS function and vtls provides
Curl_ssl_random() for this to use.
Discussion: https://curl.haxx.se/mail/lib-2016-11/0119.html
Previously, the [host] part was just ignored which made libcurl accept
strange URLs misleading users. like "file://etc/passwd" which might've
looked like it refers to "/etc/passwd" but is just "/passwd" since the
"etc" is an ignored host name.
Reported-by: Mike Crowe
Assisted-by: Kamil Dudka
- Fix GnuTLS code for CURL_SSLVERSION_TLSv1_2 that broke when the
TLS 1.3 support was added in 6ad3add.
- Homogenize across code for all backends the error message when TLS 1.3
is not available to "<backend>: TLS 1.3 is not yet supported".
- Return an error when a user-specified ssl version is unrecognized.
---
Prior to this change our code for some of the backends used the
'default' label in the switch statement (ie ver unrecognized) for
ssl.version and treated it the same as CURL_SSLVERSION_DEFAULT.
Bug: https://curl.haxx.se/mail/lib-2016-11/0048.html
Reported-by: Kamil Dudka
We're mostly saying just "curl" in lower case these days so here's a big
cleanup to adapt to this reality. A few instances are left as the
project could still formally be considered called cURL.
- Call Curl_initinfo on init and duphandle.
Prior to this change the statistical and informational variables were
simply zeroed by calloc on easy init and duphandle. While zero is the
correct default value for almost all info variables, there is one where
it isn't (filetime initializes to -1).
Bug: https://github.com/curl/curl/issues/1103
Reported-by: Neal Poole
...to use the public function curl_strnequal(). This isn't ideal because
it adds extra overhead to any internal calls to checkprefix.
follow-up to 95bd2b3e
As they are after all part of the public API. Saves space and reduces
complexity. Remove the strcase defines from the curlx_ family.
Suggested-by: Dan Fandrich
Idea: https://curl.haxx.se/mail/lib-2016-10/0136.html
This should fix the "warning: 'curl_strequal' redeclared without
dllimport attribute: previous dllimport ignored" message and subsequent
link error on Windows because of the missing CURL_EXTERN on the
prototype.
These two public functions have been mentioned as deprecated since a
very long time but since they are still part of the API and ABI we need
to keep them around.
... to make it less likely that we forget that the function actually
does case insentive compares. Also replaced several invokes of the
function with a plain strcmp when case sensitivity is not an issue (like
comparing with "-").
If the requested size is zero, bail out with error instead of doing a
realloc() that would cause a double-free: realloc(0) acts as a free()
and then there's a second free in the cleanup path.
CVE-2016-8619
Bug: https://curl.haxx.se/docs/adv_20161102E.html
Reported-by: Cure53
Previously it only held references to them, which was reckless as the
thread lock was released so the cookies could get modified by other
handles that share the same cookie jar over the share interface.
CVE-2016-8623
Bug: https://curl.haxx.se/docs/adv_20161102I.html
Reported-by: Cure53
- Change initial message box to mention delay when downloading/parsing.
Since there is no progress meter it was somewhat unexpected that after
choosing a filename nothing appears to happen, when actually the cert
data is in the process of being downloaded and parsed.
- Warn if OpenSSL is not present.
- Use a UTF-8 stream to make the ca-bundle data.
- Save the UTF-8 ca-bundle stream as binary so that no BOM is added.
---
This is a follow-up to d2c6d15 which switched mk-ca-bundle.vbs output to
ANSI due to corrupt UTF-8 output, now fixed.
This change completes making the default certificate bundle output of
mk-ca-bundle.vbs as close as possible to that of mk-ca-bundle.pl, which
should make it easier to review any difference between their output.
Ref: https://github.com/curl/curl/pull/1012
Bring the VBScript version more in line with the perl version:
- Change timestamp to UTC.
- Change URL retrieval to HTTPS-only by default.
- Comment out the options that disabled SSL cert checking by default.
- Assume OpenSSL is present, get SHA256. And add a flag to toggle it.
- Fix cert issuer name output.
The cert issuer output is now ansi, converted from UTF-8. Prior to this
it was corrupt UTF-8. It turns out though we can work with UTF-8 the
FSO object that writes ca-bundle can't write UTF-8, so there will have
to be some alternative if UTF-8 is needed (like an ADODB.Stream).
- Disable the certificate text info feature.
The certificate text info doesn't work properly with any recent OpenSSL.
- Change all predefined Mozilla URLs to HTTPS (Gregory Szorc).
- New option -k to allow URLs other than HTTPS and enable HTTP fallback.
Prior to this change the default URL retrieval mode was to fall back to
HTTP if HTTPS didn't work.
Reported-by: Gregory Szorc
Closes#1012
Several independent reports on infinite loops hanging in the
close_all_connections() function when closing a multi handle, can be
fixed by first marking the connection to get closed before calling
Curl_disconnect.
This is more fixing-the-symptom rather than the underlying problem
though.
Bug: https://curl.haxx.se/mail/lib-2016-10/0011.html
Bug: https://curl.haxx.se/mail/lib-2016-10/0059.html
Reported-by: Dan Fandrich, Valentin David, Miloš Ljumović
In short the easy handle needs to be disconnected from its connection at
this point since the connection still is serving other easy handles.
In our app we can reliably reproduce a crash in our http2 stress test
that is fixed by this change. I can't easily reproduce the same test in
a small example.
This is the gdb/asan output:
==11785==ERROR: AddressSanitizer: heap-use-after-free on address 0xe9f4fb80 at pc 0x09f41f19 bp 0xf27be688 sp 0xf27be67c
READ of size 4 at 0xe9f4fb80 thread T13 (RESOURCE_HTTP)
#0 0x9f41f18 in curl_multi_remove_handle /path/to/source/3rdparty/curl/lib/multi.c:666
0xe9f4fb80 is located 0 bytes inside of 1128-byte region [0xe9f4fb80,0xe9f4ffe8)
freed by thread T13 (RESOURCE_HTTP) here:
#0 0xf7b1b5c2 in __interceptor_free /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_malloc_linux.cc:45
#1 0x9f7862d in conn_free /path/to/source/3rdparty/curl/lib/url.c:2808
#2 0x9f78c6a in Curl_disconnect /path/to/source/3rdparty/curl/lib/url.c:2876
#3 0x9f41b09 in multi_done /path/to/source/3rdparty/curl/lib/multi.c:615
#4 0x9f48017 in multi_runsingle /path/to/source/3rdparty/curl/lib/multi.c:1896
#5 0x9f490f1 in curl_multi_perform /path/to/source/3rdparty/curl/lib/multi.c:2123
#6 0x9c4443c in perform /path/to/source/src/net/resourcemanager/ResourceManagerCurlThread.cpp:854
#7 0x9c445e0 in ...
#8 0x9c4cf1d in ...
#9 0xa2be6b5 in ...
#10 0xf7aa5780 in asan_thread_start /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_interceptors.cc:226
#11 0xf4d3a16d in __clone (/lib/i386-linux-gnu/libc.so.6+0xe716d)
previously allocated by thread T13 (RESOURCE_HTTP) here:
#0 0xf7b1ba27 in __interceptor_calloc /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_malloc_linux.cc:70
#1 0x9f7dfa6 in allocate_conn /path/to/source/3rdparty/curl/lib/url.c:3904
#2 0x9f88ca0 in create_conn /path/to/source/3rdparty/curl/lib/url.c:5797
#3 0x9f8c928 in Curl_connect /path/to/source/3rdparty/curl/lib/url.c:6438
#4 0x9f45a8c in multi_runsingle /path/to/source/3rdparty/curl/lib/multi.c:1411
#5 0x9f490f1 in curl_multi_perform /path/to/source/3rdparty/curl/lib/multi.c:2123
#6 0x9c4443c in perform /path/to/source/src/net/resourcemanager/ResourceManagerCurlThread.cpp:854
#7 0x9c445e0 in ...
#8 0x9c4cf1d in ...
#9 0xa2be6b5 in ...
#10 0xf7aa5780 in asan_thread_start /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_interceptors.cc:226
#11 0xf4d3a16d in __clone (/lib/i386-linux-gnu/libc.so.6+0xe716d)
SUMMARY: AddressSanitizer: heap-use-after-free /path/to/source/3rdparty/curl/lib/multi.c:666 in curl_multi_remove_handle
Shadow bytes around the buggy address:
0x3d3e9f20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9f30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9f40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9f50: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
0x3d3e9f60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x3d3e9f70:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9f80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9f90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9fa0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9fb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x3d3e9fc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==11785==ABORTING
Thread 14 "RESOURCE_HTTP" received signal SIGABRT, Aborted.
[Switching to Thread 0xf27bfb40 (LWP 12324)]
0xf7fd8be9 in __kernel_vsyscall ()
(gdb) bt
#0 0xf7fd8be9 in __kernel_vsyscall ()
#1 0xf4c7ee89 in __GI_raise (sig=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#2 0xf4c803e7 in __GI_abort () at abort.c:89
#3 0xf7b2ef2e in __sanitizer::Abort () at /opt/toolchain/src/gcc-6.2.0/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cc:122
#4 0xf7b262fa in __sanitizer::Die () at /opt/toolchain/src/gcc-6.2.0/libsanitizer/sanitizer_common/sanitizer_common.cc:145
#5 0xf7b21ab3 in __asan::ScopedInErrorReport::~ScopedInErrorReport (this=0xf27be171, __in_chrg=<optimized out>) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_report.cc:689
#6 0xf7b214a5 in __asan::ReportGenericError (pc=166993689, bp=4068206216, sp=4068206204, addr=3925146496, is_write=false, access_size=4, exp=0, fatal=true) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_report.cc:1074
#7 0xf7b21fce in __asan::__asan_report_load4 (addr=3925146496) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_rtl.cc:129
#8 0x09f41f19 in curl_multi_remove_handle (multi=0xf3406080, data=0xde582400) at /path/to/source3rdparty/curl/lib/multi.c:666
#9 0x09f6b277 in Curl_close (data=0xde582400) at /path/to/source3rdparty/curl/lib/url.c:415
#10 0x09f3354e in curl_easy_cleanup (data=0xde582400) at /path/to/source3rdparty/curl/lib/easy.c:860
#11 0x09c6de3f in ...
#12 0x09c378c5 in ...
#13 0x09c48133 in ...
#14 0x09c4d092 in ...
#15 0x0a2be6b6 in ...
#16 0xf7aa5781 in asan_thread_start (arg=0xf2d22938) at /opt/toolchain/src/gcc-6.2.0/libsanitizer/asan/asan_interceptors.cc:226
#17 0xf5de52b5 in start_thread (arg=0xf27bfb40) at pthread_create.c:333
#18 0xf4d3a16e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:114
Fixes#1083
The closure handle only ever has default timeouts set. To improve the
state somewhat we clone the timeouts from each added handle so that the
closure handle always has the same timeouts as the most recently added
easy handle.
Fixes#739
Curl_select_ready() was the former API that was replaced with
Curl_select_check() a while back and the former arg setup was provided
with a define (in order to leave existing code unmodified).
Now we instead offer SOCKET_READABLE and SOCKET_WRITABLE for the most
common shortcuts where only one socket is checked. They're also more
visibly macros.
- Change back behavior so that pipelining is considered possible for
connections that have not yet reached the protocol level.
This is a follow-up to e5f0b1a which had changed the behavior of
checking if pipelining is possible to ignore connections that had
'bits.close' set. Connections that have not yet reached the protocol
level also have that bit set, and we need to consider pipelining
possible on those connections.
This fixes a merge error in commit 7f3df80 caused by commit 332e8d6.
Additionally, this changes Curl_verify_windows_version for Windows App
builds to assume to always be running on the target Windows version.
There seems to be no way to determine the Windows version from a
UWP app. Neither GetVersion(Ex), nor VerifyVersionInfo, nor the
Version Helper functions are supported.
Bug: https://github.com/curl/curl/pull/820#issuecomment-250889878
Reported-by: Paul Joyce
Closes https://github.com/curl/curl/pull/1048
No longer attempt to use "doomed" to-be-closed connections when
pipelining. Prior to this change connections marked for deletion (e.g.
timeout) would be erroneously used, resulting in sporadic crashes.
As originally reported and fixed by Carlo Wood (origin unknown).
Bug: https://github.com/curl/curl/issues/627
Reported-by: Rider Linden
Closes https://github.com/curl/curl/pull/1075
Participation-by: nopjmp@users.noreply.github.com
Not all reply messages were properly checked for their lengths, which
made it possible to access uninitialized memory (but this does not lead
to out of boundary accesses).
Closes#1052
... it no longer takes printf() arguments since it was only really taken
advantage by one user and it was not written and used in a safe
way. Thus the 'f' is removed from the function name and the proto is
changed.
Although the current code wouldn't end up in badness, it was a risk that
future changes could end up springf()ing too large data or passing in a
format string inadvertently.
The previous use of snprintf() could make libcurl silently truncate some
input data and not report that back on overly large input, which could
make data get sent over the network in a bad format.
Example:
$ curl --form 'a=b' -H "Content-Type: $(perl -e 'print "A"x4100')"
Cokie with the same domain but different tailmatching property are now
considered different and do not replace each other. If header contains
following lines then two cookies will be set: Set-Cookie: foo=bar;
domain=.foo.com; expires=Thu Mar 3 GMT 8:56:27 2033 Set-Cookie: foo=baz;
domain=foo.com; expires=Thu Mar 3 GMT 8:56:27 2033
This matches Chrome, Opera, Safari, and Firefox behavior. When sending
stored tokens to foo.com Chrome, Opera, Firefox store send them in the
stored order, while Safari pre-sort the cookies.
Closes#1050
Add the new option CURLOPT_KEEP_SENDING_ON_ERROR to control whether
sending the request body shall be completed when the server responds
early with an error status code.
This is suitable for manual NTLM authentication.
Reviewed-by: Jay Satiro
Closes https://github.com/curl/curl/pull/904
As it seems to be a rarely used cipher suite (for securely established
but _unencrypted_ connections), I believe it is fine not to provide an
alias for the misspelled variant.
LibreSSL defines `OPENSSL_VERSION_NUMBER` as `0x20000000L` for all
versions returning `LibreSSL/2.0.0` for any LibreSSL version.
This change provides a local OpenSSL_version_num function replacement
returning LIBRESSL_VERSION_NUMBER instead.
Closes#1029
The OpenSSL function CRYTPO_cleanup_all_ex_data() cannot be called
multiple times without crashing - and other libs might call it! We
basically cannot call it without risking a crash. The function is a
no-op since OpenSSL 1.1.0.
Not calling this function only risks a small memory leak with OpenSSL <
1.1.0.
Bug: https://curl.haxx.se/mail/lib-2016-09/0045.html
Reported-by: Todd Short
OpenSSL 1.0.1 and 1.0.2 build an error queue that is stored per-thread
so we need to clean it when easy handles are freed, in case the thread
will be killed in which the easy handle was used. All OpenSSL code in
libcurl should extract the error in association with the error already
so clearing this queue here should be harmless at worst.
Fixes#964
NTLM support with mbedTLS was added in 497e7c9 but requires that mbedTLS
is built with the MD4 functions available, which it isn't in default
builds. This now adapts if the funtion isn't there and builds libcurl
without NTLM support if so.
Fixes#1004
... like when a HTTP/0.9 response comes back without any headers at all
and just a body this now prevents that body from being sent to the
callback etc.
Adapted test 1144 to verify.
Fixes#973
Assisted-by: Ray Satiro
Detect support for compiler symbol visibility flags and apply those
according to CURL_HIDDEN_SYMBOLS option.
It should work true to the autotools build except it tries to unhide
symbols on Windows when requested and prints warning if it fails.
Ref: https://github.com/curl/curl/issues/981#issuecomment-242665951
Reported-by: Daniel Stenberg
... by partially reverting f975f06033. The allocation could be made by
OpenSSL so the free must be made with OPENSSL_free() to avoid problems.
Reported-by: Harold Stuart
Fixes#1005
... by making sure we don't count down the "upload left" counter when the
uploaded size is unknown and then it can be allowed to continue forever.
Fixes#996
Since we're using CURLE_FTP_WEIRD_SERVER_REPLY in imap, pop3 and smtp as
more of a generic "failed to parse" introduce an alias without FTP in
the name.
Closes https://github.com/curl/curl/pull/975
This hash is used to verify the original downloaded certificate bundle
and also included in the generated bundle's comment header. Also
rename related internal symbols to algorithm-agnostic names.
CURLINFO_SSL_VERIFYRESULT does not get the certificate verification
result when SSL_connect fails because of a certificate verification
error.
This fix saves the result of SSL_get_verify_result so that it is
returned by CURLINFO_SSL_VERIFYRESULT.
Closes https://github.com/curl/curl/pull/995
While noErr and errSecSuccess are defined as the same value, the API
documentation states that SecPKCS12Import() returns errSecSuccess if
there were no errors in importing. Ensure that a future change of the
defined value doesn't break (however unlikely) and be consistent with
the API docs.
Speed limits (from CURLOPT_MAX_RECV_SPEED_LARGE &
CURLOPT_MAX_SEND_SPEED_LARGE) were applied simply by comparing limits
with the cumulative average speed of the entire transfer; While this
might work at times with good/constant connections, in other cases it
can result to the limits simply being "ignored" for more than "short
bursts" (as told in man page).
Consider a download that goes on much slower than the limit for some
time (because bandwidth is used elsewhere, server is slow, whatever the
reason), then once things get better, curl would simply ignore the limit
up until the average speed (since the beginning of the transfer) reached
the limit. This could prove the limit useless to effectively avoid
using the entire bandwidth (at least for quite some time).
So instead, we now use a "moving starting point" as reference, and every
time at least as much as the limit as been transferred, we can reset
this starting point to the current position. This gets a good limiting
effect that applies to the "current speed" with instant reactivity (in
case of sudden speed burst).
Closes#971
* Added description to Curl_sspi_free_identity()
* Added parameter and return explanations to Curl_sspi_global_init()
* Added parameter explaination to Curl_sspi_global_cleanup()
With HTTP/2 each transfer is made in an indivial logical stream over the
connection, making most previous errors that caused the connection to get
forced-closed now instead just kill the stream and not the connection.
Fixes#941
... instead of if() before the switch(), add a default to the switch so
that the compilers don't warn on "warning: enumeration value
'PLATFORM_DONT_CARE' not handled in switch" anymore.
- Disable ALPN on Wine.
- Don't pass input secbuffer when ALPN is disabled.
When ALPN support was added a change was made to pass an input secbuffer
to initialize the context. When ALPN is enabled the buffer contains the
ALPN information, and when it's disabled the buffer is empty. In either
case this input buffer caused problems with Wine and connections would
not complete.
Bug: https://github.com/curl/curl/issues/983
Reported-by: Christian Fillion
Serialise the call to PK11_FindSlotByName() to avoid spurious errors in
a multi-threaded environment. The underlying cause is a race condition
in nssSlot_IsTokenPresent().
Bug: https://bugzilla.mozilla.org/1297397Closes#985
When we're uploading using FTP and the server issues a tiny pause
between opening the connection to the client's secondary socket, the
client's initial poll() times out, which leads to second poll() which
does not wait for POLLIN on the secondary socket. So that poll() also
has to time out, creating a long (200ms) pause.
This patch adds the correct flag to the secondary socket, making the
second poll() correctly wait for the connection there too.
Signed-off-by: Ales Novak <alnovak@suse.cz>
Closes#978
Instead of displaying the requested hostname the one returned
by the SOCKS5 proxy server is used in case of connection error.
The requested hostname is displayed earlier in the connection sequence.
The upper-value of the port is moved to a temporary variable and
replaced with a 0-byte to make sure the hostname is 0-terminated.
Hooked up the HTTP authentication layer to query the new 'is mechanism
supported' functions when deciding what mechanism to use.
As per commit 00417fd66c existing functionality is maintained for now.
Hooked up the SASL authentication layer to query the new 'is mechanism
supported' functions when deciding what mechanism to use.
For now existing functionality is maintained.
As Windows SSPI authentication calls fail when a particular mechanism
isn't available, introduced these functions for DIGEST, NTLM, Kerberos 5
and Negotiate to allow both HTTP and SASL authentication the opportunity
to query support for a supported mechanism before selecting it.
For now each function returns TRUE to maintain compatability with the
existing code when called.
I discovered some people have been using "https://example.com" style
strings as proxy and it "works" (curl doesn't complain) because curl
ignores unknown schemes and then assumes plain HTTP instead.
I think this misleads users into believing curl uses HTTPS to proxies
when it doesn't. Now curl rejects proxy strings using unsupported
schemes instead of just ignoring and defaulting to HTTP.
Undo change introduced in d4643d6 which caused iPAddress match to be
ignored if dNSName was present but did not match.
Also, if iPAddress is present but does not match, and dNSName is not
present, fail as no-match. Prior to this change in such a case the CN
would be checked for a match.
Bug: https://github.com/curl/curl/issues/959
Reported-by: wmsch@users.noreply.github.com
Mark's new document about HTTP Retries
(https://mnot.github.io/I-D/httpbis-retry/) made me check our code and I
spotted that we don't retry failed HEAD requests which seems totally
inconsistent and I can't see any reason for that separate treatment.
So, no separate treatment for HEAD starting now. A HTTP request sent
over a reused connection that gets cut off before a single byte is
received will be retried on a fresh connection.
Made-aware-by: Mark Nottingham
Makes libcurl work in communication with gstreamer-based RTSP
servers. The original code validates the session id to be in accordance
with the RFC. I think it is better not to do that:
- For curl the actual content is a don't care.
- The clarity of the RFC is debatable, is $ allowed or only as \$, that
is imho not clear
- Gstreamer seems to url-encode the session id but % is not allowed by
the RFC
- less code
With this patch curl will correctly handle real-life lines like:
Session: biTN4Kc.8%2B1w-AF.; timeout=60
Bug: https://curl.haxx.se/mail/lib-2016-08/0076.html
All compilers used by cmake in Windows should support large files.
- Add test SIZEOF_OFF_T
- Remove outdated test SIZEOF_CURL_OFF_T
- Turn on USE_WIN32_LARGE_FILES in Windows
- Check for 'Largefile' during the features output
Since the server can at any time send a HTTP/2 frame to us, we need to
wait for the socket to be readable during all transfers so that we can
act on incoming frames even when uploading etc.
Reminded-by: Tatsuhiro Tsujikawa
In order to make MBEDTLS_DEBUG work, the debug threshold must be unequal
to 0. This patch also adds a comment how mbedtls must be compiled in
order to make debugging work, and explains the possible debug levels.
After a few wasted hours hunting down the reason for slowness during a
TLS handshake that turned out to be because of TCP_NODELAY not being
set, I think we have enough motivation to toggle the default for this
option. We now enable TCP_NODELAY by default and allow applications to
switch it off.
This also makes --tcp-nodelay unnecessary, but --no-tcp-nodelay can be
used to disable it.
Thanks-to: Tim Rühsen
Bug: https://curl.haxx.se/mail/lib-2016-06/0143.html
When input stream for curl is stdin and input stream is not a file but
generated by a script then curl can truncate data transfer to arbitrary
size since a partial packet is treated as end of transfer by TFTP.
Fixes#857
Previously, passing a timeout of zero to Curl_expire() was a magic code
for clearing all timeouts for the handle. That is now instead made with
the new Curl_expire_clear() function and thus a 0 timeout is fine to set
and will trigger a timeout ASAP.
This will help removing short delays, in particular notable when doing
HTTP/2.
Regression added in 790d6de485. The was then added to avoid one
particular transfer to starve out others. But when aborting due to
reading the maxcount, the connection must be marked to be read from
again without first doing a select as for some protocols (like SFTP/SCP)
the data may already have been read off the socket.
Reported-by: Dan Donahue
Bug: https://curl.haxx.se/mail/lib-2016-07/0057.html
If a call to GetSystemDirectory fails, the `path` pointer that was
previously allocated would be leaked. This makes sure that `path` is
always freed.
Closes#938
This is a follow up to the parent commit dcdd4be which fixes one leak
but creates another by failing to free the credentials handle if out of
memory. Also there's a second location a few lines down where we fail to
do same. This commit fixes both of those issues.
- the expression of an 'if' was always true
- a 'while' contained a condition that was always true
- use 'if(k->exp100 > EXP100_SEND_DATA)' instead of 'if(k->exp100)'
- fixed a typo
Closes#889
... as otherwise we could get a 0 which would count as no error and we'd
wrongly continue and could end up segfaulting.
Bug: https://curl.haxx.se/mail/lib-2016-06/0052.html
Reported-by: 暖和的和暖
Prior to this change we called Curl_ssl_getsessionid and
Curl_ssl_addsessionid regardless of whether session ID reusing was
enabled. According to comments that is in case session ID reuse was
disabled but then later enabled.
The old way was not intuitive and probably not something users expected.
When a user disables session ID caching I'd guess they don't expect the
session ID to be cached anyway in case the caching is later enabled.
- Enable protocol family logic for IPv6 resolves even when support
for synthesized addresses is enabled.
This is a follow up to the parent commit that added support for
synthesized IPv6 addresses from IPv4 on iOS/OS X. The protocol family
logic needed for IPv6 was inadvertently excluded if support for
synthesized addresses was enabled.
Bug: https://github.com/curl/curl/issues/863
Ref: https://github.com/curl/curl/pull/866
Ref: https://github.com/curl/curl/pull/867
Use getaddrinfo() to resolve the IPv4 address literal on iOS/Mac OS X.
If the current network interface doesn’t support IPv4, but supports
IPv6, NAT64, and DNS64.
Closes#866Fixes#863
Calling QueryContextAttributes with SECPKG_ATTR_APPLICATION_PROTOCOL
fails on Windows < 8.1 so we need to disable ALPN on these OS versions.
Inspiration provide by: Daniel Seither
Closes#848Fixes#840
- Change the parser to not require a minor version for HTTP/2.
HTTP/2 connection reuse broke when we changed from HTTP/2.0 to HTTP/2
in 8243a95 because the parser still expected a minor version.
Bug: https://github.com/curl/curl/issues/855
Reported-by: Andrew Robbins, Frank Gevaerts
Sessionid cache management is inseparable from managing individual
session lifetimes. E.g. for reference-counted sessions (like those in
SChannel and OpenSSL engines) every session addition and removal
should be accompanied with refcount increment and decrement
respectively. Failing to do so synchronously leads to a race condition
that causes symptoms like use-after-free and memory corruption.
This commit:
- makes existing session cache locking explicit, thus allowing
individual engines to manage lock's scope.
- fixes OpenSSL and SChannel engines by putting refcount management
inside this lock's scope in relevant places.
- adds these explicit locking calls to other engines that use
sessionid cache to accommodate for this change. Note, however,
that it is unknown whether any of these engines could also have
this race.
Bug: https://github.com/curl/curl/issues/815Fixes#815Closes#847
Mostly in order to support broken web sites that redirect to broken URLs
that are accepted by browsers.
Browsers are typically even more leniant than this as the WHATWG URL
spec they should allow an _infinite_ amount. I tested 8000 slashes with
Firefox and it just worked.
Added test case 1141, 1142 and 1143 to verify the new parser.
Closes#791
Regression from the previous *printf() rearrangements, this file missed to
include the correct header to make sure snprintf() works universally.
Reported-by: Moti Avrahami
Bug: https://curl.haxx.se/mail/lib-2016-05/0196.html
While compiling lib/curl_multibyte.c with '-DUSE_WIN32_IDN' etc. I was
getting:
f:\mingw32\src\inet\curl\lib\memdebug.h(38): error C2054: expected '('
to follow 'CURL_EXTERN'
f:\mingw32\src\inet\curl\lib\memdebug.h(38): error C2085:
'curl_domalloc': not in formal parameter list
...as otherwise the TLS libs will skip the CN/SAN check and just allow
connection to any server. curl previously skipped this function when SNI
wasn't used or when connecting to an IP address specified host.
CVE-2016-3739
Bug: https://curl.haxx.se/docs/adv_20160518A.html
Reported-by: Moti Avrahami
CID 1024412: Memory - illegal accesses (OVERRUN). Claimed to happen when
we run over 'workend' but the condition says <= workend and for all I
can see it should be safe. Compensating for the warning by adding a byte
margin in the buffer.
Also, removed the extra brace level indentation in the code and made it
so that 'workend' is only assigned once within the function.
The proper FTP wildcard init is now more properly done in Curl_pretransfer()
and the corresponding cleanup in Curl_close().
The previous place of init/cleanup code made the internal pointer to be NULL
when this feature was used with the multi_socket() API, as it was made within
the curl_multi_perform() function.
Reported-by: Jonathan Cardoso Machado
Fixes#800
Prior to this change a width arg could be erroneously output, and also
width and precision args could not be used together without crashing.
"%0*d%s", 2, 9, "foo"
Before: "092"
After: "09foo"
"%*.*s", 5, 2, "foo"
Before: crash
After: " fo"
Test 557 is updated to verify this and more
The new way of disabling certificate verification doesn't work on
Mountain Lion (OS X 10.8) so we need to use the old way in that version
too. I've tested this solution on versions 10.7.5, 10.8, 10.9, 10.10.2
and 10.11.
Closes#802
curl's representation of HTTP/2 responses involves transforming the
response to a format that is similar to HTTP/1.1. Prior to this change,
curl would do this by separating header names and values with only a
colon, without introducing a space after the colon.
While this is technically a valid way to represent a HTTP/1.1 header
block, it is much more common to see a space following the colon. This
change introduces that space, to ensure that incautious tools are safely
able to parse the header block.
This also ensures that the difference between the HTTP/1.1 and HTTP/2
response layout is as minimal as possible.
Bug: https://github.com/curl/curl/issues/797Closes#798Fixes#797
... introduced in curl-7_48_0-293-g2968c83:
Error: COMPILER_WARNING:
lib/vtls/openssl.c: scope_hint: In function ‘Curl_ossl_check_cxn’
lib/vtls/openssl.c:767:15: warning: conversion to ‘int’ from ‘ssize_t’
may alter its value [-Wconversion]
- In the case of recv error, limit returning 'connection still in place'
to EINPROGRESS, EAGAIN and EWOULDBLOCK.
This is an improvement on the parent commit which changed the openssl
connection check to use recv MSG_PEEK instead of SSL_peek.
Ref: https://github.com/curl/curl/commit/856baf5#comments
Calling SSL_peek can cause bytes to be read from the raw socket which in
turn can upset the select machinery that determines whether there's data
available on the socket.
Since Curl_ossl_check_cxn only tries to determine whether the socket is
alive and doesn't actually need to see the bytes SSL_peek seems like
the wrong function to call.
We're able to occasionally reproduce a connect timeout due to this
bug. What happens is that Curl doesn't know to call SSL_connect again
after the peek happens since data is buffered in the SSL buffer and thus
select won't fire for this socket.
Closes#795
Only protocols that actually have a protocol registered for ALPN and NPN
should try to get that negotiated in the TLS handshake. That is only
HTTPS (well, http/1.1 and http/2) right now. Previously ALPN and NPN
would wrongly be used in all handshakes if libcurl was built with it
enabled.
Reported-by: Jay Satiro
Fixes#789
Sometimes, in systems with both ipv4 and ipv6 addresses but where the
network doesn't support ipv6, Curl_is_connected returns an error
(intermittently) even if the ipv4 socket connects successfully.
This happens because there's a for-loop that iterates on the sockets but
the error variable is not resetted when the ipv4 is checked and is ok.
This patch fixes this problem by setting error to 0 when checking the
second socket and not having a result yet.
Fixes#794
curl_printf.h defines printf to curl_mprintf, etc. This can cause
problems with external headers which may use
__attribute__((format(printf, ...))) markers etc.
To avoid that they cause problems with system includes, we include
curl_printf.h after any system headers. That makes the three last
headers to always be, and we keep them in this order:
curl_printf.h
curl_memory.h
memdebug.h
None of them include system headers, they all do funny #defines.
Reported-by: David Benjamin
Fixes#743
Mostly because they're not needed, because memdebug.h is always included
last of all headers so the others already included the correct ones.
But also, starting now we don't want this to accidentally include any
system headers, as the header included _before_ this header may add
defines and other fun stuff that we won't want used in system includes.
Previously, connections were closed immediately before the user had a
chance to extract the socket when the proxy required Negotiate
authentication.
This regression was brought in with the security fix in commit
79b9d5f1a4Closes#655
WinSock destroys recv() buffer if send() is failed. As result - server
response may be lost if server sent it while curl is still sending
request. This behavior noticeable on HTTP server short replies if
libcurl use several send() for request (usually for POST request).
To workaround this problem, libcurl use recv() before every send() and
keeps received data in intermediate buffer for further processing.
Fixes: #657Closes: #668
This commit fixes a Clang warning introduced in curl-7_48_0-190-g8f72b13:
Error: CLANG_WARNING:
lib/connect.c:1120:11: warning: The right operand of '==' is a garbage value
1118| }
1119|
1120|-> if(-1 == rc)
1121| error = SOCKERRNO;
1122| }
... but output non-stripped version of the line, even if that then can
make the script identify the wrong position in the line at
times. Showing the line stripped (ie without comments) is just too
surprising.
- Error if a header line is larger than supported.
- Warn if cumulative header line length may be larger than supported.
- Allow spaces when parsing the path component.
- Make sure each header line ends in \r\n. This fixes an out of bounds.
- Disallow header continuation lines until we decide what to do.
Ref: https://github.com/curl/curl/issues/659
Ref: https://github.com/curl/curl/pull/663
This commit ensures that streams which was closed in on_stream_close
callback gets passed to http2_handle_stream_close. Previously, this
might not happen. To achieve this, we increment drain property to
forcibly call recv function for that stream.
To more accurately check that we have no pending event before shutting
down HTTP/2 session, we sum up drain property into
http_conn.drain_total. We only shutdown session if that value is 0.
With this commit, when stream was closed before reading response
header fields, error code CURLE_HTTP2_STREAM is returned even if
HTTP/2 level error is NO_ERROR. This signals the upper layer that
stream was closed by error just like TCP connection close in HTTP/1.
Ref: https://github.com/curl/curl/issues/659
Ref: https://github.com/curl/curl/pull/663
This commit ensures that data from network are processed before HTTP/2
session is terminated. This is achieved by pausing nghttp2 whenever
different stream than current easy handle receives data.
This commit also fixes the bug that sometimes processing hangs when
multiple HTTP/2 streams are multiplexed.
Ref: https://github.com/curl/curl/issues/659
Ref: https://github.com/curl/curl/pull/663
Previously, when a stream was closed with other than NGHTTP2_NO_ERROR
by RST_STREAM, underlying TCP connection was dropped. This is
undesirable since there may be other streams multiplexed and they are
very much fine. This change introduce new error code
CURLE_HTTP2_STREAM, which indicates stream error that only affects the
relevant stream, and connection should be kept open. The existing
CURLE_HTTP2 means connection error in general.
Ref: https://github.com/curl/curl/issues/659
Ref: https://github.com/curl/curl/pull/663
... but ignore EAGAIN if the stream has ended so that we don't end up in
a loop. This is a follow-up to c8ab613 in order to avoid the problem
d261652 was made to fix.
Reported-by: Jay Satiro
Clues-provided-by: Tatsuhiro Tsujikawa
Discussed in #750
As these two options provide identical functionality, the former for
SOCK5 proxies and the latter for HTTP proxies, merged the two options
together.
As such CURLOPT_SOCKS5_GSSAPI_SERVICE is marked as deprecated as of
7.49.0.
Calculate the service name and proxy service names locally, rather than
in url.c which will allow for us to support overriding the service name
for other protocols such as FTP, IMAP, POP3 and SMTP.
It turns out the google GFE HTTP/2 servers send a PING frame immediately
after a stream ends and its last DATA has been received by curl. So if
we don't drain that from the socket, it makes the socket readable in
subsequent checks and libcurl then (wrongly) assumes the connection is
dead when trying to reuse the connection.
Reported-by: Joonas Kuorilehto
Discussed in #750
Although this should never happen due to the relationship between the
'mech' and 'resp' variables, and the way they are allocated together,
it does cause problems for code analysis tools:
V595 The 'mech' pointer was utilized before it was verified against
nullptr. Check lines: 376, 381. curl_sasl.c 376
Bug: https://github.com/curl/curl/issues/745
Reported-by: Alexis La Goutte
This wouldn't cause a problem because of the way the function is called,
but prior to this change, we were processing the challenge message when
the credentials were NULL rather than when the challenge message was
populated.
This also brings this part of the Kerberos 5 code in line with the
Negotiate code.
Although mutual authentication is currently turned off and can only be
enabled by changing libcurl source code, authentication using Kerberos
5 has been broken since commit 79543caf90 in this use case.
This wouldn't cause a problem because of the way the function is called,
but prior to this change, we were processing the challenge message when
the credentials were NULL rather than when the challenge message was
populated.
This also brings this part of the Kerberos 5 code in line with the
Negotiate code.
