As the loop discards cookies without domain set. This bug would lead to
qsort() trying to sort uninitialized pointers. We have however not found
it a security problem.
Reported-by: Paul Dreik
Closes#4386
If the input hostname is "[", hlen will underflow to max of size_t when
it is subtracted with 2.
hostname[hlen] will then cause a warning by ubsanitizer:
runtime error: addition of unsigned offset to 0x<snip> overflowed to
0x<snip>
I think that in practice, the generated code will work, and the output
of hostname[hlen] will be the first character "[".
This can be demonstrated by the following program (tested in both clang
and gcc, with -O3)
int main() {
char* hostname=strdup("[");
size_t hlen = strlen(hostname);
hlen-=2;
hostname++;
printf("character is %d\n",+hostname[hlen]);
free(hostname-1);
}
I found this through fuzzing, and even if it seems harmless, the proper
thing is to return early with an error.
Closes#4389
CURLU_NO_AUTHORITY is intended for use with unknown schemes (i.e. not
"file:///") to override cURL's default demand that an authority exists.
Closes#4349
If the :authority pseudo header field doesn't contain an explicit port,
we assume it is valid for the default port, instead of rejecting the
request for all ports.
Ref: https://curl.haxx.se/mail/lib-2019-09/0041.htmlCloses#4365
If you set the same URL for target as for DoH (and it isn't a DoH
server), like "https://example.com" in both, the easy handles used for
the DoH requests could be left "dangling" and end up not getting freed.
Reported-by: Paul Dreik
Closes#4366
The undefined behaviour is annoying when running fuzzing with
sanitizers. The codegen is the same, but the meaning is now not up for
dispute. See https://cppinsights.io/s/516a2ff4
By incrementing the pointer first, both gcc and clang recognize this as
a bswap and optimizes it to a single instruction. See
https://godbolt.org/z/994ZpxCloses#4350
Added unit test case 1655 to verify.
Close#4352
the code correctly finds the flaws in the old code,
if one temporarily restores doh.c to the old version.
This is a protocol violation but apparently there are legacy proprietary
servers doing this.
Added test 336 and 337 to verify.
Reported-by: Philippe Marguinaud
Closes#4339
For FTPS transfers, curl gets close_notify on the data connection
without that being a signal to close the control connection!
Regression since 3f5da4e59a (7.65.0)
Reported-by: Zenju on github
Reviewed-by: Jay Satiro
Fixes#4329Closes#4340
Despite ldapp_err2string being documented by MS as returning a
PCHAR (char *), when UNICODE it is mapped to ldap_err2stringW and
returns PWCHAR (wchar_t *).
We have lots of code that expects ldap_err2string to return char *,
most of it failf used like this:
failf(data, "LDAP local: Some error: %s", ldap_err2string(rc));
Closes https://github.com/curl/curl/pull/4272
It needs to parse correctly. Otherwise it could be tricked into letting
through a-f using host names that libcurl would then resolve. Like
'[ab.be]'.
Reported-by: Thomas Vegas
Closes#4315
OpenSSL 1.1.0 adds SSL_CTX_set_<min|max>_proto_version() that we now use
when available. Existing code is preserved for older versions of
OpenSSL.
Closes#4304
Otherwise, a three byte response would make the smtp_state_ehlo_resp()
function misbehave.
Credit to OSS-Fuzz
Bug: https://crbug.com/oss-fuzz/16918
Assisted-by: Max Dymond
Closes#4287
This allows the function to figure out if a unix domain socket has a
file name or not associated with it! When a socket is created with
socketpair(), as done in the fuzzer testing, the path struct member is
uninitialized and must not be accessed.
Bug: https://crbug.com/oss-fuzz/16699Closes#4283
It could otherwise return an error even when closed correctly if GOAWAY
had been received previously.
Reported-by: Tom van der Woerdt
Fixes#4267Closes#4268
For a long time (since 7.28.1) we've returned error when setting the
value to 1 to make applications notice that we stopped supported the old
behavior for 1. Starting now, we treat 1 and 2 exactly the same.
Closes#4241
The quiche debug callback is global and can only be initialized once, so
make sure we don't do it multiple times (e.g. if multiple requests are
executed).
In addition this initializes the callback before the connection is
created, so we get logs for the handshake as well.
Closes#4236
When a username and password are provided in the URL, they were wrongly
removed from the stored URL so that subsequent uses of the same URL
wouldn't find the crendentials. This made doing HTTP auth with multiple
connections (like Digest) mishave.
Regression from 46e164069d (7.62.0)
Test case 335 added to verify.
Reported-by: Mike Crowe
Fixes#4228Closes#4229
SSL_VersionRangeGetDefault returns (TLSv1.0, TLSv1.2) as supported
range in NSS 3.45. It looks like the intention is to raise the minimum
version rather than lowering the maximum, so adjust accordingly. Note
that the caller (nss_setup_connect) initializes the version range to
(TLSv1.0, TLSv1.3), so there is no need to check for >= TLSv1.0 again.
Closes#4187
Reviewed-by: Daniel Stenberg
Reviewed-by: Kamil Dudka
This commit makes sending HTTP/3 request with nghttp3 work. It
minimally receives HTTP response and calls nghttp3 callbacks, but no
processing is made at the moment.
Closes#4215
This avoids EBADF errors from EPOLL_CTL_DEL operations in the
ephiperfifo.c example. EBADF is dangerous in multi-threaded
applications where I rely on epoll_ctl to operate on the same
epoll description from different threads.
Follow-up to eb9a604f8d
Bug: https://curl.haxx.se/mail/lib-2019-08/0026.htmlCloses#4211
So that users can mask in/out specific HTTP versions when Alt-Svc is
used.
- Removed "h2c" and updated test case accordingly
- Changed how the altsvc struct is laid out
- Added ifdefs to make the unittest run even in a quiche-tree
Closes#4201
RFC 7838 section 5:
When using an alternative service, clients SHOULD include an Alt-Used
header field in all requests.
Removed CURLALTSVC_ALTUSED again (feature is still EXPERIMENTAL thus
this is deemed ok).
You can disable sending this header just like you disable any other HTTP
header in libcurl.
Closes#4199
Even though it cannot fall-back to a lower HTTP version automatically. The
safer way to upgrade remains via CURLOPT_ALTSVC.
CURLOPT_H3 no longer has any bits that do anything and might be removed
before we remove the experimental label.
Updated the curl tool accordingly to use "--http3".
Closes#4197
This is only the libcurl part that provides the information. There's no
user of the parsed value. This change includes three new tests for the
parser.
Ref: #3794
Added the ability for the calling program to specify the authorisation
identity (authzid), the identity to act as, in addition to the
authentication identity (authcid) and password when using SASL PLAIN
authentication.
Fixes#3653Closes#3790
NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.
Closes https://github.com/curl/curl/pull/4186
Repeatedly we see problems where using curl_multi_wait() is difficult or
just awkward because if it has no file descriptor to wait for
internally, it returns immediately and leaves it to the caller to wait
for a small amount of time in order to avoid occasional busy-looping.
This is often missed or misunderstood, leading to underperforming
applications.
This change introduces curl_multi_poll() as a replacement drop-in
function that accepts the exact same set of arguments. This function
works identically to curl_multi_wait() - EXCEPT - for the case when
there's nothing to wait for internally, as then this function will by
itself wait for a "suitable" short time before it returns. This
effectiely avoids all risks of busy-looping and should also make it less
likely that apps "over-wait".
This also changes the curl tool to use this funtion internally when
doing parallel transfers and changes curl_easy_perform() to use it
internally.
Closes#4163
As the plan has been laid out in DEPRECATED. Update docs accordingly and
verify in test 1174. Now requires the option to be set to allow HTTP/0.9
responses.
Closes#4191
Allow pretty much anything to be part of the ALPN identifier. In
particular minus, which is used for "h3-20" (in-progress HTTP/3
versions) etc.
Updated test 356.
Closes#4182
If HTTPAUTH_GSSNEGOTIATE was used for a POST request and
gss_init_sec_context() failed, the POST request was sent
with empty body. This commit also restores the original
behavior of `curl --fail --negotiate`, which was changed
by commit 6c60355323.
Add regression tests 2077 and 2078 to cover this.
Fixes#3992Closes#4171
It was used (intended) to pass in the size of the 'socks' array that is
also passed to these functions, but was rarely actually checked/used and
the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries
that should be used instead.
Closes#4169
... to make CURLOPT_MAX_RECV_SPEED_LARGE and
CURLOPT_MAX_SEND_SPEED_LARGE work correctly on subsequent transfers that
reuse the same handle.
Fixed-by: Ironbars13 on github
Fixes#4084Closes#4161
If using the read callback for HTTP_POST, and POSTFIELDSIZE is not set,
automatically add a Transfer-Encoding: chunked header, same as it is
already done for HTTP_PUT, HTTP_POST_FORM and HTTP_POST_MIME. Update
test 1514 according to the new behaviour.
Closes#4138
- In curl_easy_reset attempt to resize the receive buffer to its default
size. If realloc fails then continue using the previous size.
Prior to this change curl_easy_reset did not properly handle resetting
the receive buffer (data->state.buffer). It reset the variable holding
its size (data->set.buffer_size) to the default size (READBUFFER_SIZE)
but then did not actually resize the buffer. If a user resized the
buffer by using CURLOPT_BUFFERSIZE to set the size smaller than the
default, later called curl_easy_reset and attempted to reuse the handle
then a heap overflow would very likely occur during that handle's next
transfer.
Reported-by: Felix Hädicke
Fixes https://github.com/curl/curl/issues/4143
Closes https://github.com/curl/curl/pull/4145
Specifying O_APPEND in conjunction with O_TRUNC and O_CREAT does not
make much sense. And this combination of flags is not accepted by all
SFTP servers (at least not Apache SSHD).
Fixes#4147Closes#4148
Curl_disconnect bails out if conn->easyq is not empty, detach_connection
needs to be called first to remove the current easy from the queue.
Fixes#4144Closes#4151
USe configure --with-ngtcp2 or --with-quiche
Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>
Closes#3500
Several reasons:
- we can't add everyone who's helping out so its unfair to just a few
selected ones.
- we already list all helpers in THANKS and in RELEASE-NOTES for each
release
- we don't want to give the impression that some parts of the code is
"owned" or "controlled" by specific persons
Assisted-by: Daniel Gustafsson
Closes#4129
PK11_IsPresent() checks for the token for the given slot is available,
and sets needlogin flags for the PK11_Authenticate() call. Should it
return false, we should however treat it as an error and bail out.
Closes https://github.com/curl/curl/pull/4110
All protocols except for CURLPROTO_FILE/CURLPROTO_SMB and their TLS
counterpart were allowed for redirect. This vastly broadens the
exploitation surface in case of a vulnerability such as SSRF [1], where
libcurl-based clients are forced to make requests to arbitrary hosts.
For instance, CURLPROTO_GOPHER can be used to smuggle any TCP-based
protocol by URL-encoding a payload in the URI. Gopher will open a TCP
connection and send the payload.
Only HTTP/HTTPS and FTP are allowed. All other protocols have to be
explicitly enabled for redirects through CURLOPT_REDIR_PROTOCOLS.
[1]: https://www.acunetix.com/blog/articles/server-side-request-forgery-vulnerability/
Signed-off-by: Linos Giannopoulos <lgian@skroutz.gr>
Closes#4094
With CURLOPT_TIMECONDITION set, a header is automatically added (e.g.
If-Modified-Since). Allow this to be replaced or suppressed with
CURLOPT_HTTPHEADER.
Fixes#4103Closes#4109
- Return CURLE_REMOTE_ACCESS_DENIED for SMB access denied on file open.
Prior to this change CURLE_REMOTE_FILE_NOT_FOUND was returned instead.
Closes https://github.com/curl/curl/pull/4095
There were a leftover few prototypes of Curl_ functions that we used to
export but no longer do, this removes those prototypes and cleans up any
comments still referring to them.
Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free()
Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn()
were made static in 05b100aee2.
Curl_http_perhapsrewind() made static in 574aecee20.
For the remainder, I didn't trawl the Git logs hard enough to capture
their exact time of deletion, but they were all gone: Curl_splayprint(),
Curl_http2_send_request(), Curl_global_host_cache_dtor(),
Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(),
Curl_http_auth_stage() and Curl_close_connections().
Closes#4096
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
The file suffix for dynamically loadable objects on macOS is .dylib,
which need to be added for the module definitions in order to get the
NSS TLS backend to work properly on macOS.
Closes https://github.com/curl/curl/pull/4046
The value of the maxPTDs parameter to PR_Init() has since at least
NSPR 2.1, which was released sometime in 1998, been marked ignored
as is accordingly not used in the initialization code. Setting it
to a value when calling PR_Init() is thus benign, but indicates an
intent which may be misleading. Reset the value to zero to improve
clarity.
Closes https://github.com/curl/curl/pull/4054
Change the logic around such that we only keep CRLs that NSS actually
ended up caching around for later deletion. If CERT_CacheCRL() fails
then there is little point in delaying the freeing of the CRL as it
is not used.
Closes https://github.com/curl/curl/pull/4053
Some editors and IDEs assume that source files use UTF-8 file encodings.
It also fixes the build with MSVC when /utf-8 command line option is
used (this option is mandatory for some other open-source projects, this
is useful when using the same options is desired for building all
libraries of a project).
Closes https://github.com/curl/curl/pull/4087
... since that needs UI_OpenSSL() which isn't provided when OpenSSL is
built with OPENSSL_NO_UI_CONSOLE which happens when OpenSSL is built for
UWP (with "VC-WIN32-UWP").
Reported-by: Vasily Lobaskin
Fixes#4073Closes#4077
The header buffer size calculation can from static analysis seem to
overlow as it performs an addition between two size_t variables and
stores the result in a size_t variable. Overflow is however guarded
against elsewhere since the input to the addition is regulated by
the maximum read buffer size. Clarify this with a comment since the
question was asked.
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
To make sure a HTTP/2 stream registers the end of stream.
Bug #4043 made me find this problem but this fix doesn't correct the
reported issue.
Closes#4068
Certinfo gives the same result for all OpenSSL versions.
Also made printing RSA pubkeys consistent with older versions.
Reported-by: Michael Wallner
Fixes#3706Closes#4030
OpenSSL used to call exit(1) on syntax errors in OPENSSL_config(),
which is why we switched to CONF_modules_load_file() and introduced
a comment stating why. This behavior was however changed in OpenSSL
commit abdd677125f3a9e3082f8c5692203590fdb9b860, so remove the now
outdated and incorrect comment. The mentioned commit also declares
OPENSSL_config() deprecated so keep the current coding.
Closes#4033
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Even though the variable was used in a DEBUGASSERT, GCC 8 warned in
debug mode:
krb5.c:324:17: error: unused variable 'maj' [-Werror=unused-variable]
Just suppress the warning and declare the variable unconditionally
instead of only for DEBUGBUILD (which also missed the check for
HAVE_ASSERT_H).
Closes https://github.com/curl/curl/pull/4020
- The transfer hashes weren't using the correct keys so removing entries
failed.
- Simplified the iteration logic over transfers sharing the same socket and
they now simply are set to expire and thus get handled in the "regular"
timer loop instead.
Reported-by: Tom van der Woerdt
Fixes#4012Closes#4014
Old connections are meant to expire from the connection cache after
CURLOPT_MAXAGE_CONN seconds. However, they actually expire after 1000x
that value. This occurs because a time value measured in milliseconds is
accidentally divided by 1M instead of by 1,000.
Closes https://github.com/curl/curl/pull/4013
Remove support for, references to and use of "cyaSSL" from the source
and docs. wolfSSL is the current name and there's no point in keeping
references to ancient history.
Assisted-by: Daniel Gustafsson
Closes#3903
Since more than one socket can be used by each transfer at a given time,
each sockhash entry how has its own hash table with transfers using that
socket.
In addition, the sockhash entry can now be marked 'blocked = TRUE'"
which then makes the delete function just set 'removed = TRUE' instead
of removing it "for real", as a way to not rip out the carpet under the
feet of a parent function that iterates over the transfers of that same
sockhash entry.
Reported-by: Tom van der Woerdt
Fixes#3961Fixes#3986Fixes#3995Fixes#4004Closes#3997
... so that timeouts or other state machine actions get going again
after a changing pause state. For example, if the last delivery was
paused there's no pending socket activity.
Reported-by: sstruchtrup on github
Fixes#3994Closes#4001
Responses with status codes 1xx, 204 or 304 don't have a response body. For
these, don't parse these headers:
- Content-Encoding
- Content-Length
- Content-Range
- Last-Modified
- Transfer-Encoding
This change ensures that HTTP/2 upgrades work even if a
"Content-Length: 0" or a "Transfer-Encoding: chunked" header is present.
Co-authored-by: Daniel Stenberg
Closes#3702Fixes#3968Closes#3977
An inner loop within the singlesocket() function wrongly re-used the
variable for the outer loop which then could cause an infinite
loop. Change to using a separate variable!
Reported-by: Eric Wu
Fixes#3970Closes#3973
Various functions called within Curl_http2_done() can have the
side-effect of setting the Easy connection into drain mode (by calling
drain_this()). However, the last time we unset this for a transfer (by
calling drained_transfer()) is at the beginning of Curl_http2_done().
If the Curl_easy is reused for another transfer, it is then stuck in
drain mode permanently, which in practice makes it unable to write any
data in the new transfer.
This fix moves the last call to drained_transfer() to later in
Curl_http2_done(), after the functions that could potentially call for a
drain.
Fixes#3966Closes#3967
Reported-by: Josie-H
This fixes the static dependency on iphlpapi.lib and allows curl to
build for targets prior to Windows Vista.
This partially reverts 170bd047.
Fixes#3960Closes#3958
... so that it has a sensible value when ConnectionExists() is called which
needs it set to differentiate host "bundles" correctly on port number!
Also, make conncache:hashkey() use correct port for bundles that are proxy vs
host connections.
Probably a regression from 7.62.0
Reported-by: Tom van der Woerdt
Fixes#3956Closes#3957
Only HTTP proxy use where multiple host names can be used over the same
connection should use the proxy host name for bundles.
Reported-by: Tom van der Woerdt
Fixes#3951Closes#3955
They need to be removed from the socket hash linked list with more care.
When sh_delentry() is called to remove a sockethash entry, remove all
individual transfers from the list first. To enable this, each Curl_easy struct
now stores a pointer to the sockethash entry to know how to remove itself.
Reported-by: Tom van der Woerdt and Kunal Ekawde
Fixes#3952Fixes#3904Closes#3953
Microsoft added support for Unix Domain Sockets in Windows 10 1803
(RS4). Rather than expect the user to enable Unix Domain Sockets by
uncommenting the #define that was added in 0fd6221f we use the RS4
pre-processor variable that is present in newer versions of the
Windows SDK.
Closes#3939
- Revert all commits related to the SASL authzid feature since the next
release will be a patch release, 7.65.1.
Prior to this change CURLOPT_SASL_AUTHZID / --sasl-authzid was destined
for the next release, assuming it would be a feature release 7.66.0.
However instead the next release will be a patch release, 7.65.1 and
will not contain any new features.
After the patch release after the reverted commits can be restored by
using cherry-pick:
git cherry-pick a14d72ca9499ff8c1cc36c2a8d520edf690
Details for all reverted commits:
Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()."
This reverts commit 0edf6907ae.
Revert "tests: Fix the line endings for the SASL alt-auth tests"
This reverts commit c2a8d52a13.
Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples"
This reverts commit 8c1cc369d0.
Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool"
This reverts commit a9499ff136.
Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID"
This reverts commit a14d72ca2f.
This reverts commit 3b06e68b77.
Clearly this change wasn't good enough as it broke CURLOPT_LOW_SPEED_LIMIT +
CURLOPT_LOW_SPEED_TIME
Reported-by: Dave Reisner
Fixes#3927Closes#3928
Added the ability for the calling program to specify the authorisation
identity (authzid), the identity to act as, in addition to the
authentication identity (authcid) and password when using SASL PLAIN
authentication.
Fixed#3653Closes#3790
If the proxy string is given as an IPv6 numerical address with a zone
id, make sure to use that for the connect to the proxy.
Reported-by: Edmond Yu
Fixes#3482Closes#3918
When running a multi TLS backend build the version string needs more
buffer space. Make the internal ssl_buffer stack buffer match the one
in Curl_multissl_version() to allow for the longer string. For single
TLS backend builds there is no use in extended to buffer. This is a
fallout from #3863 which fixes up the multi_ssl string generation to
avoid a buffer overflow when the buffer is too small.
Closes#3875
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Currently when the server responds with 401 on NTLM authenticated
connection (re-used) we consider it to have failed. However this is
legitimate and may happen when for example IIS is set configured to
'authPersistSingleRequest' or when the request goes thru a proxy (with
'via' header).
Implemented by imploying an additional state once a connection is
re-used to indicate that if we receive 401 we need to restart
authentication.
Missed in fe6049f0.
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.
Ref: #3876Closes#3883
Given that Curl_disconnect() calls Curl_http_auth_cleanup_ntlm() prior
to calling conn_shutdown() and it in turn performs this, there is no
need to perform the same action in conn_shutdown().
Closes#3881
In Curl_multissl_version() it was possible to overflow the passed in
buffer if the generated version string exceeded the size of the buffer.
Fix by inverting the logic, and also make sure to not exceed the local
buffer during the string generation.
Closes#3863
Reported-by: nevv on HackerOne/curl
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Codacy/CppCheck warns about this. Consistently use parentheses as we
already do in some places to silence the warning.
Closes https://github.com/curl/curl/pull/3866
Due to limitations in Curl_resolver_wait_resolv(), it doesn't work for
DOH resolves. This fix disables DOH for those.
Limitation added to KNOWN_BUGS.
Fixes#3850Closes#3857
This reverts commit b0972bc.
- No longer show verbose output for the conncache closure handle.
The offending commit was added so that the conncache closure handle
would inherit verbose mode from the user's easy handle. (Note there is
no way for the user to set options for the closure handle which is why
that was necessary.) Other debug settings such as the debug function
were not also inherited since we determined that could lead to crashes
if the user's per-handle private data was used on an unexpected handle.
The reporter here says he has a debug function to capture the verbose
output, and does not expect or want any output to stderr; however
because the conncache closure handle does not inherit the debug function
the verbose output for that handle does go to stderr.
There are other plausible scenarios as well such as the user redirects
stderr on their handle, which is also not inherited since it could lead
to crashes when used on an unexpected handle.
Short of allowing the user to set options for the conncache closure
handle I don't think there's much we can safely do except no longer
inherit the verbose setting.
Bug: https://curl.haxx.se/mail/lib-2019-05/0021.html
Reported-by: Kristoffer Gleditsch
Ref: https://github.com/curl/curl/pull/3598
Ref: https://github.com/curl/curl/pull/3618
Closes https://github.com/curl/curl/pull/3856
Older versions of OpenSSL report FIPS availabilty via an OPENSSL_FIPS
define. It uses this define to determine whether to publish -fips at
the end of the version displayed. Applications that utilize the version
reported by OpenSSL will see a mismatch if they compare it to what curl
reports, as curl is not modifying the version in the same way. This
change simply adds a check to see if OPENSSL_FIPS is defined, and will
alter the reported version to match what OpenSSL itself provides. This
only appears to be applicable in versions of OpenSSL <1.1.1
Closes#3771
Currently you can do things like --cert <(cat ./cert.crt) with (at least) the
openssl backend, but that doesn't work for nss because is_file rejects fifos.
I don't actually know if this is sufficient, nss might do things internally
(like seeking back) that make this not work, so actual testing is needed.
Closes#3807
... to make the host name "usable". Store the scope id and put it back
when extracting a URL out of it.
Also makes curl_url_set() syntax check CURLUPART_HOST.
Fixes#3817Closes#3822
As soon as a TLS backend gets ALPN conformation about the specific HTTP
version it can now set the multiplex situation for the "bundle" and
trigger moving potentially queued up transfers to the CONNECT state.
With transfers being queued up, we only move one at a a time back to the
CONNECT state but now we mark moved transfers so that when a moved
transfer is confirmed "successful" (it connected) it will trigger the
move of another pending transfer. Previously, it would otherwise wait
until the transfer was done before doing this. This makes queued up
pending transfers get processed (much) faster.
In case the name pointer isn't set (due to memory pressure most likely)
we need to skip the prefix matching and reject with a badcookie to avoid
a possible NULL pointer dereference.
Closes#3820#3821
Reported-by: Jonathan Moerman
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
This limits all accepted input strings passed to libcurl to be less than
CURL_MAX_INPUT_LENGTH (8000000) bytes, for these API calls:
curl_easy_setopt() and curl_url_set().
The 8000000 number is arbitrary picked and is meant to detect mistakes
or abuse, not to limit actual practical use cases. By limiting the
acceptable string lengths we also reduce the risk of integer overflows
all over.
NOTE: This does not apply to `CURLOPT_POSTFIELDS`.
Test 1559 verifies.
Closes#3805
Just like we do for mbed TLS, use our local implementation of MD4 when
OpenSSL doesn't support it. This allows a type-3 message to include the
NT response.
RFC 4616 specifies the authzid is optional in the client authentication
message and that the server will derive the authorisation identity
(authzid) from the authentication identity (authcid) when not specified
by the client.
ALTSVC requires Curl_get_line which is defined in lib/cookie.c inside a #if
check of HTTP and COOKIES. That makes Curl_get_line undefined if COOKIES is
disabled. Fix by splitting out the function into a separate file which can
be included where needed.
Closes#3717
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
The indentation from 211d5329 and 57d6d253 was a little strange as
parts didn't align correctly, uses 4 spaces rather than 2. Checked
the indentation of the original source so it aligns, albeit, using
curl style.
Commit 9081014 fixed most of the confusing issues between scope id and
scope however 844896d added bad limits checking assuming that the scope
is being set and not the scope id.
I have fixed the documentation so it all refers to scope ids.
In addition Curl_if2ip refered to the scope id as remote_scope_id which
is incorrect, so I renamed it to local_scope_id.
Adjusted-by: Daniel Stenberg
Closes#3655Closes#3765Fixes#3713
Only allow well formed decimal numbers in the input.
Document that the number MUST be between 1 and 65535.
Add tests to test 1560 to verify the above.
Ref: https://github.com/curl/curl/issues/3753Closes#3762
Without this, detecting and avoid reusing a closed TLS connection
(without a previous GOAWAY) when doing HTTP/2 is tricky.
Reported-by: Tom van der Woerdt
Fixes#3750Closes#3763
Functionally this doesn't change anything as we still use the username
for both the authorisation identity and the authentication identity.
Closes#3757
Remove the code too. The functionality has been disabled in code since
7.62.0. Setting this option will from now on simply be ignored and have
no function.
Closes#3654
- remove unused variables
- declare conditionally used variables conditionally
- suppress unused variable warnings in the CMake tests
- remove dead variable stores
- consistently use WIN32 macro to detect Windows
Closes https://github.com/curl/curl/pull/3739
Make functions no-ops if neither both USE_THREADS_POSIX and
HAVE_PTHREAD_H nor both USE_THREADS_WIN32 and HAVE_PROCESS_H are
defined. Previously, if only one of them was defined, there was either
code compiled that did nothing useful or the wrong header included for
the functions used.
Also, move POLARSSL_MUTEX_T define to implementation file as it's not
used externally.
Closes https://github.com/curl/curl/pull/3739
Fixes#3745Closes#3746
The following snippet
```
int main()
{
CURL* hCurlHandle = curl_easy_init();
curl_easy_setopt(hCurlHandle, CURLOPT_URL, "http://example.com");
curl_easy_setopt(hCurlHandle, CURLOPT_PROXY, "1");
curl_easy_perform(hCurlHandle);
curl_easy_cleanup(hCurlHandle);
return 0;
}
```
triggers the following Valgrind warning
```
==4125== Invalid read of size 8
==4125== at 0x4E7D1EE: Curl_llist_remove (llist.c:97)
==4125== by 0x4E7EF5C: detach_connnection (multi.c:798)
==4125== by 0x4E80545: multi_runsingle (multi.c:1451)
==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072)
==4125== by 0x4E766A0: easy_transfer (easy.c:625)
==4125== by 0x4E76915: easy_perform (easy.c:719)
==4125== by 0x4E7697C: curl_easy_perform (easy.c:738)
==4125== by 0x4008BE: main (in /home/even/curl/test)
==4125== Address 0x9b3d1d0 is 1,120 bytes inside a block of size 1,600 free'd
==4125== at 0x4C2ECF0: free (vg_replace_malloc.c:530)
==4125== by 0x4E62C36: conn_free (url.c:756)
==4125== by 0x4E62D34: Curl_disconnect (url.c:818)
==4125== by 0x4E48DF9: Curl_once_resolved (hostip.c:1097)
==4125== by 0x4E8052D: multi_runsingle (multi.c:1446)
==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072)
==4125== by 0x4E766A0: easy_transfer (easy.c:625)
==4125== by 0x4E76915: easy_perform (easy.c:719)
==4125== by 0x4E7697C: curl_easy_perform (easy.c:738)
==4125== by 0x4008BE: main (in /home/even/curl/test)
==4125== Block was alloc'd at
==4125== at 0x4C2F988: calloc (vg_replace_malloc.c:711)
==4125== by 0x4E6438E: allocate_conn (url.c:1654)
==4125== by 0x4E685B4: create_conn (url.c:3496)
==4125== by 0x4E6968F: Curl_connect (url.c:4023)
==4125== by 0x4E802E7: multi_runsingle (multi.c:1368)
==4125== by 0x4E8197C: curl_multi_perform (multi.c:2072)
==4125== by 0x4E766A0: easy_transfer (easy.c:625)
==4125== by 0x4E76915: easy_perform (easy.c:719)
==4125== by 0x4E7697C: curl_easy_perform (easy.c:738)
==4125== by 0x4008BE: main (in /home/even/curl/test)
```
This has been bisected to commit 2f44e94
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14109
Credit to OSS Fuzz
Just remove the redundant condition, which also makes it clear that
k->buf is always 0-terminated if this break is not hit.
Closes https://github.com/curl/curl/pull/3732
- Fix clang string-plus-int warning.
Clang 8 warns about adding a string to an int does not append to the
string. Indeed it doesn't, but that was not the intention either. Use
array indexing as suggested to silence the warning. There should be no
functional changes.
(In other words clang warns about "foo"+2 but not &"foo"[2] so use the
latter.)
smtp.c:1221:29: warning: adding 'int' to a string does not append to the
string [-Wstring-plus-int]
eob = strdup(SMTP_EOB + 2);
~~~~~~~~~~~~~~~~^~~~
Closes https://github.com/curl/curl/pull/3729
OAUTHBEARER tokens were incorrectly generated in a format similar to
XOAUTH2 tokens. These changes make OAUTHBEARER tokens conform to the
RFC7628.
Fixes: #2487
Reported-by: Paolo Mossino
Closes https://github.com/curl/curl/pull/3377
- make sure an already "owned" connection isn't returned unless
multiplexed.
- clear ->data when returning the connection to the cache again
Regression since 7.62.0 (probably in commit 1b76c38904)
Bug: https://curl.haxx.se/mail/lib-2019-03/0064.htmlCloses#3686
AmiSSL is an Amiga native library which provides a wrapper over OpenSSL.
It also requires all programs using it to use bsdsocket.library
directly, rather than accessing socket functions through clib, which
libcurl was not necessarily doing previously. Configure will now check
for the headers and ensure they are included if found.
Closes#3677
* Adjusted unit tests 2056, 2057
* do not generally close connections with CURLAUTH_NEGOTIATE after every request
* moved negotiatedata from UrlState to connectdata
* Added stream rewind logic for CURLAUTH_NEGOTIATE
* introduced negotiatedata::GSS_AUTHDONE and negotiatedata::GSS_AUTHSUCC
* Consider authproblem state for CURLAUTH_NEGOTIATE
* Consider reuse_forbid for CURLAUTH_NEGOTIATE
* moved and adjusted negotiate authentication state handling from
output_auth_headers into Curl_output_negotiate
* Curl_output_negotiate: ensure auth done is always set
* Curl_output_negotiate: Set auth done also if result code is
GSS_S_CONTINUE_NEEDED/SEC_I_CONTINUE_NEEDED as this result code may
also indicate the last challenge request (only works with disabled
Expect: 100-continue and CURLOPT_KEEP_SENDING_ON_ERROR -> 1)
* Consider "Persistent-Auth" header, detect if not present;
Reset/Cleanup negotiate after authentication if no persistent
authentication
* apply changes introduced with #2546 for negotiate rewind logic
Fixes#1261Closes#1975
The check that prevents payload from sending in case of authentication
doesn't check properly if the authentication is done or not.
They're cases where the proxy respond "200 OK" before sending
authentication challenge. This change takes care of that.
Fixes#2431Closes#3669
Coverity warned for two potentional "Use after free" cases. Both are false
positives because the memory wasn't used, it was only the actual pointer
value that was logged.
The fix still changes the order of execution to avoid the warnings.
Coverity CID 1443033 and 1443034
Closes#3671
This code was once used for the non multi-interface using code path, but
ever since easy_perform was turned into a wrapper around the multi
interface, this code path never runs.
Closes#3666
- Inherit SSL options for the doh handle but not SSL client certs,
SSL ALPN/NPN, SSL engine, SSL version, SSL issuer cert,
SSL pinned public key, SSL ciphers, SSL id cache setting,
SSL kerberos or SSL gss-api settings.
- Fix inheritance of verbose setting.
- Inherit NOSIGNAL.
There is no way for the user to set options for the doh (DNS-over-HTTPS)
handles and instead we inherit some options from the user's easy handle.
My thinking for the SSL options not inherited is they are most likely
not intended by the user for the DOH transfer. I did inherit insecure
because I think that should still be in control of the user.
Prior to this change doh did not work for me because CAINFO was not
inherited. Also verbose was set always which AFAICT was a bug (#3660).
Fixes https://github.com/curl/curl/issues/3660
Closes https://github.com/curl/curl/pull/3661
Since it can't be NULL and it makes Coverity believe we lack proper NULL
checks. Verified by test 659, landed in commit 15401fa886.
Pointed out by Coverity CID 1442746.
Assisted-by: Dan Fandrich
Fixes#3617Closes#3642
If the state machine isn't complete, didn't fail and it didn't return
due to blocking it can just as well loop again.
This addresses the problem with SFTP directory listings where we would
otherwise return back to the parent and as the multi state machine
doesn't have any code for using CURLM_CALL_MULTI_PERFORM for as long the
doing phase isn't complete, it would return out when in reality there
was more data to deal with.
Fixes#3506Closes#3644
- Change closure handle to receive verbose setting from the easy handle
most recently added via curl_multi_add_handle.
The closure handle is a special easy handle used for closing cached
connections. It receives limited settings from the easy handle most
recently added to the multi handle. Prior to this change that did not
include verbose which was a problem because on connection shutdown
verbose mode was not acknowledged.
Ref: https://github.com/curl/curl/pull/3598
Co-authored-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/3618
- no need to have them protocol specific
- no need to set pointers to them with the Curl_setup_transfer() call
- make Curl_setup_transfer() operate on a transfer pointer, not
connection
- switch some counters from long to the more proper curl_off_t type
Closes#3627
When a transfer is done, the resolver thread will be brought down. That
could accidentally generate an error message in the error buffer even
though this is not an error situationand the transfer would still return
OK. An application that still reads the error buffer could find a
"Could not resolve host: [host name]" message there and get confused.
Reported-by: Michael Schmid
Fixes#3629Closes#3630
Without it set, we would unwillingly triger the "HTTP error before end
of send, stop sending" condition even if the entire POST body had been
sent (since it wouldn't know the expected size) which would
unnecessarily log that message and close the connection when it didn't
have to.
Reported-by: Matt McClure
Bug: https://curl.haxx.se/mail/archive-2019-02/0023.htmlCloses#3624
Follow-up to 8eddb8f425.
If the cookieinfo pointer is NULL there really is nothing to save.
Without this fix, we got a problem when a handle was using shared object
with cookies and is told to "FLUSH" it to file (which worked) and then
the share object was removed and when the easy handle was closed just
afterwards it has no cookieinfo and no cookies so it decided to save an
empty jar (overwriting the file just flushed).
Test 1905 now verifies that this works.
Assisted-by: Michael Wallner
Assisted-by: Marcel Raad
Closes#3621
This allows the compiler to pack and align the structs better in
memory. For a rather feature-complete build on x86_64 Linux, gcc 8.1.2
makes the Curl_easy struct 4.9% smaller. From 6312 bytes to 6000.
Removed an unused struct field.
No functionality changes.
Closes#3610
Instead of using a fixed 256 byte buffer in the connectdata struct.
In my build, this reduces the size of the connectdata struct by 11.8%,
from 2160 to 1904 bytes with no functionality or performance loss.
This also fixes a bug in schannel's Curl_verify_certificate where it
called Curl_sspi_strerror when it should have called Curl_strerror for
string from GetLastError. the only effect would have been no text or the
wrong text being shown for the error.
Co-authored-by: Jay Satiro
Closes#3612
Add support for Ephemeral elliptic curve Diffie-Hellman key exchange
algorithm option when selecting ciphers. This became available on the
Win10 SDK.
Closes https://github.com/curl/curl/pull/3608
Failing to do so would make the CURLINFO_TOTAL_TIME timeout to not get
updated correctly and could end up getting reported to the application
completely wrong (way too small).
Reported-by: accountantM on github
Fixes#3602Closes#3605
- Split off connection shutdown procedure from Curl_disconnect into new
function conn_shutdown.
- Change the shutdown procedure to close the sockets before
disassociating the transfer.
Prior to this change the sockets were closed after disassociating the
transfer so SOCKETFUNCTION wasn't called since the transfer was already
disassociated. That likely came about from recent work started in
Jan 2019 (#3442) to separate transfers from connections.
Bug: https://curl.haxx.se/mail/lib-2019-02/0101.html
Reported-by: Pavel Löbl
Closes https://github.com/curl/curl/issues/3597
Closes https://github.com/curl/curl/pull/3598
RFC 7540 says we should verify that the push is for an "authoritative"
server. We make sure of this by only allowing push with an :athority
header that matches the host that was asked for in the URL.
Fixes#3577
Reported-by: Nicolas Grekas
Bug: https://curl.haxx.se/mail/lib-2019-02/0057.htmlCloses#3581
The variable wasn't properly reset within the loop and thus could remain
set for sockets that hadn't been set before and miss notifying the app.
This is a follow-up to 4c35574 (shipped in curl 7.64.0)
Reported-by: buzo-ffm on github
Detected-by: Jan Alexander Steffens
Fixes#3585Closes#3589
- rename 'n' to buflen in functions, and use size_t for them. Don't pass
in negative buffer lengths.
- move most function comments to above the function starts like we use
to
- remove several unnecessary typecasts (especially of NULL)
Reviewed-by: Patrick Monnerat
Closes#3582
Previously the function would edit the provided header in-place when a
semicolon is used to signify an empty header. This made it impossible to
use the same set of custom headers in multiple threads simultaneously.
This approach now makes a local copy when it needs to edit the string.
Reported-by: d912e3 on github
Fixes#3578Closes#3579
- Change the behavior of win32_init so that the required initialization
procedures are not affected by CURL_GLOBAL_WIN32 flag.
libcurl via curl_global_init supports initializing for win32 with an
optional flag CURL_GLOBAL_WIN32, which if omitted was meant to stop
Winsock initialization. It did so internally by skipping win32_init()
when that flag was set. Since then win32_init() has been expanded to
include required initialization routines that are separate from
Winsock and therefore must be called in all cases. This commit fixes
it so that CURL_GLOBAL_WIN32 only controls the optional win32
initialization (which is Winsock initialization, according to our doc).
The only users affected by this change are those that don't pass
CURL_GLOBAL_WIN32 to curl_global_init. For them this commit removes the
risk of a potential crash.
Ref: https://github.com/curl/curl/pull/3573
Fixes https://github.com/curl/curl/issues/3313
Closes https://github.com/curl/curl/pull/3575
The draft-ietf-httpbis-rfc6265bis-02 draft, specify a set of prefixes
and how they should affect cookie initialization, which has been
adopted by the major browsers. This adds support for the two prefixes
defined, __Host- and __Secure, and updates the testcase with the
supplied examples from the draft.
Closes#3554
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
If mbedtls_ssl_get_session() fails, it may still have allocated
memory that needs to be freed to avoid leaking. Call the library
API function to release session resources on this errorpath as
well as on Curl_ssl_addsessionid() errors.
Closes: #3574
Reported-by: Michał Antoniak <M.Antoniak@posnet.com>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
... and avoid use of static variables that aren't thread safe.
Fixes regression from e9ababd4f5 (present in the 7.64.0 release)
Reported-by: Paul Groke
Fixes#3572Closes#3573
- Save the original conn->data before it's changed to the specified
data transfer for the connection check and then restore it afterwards.
This is a follow-up to 38d8e1b 2019-02-11.
History:
It was discovered a month ago that before checking whether to extract a
dead connection that that connection should be associated with a "live"
transfer for the check (ie original conn->data ignored and set to the
passed in data). A fix was landed in 54b201b which did that and also
cleared conn->data after the check. The original conn->data was not
restored, so presumably it was thought that a valid conn->data was no
longer needed.
Several days later it was discovered that a valid conn->data was needed
after the check and follow-up fix was landed in bbae24c which partially
reverted the original fix and attempted to limit the scope of when
conn->data was changed to only when pruning dead connections. In that
case conn->data was not cleared and the original conn->data not
restored.
A month later it was discovered that the original fix was somewhat
correct; a "live" transfer is needed for the check in all cases
because original conn->data could be null which could cause a bad deref
at arbitrary points in the check. A fix was landed in 38d8e1b which
expanded the scope to all cases. conn->data was not cleared and the
original conn->data not restored.
A day later it was discovered that not restoring the original conn->data
may lead to busy loops in applications that use the event interface, and
given this observation it's a pretty safe assumption that there is some
code path that still needs the original conn->data. This commit is the
follow-up fix for that, it restores the original conn->data after the
connection check.
Assisted-by: tholin@users.noreply.github.com
Reported-by: tholin@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/3542Closes#3559
On non-ascii platforms, the chunked hex header was measured for char code
conversion length, even for chunked trailers that do not have an hex header.
In addition, the efective length is already known: use it.
Since the hex length can be zero, only convert if needed.
Reported by valgrind.
Convert numerous infof() calls into debug-build only messages since they
are annoyingly verbose for regular applications. Removed a few.
Bug: https://curl.haxx.se/mail/lib-2019-02/0027.html
Reported-by: Volker Schmid
Closes#3552
There is no benefit to holding the data sharelock when freeing the
addrinfo in case it fails, so ensure releaseing it as soon as we can
rather than holding on to it. This also aligns the code with other
consumers of sharelocks.
Closes#3516
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
The http2 code for connection checking needs a transfer to use. Make
sure a working one is set before handler->connection_check() is called.
Reported-by: jnbr on github
Fixes#3541Closes#3547
urlapi: turn three local-only functions into statics
conncache: make conncache_find_first_connection static
multi: make detach_connnection static
connect: make getaddressinfo static
curl_ntlm_core: make hmac_md5 static
http2: make two functions static
http: make http_setup_conn static
connect: make tcpnodelay static
tests: make UNITTEST a thing to mark functions with, so they can be static for
normal builds and non-static for unit test builds
... and mark Curl_shuffle_addr accordingly.
url: make up_free static
setopt: make vsetopt static
curl_endian: make write32_le static
rtsp: make rtsp_connisdead static
warnless: remove unused functions
memdebug: remove one unused function, made another static
If the incoming len 5, but the buffer does not have a termination
after 5 bytes, the strtol() call may keep reading through the line
buffer until is exceeds its boundary. Fix by ensuring that we are
using a bounded read with a temporary buffer on the stack.
Bug: https://curl.haxx.se/docs/CVE-2019-3823.html
Reported-by: Brian Carpenter (Geeknik Labs)
CVE-2019-3823
Attempt to add support for Secure Channel binding when negotiate
authentication is used. The problem to solve is that by default IIS
accepts channel binding and curl doesn't utilise them. The result was a
401 response. Scope affects only the Schannel(winssl)-SSPI combination.
Fixes https://github.com/curl/curl/issues/3503
Closes https://github.com/curl/curl/pull/3509
Stick to "Schannel" everywhere. The configure option --with-winssl is
kept to allow existing builds to work but --with-schannel is added as an
alias.
Closes#3504
mbedTLS doesn't have a sigpipe management. If a write/read occurs when
the remote closes the socket, the signal is raised and kills the
application. Use the curl mecanisms fix this behavior.
Signed-off-by: Jeremie Rapin <j.rapin@overkiz.com>
Closes#3502
Compiling with msvc /analyze and a recent Windows SDK warns against
using GetTickCount (Suggests to use GetTickCount64 instead.)
Since GetTickCount is only being used when GetTickCount64 isn't
available, I am disabling that warning.
Fixes https://github.com/curl/curl/issues/3437
Closes https://github.com/curl/curl/pull/3440
CURLOPT_SSH_KNOWNHOSTS and CURLOPT_SSH_KEYFUNCTION are supported for
libssh as well. So accepting these options only when compiling with
libssh2 is wrong here.
Fixes#3493Closes#3494
By default, libssh creates a new socket, instead of using the socket
created by curl for SSH connections.
Pass the socket created by curl to libssh using ssh_options_set() with
SSH_OPTIONS_FD directly after ssh_new(). So libssh uses our socket
instead of creating a new one.
This approach is very similar to what is done in the libssh2 code, where
the socket created by curl is passed to libssh2 when
libssh2_session_startup() is called.
Fixes#3491Closes#3495
There is no real gain in performing memcmp() comparisons on single
characters, so change these to array subscript inspections which
saves a call and makes the code clearer.
Closes#3486
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
Windows extended potection (aka ssl channel binding) is required
to login to ntlm IIS endpoint, otherwise the server returns 401
responses.
Fixes#3280Closes#3321
When a ssh session startup fails, it is useful to know why it has
failed. This commit changes the message from:
"Failure establishing ssh session"
to something like this, for example:
"Failure establishing ssh session: -5, Unable to exchange encryption keys"
Closes#3481
.... to not pass in a const in the second argument as that's not how it
is supposed to be used and might cause compiler warnings.
Reported-by: Pavel Pavlov
Fixes#3477Closes#3478
extract_if_dead() dead is called from two functions, and only one of
them should get conn->data updated and now neither call path clears it.
scan-build found a case where conn->data would be NULL dereferenced in
ConnectionExists() otherwise.
Closes#3473
