... instead of having it static within the Curl_easy struct. This takes
away 1176 bytes (18%) from the Curl_easy struct that aren't used very
often and instead makes the code allocate it when needed.
Closes#6492
The SOCKS code now uses the generic download buffer for temporary
storage during the connection procedure, instead of having its own
private 600 byte buffer that adds to the connectdata struct size. This
works fine because this point the buffer is allocated but is not use for
download yet since the connection hasn't completed.
This reduces the connection struct size by 22% on a 64bit arch!
The SOCKS buffer needs to be at least 600 bytes, and the download buffer
is guaranteed to never be smaller than 1000 bytes.
Closes#6491
By making the `magic` identifier the same size and at the same place
within the structs (easy, multi, share), libcurl will be able to more
reliably detect and safely error out if an application passes in the
wrong handle to APIs. Easier to detect and less likely to cause crashes
if done.
Such mixups can't be detected at compile-time due to them being
typedefed void pointers - unless `CURL_STRICTER` is defined.
Closes#6484
Since curl's own memory debugging system redefines free() calls to track
and fiddle with memory, it cannot be used on memory allocated by 3rd
party libraries.
Third party libraries SHOULD NOT require free() to release allocated
resources for this reason - and libs can use separate healp allocators
on some systems (like Windows) so free() doesn't necessarily work
anyway.
Filed as an issue with libssh: https://bugs.libssh.org/T268Closes#6481
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.
- We mostly operate on transfers and not connections.
- We need the transfer handle to log, store data and more. Everything in
libcurl is driven by a transfer (the CURL * in the public API).
- This work clarifies and separates the transfers from the connections
better.
- We should avoid "conn->data". Since individual connections can be used
by many transfers when multiplexing, making sure that conn->data
points to the current and correct transfer at all times is difficult
and has been notoriously error-prone over the years. The goal is to
ultimately remove the conn->data pointer for this reason.
Closes#6425
... so that a function can first use MIMEPOST and then set it to NULL to
reset it back to a blank POST.
Added test 584 to verify the fix.
Reported-by: Christoph M. Becker
Fixes#6455Closes#6456
... instead of at end of the DO state. This makes the timer more
accurate for the protocols that use the DOING state (such as FTP), and
simplifies how the function (now called init_perform) is called.
The timer will then include the entire procedure up to PERFORM -
including all instructions for getting the transfer started.
Closes#6454
- During the end-of-headers response phase do not mark the tunnel
complete unless the response body was completely parsed/ignored.
Prior to this change if the entirety of a CONNECT response with chunked
encoding was not received by the time the final header was parsed then
the connection would be marked done prematurely, before all the chunked
data could be read in and ignored (since this is what we do with any
CONNECT response body) and the connection could not be used.
Bug: https://curl.se/mail/lib-2021-01/0033.html
Reported-by: Fabian Keil
Closes https://github.com/curl/curl/pull/6432
When doing a request with a request body expecting a 401/407 back, that
initial request is sent with a zero content-length. Test 177 and more.
Closes#6424
... so that Retry-After and other meta-content can still be used.
Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.
Fixes#6408Closes#6409
... to make build tools/valgrind warn if no curl_global_cleanup is
called.
This is conditionally only done for debug builds with the env variable
CURL_GLOBAL_INIT set.
Closes#6410
... and not in the connection setup, as for multiplexed transfers the
connection setup might be skipped and then the transfer would end up
without the set user-agent!
Reported-by: Flameborn on github
Assisted-by: Andrey Gursky
Assisted-by: Jay Satiro
Assisted-by: Mike Gelfand
Fixes#6312Closes#6417
The wolfSSL TLS library defines NO_OLD_TLS in some of their build
configurations and that causes the library to be built without TLS 1.1.
For example if MD5 is explicitly disabled when building wolfSSL then
that defines NO_OLD_TLS and the library is built without TLS 1.1 [1].
Prior to this change attempting to build curl with a wolfSSL that was
built with NO_OLD_TLS would cause a build link error undefined reference
to wolfTLSv1_client_method.
[1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366
Bug: https://curl.se/mail/lib-2020-12/0121.html
Reported-by: Julian Montes
Closes https://github.com/curl/curl/pull/6388
When doing HTTP authentication and a port number set with CURLOPT_PORT,
the code would previously have the URL's port number override as if it
had been a redirect to an absolute URL.
Added test 1568 to verify.
Reported-by: UrsusArctos on github
Fixes#6397Closes#6400
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.
Closes#6395
Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix
(similar to the existing '-' prefix for removing entries) to add
DNS cache entries that will time out just like entries that are added
by libcurl itself.
Append " (non-permanent)" to info log message in case a non-permanent
entry is added.
Adjust relevant comments to reflect the new behavior.
Adjust documentation.
Extend unit1607 to test the new functionality.
Closes#6294
Paused transfers should not be stopped due to slow speed even when
CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is
now reset when the transfer is unpaused - as otherwise it would easily
just trigger immediately after unpausing.
Reported-by: Harry Sintonen
Fixes#6358Closes#6359
... as the socket might be readable all the time when paused and thus
causing a busy-loop.
Reported-by: Harry Sintonen
Reviewed-by: Jay Satiro
Fixes#6356Closes#6357
It is a security process for HTTP.
It doesn't seems to be standard, but it is used by some cloud providers.
Aws:
https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
Outscale:
https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request
GCP (I didn't test that this code work with GCP though):
https://cloud.google.com/storage/docs/access-control/signing-urls-manually
most of the code is in lib/http_v4_signature.c
Information require by the algorithm:
- The URL
- Current time
- some prefix that are append to some of the signature parameters.
The data extracted from the URL are: the URI, the region,
the host and the API type
example:
https://api.eu-west-2.outscale.com/api/latest/ReadNets
~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
^ ^ ^
/ \ URI
API type region
Small description of the algorithm:
- make canonical header using content type, the host, and the date
- hash the post data
- make canonical_request using custom request, the URI,
the get data, the canonical header, the signed header
and post data hash
- hash canonical_request
- make str_to_sign using one of the prefix pass in parameter,
the date, the credential scope and the canonical_request hash
- compute hmac from date, using secret key as key.
- compute hmac from region, using above hmac as key
- compute hmac from api_type, using above hmac as key
- compute hmac from request_type, using above hmac as key
- compute hmac from str_to_sign using above hmac as key
- create Authorization header using above hmac, prefix pass in parameter,
the date, and above hash
Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
Closes#5703
It seems current hmac implementation use md5 for the hash,
V4 signature require sha256, so I've added the needed struct in
this commit.
I've added the functions that do the hmac in v4 signature file
as a static function ,in the next patch of the serie,
because it's used only by this file.
Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
The linux kernel does not report all ICMP errors back to userspace due
to historical reasons.
IP*_RECVERR sockopt must be turned on to have the correct behaviour
which is to pass all ICMP errors to userspace.
See https://bugzilla.kernel.org/show_bug.cgi?id=202355Closes#6341
When the initial request isn't possible to send in its entirety, the
remainder of request would be delivered to the debug callback as data
and would wrongly be counted internally as body-bytes sent.
Extended test 1295 to verify.
Closes#6328
When failing in TOOFAST, the multi_done() wasn't called so the same
cleanup and handling wasn't done like when it fails in PERFORM, which in
the case of FTP could mean that the control connection wouldn't be
marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused
ftp_disconnect() to use it to send "QUIT", which could end up waiting
for a response a long time before giving up!
Reported-by: Tomas Berger
Fixes#6333Closes#6337
This commit introduces a "gophers" handler inside the gopher protocol if
USE_SSL is defined. This protocol is no different than the usual gopher
prococol, with the added TLS encapsulation upon connecting. The protocol
has been adopted in the gopher community, and many people have enabled
TLS in their gopher daemons like geomyidae(8), and clients, like clic(1)
and hurl(1).
I have not implemented test units for this protocol because my knowledge
of Perl is sub-par. However, for someone more knowledgeable it might be
fairly trivial, because the same test that tests the plain gopher
protocol can be used for "gophers" just by adding a TLS listener.
Signed-off-by: parazyd <parazyd@dyne.org>
Closes#6208
The error is shown with infof rather than failf so that the user will
see the extended error message information only in verbose mode, and
will still see the standard CURLE_AUTH_ERROR message. For example:
---
* schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED
(0x8009030A) - The per-message Quality of Protection is not supported by
the security package
* multi_done
* Connection #1 to host 127.0.0.1 left intact
curl: (94) An authentication function returned an error
---
Ref: https://github.com/curl/curl/issues/6302
Closes https://github.com/curl/curl/pull/6315
If supported, defer port selection until connect() time
if --interface is given and source port is 0.
Reproducer:
* start fast webserver on port 80
* starve system of ephemeral ports
$ sysctl net.ipv4.ip_local_port_range="60990 60999"
* start a curl/libcurl "crawler"
$curl --keepalive --parallel --parallel-immediate --head --interface
127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt"
current result:
(possible some successful data)
curl: (45) bind failed with errno 98: Address already in use
result after patch:
(complete success or few connections failing, higlhy depending on load)
Fail only when all the possible 4-tuple combinations are exhausted,
which is impossible to do when port is selected at bind() time becuse
the kernel does not know if socket will be listen()'ed on or connect'ed
yet.
Closes#6295
Valgrind will complain that ssrem buffer usage if not explicit
initialized, hence initialize it to zero.
This completes the change intially started in commit 2c0d721215 ('ftp:
retry getpeername for FTP with TCP_FASTOPEN') where the ssloc buffer has
a similar memset to zero.
Signed-off-by: Hans-Christian Noren Egtvedt <hegtvedt@cisco.com>
Closes#6289
The command line tool also independently sets --ftp-skip-pasv-ip by
default.
Ten test cases updated to adapt the modified --libcurl output.
Bug: https://curl.se/docs/CVE-2020-8284.html
CVE-2020-8284
Reported-by: Varnavas Papaioannou
... as it makes the URL parser accept "very-long-hostname://" as a valid
host name and we don't want that. The parser now only accepts a blank
(no digits) after the colon if the URL starts with a scheme.
Reported-by: d4d on hackerone
Closes#6283
This reverts commit d2a7d7c185.
This commit also reverts the subsequent follow-ups to that commit, which
were all done within windows #ifdefs that are removed in this
change. Marc helped me verify this.
Fixes#6146Closes#6281
In the case of TFO, the remote host name is not resolved at the
connetion time.
For FTP that has lead to missing hostname for the secondary connection.
Therefore the name resolution is done at the time, when FTP requires it.
Fixes#6252Closes#6265Closes#6282
This field needs to be wide enough to hold sockaddr_in6 when
connecting via IPv6. Otherwise, ngtcp2_conn_read_pkt will drop the
packets because of the address mismatch:
I00000022 [...] con ignore packet from unknown path
We can safely assume that struct sockaddr_storage is available, as it
is used in the public interface of ngtcp2.
Closes#6250
The resolve call is done with the right port number, but the subsequent
check used the wrong one, which then could find a previous resolve which
would return and leave the fresh resolve "incomplete" and leaking
memory.
Fixes#6247Closes#6253
file_disconnect() is identical with file_do() except the function header
but as the arguments are unused anyway so why not just return file_do()
directly!
Reviewed-by: Daniel Stenberg
Closes#6249
The function has been removed from common usage. Also removed comment in
gopher.c that still referenced it.
Reported-by: Rikard Falkeborn
Fixes#6242Closes#6243
To reduce use of types that can't be checked at compile time. Also
removes several typecasts.
... and rename the struct field from 'os_specific' to 'tdata'.
Closes#6239
Reviewed-by: Jay Satiro
EVP_MD_CTX_create will allocate memory for the context and returns
NULL in case the allocation fails. Make sure to catch any allocation
failures and exit early if so.
In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex
as the latter is intended for ENGINE selection which we don't do.
Closes#6224
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
curl CMake was setting the the EXE flags for static libraries which made
the /manifest:no flag ended up when linking the static library, which is
not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit
with an error.
The better way to handle this is to make sure that we pass the correct
linker flags to CMAKE_STATIC_LINKER_FLAGS instead.
Reviewed-by: Jakub Zakrzewski
Closes#6195
... instead of the limiting it to just the max input size. As every
input byte can be expanded to 3 output bytes, this could limit the input
string to 2.66 MB instead of the intended 8 MB.
Reported-by: Marc Schlatter
Closes#6192
By setting the speed limit time stamps unconditionally at transfer
start, we can start off a transfer without speed limits and yet allow
them to get set during transfer and have an effect.
Reported-by: Kael1117 on github
Fixes#6162Closes#6184
The line variable will always be NULL in the error path, so remove
the free call since it's pointless.
Closes#6170
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle
Closes#5896
... when the chunked framing was added, the size of the "body part" of
the data was calculated wrongly so the debug callback would get told a
header chunk a few bytes too big that would also contain the first few
bytes of the request body.
Reported-by: Dirk Wetter
Ref: #6144Closes#6147
Build breaks because the http_proxy field is missing:
vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy'
Regression from #6021, shipped in curl 7.73.0
Closes#6125
* fix two build errors due to mismatch between function
declarations and their definitions
* silence two mismatched signs warnings via casts
Approved-by: Daniel Stenberg
Closes#6093
... and remove the objnames scripts they tested. They're not used for
anything anymore so testing them serves no purpose!
Reported-by: Marc Hörsken
Fixes#6080Closes#6081
- Change get_winapi_error() to return the error string in the local
codepage instead of UTF-8 encoding.
Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it
also changed the error string's encoding from local codepage to UTF-8.
We return the local codepage version of the error string because if it
is output to the user's terminal it will likely be with functions which
expect the local codepage (eg fprintf, failf, infof).
This is essentially a partial revert of bed5f84. The support for xbox
remains but the error string is reverted back to local codepage.
Ref: https://github.com/curl/curl/pull/6005
Reviewed-by: Marcel Raad
Closes#6065
Whitespace is spelled without a space between white and space, so
make sure to consistently spell it that way across the codebase.
Closes#6023
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
There used to be a way to have memdebug fill allocated memory. 9 years
later this has no value there (valgrind and ASAN etc are way better). If
people need to know about it they can have a look at VCS logs.
Closes#5973
This matches what is returned in other TLS backends in the same
situation.
Reviewed-by: Jay Satiro
Reviewed-by: Emil Engler
Follow-up to 5a3efb1
Reported-by: iammrtau on github
Fixes#6003Closes#6018
Added test 348 to verify. Added a 'STOR' command to the test FTP
server to enable test 348. Documented the command in FILEFORMAT.md
Reported-by: Duncan Wilcox
Fixes#6016Closes#6017
When using HTTPS proxy, SSL is used but not in the view of the FTP
protocol handler itself so separate the connection's use of SSL from the
FTP control connection's sue.
Reported-by: Mingtao Yang
Fixes#5523Closes#6006
... reuses the same dynamic buffer instead of doing repeated malloc/free
cycles.
Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls
after this change in my test setup (132 => 125), curl 7.72.0 needed 140
calls for this.
Test case 103 makes 9 less allocations now (130). Down from 149 in
7.72.0.
Closes#6004
... by introducing a printf() function that appends directly into a
dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if
the buffer is already big enough it can just printf directly into it.
Since this less-malloc version requires tthe use of a library internal
printf function, we only provide this version when building libcurl and
not for the dynbuf code that is used when building the curl tool.
Closes#5998
... as that counter is subsequently used to detect if nothing was
returned from the peer. This made curl return CURLE_OK when it should
have returned CURLE_GOT_NOTHING.
Fixes#5992
Reported-by: Tom van der Woerdt
Closes#5994
Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the
option is, yeah, not known. Clarified this in the setopt man page too.
Closes#5993
These two files were always tightly connected and it was hard to
understand what went into which. This also allows us to make the
ftpsend() function static (moved from ftp.c).
Removed security.c
Renamed curl_sec.h to krb5.h
Closes#5987
By avoiding an unnecessary error check and the temp use of the tm
struct, the time2epoch conversion function gets a little bit faster.
When repeating test 517, the updated version is perhaps 1% faster (on
one particular build on one particular architecture).
Closes#5985
If the received PASV response doesn't match the expected pattern, we
could end up reading uninitialized integers for IP address and port
number.
Issue pointed out by muse.dev
Closes#5972
Also skip pre-checking sockets to set timeout_ms to 0
after the first socket has been detected to be ready.
Reviewed-by: rcombs on github
Reviewed-by: Daniel Stenberg
Follow up to #5886
Since the struct is quite large (1 long and 10 ints) we
declare it once at the beginning of the function instead
of multiple times inside loops to avoid stack movements.
Reviewed-by: Viktor Szakats
Reviewed-by: Daniel Stenberg
Closes#5886
This is primarily interesting for cases where CURLOPT_NOBODY is set as
previously curl would not return an error for this case.
MDTM getting 550 now also returns this error (it returned
CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for
missing files across protocols and specific FTP commands.
libcurl already returns error on a 550 as a MDTM response (when
CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would
happen subsequently anyway since the RETR command would fail.
Add test 1913 and 1914 to verify. Updated several tests accordingly due
to the updated SIZE behavior.
Reported-by: Tomas Berger
Fixes#5953Closes#5957
... when continuing a transfer from a FETCH response.
When the size of the file was small enough that the entirety of the
transfer happens in a single go and schannel buffers holds the entire
data. However, it wasn't completely read in Curl_pp_readresp since a
line break was found before that could happen. So, by the time we are in
imap_state_fetch_resp - there's data in buffers that needs to be read
via Curl_read but nothing to read from the socket. After we setup a
transfer (Curl_setup_transfer), curl just waits on the socket state to
change - which doesn't happen since no new data ever comes.
Closes#5961
Previously there were several locations that called
nghttp2_session_mem_recv and handled responses slightly differently.
Those have been converted to call the existing
h2_process_pending_input() function.
Moved the end-of-session check to h2_process_pending_input() since the
only place the end-of-session state can change is after nghttp2
processes additional input frames.
This will likely fix the fuzzing error. While I don't have a root cause
the out-of-bounds read seems like a use after free, so moving the
nghttp2_session_check_request_allowed() call to a location with a
guaranteed nghttp2 session seems reasonable.
Also updated a few nghttp2 callsites to include error messages and added
a few additional error checks.
Closes#5648
configure --enable-debug now enables -Wassign-enum with clang,
identifying several enum "abuses" also fixed.
Reported-by: Gisle Vanem
Bug: 879007f811 (commitcomment-42087553)Closes#5929
If the error reason from the lib is
SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return
CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR.
This unifies the libcurl return code and makes libressl run test 313
(CRL testing) fine.
Closes#5934
The cache content is not duplicated, like other caches, but the setting
and specified file name are.
Test 1908 is extended to verify this somewhat. Since the duplicated
handle gets the same file name, the test unfortunately overwrites the
same file twice (with different contents) which makes it hard to check
automatically.
Closes#5923
The code section that is OpenSSL 3+ specific now uses the same logic as
is used in the version < 3 section. It caused a compiler error without
it.
Closes#5907
Drop dynamic loading of ws2_32.dll and instead rely on the
imported version which is now required to be at least 2.2.
Reviewed-by: Marcel Raad
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Reviewed-by: Viktor Szakats
Closes#5854
IPv6, telnet and now also the multi API require WinSock
version 2 which is available starting with Windows 95.
Therefore we think it is time to drop support for version 1.
Reviewed-by: Marcel Raad
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Reviewed-by: Viktor Szakats
Follow up to #5634Closes#5854
The poll emulation via select already consumes POLLRDNORM,
POLLWRNORM and POLLRDBAND as input events. Therefore it
should also return them as output events if signaled.
Also fix indentation in input event handling block.
Assisted-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Replaces #5852Closes#5883
`USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES
backend is fine, but was excluded before.
This also fixes test 1013 as the condition for SMB support in
configure.ac didn't match the condition in the source code. Now it
does.
Fixes https://github.com/curl/curl/issues/1262
Closes https://github.com/curl/curl/pull/5771
poll and select already return -1 on error according to POSIX,
so there is no need to perform a <0 to -1 conversion in code.
Also we can just use one check with <= 0 on the return code.
Assisted-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Replaces #5852Closes#5880
Learn from the way Cygwin handles and maps the WinSock events
to simulate correct and complete poll and select behaviour
according to Richard W. Stevens Network Programming book.
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Follow up to #5634Closes#5867
Setting a timeout to INT_MAX could cause an immediate error to get
returned as timeout because of an overflow when different values of
'now' were used.
This is primarily fixed by having Curl_pgrsTime() return the "now" when
TIMER_STARTSINGLE is set so that the parent function will continue using
that time.
Reported-by: Ionuț-Francisc Oancea
Fixes#5583Closes#5847
USE_TLS_SRP will be true if *any* selected TLS backend can use SRP
HAVE_OPENSSL_SRP is defined when OpenSSL can use it
HAVE_GNUTLS_SRP is defined when GnuTLS can use it
Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is
set if at least one of the supported backends offers SRP.
Reported-by: Stefan Strogin
Fixes#5865Closes#5870
const struct curl_easyoption *curl_easy_option_by_name(const char *name);
const struct curl_easyoption *curl_easy_option_by_id (CURLoption id);
const struct curl_easyoption *
curl_easy_option_next(const struct curl_easyoption *prev);
The purpose is to provide detailed enough information to allow for
example libcurl bindings to get option information at run-time about
what easy options that exist and what arguments they expect.
Assisted-by: Jeroen Ooms
Closes#5365
The get_cert_location function allocates memory only on success.
Previously get_cert_location was able to allocate memory and return
error. It wasn't obvious and in this case the memory wasn't
released.
Fixes#5855Closes#5860
Check readiness of all sockets before waiting on them
to avoid locking in case the one-time event FD_WRITE
was already consumed by a previous wait operation.
More information about WinSock network events:
https://docs.microsoft.com/en-us/windows/win32/api/
winsock2/nf-winsock2-wsaeventselect#return-value
Closes#5634
This avoids using a pair of TCP ports to provide wakeup functionality
for every multi instance on Windows, where socketpair() is emulated
using a TCP socket on loopback which could in turn lead to socket
resource exhaustion.
A previous version of this patch failed to account for how in WinSock,
FD_WRITE is set only once when writing becomes possible and not again
until after a send has failed due to the buffer filling. This contrasts
to how FD_READ and FD_OOB continue to be set until the conditions they
refer to no longer apply. This meant that if a user wrote some data to
a socket, but not enough data to completely fill its send buffer, then
waited on that socket to become writable, we'd erroneously stall until
their configured timeout rather than returning immediately.
This version of the patch addresses that issue by checking each socket
we're waiting on to become writable with select() before the wait, and
zeroing the timeout if it's already writable.
Assisted-by: Marc Hörsken
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Tested-by: Gergely Nagy
Tested-by: Rasmus Melchior Jacobsen
Tested-by: Tomas Berger
Replaces #5397
Reverts #5632Closes#5634
Change Curl_socket_check to use select-fallback in Curl_poll
instead of implementing it in Curl_socket_check and Curl_poll.
Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Replaces #5262 and #5492Closes#5707
This commit changes Curl_socket_check to use POLLPRI to
check for connect failure on the write socket, because
POLLPRI maps to fds_err. This is in line with select(2).
The select-based socket check correctly checks for connect
failures by adding the write socket also to fds_err.
The poll-based implementation (which internally can itself
fallback to select again) did not previously check for
connect failure by using POLLPRI with the write socket.
See the follow up commit to this for more information.
This commit makes sure connect failures can be detected
and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel.
Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Replaces #5509
Prepares #5707
The buffer only exists during transfer and then we shouldn't change the
size (the setopt is not documented to work then).
Reported-by: Harry Sintonen
Closes#5842
Failures clearly returned from a (SOCKS) proxy now causes this return
code. Previously the situation was not very clear as what would be
returned and when.
In addition: when this error code is returned, an application can use
CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then
returns a value from the new 'CURLproxycode' enum.
Closes#5770
... not newline separated from the previous line. This makes it output
asterisk prefixed properly like other verbose putput!
Reported-by: jmdavitt on github
Fixes#5826Closes#5827
This flag was applied to the connection struct that is released on
retry. These changes move the retry counter into Curl_easy struct that
lives across retries and retains the new connection.
Reported-by: Cherish98 on github
Fixes#5794Closes#5800
The debug output used ssherr instead of sftperr which not only outputs
the wrong error code but also casues a warning on Windows.
Follow-up to 7370b4e39f
Reported-by: Gisle Vanem
Bug: 7370b4e39f (r41334700)Closes#5799
The shutdown function is for downgrading a connection from TLS to plain,
and this is not requested here.
Have ssl_close reset the TLS connection state.
This partially reverts commit f002c850d9
Reported-by: Rasmus Melchior Jacobsen
Reported-by: Denis Goleshchikhin
Fixes#5797
The previous h2 trailer fix in 54a2b63 was wrong and caused a
regression: it cannot deal with trailers immediately when read since
they may be read off the connection by the wrong 'data' owner.
This change reverts the logic back to gathering all trailers into a
single buffer, like before 54a2b63.
Reported-by: Tadej Vengust
Fixes#5663Closes#5769
Classic mingw and 10y+ old versions of mingw-w64 don't ship with
Windows headers having the typedef necessary for Unix Sockets
support, so try detecting these environments to disable this
feature.
Ref: cf6afc5717/
Reviewed-by: Daniel Stenberg
Fixes#5674Closes#5758
Previously any connect-only connections in a multi handle would be kept
alive until the multi handle was closed. Since these connections cannot
be re-used, they can be marked for closure when the associated easy
handle is removed from the multi handle.
Closes#5749
They're not thread-safe so they should not be used in libcurl code.
Explictly enabled when deemed necessary and in examples and tests
Reviewed-by: Nicolas Sterchele
Closes#5732
Previously a file that isn't user-readable but is user-writable would
not be properly avoided and would get overwritten.
Reported-by: BrumBrum on hackerone
Assisted-by: Jay Satiro
Bug: https://hackerone.com/reports/926638Closes#5731
Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1,
2013-08-02), the easy handle list is not circular but ends with
->next pointing to NULL.
Reported-by: Masaya Suzuki <masayasuzuki@google.com>
Closes#5737
Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented
action but before 7.71.0 that used to switch back to GET and with this
change (assuming the method is still set to HEAD) this behavior is
brought back.
Reported-by: causal-agent on github
Fixes#5725Closes#5728
Stack overflows can occur with precisions for integers and floats.
Proof of concepts:
- curl_mprintf("%d, %.*1$d", 500, 1);
- curl_mprintf("%d, %+0500.*1$f", 500, 1);
Ideally, compile with -fsanitize=address which makes this undefined
behavior a bit more defined for debug purposes.
The format strings are valid. The overflows occur due to invalid
arguments. If these arguments are variables with contents controlled
by an attacker, the function's stack can be corrupted.
Also see CVE-2016-9586 which partially fixed the float aspect.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Closes https://github.com/curl/curl/pull/5722
Verify that specified parameters are in range. If parameters are too
large, fail early on and avoid out of boundary accesses.
Also do not read behind boundaries of illegal format strings.
These are defensive measures since it is expected that format strings
are well-formed. Format strings should not be modifiable by user
input due to possible generic format string attacks.
Closes https://github.com/curl/curl/pull/5722
They are marked as deprecated for -mmacosx-version-min >= 10.15,
which might result in warnings-as-errors.
Closes https://github.com/curl/curl/pull/5695
It confuses code analyzers with its use of -1 for unsigned value. Also,
a check that's not normally used in strdup() code - and not necessary.
Closes#5697
Provide the HTTP method that was used on the latest request, which might
be relevant for users when there was one or more redirects involved.
Closes#5511
Use the unsigned type (size_t) in the arithmetic of pointers. In this
context, the signed type (ssize_t) is used unnecessarily.
Authored-by: ihsinme on github
Closes#5654
include zstd curl patch for Makefile.m32 from vszakats
and include Add CMake support for zstd from Peter Wu
Helped-by: Viktor Szakats
Helped-by: Peter Wu
Closes#5453
`http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn`
if `CURL_DISABLE_PROXY` is enabled. Repair the build with that
configuration.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Closes#5645
Well-behaving HTTP2 servers send two GOAWAY messages. The first
message is a warning that indicates that the server is going to
stop accepting streams. The second one actually closes the stream.
nghttp2 reports this state (and the other state of no more stream
identifiers) via the call nghttp2_session_check_request_allowed().
In this state the client should not create more streams on the
session (tcp connection), and in curl this means that the server
has requested that the connection is closed.
It would be also be possible to put the connclose() call into the
on_http2_frame_recv() function that triggers on the GOAWAY message.
This fixes a bug seen when the client sees the following sequence of
frames:
// advisory GOAWAY
HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1]
... some additional frames
// final GOAWAY
HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ]
Before this change, curl will attempt to reuse the connection even
after the last stream, will encounter this error:
* Found bundle for host localhost: 0x5595f0a694e0 [can multiplex]
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (::1) port 10443 (#0)
* Using Stream ID: 9 (easy handle 0x5595f0a72e30)
> GET /index.html?5 HTTP/2
> Host: localhost:10443
> user-agent: curl/7.68.0
> accept: */*
>
* stopped the pause stream!
* Connection #0 to host localhost left intact
curl: (16) Error in the HTTP2 framing layer
This error may posion the connection cache, causing future requests
which resolve to the same curl connection to go through the same error
path.
Closes#5643
Confusingly, nghttp2 has two different error code enums:
- nghttp2_error, to be used with nghttp2_strerror
- nghttp2_error_code, to be used with nghttp2_http2_strerror
Closes#5641
Since commit f3d501dc67, if proxy support is disabled, MSVC warns:
url.c : warning C4701: potentially uninitialized local variable
'hostaddr' used
url.c : error C4703: potentially uninitialized local pointer variable
'hostaddr' used
That could actually only happen if both `conn->bits.proxy` and
`CURL_DISABLE_PROXY` were enabled.
Initialize it to NULL to silence the warning.
Closes https://github.com/curl/curl/pull/5638
Updated terminology in docs, comments and phrases to refer to C strings
as "null-terminated". Done to unify with how most other C oriented docs
refer of them and what users in general seem to prefer (based on a
single highly unscientific poll on twitter).
Reported-by: coinhubs on github
Fixes#5598Closes#5608
Don't reference fields that do not exist. Fixes build failure:
vtls/mbedtls.c: In function 'mbed_connect_step1':
vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy'
Closes#5615
...previously CURLINFO_EFFECTIVE_URL would report the URL of the
original "mother transfer", not the actually pushed resource.
Reported-by: Jonathan Cardoso Machado
Fixes#5589Closes#5591
- Include wincrypt before OpenSSL includes so that the latter can
properly handle any conflicts between the two.
Closes https://github.com/curl/curl/pull/5606
Replace "Failed writing body (X != Y)" with
"Failure writing output to destination". Possibly slightly less cryptic.
Reported-by: coinhubs on github
Fixes#5594Closes#5596
Follow-up to c4e6968127
When a new transfer is created, as a resuly of an acknowledged push,
that transfer needs a download buffer allocated.
Closes#5590
This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does
not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default
locations. Instead the CA store can now be used at the same time.
The change is due to the impending release. The issue is still being
discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and
is now documented as experimental.
Ref: bc052cc (parent commit)
Ref: https://github.com/curl/curl/issues/5585
For QUIC but also for regular TCP when the second family runs out of IPs
with a failure while the first family is still trying to connect.
Separated the timeout handling for IPv4 and IPv6 connections when they
both have a number of addresses to iterate over.
This avoids using a pair of TCP ports to provide wakeup functionality
for every multi instance on Windows, where socketpair() is emulated
using a TCP socket on loopback which could in turn lead to socket
resource exhaustion.
Reviewed-by: Gergely Nagy
Reviewed-by: Marc Hörsken
Closes#5397
When wolfSSL is built with its OpenSSL API layer, it fetures the same DES*
functions that OpenSSL has. This change take advantage of that.
Co-authored-by: Daniel Stenberg
Closes#5556Fixes#5548
Since the connection can be used by many independent requests (using
HTTP/2 or HTTP/3), things like user-agent and other transfer-specific
data MUST NOT be kept connection oriented as it could lead to requests
getting the wrong string for their requests. This struct data was
lingering like this due to old HTTP1 legacy thinking where it didn't
mattered..
Fixes#5566Closes#5567
When asking for a specific feature to be shared in the share object,
that bit was previously set unconditionally even if the shared feature
failed or otherwise wouldn't work.
Closes#5554
Instead of discussing if there's value or meaning (implied or not) in
the colors, let's use words without the same possibly negative
associations.
Closes#5546
The SOCKS4/5 state machines weren't properly terminated when the proxy
connection got closed, leading to a busy-loop.
Reported-By: zloi-user on github
Fixes#5532Closes#5542
To reduce the amount of allocations needed for creating a Curl_addrinfo
struct, make a single larger malloc instead of three separate smaller
ones.
Closes#5533
quiche now requires the application to explicitly set the keylog path
for each connection, rather than reading the environment variable
itself.
Closes#5541
Now that all functions in select.[ch] take timediff_t instead
of the limited int or long, we can remove type conversions
and related preprocessor checks to silence compiler warnings.
Avoiding conversions from time_t was already done in 842f73de.
Based upon #5262
Supersedes #5214, #5220 and #5221
Follow up to #5343 and #5479Closes#5490
On some systems, openssl 1.0.x is still the default, but it has been
patched to contain all the recent security fixes. As a result of this
patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be
defined, while the previous behavior of openssl to not look at trusted
chains first, remains.
Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to
probe for the behavior of openssl based on the existence ofmacros.
Closes#5530
