Whitespace is spelled without a space between white and space, so
make sure to consistently spell it that way across the codebase.
Closes#6023
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
There used to be a way to have memdebug fill allocated memory. 9 years
later this has no value there (valgrind and ASAN etc are way better). If
people need to know about it they can have a look at VCS logs.
Closes#5973
This matches what is returned in other TLS backends in the same
situation.
Reviewed-by: Jay Satiro
Reviewed-by: Emil Engler
Follow-up to 5a3efb1
Reported-by: iammrtau on github
Fixes#6003Closes#6018
Added test 348 to verify. Added a 'STOR' command to the test FTP
server to enable test 348. Documented the command in FILEFORMAT.md
Reported-by: Duncan Wilcox
Fixes#6016Closes#6017
When using HTTPS proxy, SSL is used but not in the view of the FTP
protocol handler itself so separate the connection's use of SSL from the
FTP control connection's sue.
Reported-by: Mingtao Yang
Fixes#5523Closes#6006
... reuses the same dynamic buffer instead of doing repeated malloc/free
cycles.
Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls
after this change in my test setup (132 => 125), curl 7.72.0 needed 140
calls for this.
Test case 103 makes 9 less allocations now (130). Down from 149 in
7.72.0.
Closes#6004
... by introducing a printf() function that appends directly into a
dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if
the buffer is already big enough it can just printf directly into it.
Since this less-malloc version requires tthe use of a library internal
printf function, we only provide this version when building libcurl and
not for the dynbuf code that is used when building the curl tool.
Closes#5998
... as that counter is subsequently used to detect if nothing was
returned from the peer. This made curl return CURLE_OK when it should
have returned CURLE_GOT_NOTHING.
Fixes#5992
Reported-by: Tom van der Woerdt
Closes#5994
Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the
option is, yeah, not known. Clarified this in the setopt man page too.
Closes#5993
These two files were always tightly connected and it was hard to
understand what went into which. This also allows us to make the
ftpsend() function static (moved from ftp.c).
Removed security.c
Renamed curl_sec.h to krb5.h
Closes#5987
By avoiding an unnecessary error check and the temp use of the tm
struct, the time2epoch conversion function gets a little bit faster.
When repeating test 517, the updated version is perhaps 1% faster (on
one particular build on one particular architecture).
Closes#5985
If the received PASV response doesn't match the expected pattern, we
could end up reading uninitialized integers for IP address and port
number.
Issue pointed out by muse.dev
Closes#5972
Also skip pre-checking sockets to set timeout_ms to 0
after the first socket has been detected to be ready.
Reviewed-by: rcombs on github
Reviewed-by: Daniel Stenberg
Follow up to #5886
Since the struct is quite large (1 long and 10 ints) we
declare it once at the beginning of the function instead
of multiple times inside loops to avoid stack movements.
Reviewed-by: Viktor Szakats
Reviewed-by: Daniel Stenberg
Closes#5886
This is primarily interesting for cases where CURLOPT_NOBODY is set as
previously curl would not return an error for this case.
MDTM getting 550 now also returns this error (it returned
CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for
missing files across protocols and specific FTP commands.
libcurl already returns error on a 550 as a MDTM response (when
CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would
happen subsequently anyway since the RETR command would fail.
Add test 1913 and 1914 to verify. Updated several tests accordingly due
to the updated SIZE behavior.
Reported-by: Tomas Berger
Fixes#5953Closes#5957
... when continuing a transfer from a FETCH response.
When the size of the file was small enough that the entirety of the
transfer happens in a single go and schannel buffers holds the entire
data. However, it wasn't completely read in Curl_pp_readresp since a
line break was found before that could happen. So, by the time we are in
imap_state_fetch_resp - there's data in buffers that needs to be read
via Curl_read but nothing to read from the socket. After we setup a
transfer (Curl_setup_transfer), curl just waits on the socket state to
change - which doesn't happen since no new data ever comes.
Closes#5961
Previously there were several locations that called
nghttp2_session_mem_recv and handled responses slightly differently.
Those have been converted to call the existing
h2_process_pending_input() function.
Moved the end-of-session check to h2_process_pending_input() since the
only place the end-of-session state can change is after nghttp2
processes additional input frames.
This will likely fix the fuzzing error. While I don't have a root cause
the out-of-bounds read seems like a use after free, so moving the
nghttp2_session_check_request_allowed() call to a location with a
guaranteed nghttp2 session seems reasonable.
Also updated a few nghttp2 callsites to include error messages and added
a few additional error checks.
Closes#5648
configure --enable-debug now enables -Wassign-enum with clang,
identifying several enum "abuses" also fixed.
Reported-by: Gisle Vanem
Bug: 879007f811 (commitcomment-42087553)Closes#5929
If the error reason from the lib is
SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return
CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR.
This unifies the libcurl return code and makes libressl run test 313
(CRL testing) fine.
Closes#5934
The cache content is not duplicated, like other caches, but the setting
and specified file name are.
Test 1908 is extended to verify this somewhat. Since the duplicated
handle gets the same file name, the test unfortunately overwrites the
same file twice (with different contents) which makes it hard to check
automatically.
Closes#5923
The code section that is OpenSSL 3+ specific now uses the same logic as
is used in the version < 3 section. It caused a compiler error without
it.
Closes#5907
Drop dynamic loading of ws2_32.dll and instead rely on the
imported version which is now required to be at least 2.2.
Reviewed-by: Marcel Raad
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Reviewed-by: Viktor Szakats
Closes#5854
IPv6, telnet and now also the multi API require WinSock
version 2 which is available starting with Windows 95.
Therefore we think it is time to drop support for version 1.
Reviewed-by: Marcel Raad
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Reviewed-by: Viktor Szakats
Follow up to #5634Closes#5854
The poll emulation via select already consumes POLLRDNORM,
POLLWRNORM and POLLRDBAND as input events. Therefore it
should also return them as output events if signaled.
Also fix indentation in input event handling block.
Assisted-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Replaces #5852Closes#5883
`USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES
backend is fine, but was excluded before.
This also fixes test 1013 as the condition for SMB support in
configure.ac didn't match the condition in the source code. Now it
does.
Fixes https://github.com/curl/curl/issues/1262
Closes https://github.com/curl/curl/pull/5771
poll and select already return -1 on error according to POSIX,
so there is no need to perform a <0 to -1 conversion in code.
Also we can just use one check with <= 0 on the return code.
Assisted-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Replaces #5852Closes#5880
Learn from the way Cygwin handles and maps the WinSock events
to simulate correct and complete poll and select behaviour
according to Richard W. Stevens Network Programming book.
Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad
Follow up to #5634Closes#5867
Setting a timeout to INT_MAX could cause an immediate error to get
returned as timeout because of an overflow when different values of
'now' were used.
This is primarily fixed by having Curl_pgrsTime() return the "now" when
TIMER_STARTSINGLE is set so that the parent function will continue using
that time.
Reported-by: Ionuț-Francisc Oancea
Fixes#5583Closes#5847
USE_TLS_SRP will be true if *any* selected TLS backend can use SRP
HAVE_OPENSSL_SRP is defined when OpenSSL can use it
HAVE_GNUTLS_SRP is defined when GnuTLS can use it
Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is
set if at least one of the supported backends offers SRP.
Reported-by: Stefan Strogin
Fixes#5865Closes#5870
const struct curl_easyoption *curl_easy_option_by_name(const char *name);
const struct curl_easyoption *curl_easy_option_by_id (CURLoption id);
const struct curl_easyoption *
curl_easy_option_next(const struct curl_easyoption *prev);
The purpose is to provide detailed enough information to allow for
example libcurl bindings to get option information at run-time about
what easy options that exist and what arguments they expect.
Assisted-by: Jeroen Ooms
Closes#5365
The get_cert_location function allocates memory only on success.
Previously get_cert_location was able to allocate memory and return
error. It wasn't obvious and in this case the memory wasn't
released.
Fixes#5855Closes#5860
Check readiness of all sockets before waiting on them
to avoid locking in case the one-time event FD_WRITE
was already consumed by a previous wait operation.
More information about WinSock network events:
https://docs.microsoft.com/en-us/windows/win32/api/
winsock2/nf-winsock2-wsaeventselect#return-value
Closes#5634
This avoids using a pair of TCP ports to provide wakeup functionality
for every multi instance on Windows, where socketpair() is emulated
using a TCP socket on loopback which could in turn lead to socket
resource exhaustion.
A previous version of this patch failed to account for how in WinSock,
FD_WRITE is set only once when writing becomes possible and not again
until after a send has failed due to the buffer filling. This contrasts
to how FD_READ and FD_OOB continue to be set until the conditions they
refer to no longer apply. This meant that if a user wrote some data to
a socket, but not enough data to completely fill its send buffer, then
waited on that socket to become writable, we'd erroneously stall until
their configured timeout rather than returning immediately.
This version of the patch addresses that issue by checking each socket
we're waiting on to become writable with select() before the wait, and
zeroing the timeout if it's already writable.
Assisted-by: Marc Hörsken
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Tested-by: Gergely Nagy
Tested-by: Rasmus Melchior Jacobsen
Tested-by: Tomas Berger
Replaces #5397
Reverts #5632Closes#5634
Change Curl_socket_check to use select-fallback in Curl_poll
instead of implementing it in Curl_socket_check and Curl_poll.
Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Replaces #5262 and #5492Closes#5707
This commit changes Curl_socket_check to use POLLPRI to
check for connect failure on the write socket, because
POLLPRI maps to fds_err. This is in line with select(2).
The select-based socket check correctly checks for connect
failures by adding the write socket also to fds_err.
The poll-based implementation (which internally can itself
fallback to select again) did not previously check for
connect failure by using POLLPRI with the write socket.
See the follow up commit to this for more information.
This commit makes sure connect failures can be detected
and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel.
Reviewed-by: Daniel Stenberg
Reviewed-by: Jay Satiro
Replaces #5509
Prepares #5707
The buffer only exists during transfer and then we shouldn't change the
size (the setopt is not documented to work then).
Reported-by: Harry Sintonen
Closes#5842
Failures clearly returned from a (SOCKS) proxy now causes this return
code. Previously the situation was not very clear as what would be
returned and when.
In addition: when this error code is returned, an application can use
CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then
returns a value from the new 'CURLproxycode' enum.
Closes#5770
... not newline separated from the previous line. This makes it output
asterisk prefixed properly like other verbose putput!
Reported-by: jmdavitt on github
Fixes#5826Closes#5827
This flag was applied to the connection struct that is released on
retry. These changes move the retry counter into Curl_easy struct that
lives across retries and retains the new connection.
Reported-by: Cherish98 on github
Fixes#5794Closes#5800
The debug output used ssherr instead of sftperr which not only outputs
the wrong error code but also casues a warning on Windows.
Follow-up to 7370b4e39f
Reported-by: Gisle Vanem
Bug: 7370b4e39f (r41334700)Closes#5799
The shutdown function is for downgrading a connection from TLS to plain,
and this is not requested here.
Have ssl_close reset the TLS connection state.
This partially reverts commit f002c850d9
Reported-by: Rasmus Melchior Jacobsen
Reported-by: Denis Goleshchikhin
Fixes#5797
The previous h2 trailer fix in 54a2b63 was wrong and caused a
regression: it cannot deal with trailers immediately when read since
they may be read off the connection by the wrong 'data' owner.
This change reverts the logic back to gathering all trailers into a
single buffer, like before 54a2b63.
Reported-by: Tadej Vengust
Fixes#5663Closes#5769
Classic mingw and 10y+ old versions of mingw-w64 don't ship with
Windows headers having the typedef necessary for Unix Sockets
support, so try detecting these environments to disable this
feature.
Ref: cf6afc5717/
Reviewed-by: Daniel Stenberg
Fixes#5674Closes#5758
Previously any connect-only connections in a multi handle would be kept
alive until the multi handle was closed. Since these connections cannot
be re-used, they can be marked for closure when the associated easy
handle is removed from the multi handle.
Closes#5749
They're not thread-safe so they should not be used in libcurl code.
Explictly enabled when deemed necessary and in examples and tests
Reviewed-by: Nicolas Sterchele
Closes#5732
Previously a file that isn't user-readable but is user-writable would
not be properly avoided and would get overwritten.
Reported-by: BrumBrum on hackerone
Assisted-by: Jay Satiro
Bug: https://hackerone.com/reports/926638Closes#5731
Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1,
2013-08-02), the easy handle list is not circular but ends with
->next pointing to NULL.
Reported-by: Masaya Suzuki <masayasuzuki@google.com>
Closes#5737
Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented
action but before 7.71.0 that used to switch back to GET and with this
change (assuming the method is still set to HEAD) this behavior is
brought back.
Reported-by: causal-agent on github
Fixes#5725Closes#5728
Stack overflows can occur with precisions for integers and floats.
Proof of concepts:
- curl_mprintf("%d, %.*1$d", 500, 1);
- curl_mprintf("%d, %+0500.*1$f", 500, 1);
Ideally, compile with -fsanitize=address which makes this undefined
behavior a bit more defined for debug purposes.
The format strings are valid. The overflows occur due to invalid
arguments. If these arguments are variables with contents controlled
by an attacker, the function's stack can be corrupted.
Also see CVE-2016-9586 which partially fixed the float aspect.
Signed-off-by: Tobias Stoeckmann <tobias@stoeckmann.org>
Closes https://github.com/curl/curl/pull/5722
Verify that specified parameters are in range. If parameters are too
large, fail early on and avoid out of boundary accesses.
Also do not read behind boundaries of illegal format strings.
These are defensive measures since it is expected that format strings
are well-formed. Format strings should not be modifiable by user
input due to possible generic format string attacks.
Closes https://github.com/curl/curl/pull/5722
They are marked as deprecated for -mmacosx-version-min >= 10.15,
which might result in warnings-as-errors.
Closes https://github.com/curl/curl/pull/5695
It confuses code analyzers with its use of -1 for unsigned value. Also,
a check that's not normally used in strdup() code - and not necessary.
Closes#5697
Provide the HTTP method that was used on the latest request, which might
be relevant for users when there was one or more redirects involved.
Closes#5511
Use the unsigned type (size_t) in the arithmetic of pointers. In this
context, the signed type (ssize_t) is used unnecessarily.
Authored-by: ihsinme on github
Closes#5654
include zstd curl patch for Makefile.m32 from vszakats
and include Add CMake support for zstd from Peter Wu
Helped-by: Viktor Szakats
Helped-by: Peter Wu
Closes#5453
`http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn`
if `CURL_DISABLE_PROXY` is enabled. Repair the build with that
configuration.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Closes#5645
Well-behaving HTTP2 servers send two GOAWAY messages. The first
message is a warning that indicates that the server is going to
stop accepting streams. The second one actually closes the stream.
nghttp2 reports this state (and the other state of no more stream
identifiers) via the call nghttp2_session_check_request_allowed().
In this state the client should not create more streams on the
session (tcp connection), and in curl this means that the server
has requested that the connection is closed.
It would be also be possible to put the connclose() call into the
on_http2_frame_recv() function that triggers on the GOAWAY message.
This fixes a bug seen when the client sees the following sequence of
frames:
// advisory GOAWAY
HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1]
... some additional frames
// final GOAWAY
HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ]
Before this change, curl will attempt to reuse the connection even
after the last stream, will encounter this error:
* Found bundle for host localhost: 0x5595f0a694e0 [can multiplex]
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (::1) port 10443 (#0)
* Using Stream ID: 9 (easy handle 0x5595f0a72e30)
> GET /index.html?5 HTTP/2
> Host: localhost:10443
> user-agent: curl/7.68.0
> accept: */*
>
* stopped the pause stream!
* Connection #0 to host localhost left intact
curl: (16) Error in the HTTP2 framing layer
This error may posion the connection cache, causing future requests
which resolve to the same curl connection to go through the same error
path.
Closes#5643
Confusingly, nghttp2 has two different error code enums:
- nghttp2_error, to be used with nghttp2_strerror
- nghttp2_error_code, to be used with nghttp2_http2_strerror
Closes#5641
Since commit f3d501dc67, if proxy support is disabled, MSVC warns:
url.c : warning C4701: potentially uninitialized local variable
'hostaddr' used
url.c : error C4703: potentially uninitialized local pointer variable
'hostaddr' used
That could actually only happen if both `conn->bits.proxy` and
`CURL_DISABLE_PROXY` were enabled.
Initialize it to NULL to silence the warning.
Closes https://github.com/curl/curl/pull/5638
Updated terminology in docs, comments and phrases to refer to C strings
as "null-terminated". Done to unify with how most other C oriented docs
refer of them and what users in general seem to prefer (based on a
single highly unscientific poll on twitter).
Reported-by: coinhubs on github
Fixes#5598Closes#5608
Don't reference fields that do not exist. Fixes build failure:
vtls/mbedtls.c: In function 'mbed_connect_step1':
vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy'
Closes#5615
...previously CURLINFO_EFFECTIVE_URL would report the URL of the
original "mother transfer", not the actually pushed resource.
Reported-by: Jonathan Cardoso Machado
Fixes#5589Closes#5591
- Include wincrypt before OpenSSL includes so that the latter can
properly handle any conflicts between the two.
Closes https://github.com/curl/curl/pull/5606
Replace "Failed writing body (X != Y)" with
"Failure writing output to destination". Possibly slightly less cryptic.
Reported-by: coinhubs on github
Fixes#5594Closes#5596
Follow-up to c4e6968127
When a new transfer is created, as a resuly of an acknowledged push,
that transfer needs a download buffer allocated.
Closes#5590
This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does
not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default
locations. Instead the CA store can now be used at the same time.
The change is due to the impending release. The issue is still being
discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and
is now documented as experimental.
Ref: bc052cc (parent commit)
Ref: https://github.com/curl/curl/issues/5585
For QUIC but also for regular TCP when the second family runs out of IPs
with a failure while the first family is still trying to connect.
Separated the timeout handling for IPv4 and IPv6 connections when they
both have a number of addresses to iterate over.
This avoids using a pair of TCP ports to provide wakeup functionality
for every multi instance on Windows, where socketpair() is emulated
using a TCP socket on loopback which could in turn lead to socket
resource exhaustion.
Reviewed-by: Gergely Nagy
Reviewed-by: Marc Hörsken
Closes#5397
When wolfSSL is built with its OpenSSL API layer, it fetures the same DES*
functions that OpenSSL has. This change take advantage of that.
Co-authored-by: Daniel Stenberg
Closes#5556Fixes#5548
Since the connection can be used by many independent requests (using
HTTP/2 or HTTP/3), things like user-agent and other transfer-specific
data MUST NOT be kept connection oriented as it could lead to requests
getting the wrong string for their requests. This struct data was
lingering like this due to old HTTP1 legacy thinking where it didn't
mattered..
Fixes#5566Closes#5567
When asking for a specific feature to be shared in the share object,
that bit was previously set unconditionally even if the shared feature
failed or otherwise wouldn't work.
Closes#5554
Instead of discussing if there's value or meaning (implied or not) in
the colors, let's use words without the same possibly negative
associations.
Closes#5546
The SOCKS4/5 state machines weren't properly terminated when the proxy
connection got closed, leading to a busy-loop.
Reported-By: zloi-user on github
Fixes#5532Closes#5542
To reduce the amount of allocations needed for creating a Curl_addrinfo
struct, make a single larger malloc instead of three separate smaller
ones.
Closes#5533
quiche now requires the application to explicitly set the keylog path
for each connection, rather than reading the environment variable
itself.
Closes#5541
Now that all functions in select.[ch] take timediff_t instead
of the limited int or long, we can remove type conversions
and related preprocessor checks to silence compiler warnings.
Avoiding conversions from time_t was already done in 842f73de.
Based upon #5262
Supersedes #5214, #5220 and #5221
Follow up to #5343 and #5479Closes#5490
On some systems, openssl 1.0.x is still the default, but it has been
patched to contain all the recent security fixes. As a result of this
patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be
defined, while the previous behavior of openssl to not look at trusted
chains first, remains.
Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to
probe for the behavior of openssl based on the existence ofmacros.
Closes#5530
Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the
reply ATYP is X'01'. This resulted in erroneously expecting more bytes
when the request length is greater than the reply length (e.g., when
remotely resolving the hostname).
Closes#5527
When the method is updated inside libcurl we must still not change the
method as set by the user as then repeated transfers with that same
handle might not execute the same operation anymore!
This fixes the libcurl part of #5462
Test 1633 added to verify.
Closes#5499
`http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY`
is enabled. Repair the build with that configuration.
Follow-up to f3d501dc67Closes#5498
"Null-checking k->str suggests that it may be null, but it has already
been dereferenced on all paths leading to the check" - and it can't
legally be NULL at this point. Remove check.
Detected by Coverity CID 1463884
Closes#5495
Since Win32 almost always will also have USE_WINSOCK,
we can reduce complexity and always use Sleep there.
Assisted-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Follow up to #5343Closes#5489
