... since the state machine might go to RATELIMITING and then back to
PERFORMING doing once-per-transfer inits in that function is wrong and
it caused problems with receiving chunked HTTP and it set the
PRETRANSFER time much too often...
Regression from b68dc34af3 (shipped in 7.75.0)
Reported-by: Amaury Denoyelle
Fixes#6640Closes#6641
(Unless 32-bit `time_t` is selected manually via the `_USE_32BIT_TIME_T`
mingw macro.)
Previously, 64-bit `time_t` was enabled on VS2005 and newer only, and
32-bit `time_t` was used on all other Windows builds.
Assisted-by: Jay Satiro
Closes#6636
- Use atexit to register a dbg cleanup function that closes the logfile.
LeakSantizier (LSAN) calls _exit() instead of exit() when a leak is
detected on exit so the logfile must be closed explicitly or data could
be lost. Though _exit() does not call atexit handlers such as this,
LSAN's call to _exit() comes after the atexit handlers are called.
Prior to this change the logfile was not explicitly closed so it was
possible that if LSAN detected a leak and called _exit (which does
not flush or close files like exit) then the logfile could be missing
data. That could then cause curl's memanalyze to report false leaks
(eg a malloc was recorded to the logfile but the corresponding free was
discarded from the buffer instead of written to the logfile, then
memanalyze reports that as a leak).
Ref: https://github.com/google/sanitizers/issues/1374
Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541
Closes https://github.com/curl/curl/pull/6620
- Change the Windows char <-> UTF-8 conversion functions to return an
allocated copy of the passed in string instead of the original.
Prior to this change the curlx_convert_ functions would, as what I
assume was an optimization, not make a copy of the passed in string if
no conversion was required. No conversion is required in non-UNICODE
Windows builds since our tchar strings are type char and remain in
whatever the passed in encoding is, which is assumed to be UTF-8 but may
be other encoding.
In contrast the UNICODE Windows builds require conversion
(wchar <-> char) and do return a copy. That inconsistency could lead to
programming errors where the developer expects a copy, and does not
realize that won't happen in all cases.
Closes https://github.com/curl/curl/pull/6602
- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
attribute with the referrer (if there was any)
Closes#6591
... with the help of Curl_resolver_error() which now is moved from
asyn-thead.c and is provided globally for this purpose.
Follow-up to 35ca04ce1b
Makes test 1188 work for c-ares builds
Closes#6626
This caused a memory leak as the session id cache entry was still
erroneously stored with a NULL sessionid and that would later be treated
as not needed to get freed.
Reported-by: Gisle Vanem
Fixes#6616Closes#6617
While working on documenting the states it dawned on me that step one is
to use more descriptive names on the states. This also changes prefix on
the states to make them shorter in the source.
State names NOT ending with *ing are transitional ones.
Closes#6612
The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references to conn->data.
Closes#6608
- Share the shared object from the user's easy handle with the DOH
handles.
Prior to this change if the user had set a shared object with shared
cached DNS (CURL_LOCK_DATA_DNS) for their easy handle then that wasn't
used by any associated DOH handles, since they used the multi's default
hostcache.
This change means all the handles now use the same hostcache, which is
either the shared hostcache from the user created shared object if it
exists or if not then the multi's default hostcache.
Reported-by: Manuj Bhatia
Fixes https://github.com/curl/curl/issues/6589
Closes https://github.com/curl/curl/pull/6607
... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connection to be the connectdata struct, but when we drive transfers
the code still needs to know the pointer to the transfer. We can change
the user data to become the Curl_easy handle, but with older nghttp2
version we cannot dynamically update that pointer properly when
different transfers are used over the same connection.
Closes#6520
We still make the trace callback function get the connectdata struct
passed to it, since the callback is anchored on the connection.
Repeatedly updating the callback pointer to set 'data' with
SSL_CTX_set_msg_callback_arg() doesn't seem to work, probably because
there might already be messages in the queue with the old pointer.
This code therefore makes sure to set the "logger" handle before using
OpenSSL calls so that the right easy handle gets used for tracing.
Closes#6522
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
same as their respective counterparts.
- New curl tool options --doh-insecure and --doh-cert-status do the same
as their respective counterparts.
Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.
Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676
Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
- Guard some Curl_async accesses with USE_CURL_ASYNC instead of
!CURLRES_SYNCH.
This is another follow-up to 8335c64 which moved the async struct from
the connectdata struct into the Curl_easy struct. A previous follow-up
6cd167a fixed building for sync resolver by guarding some async struct
accesses with !CURLRES_SYNCH. The problem is since DOH (DNS-over-HTTPS)
is available as an asynchronous secondary resolver the async struct may
be used even when libcurl is built for the sync resolver. That means
that CURLRES_SYNCH and USE_CURL_ASYNC may be defined at the same time.
Closes https://github.com/curl/curl/pull/6603
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.
The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*
Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.
Fixes#6542Closes#6545
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.
Also: refuse non-supported HTTP versions. Verified with test 129.
Closes#6585
and rename it from 'ftp_list_only' since it is also used for SSH and
POP3. The state is updated internally for 'type=D' FTP URLs.
Added test case 1570 to verify.
Closes#6578
... and make sure the code never updates 'set.prefer_ascii' as it breaks
handle reuse which should use the setting as the user specified it.
Added test 1569 to verify: it first makes an FTP transfer with ';type=A'
and then another without type on the same handle and the second should
then use binary. Previously, curl failed this.
Closes#6578
If libcurl is built with Unicode support for Windows then it is assumed
the filename string is Unicode in UTF-8 encoding and it is converted to
UTF-16 to be passed to the wide character version of the respective
function (eg wstat). However the filename string may actually be in the
local encoding so, even if it successfully converted to UTF-16, if it
could not be stat/accessed then try again using the local code page
version of the function (eg wstat fails try stat).
We already do this with fopen (ie wfopen fails try fopen), so I think it
makes sense to extend it to stat and access functions.
Closes https://github.com/curl/curl/pull/6514
- Use _imp.lib suffix only for Microsoft's compiler (MSVC).
Prior to this change library suffix _imp.lib was used for the import
library on Windows regardless of compiler.
With this change the other compilers should now use their default
suffix which should be .dll.a.
This change is motivated by the usage of pkg-config on MSYS2.
Indeed, when 'pkg-config --libs libcurl' is used, -lcurl is
passed to ld. The documentation of ld on Windows :
https://sourceware.org/binutils/docs/ld/WIN32.html
lists, in the 'direct linking to a dll' section, the pattern
of the searched import library, and libcurl_imp.lib is not there.
Closes https://github.com/curl/curl/pull/6225
Since the set value then risks getting used like that when the easy
handle is reused by the application.
Also: renamed the struct field from 'ftp_append' to 'remote_append'
since it is also used for SSH protocols.
Closes#6579
... as we ignore it anyway because servers don't report the correct size
and proftpd even blatantly returns a 550.
Updates a set of tests accordingly.
Reported-by: awesomenode on github
Fixes#6564Closes#6565
- Separate ngtcp2_transport_params.
ngtcp2/ngtcp2@05d7adc made ngtcp2_transport_params separate from
ngtcp2_settings.
ngtcp2 master is required to build curl with http3 support.
Closes#6554
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.
Closes#6524
... because it turns out several servers out there don't actually behave
correctly otherwise in spite of the fact that the SNI field is
specifically said to be case insensitive in RFC 6066 section 3.
Reported-by: David Earl
Fixes#6540Closes#6543
- Update build instructions in packages/DOS/README
- Extend 'VPATH' with 'vquic' and 'vssh'.
- Allow 'Makefile.dist' to build both 'lib' and 'src'.
- Allow using the Windows hosted djgpp cross compiler to build for MSDOS
under Windows.
- 'USE_SSL' -> 'USE_OPENSSL'
- Added a 'link_EXE' macro. Etc, etc.
- Linking 'curl.exe' needs '$(CURLX_CFILES)' too.
- Do not pick-up '../lib/djgpp/*.o' files. Recompile locally.
- Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'.
- Remove 'djgpp-clean'
- Adapt to new C-ares directory structure
- Use conditional variable assignments
Clarify the 'conditional variable assignment' in 'common.dj'.
Closes https://github.com/curl/curl/pull/6382
This is a follow-up to 8315343 which several days ago moved the resolver
pointer into the async struct but did not update the code that uses it
when getaddrinfo is not present.
Closes https://github.com/curl/curl/pull/6536
As the info is already stored in the transfer handle anyway, there's no
need to carry around a duplicate buffer for the life-time of the handle.
Closes#6534
... and use 'int' for ports. We don't use 'unsigned short' since -1 is
still often used internally to signify "unknown value" and 0 - 65535 are
all valid port numbers.
Closes#6534
The old function should not be used anywhere anymore (the only remaining
gskit use has to be fixed to instead use Curl_poll or none at all).
The static function version is now called our_select() and is only built
if necessary.
Closes#6531
Readdir data, filenames and attributes are strictly related to the
transfer and not the connection. This also reduces the total size of the
fixed connectdata struct.
Closes#6519
On Windows an error number may be greater than INT_MAX and negative once
cast to int.
The assertion is checked only in debug builds.
Closes https://github.com/curl/curl/pull/6504
... if Curl_doh() returned a NULL, this function gets called anyway as
in a asynch procedure. Then the doh struct pointer is NULL and signifies
an OOM situation.
Follow-up to 6246a1d8c6
- Reorder some internal struct members so that less padding is used.
This is an attempt at saving a bit of space by packing some structs
(using pahole to find the holes) where it might make sense to do
so without losing readability.
I.e., I tried to avoid separating fields that seem grouped
together (like the cwd... fields in struct ftp_conn for instance).
Also abstained from touching fields behind conditional macros as
that quickly can get complicated.
Closes https://github.com/curl/curl/pull/6483
