- Save the original conn->data before it's changed to the specified
data transfer for the connection check and then restore it afterwards.
This is a follow-up to 38d8e1b 2019-02-11.
History:
It was discovered a month ago that before checking whether to extract a
dead connection that that connection should be associated with a "live"
transfer for the check (ie original conn->data ignored and set to the
passed in data). A fix was landed in 54b201b which did that and also
cleared conn->data after the check. The original conn->data was not
restored, so presumably it was thought that a valid conn->data was no
longer needed.
Several days later it was discovered that a valid conn->data was needed
after the check and follow-up fix was landed in bbae24c which partially
reverted the original fix and attempted to limit the scope of when
conn->data was changed to only when pruning dead connections. In that
case conn->data was not cleared and the original conn->data not
restored.
A month later it was discovered that the original fix was somewhat
correct; a "live" transfer is needed for the check in all cases
because original conn->data could be null which could cause a bad deref
at arbitrary points in the check. A fix was landed in 38d8e1b which
expanded the scope to all cases. conn->data was not cleared and the
original conn->data not restored.
A day later it was discovered that not restoring the original conn->data
may lead to busy loops in applications that use the event interface, and
given this observation it's a pretty safe assumption that there is some
code path that still needs the original conn->data. This commit is the
follow-up fix for that, it restores the original conn->data after the
connection check.
Assisted-by: tholin@users.noreply.github.com
Reported-by: tholin@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/3542Closes#3559
On non-ascii platforms, the chunked hex header was measured for char code
conversion length, even for chunked trailers that do not have an hex header.
In addition, the efective length is already known: use it.
Since the hex length can be zero, only convert if needed.
Reported by valgrind.
- Pass an empty string to CURLOPT_ACCEPT_ENCODING to use the default
supported encodings.
Prior to this change the specific encodings of gzip and deflate were set
but there's no guarantee they'd be supported by the user's libcurl.
Convert numerous infof() calls into debug-build only messages since they
are annoyingly verbose for regular applications. Removed a few.
Bug: https://curl.haxx.se/mail/lib-2019-02/0027.html
Reported-by: Volker Schmid
Closes#3552
The code is more or less copied from the stdout comparison code, maybe
some better reuse is possible.
test 1457 is adjusted to make the output actually match (by using --silent)
test 506 used <stderr> without actually needing it, so that <stderr> block is removed
Closes#3536
Option -F generates an intermediate representation of the mime structure
that is used later to create the libcurl mime structure and generate
the --libcurl statements.
Reported-by: Daniel Stenberg
Fixes#3532Closes#3546
Commit 3f16990ec8 followed-up a bug in b49652ac66 but was
inadvertently introducing a new bug in the ternary expression.
Close#3555
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
There is no benefit to holding the data sharelock when freeing the
addrinfo in case it fails, so ensure releaseing it as soon as we can
rather than holding on to it. This also aligns the code with other
consumers of sharelocks.
Closes#3516
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
The http2 code for connection checking needs a transfer to use. Make
sure a working one is set before handler->connection_check() is called.
Reported-by: jnbr on github
Fixes#3541Closes#3547
That is functions that are declared global but are not used from outside
of the file in which it is declared. Such functions should be made
static or even at times be removed.
It also verifies that all used curl_ prefixed functions are "blessed"
Closes#3538
urlapi: turn three local-only functions into statics
conncache: make conncache_find_first_connection static
multi: make detach_connnection static
connect: make getaddressinfo static
curl_ntlm_core: make hmac_md5 static
http2: make two functions static
http: make http_setup_conn static
connect: make tcpnodelay static
tests: make UNITTEST a thing to mark functions with, so they can be static for
normal builds and non-static for unit test builds
... and mark Curl_shuffle_addr accordingly.
url: make up_free static
setopt: make vsetopt static
curl_endian: make write32_le static
rtsp: make rtsp_connisdead static
warnless: remove unused functions
memdebug: remove one unused function, made another static
The build logs will be at https://cirrus-ci.com/github/curl/curl
Some tests are currently failing and so disabled for now. The SSH server
isn't starting for the SSH tests due to unsupported options used in its
config file. The DICT server also is failing on startup.
':' is interpreted as separator by zsh, so if used as part of the argument
or option's description it needs to be escaped.
The problem can be reproduced as follows:
% curl --reso<TAB>
% curl -E <TAB>
Bug: https://bugs.debian.org/921452
The current regex fails to match '<...>' arguments properly (e.g. those
with spaces in them), which causes an completion script with wrong
descriptions for some options.
Here's a diff of the generated completion script, comparing the previous
version to the one with this fix:
--- /usr/share/zsh/vendor-completions/_curl 2019-01-15 20:47:40.000000000 +0000
+++ _curl 2019-02-05 20:57:29.453349040 +0000
@@ -9,48 +9,48 @@
_arguments -C -S \
--happy-eyeballs-timeout-ms'[How long to wait in milliseconds for IPv6 before trying IPv4]':'<milliseconds>' \
+ --resolve'[Resolve the host+port to this address]':'<host:port:address[,address]...>' \
{-c,--cookie-jar}'[Write cookies to <filename> after operation]':'<filename>':_files \
{-D,--dump-header}'[Write the received headers to <filename>]':'<filename>':_files \
{-y,--speed-time}'[Trigger '\''speed-limit'\'' abort after this time]':'<seconds>' \
--proxy-cacert'[CA certificate to verify peer against for proxy]':'<file>':_files \
- --tls13-ciphers'[of TLS 1.3 ciphersuites> TLS 1.3 cipher suites to use]':'<list' \
+ --tls13-ciphers'[TLS 1.3 cipher suites to use]':'<list of TLS 1.3 ciphersuites>' \
{-E,--cert}'[Client certificate file and password]':'<certificate[:password]>' \
--libcurl'[Dump libcurl equivalent code of this command line]':'<file>':_files \
--proxy-capath'[CA directory to verify peer against for proxy]':'<dir>':_files \
- --proxy-negotiate'[HTTP Negotiate (SPNEGO) authentication on the proxy]':'Use' \
--proxy-pinnedpubkey'[FILE/HASHES public key to verify proxy with]':'<hashes>' \
--crlfile'[Get a CRL list in PEM format from the given file]':'<file>':_files \
- --proxy-insecure'[HTTPS proxy connections without verifying the proxy]':'Do' \
- --proxy-ssl-allow-beast'[security flaw for interop for HTTPS proxy]':'Allow' \
+ --proxy-negotiate'[Use HTTP Negotiate (SPNEGO) authentication on the proxy]' \
--abstract-unix-socket'[Connect via abstract Unix domain socket]':'<path>' \
--pinnedpubkey'[FILE/HASHES Public key to verify peer against]':'<hashes>' \
+ --proxy-insecure'[Do HTTPS proxy connections without verifying the proxy]' \
--proxy-pass'[Pass phrase for the private key for HTTPS proxy]':'<phrase>' \
+ --proxy-ssl-allow-beast'[Allow security flaw for interop for HTTPS proxy]' \
{-p,--proxytunnel}'[Operate through an HTTP proxy tunnel (using CONNECT)]' \
--socks5-hostname'[SOCKS5 proxy, pass host name to proxy]':'<host[:port]>' \
--proto-default'[Use PROTOCOL for any URL missing a scheme]':'<protocol>' \
- --proxy-tls13-ciphers'[list> TLS 1.3 proxy cipher suites]':'<ciphersuite' \
+ --proxy-tls13-ciphers'[TLS 1.3 proxy cipher suites]':'<ciphersuite list>' \
--socks5-gssapi-service'[SOCKS5 proxy service name for GSS-API]':'<name>' \
--ftp-alternative-to-user'[String to replace USER \[name\]]':'<command>' \
- --ftp-ssl-control'[SSL/TLS for FTP login, clear for transfer]':'Require' \
{-T,--upload-file}'[Transfer local FILE to destination]':'<file>':_files \
--local-port'[Force use of RANGE for local port numbers]':'<num/range>' \
--proxy-tlsauthtype'[TLS authentication type for HTTPS proxy]':'<type>' \
{-R,--remote-time}'[Set the remote file'\''s time on the local output]' \
- --retry-connrefused'[on connection refused (use with --retry)]':'Retry' \
- --suppress-connect-headers'[proxy CONNECT response headers]':'Suppress' \
- {-j,--junk-session-cookies}'[session cookies read from file]':'Ignore' \
- --location-trusted'[--location, and send auth to other hosts]':'Like' \
+ --ftp-ssl-control'[Require SSL/TLS for FTP login, clear for transfer]' \
--proxy-cert-type'[Client certificate type for HTTPS proxy]':'<type>' \
{-O,--remote-name}'[Write output to a file named as the remote file]' \
+ --retry-connrefused'[Retry on connection refused (use with --retry)]' \
+ --suppress-connect-headers'[Suppress proxy CONNECT response headers]' \
--trace-ascii'[Like --trace, but without hex output]':'<file>':_files \
--connect-timeout'[Maximum time allowed for connection]':'<seconds>' \
--expect100-timeout'[How long to wait for 100-continue]':'<seconds>' \
{-g,--globoff}'[Disable URL sequences and ranges using {} and \[\]]' \
+ {-j,--junk-session-cookies}'[Ignore session cookies read from file]' \
{-m,--max-time}'[Maximum time allowed for the transfer]':'<seconds>' \
--dns-ipv4-addr'[IPv4 address to use for DNS requests]':'<address>' \
--dns-ipv6-addr'[IPv6 address to use for DNS requests]':'<address>' \
- --ignore-content-length'[the size of the remote resource]':'Ignore' \
{-k,--insecure}'[Allow insecure server connections when using SSL]' \
+ --location-trusted'[Like --location, and send auth to other hosts]' \
--mail-auth'[Originator address of the original email]':'<address>' \
--noproxy'[List of hosts which do not use proxy]':'<no-proxy-list>' \
--proto-redir'[Enable/disable PROTOCOLS on redirect]':'<protocols>' \
@@ -62,18 +62,19 @@
--socks5-basic'[Enable username/password auth for SOCKS5 proxies]' \
--cacert'[CA certificate to verify peer against]':'<file>':_files \
{-H,--header}'[Pass custom header(s) to server]':'<header/@file>' \
+ --ignore-content-length'[Ignore the size of the remote resource]' \
{-i,--include}'[Include protocol response headers in the output]' \
--proxy-header'[Pass custom header(s) to proxy]':'<header/@file>' \
--unix-socket'[Connect through this Unix domain socket]':'<path>' \
{-w,--write-out}'[Use output FORMAT after completion]':'<format>' \
- --http2-prior-knowledge'[HTTP 2 without HTTP/1.1 Upgrade]':'Use' \
{-o,--output}'[Write to file instead of stdout]':'<file>':_files \
- {-J,--remote-header-name}'[the header-provided filename]':'Use' \
+ --preproxy'[\[protocol://\]host\[:port\] Use this proxy first]' \
--socks4a'[SOCKS4a proxy on given host + port]':'<host[:port]>' \
{-Y,--speed-limit}'[Stop transfers slower than this]':'<speed>' \
{-z,--time-cond}'[Transfer based on a time condition]':'<time>' \
--capath'[CA directory to verify peer against]':'<dir>':_files \
{-f,--fail}'[Fail silently (no output at all) on HTTP errors]' \
+ --http2-prior-knowledge'[Use HTTP 2 without HTTP/1.1 Upgrade]' \
--proxy-tlspassword'[TLS password for HTTPS proxy]':'<string>' \
{-U,--proxy-user}'[Proxy user and password]':'<user:password>' \
--proxy1.0'[Use HTTP/1.0 proxy on given port]':'<host[:port]>' \
@@ -81,52 +82,49 @@
{-A,--user-agent}'[Send User-Agent <name> to server]':'<name>' \
--egd-file'[EGD socket path for random data]':'<file>':_files \
--fail-early'[Fail on first transfer error, do not continue]' \
- --haproxy-protocol'[HAProxy PROXY protocol v1 header]':'Send' \
- --preproxy'[Use this proxy first]':'[protocol://]host[:port]' \
+ {-J,--remote-header-name}'[Use the header-provided filename]' \
--retry-max-time'[Retry only within this period]':'<seconds>' \
--socks4'[SOCKS4 proxy on given host + port]':'<host[:port]>' \
--socks5'[SOCKS5 proxy on given host + port]':'<host[:port]>' \
- --socks5-gssapi-nec'[with NEC SOCKS5 server]':'Compatibility' \
- --ssl-allow-beast'[security flaw to improve interop]':'Allow' \
--cert-status'[Verify the status of the server certificate]' \
- --ftp-create-dirs'[the remote dirs if not present]':'Create' \
{-:,--next}'[Make next URL use its separate set of options]' \
--proxy-key-type'[Private key file type for proxy]':'<type>' \
- --remote-name-all'[the remote file name for all URLs]':'Use' \
{-X,--request}'[Specify request command to use]':'<command>' \
--retry'[Retry request if transient problems occur]':'<num>' \
- --ssl-no-revoke'[cert revocation checks (WinSSL)]':'Disable' \
--cert-type'[Certificate file type (DER/PEM/ENG)]':'<type>' \
--connect-to'[Connect to host]':'<HOST1:PORT1:HOST2:PORT2>' \
--create-dirs'[Create necessary local directory hierarchy]' \
+ --haproxy-protocol'[Send HAProxy PROXY protocol v1 header]' \
--max-redirs'[Maximum number of redirects allowed]':'<num>' \
{-n,--netrc}'[Must read .netrc for user name and password]' \
+ {-x,--proxy}'[\[protocol://\]host\[:port\] Use this proxy]' \
--proxy-crlfile'[Set a CRL list for proxy]':'<file>':_files \
--sasl-ir'[Enable initial response in SASL authentication]' \
- --socks5-gssapi'[GSS-API auth for SOCKS5 proxies]':'Enable' \
+ --socks5-gssapi-nec'[Compatibility with NEC SOCKS5 server]' \
+ --ssl-allow-beast'[Allow security flaw to improve interop]' \
+ --ftp-create-dirs'[Create the remote dirs if not present]' \
--interface'[Use network INTERFACE (or address)]':'<name>' \
--key-type'[Private key file type (DER/PEM/ENG)]':'<type>' \
--netrc-file'[Specify FILE for netrc]':'<filename>':_files \
{-N,--no-buffer}'[Disable buffering of the output stream]' \
--proxy-service-name'[SPNEGO proxy service name]':'<name>' \
- --styled-output'[styled output for HTTP headers]':'Enable' \
+ --remote-name-all'[Use the remote file name for all URLs]' \
+ --ssl-no-revoke'[Disable cert revocation checks (WinSSL)]' \
--max-filesize'[Maximum file size to download]':'<bytes>' \
--negotiate'[Use HTTP Negotiate (SPNEGO) authentication]' \
--no-keepalive'[Disable TCP keepalive on the connection]' \
{-#,--progress-bar}'[Display transfer progress as a bar]' \
- {-x,--proxy}'[Use this proxy]':'[protocol://]host[:port]' \
- --proxy-anyauth'[any proxy authentication method]':'Pick' \
{-Q,--quote}'[Send command(s) to server before transfer]' \
- --request-target'[the target for this request]':'Specify' \
+ --socks5-gssapi'[Enable GSS-API auth for SOCKS5 proxies]' \
{-u,--user}'[Server user and password]':'<user:password>' \
{-K,--config}'[Read config from a file]':'<file>':_files \
{-C,--continue-at}'[Resumed transfer offset]':'<offset>' \
--data-raw'[HTTP POST data, '\''@'\'' allowed]':'<data>' \
- --disallow-username-in-url'[username in url]':'Disallow' \
--krb'[Enable Kerberos with security <level>]':'<level>' \
--proxy-ciphers'[SSL ciphers to use for proxy]':'<list>' \
--proxy-digest'[Use Digest authentication on the proxy]' \
--proxy-tlsuser'[TLS username for HTTPS proxy]':'<name>' \
+ --styled-output'[Enable styled output for HTTP headers]' \
{-b,--cookie}'[Send cookies from string/file]':'<data>' \
--data-urlencode'[HTTP POST data url encoded]':'<data>' \
--delegation'[GSS-API delegation permission]':'<LEVEL>' \
@@ -134,7 +132,10 @@
--post301'[Do not switch to GET after following a 301]' \
--post302'[Do not switch to GET after following a 302]' \
--post303'[Do not switch to GET after following a 303]' \
+ --proxy-anyauth'[Pick any proxy authentication method]' \
+ --request-target'[Specify the target for this request]' \
--trace-time'[Add time stamps to trace/verbose output]' \
+ --disallow-username-in-url'[Disallow username in url]' \
--dns-servers'[DNS server addrs to use]':'<addresses>' \
{-G,--get}'[Put the post data in the URL and use GET]' \
--limit-rate'[Limit transfer speed to RATE]':'<speed>' \
@@ -148,21 +149,21 @@
--metalink'[Process given URLs as metalink XML file]' \
--tr-encoding'[Request compressed transfer encoding]' \
--xattr'[Store metadata in extended file attributes]' \
- --ftp-skip-pasv-ip'[the IP address for PASV]':'Skip' \
--pass'[Pass phrase for the private key]':'<phrase>' \
--proxy-ntlm'[Use NTLM authentication on the proxy]' \
{-S,--show-error}'[Show error even when -s is used]' \
- --ciphers'[of ciphers> SSL ciphers to use]':'<list' \
+ --ciphers'[SSL ciphers to use]':'<list of ciphers>' \
--form-string'[Specify multipart MIME data]':'<name=string>' \
--login-options'[Server login options]':'<options>' \
--tftp-blksize'[Set TFTP BLKSIZE option]':'<value>' \
- --tftp-no-options'[not send any TFTP options]':'Do' \
{-v,--verbose}'[Make the operation more talkative]' \
+ --ftp-skip-pasv-ip'[Skip the IP address for PASV]' \
--proxy-key'[Private key for HTTPS proxy]':'<key>' \
{-F,--form}'[Specify multipart MIME data]':'<name=content>' \
--mail-from'[Mail from this address]':'<address>' \
--oauth2-bearer'[OAuth 2 Bearer Token]':'<token>' \
--proto'[Enable/disable PROTOCOLS]':'<protocols>' \
+ --tftp-no-options'[Do not send any TFTP options]' \
--tlsauthtype'[TLS authentication type]':'<type>' \
--doh-url'[Resolve host names over DOH]':'<URL>' \
--no-sessionid'[Disable SSL session-ID reusing]' \
@@ -173,14 +174,13 @@
--ftp-ssl-ccc'[Send CCC after authenticating]' \
{-4,--ipv4}'[Resolve names to IPv4 addresses]' \
{-6,--ipv6}'[Resolve names to IPv6 addresses]' \
- --netrc-optional'[either .netrc or URL]':'Use' \
--service-name'[SPNEGO service name]':'<name>' \
{-V,--version}'[Show version number and quit]' \
--data-ascii'[HTTP POST ASCII data]':'<data>' \
--ftp-account'[Account data string]':'<data>' \
- --compressed-ssh'[SSH compression]':'Enable' \
--disable-eprt'[Inhibit using EPRT or LPRT]' \
--ftp-method'[Control CWD usage]':'<method>' \
+ --netrc-optional'[Use either .netrc or URL]' \
--pubkey'[SSH Public key file name]':'<key>' \
--raw'[Do HTTP "raw"; no transfer decoding]' \
--anyauth'[Pick any authentication method]' \
@@ -189,6 +189,7 @@
--no-alpn'[Disable the ALPN TLS extension]' \
--tcp-nodelay'[Use the TCP_NODELAY option]' \
{-B,--use-ascii}'[Use ASCII/text transfer]' \
+ --compressed-ssh'[Enable SSH compression]' \
--digest'[Use HTTP Digest Authentication]' \
--proxy-tlsv1'[Use TLSv1 for HTTPS proxy]' \
--engine'[Crypto engine to use]':'<name>' \
Use long for CURLOPT_HTTP09_ALLOWED to fix the following warning:
tool_operate.c: In function 'operate_do':
../include/curl/typecheck-gcc.h:47:9: error: call to
'_curl_easy_setopt_err_long' declared with attribute warning:
curl_easy_setopt expects a long argument for this option [-Werror]
Closes https://github.com/curl/curl/pull/3534
