Commit Graph

1593 Commits

Author SHA1 Message Date
Daniel Stenberg 95cbcec8f9
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.

Closes #6798
2021-03-26 23:19:20 +01:00
Daniel Stenberg 78f642ffab
config: remove CURL_SIZEOF_CURL_OFF_T use only SIZEOF_CURL_OFF_T
Make the code consistently use a single name for the size of the
"curl_off_t" type.

Closes #6702
2021-03-11 10:11:56 +01:00
Daniel Gustafsson 313faa1bcb url: fix typo in comment
Correct a small typo which snuck in with a304051620.
2021-02-28 21:13:26 +01:00
Daniel Stenberg 70472a44de
urldata: remove the _ORIG suffix from string names
It doesn't provide any useful info but only makes the names longer.

Closes #6624
2021-02-19 08:07:19 +01:00
Daniel Stenberg d25c479e84
url: fix memory leak if OOM in the HSTS handling
Reported-by: Viktor Szakats
Bug: https://github.com/curl/curl/pull/6627#issuecomment-781626205

Closes #6628
2021-02-19 08:05:40 +01:00
Daniel Stenberg 6174fc993a
url.c: use consistent error message for failed resolve 2021-02-18 11:10:36 +01:00
Daniel Stenberg f248a13065
parse_proxy: fix a memory leak in the OOM path
Reported-by: Jay Satiro
Reviewed-by: Jay Satiro
Reviewed-by: Emil Engler

Closes #6614
Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541
2021-02-17 22:54:42 +01:00
Jay Satiro 568190f493 url: fix possible use-after-free in default protocol
Prior to this change if the user specified a default protocol and a
separately allocated non-absolute URL was used then it was freed
prematurely, before it was then used to make the replacement URL.

Bug: https://github.com/curl/curl/issues/6604#issuecomment-780138219
Reported-by: arvids-kokins-bidstack@users.noreply.github.com

Closes https://github.com/curl/curl/pull/6613
2021-02-17 16:15:11 -05:00
Daniel Stenberg cfff12a0b3
lib: remove 'conn->data' completely
The Curl_easy pointer struct entry in connectdata is now gone. Just
before commit 215db086e0 landed on January 8, 2021 there were 919
references to conn->data.

Closes #6608
2021-02-16 10:48:57 +01:00
Jay Satiro 53022e1893 doh: add options to disable ssl verification
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
  CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
  same as their respective counterparts.

- New curl tool options --doh-insecure and --doh-cert-status do the same
  as their respective counterparts.

Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.

Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676

Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
2021-02-14 18:20:48 -05:00
Daniel Stenberg 46620b9743
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545
2021-02-13 22:36:15 +01:00
Daniel Stenberg 88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Jay Satiro cb2dc1ba89 lib: use int type for more port variables
This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553
2021-02-09 02:53:31 -05:00
Daniel Stenberg 1c3def53c7
urldata: remove duplicate 'upkeep_interval_ms' from connectdata
... and rely only on the value already set in Curl_easy.

Closes #6534
2021-01-27 09:19:08 +01:00
Daniel Stenberg d6a37c23a3
urldata: remove 'local_ip' from the connectdata struct
As the info is already stored in the transfer handle anyway, there's no
need to carry around a duplicate buffer for the life-time of the handle.

Closes #6534
2021-01-27 09:19:05 +01:00
Daniel Stenberg 764c6bd3bf
urldata: remove duplicate port number storage
... and use 'int' for ports. We don't use 'unsigned short' since -1 is
still often used internally to signify "unknown value" and 0 - 65535 are
all valid port numbers.

Closes #6534
2021-01-27 09:19:01 +01:00
Daniel Stenberg 642d78026f
urldata: remove the duplicate 'ip_addr_str' field
... as the numerical IP address is already stored and kept in 'primary_ip'.

Closes #6534
2021-01-27 09:18:51 +01:00
Daniel Stenberg 560fc170ec
url: reduce conn->data references
... there are a few left but let's keep them to last

Closes #6512
2021-01-26 10:00:59 +01:00
Daniel Stenberg 8335c6417e
hostip: remove conn->data from resolver functions
This also moves the 'async' struct from the connectdata struct into the
Curl_easy struct, which seems like a better home for it.

Closes #6497
2021-01-22 08:25:09 +01:00
Daniel Stenberg 6246a1d8c6
doh: allocate state struct on demand
... instead of having it static within the Curl_easy struct. This takes
away 1176 bytes (18%) from the Curl_easy struct that aren't used very
often and instead makes the code allocate it when needed.

Closes #6492
2021-01-20 09:02:24 +01:00
Daniel Stenberg a304051620
lib: more conn->data cleanups
Closes #6479
2021-01-19 09:14:51 +01:00
Patrick Monnerat bbe3aa9f88
vtls: reduce conn->data use
Closes #6474
2021-01-19 09:14:40 +01:00
Daniel Stenberg 215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Daniel Stenberg 229930762f
url: if IDNA conversion fails, fallback to Transitional
This improves IDNA2003 compatiblity.

Reported-by: Bubu on github
Fixes #6423
Closes #6428
2021-01-11 00:21:28 +01:00
Daniel Stenberg 0936ecd0ee
pretransfer: setup the User-Agent header here
... and not in the connection setup, as for multiplexed transfers the
connection setup might be skipped and then the transfer would end up
without the set user-agent!

Reported-by: Flameborn on github
Assisted-by: Andrey Gursky
Assisted-by: Jay Satiro
Assisted-by: Mike Gelfand
Fixes #6312
Closes #6417
2021-01-05 23:46:32 +01:00
Daniel Stenberg 648712eec1
httpauth: make multi-request auth work with custom port
When doing HTTP authentication and a port number set with CURLOPT_PORT,
the code would previously have the URL's port number override as if it
had been a redirect to an absolute URL.

Added test 1568 to verify.

Reported-by: UrsusArctos on github
Fixes #6397
Closes #6400
2021-01-02 23:40:47 +01:00
Emil Engler 725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00
Daniel Stenberg 8ab78f720a
misc: fix "warning: empty expression statement has no effect"
Turned several macros into do-while(0) style to allow their use to work
find with semicolon.

Bug: 08e8455ddd (commitcomment-45433279)
Follow-up to 08e8455ddd
Reported-by: Gisle Vanem
Closes #6376
2020-12-26 23:44:17 +01:00
Daniel Stenberg aba01da639
failf: remove newline from formatting strings
... as failf adds one itself.

Also: add an assert() to failf() that triggers on a newline in the
format string!

Closes #6365
2020-12-25 23:42:16 +01:00
parazyd a1f06f32b8
gopher: Implement secure gopher protocol.
This commit introduces a "gophers" handler inside the gopher protocol if
USE_SSL is defined. This protocol is no different than the usual gopher
prococol, with the added TLS encapsulation upon connecting. The protocol
has been adopted in the gopher community, and many people have enabled
TLS in their gopher daemons like geomyidae(8), and clients, like clic(1)
and hurl(1).

I have not implemented test units for this protocol because my knowledge
of Perl is sub-par. However, for someone more knowledgeable it might be
fairly trivial, because the same test that tests the plain gopher
protocol can be used for "gophers" just by adding a TLS listener.

Signed-off-by: parazyd <parazyd@dyne.org>

Closes #6208
2020-12-15 12:58:12 +01:00
Daniel Stenberg ec9cc725d5
ftp: CURLOPT_FTP_SKIP_PASV_IP by default
The command line tool also independently sets --ftp-skip-pasv-ip by
default.

Ten test cases updated to adapt the modified --libcurl output.

Bug: https://curl.se/docs/CVE-2020-8284.html
CVE-2020-8284

Reported-by: Varnavas Papaioannou
2020-12-07 08:38:05 +01:00
Daniel Stenberg a95a6ce6b8
urldata: remove 'void *protop' and create the union 'p'
... to avoid the use of 'void *' for the protocol specific structs done
per transfer.

Closes #6238
2020-11-23 16:16:16 +01:00
Daniel Stenberg 20558ff269
url: make sure an HSTS upgrade updates URL and scheme correctly
Closes #6175
2020-11-06 08:22:31 +01:00
Daniel Stenberg 4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg 2cfc4ed983
hsts: add read/write callbacks
- read/write callback options
- man pages for the 4 new setopts
- test 1915 verifies the callbacks

Closes #5896
2020-11-03 16:08:48 +01:00
Daniel Stenberg 7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Daniel Stenberg 96450a1a33
alt-svc: enable by default
Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported.

alt-svc support in curl is no longer considered experimental

Closes #5868
2020-10-25 23:08:54 +01:00
Daniel Stenberg 6434a73984
Curl_handler: add 'family' to each protocol
Makes get_protocol_family() faster and it moves the knowledge about the
"families" to each protocol handler, where it belongs.

Closes #5986
2020-09-21 23:30:16 +02:00
Gergely Nagy 182ff2d63c
vtls: deduplicate client certificates in ssl_config_data
Closes #5629
2020-09-14 12:56:47 +02:00
Diven Qi 3532262edd
url: use blank credentials when using proxy w/o username and password
Fixes proxy regression brought in commit ad829b21ae (7.71.0)

Fixed #5911
Closes #5914
2020-09-08 00:48:09 +02:00
Daniel Stenberg f93455eb04
altsvc: clone setting in curl_easy_duphandle
The cache content is not duplicated, like other caches, but the setting
and specified file name are.

Test 1908 is extended to verify this somewhat. Since the duplicated
handle gets the same file name, the test unfortunately overwrites the
same file twice (with different contents) which makes it hard to check
automatically.

Closes #5923
2020-09-06 09:54:54 +02:00
Martin Bašti 3eff1c5092
http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set
... in case NO_PROXY takes an effect

Without this patch, the following command crashes:

    $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \
        git clone https://github.com/curl/curl.git

Minimal libcurl-based reproducer:

    #include <curl/curl.h>

    int main() {
      CURL *curl = curl_easy_init();
      if(curl) {
        CURLcode ret;
        curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/");
        curl_easy_setopt(curl, CURLOPT_PROXY, "example.com");
        /* set the proxy type */
        curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS);
        curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com");
        curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
        ret = curl_easy_perform(curl);
        curl_easy_cleanup(curl);
        return ret;
      }
      return -1;
    }

Assisted-by: Kamil Dudka
Bug: https://bugzilla.redhat.com/1873327
Closes #5902
2020-09-03 08:01:56 +02:00
Daniel Stenberg 9b3f888a00
llist: make it "struct Curl_llist"
As internal global names should use captical C.

Closes #5906
2020-09-02 22:41:58 +02:00
Daniel Stenberg e37e446868
mqtt: enable by default
No longer considered experimental.

Closes #5858
2020-08-31 09:45:09 +02:00
Michael Baentsch ede125b7b7
tls: add CURLOPT_SSL_EC_CURVES and --curves
Closes #5892
2020-08-30 17:24:04 +02:00
Daniel Stenberg a337355487
url: remove funny embedded comments in Curl_disonnect calls 2020-08-30 17:03:48 +02:00
Chris Paulson-Ellis 728f8d3bdc
conn: check for connection being dead before reuse
Prevents incorrect reuse of an HTTP connection that has been prematurely
shutdown() by the server.

Partial revert of 755083d00d

Fixes #5884
Closes #5893
2020-08-30 16:59:16 +02:00
Marcel Raad f504f18641 ntlm: fix condition for curl_ntlm_core usage
`USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES
backend is fine, but was excluded before.

This also fixes test 1013 as the condition for SMB support in
configure.ac didn't match the condition in the source code. Now it
does.

Fixes https://github.com/curl/curl/issues/1262
Closes https://github.com/curl/curl/pull/5771
2020-08-29 10:08:32 +02:00
Daniel Stenberg 3c9e021f86
Curl_easy: remember last connection by id, not by pointer
CVE-2020-8231

Bug: https://curl.haxx.se/docs/CVE-2020-8231.html

Reported-by: Marc Aldorasi
Closes #5824
2020-08-17 14:33:09 +02:00
Jay Satiro a12a16151a url: fix CURLU and location following
Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was
incorrectly used for the location follow, resulting in infinite requests
to the original location.

Reported-by: sspiri@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/5709
Closes https://github.com/curl/curl/pull/5713
2020-07-30 11:57:35 -04:00