1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-18 07:25:14 -05:00
Commit Graph

813 Commits

Author SHA1 Message Date
Daniel Stenberg
3c5f9ba899
url: only reuse TLS connections with matching pinning
If the requests have different CURLOPT_PINNEDPUBLICKEY strings set, the
connection should not be reused.

Bug: https://curl.haxx.se/mail/lib-2019-09/0061.html
Reported-by: Sebastian Haglund

Closes #4347
2019-09-19 14:58:24 +02:00
Daniel Stenberg
4ac2884003
urldata: avoid 'generic', use dedicated pointers
For the 'proto' union within the connectdata struct.

Closes #4290
2019-09-03 23:00:51 +02:00
Daniel Stenberg
68fab35c73
http: fix use of credentials from URL when using HTTP proxy
When a username and password are provided in the URL, they were wrongly
removed from the stored URL so that subsequent uses of the same URL
wouldn't find the crendentials. This made doing HTTP auth with multiple
connections (like Digest) mishave.

Regression from 46e164069d (7.62.0)

Test case 335 added to verify.

Reported-by: Mike Crowe

Fixes #4228
Closes #4229
2019-08-16 10:11:22 +02:00
Daniel Stenberg
f2cc26456b
ngtcp2: do QUIC connections happy-eyeballs friendly 2019-08-14 23:14:12 +02:00
Daniel Stenberg
98c3f14871
alt-svc: send Alt-Used: in redirected requests
RFC 7838 section 5:

   When using an alternative service, clients SHOULD include an Alt-Used
   header field in all requests.

Removed CURLALTSVC_ALTUSED again (feature is still EXPERIMENTAL thus
this is deemed ok).

You can disable sending this header just like you disable any other HTTP
header in libcurl.

Closes #4199
2019-08-08 08:24:37 +02:00
Daniel Stenberg
026840e35c
CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly
Even though it cannot fall-back to a lower HTTP version automatically. The
safer way to upgrade remains via CURLOPT_ALTSVC.

CURLOPT_H3 no longer has any bits that do anything and might be removed
before we remove the experimental label.

Updated the curl tool accordingly to use "--http3".

Closes #4197
2019-08-08 08:17:24 +02:00
Daniel Stenberg
f933449d3b
CURLINFO_RETRY_AFTER: parse the Retry-After header value
This is only the libcurl part that provides the information. There's no
user of the parsed value. This change includes three new tests for the
parser.

Ref: #3794
2019-08-07 14:45:39 +02:00
Daniel Stenberg
1f145b02d9
quiche: make use of the connection timeout API properly 2019-08-07 12:51:52 +02:00
Steve Holme
7c469fa537 sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID
Added the ability for the calling program to specify the authorisation
identity (authzid), the identity to act as, in addition to the
authentication identity (authcid) and password when using SASL PLAIN
authentication.

Fixes #3653
Closes #3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:20 -04:00
Daniel Stenberg
b1616dad8f
timediff: make it 64 bit (if possible) even with 32 bit time_t
... to make it hold microseconds too.

Fixes #4165
Closes #4168
2019-08-01 07:43:24 +02:00
Daniel Stenberg
a55faf33d4
cleanup: remove the 'numsocks' argument used in many places
It was used (intended) to pass in the size of the 'socks' array that is
also passed to these functions, but was rarely actually checked/used and
the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries
that should be used instead.

Closes #4169
2019-07-30 23:16:44 +02:00
Daniel Stenberg
3af0e76d1e
HTTP3: initial (experimental) support
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes #3500
2019-07-21 23:49:03 +02:00
Daniel Stenberg
094b5f3540
multi: make sure 'data' can present in several sockhash entries
Since more than one socket can be used by each transfer at a given time,
each sockhash entry how has its own hash table with transfers using that
socket.

In addition, the sockhash entry can now be marked 'blocked = TRUE'"
which then makes the delete function just set 'removed = TRUE' instead
of removing it "for real", as a way to not rip out the carpet under the
feet of a parent function that iterates over the transfers of that same
sockhash entry.

Reported-by: Tom van der Woerdt
Fixes #3961
Fixes #3986
Fixes #3995
Fixes #4004
Closes #3997
2019-06-10 00:47:48 +02:00
Frank Gevaerts
660603141c
ssl: Update outdated "openssl-only" comments for supported backends
These are for features that used to be openssl-only but were expanded
over time to support other SSL backends.

Closes #3985
2019-06-04 20:25:12 +01:00
Michael Kaufmann
2e5ceb3934
http: don't parse body-related headers bodyless responses
Responses with status codes 1xx, 204 or 304 don't have a response body. For
these, don't parse these headers:

- Content-Encoding
- Content-Length
- Content-Range
- Last-Modified
- Transfer-Encoding

This change ensures that HTTP/2 upgrades work even if a
"Content-Length: 0" or a "Transfer-Encoding: chunked" header is present.

Co-authored-by: Daniel Stenberg
Closes #3702
Fixes #3968
Closes #3977
2019-06-02 22:58:04 +02:00
Daniel Stenberg
8581e1928e
multi: track users of a socket better
They need to be removed from the socket hash linked list with more care.

When sh_delentry() is called to remove a sockethash entry, remove all
individual transfers from the list first. To enable this, each Curl_easy struct
now stores a pointer to the sockethash entry to know how to remove itself.

Reported-by: Tom van der Woerdt and Kunal Ekawde

Fixes #3952
Fixes #3904
Closes #3953
2019-05-28 12:17:02 +02:00
Jay Satiro
db8ec1fa38
Revert all SASL authzid (new feature) commits
- Revert all commits related to the SASL authzid feature since the next
  release will be a patch release, 7.65.1.

Prior to this change CURLOPT_SASL_AUTHZID  / --sasl-authzid was destined
for the next release, assuming it would be a feature release 7.66.0.
However instead the next release will be a patch release, 7.65.1 and
will not contain any new features.

After the patch release after the reverted commits can be restored by
using cherry-pick:

git cherry-pick a14d72c a9499ff 8c1cc36 c2a8d52 0edf690

Details for all reverted commits:

Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()."

This reverts commit 0edf6907ae.

Revert "tests: Fix the line endings for the SASL alt-auth tests"

This reverts commit c2a8d52a13.

Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples"

This reverts commit 8c1cc369d0.

Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool"

This reverts commit a9499ff136.

Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID"

This reverts commit a14d72ca2f.
2019-05-25 23:36:11 +02:00
Steve Holme
a14d72ca2f
sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID
Added the ability for the calling program to specify the authorisation
identity (authzid), the identity to act as, in addition to the
authentication identity (authcid) and password when using SASL PLAIN
authentication.

Fixed #3653
Closes #3790
2019-05-22 22:55:05 +01:00
Daniel Stenberg
e91e481612
libcurl: #ifdef away more code for disabled features/protocols 2019-05-17 23:24:34 +02:00
Steve Holme
e832d1ef74
http_negotiate: Move the Negotiate state out of the negotiatedata structure
Given that this member variable is not used by the SASL based protocols
there is no need to have it here.

Closes #3882
2019-05-15 00:32:42 +01:00
Steve Holme
85bef18ca1
http_ntlm: Move the NTLM state out of the ntlmdata structure
Given that this member variable is not used by the SASL based protocols
there is no need to have it here.
2019-05-15 00:31:45 +01:00
Steve Holme
0c73adfad3
url: Move the negotiate state type into a dedicated enum 2019-05-15 00:31:35 +01:00
Daniel Stenberg
0e2208ada6
wildcard: disable from build when FTP isn't present 2019-05-13 08:17:09 +02:00
Daniel Stenberg
7c70a3b1ad
http: CURL_DISABLE_HTTP_AUTH 2019-05-13 08:17:09 +02:00
Daniel Stenberg
ae4adae954
doh: CURL_DISABLE_DOH 2019-05-13 08:17:09 +02:00
Daniel Stenberg
d37b0936d1
ssh: define USE_SSH if SSH is enabled (any backend)
Closes #3846
2019-05-06 10:14:17 +02:00
Daniel Stenberg
5c8783d77f
process_pending_handles: mark queued transfers as previously pending
With transfers being queued up, we only move one at a a time back to the
CONNECT state but now we mark moved transfers so that when a moved
transfer is confirmed "successful" (it connected) it will trigger the
move of another pending transfer. Previously, it would otherwise wait
until the transfer was done before doing this. This makes queued up
pending transfers get processed (much) faster.
2019-05-01 22:51:23 +02:00
Daniel Stenberg
5fc28510a4
CURL_MAX_INPUT_LENGTH: largest acceptable string input size
This limits all accepted input strings passed to libcurl to be less than
CURL_MAX_INPUT_LENGTH (8000000) bytes, for these API calls:
curl_easy_setopt() and curl_url_set().

The 8000000 number is arbitrary picked and is meant to detect mistakes
or abuse, not to limit actual practical use cases. By limiting the
acceptable string lengths we also reduce the risk of integer overflows
all over.

NOTE: This does not apply to `CURLOPT_POSTFIELDS`.

Test 1559 verifies.

Closes #3805
2019-04-29 08:02:44 +02:00
Daniel Stenberg
e649432e72
CURLOPT_MAXAGE_CONN: set the maximum allowed age for conn reuse
... and disconnect too old ones instead of trying to reuse.

Default max age is set to 118 seconds.

Ref: #3722
Closes #3782
2019-04-21 23:06:23 +02:00
Daniel Stenberg
f73de32d46
CURLOPT_DNS_USE_GLOBAL_CACHE: remove
Remove the code too. The functionality has been disabled in code since
7.62.0. Setting this option will from now on simply be ignored and have
no function.

Closes #3654
2019-04-11 22:22:39 +02:00
Daniel Stenberg
2f44e94efb
pipelining: removed
As previously planned and documented in DEPRECATE.md, all pipelining
code is removed.

Closes #3651
2019-04-06 22:49:50 +02:00
Brad Spencer
80208d68dc
resolve: apply Happy Eyeballs philosophy to parallel c-ares queries
Closes #3699
2019-03-27 23:31:59 +01:00
Dominik Hölzl
6c60355323
Negotiate: fix for HTTP POST with Negotiate
* Adjusted unit tests 2056, 2057
* do not generally close connections with CURLAUTH_NEGOTIATE after every request
* moved negotiatedata from UrlState to connectdata
* Added stream rewind logic for CURLAUTH_NEGOTIATE
* introduced negotiatedata::GSS_AUTHDONE and negotiatedata::GSS_AUTHSUCC
* Consider authproblem state for CURLAUTH_NEGOTIATE
* Consider reuse_forbid for CURLAUTH_NEGOTIATE
* moved and adjusted negotiate authentication state handling from
  output_auth_headers into Curl_output_negotiate
* Curl_output_negotiate: ensure auth done is always set
* Curl_output_negotiate: Set auth done also if result code is
  GSS_S_CONTINUE_NEEDED/SEC_I_CONTINUE_NEEDED as this result code may
  also indicate the last challenge request (only works with disabled
  Expect: 100-continue and CURLOPT_KEEP_SENDING_ON_ERROR -> 1)
* Consider "Persistent-Auth" header, detect if not present;
  Reset/Cleanup negotiate after authentication if no persistent
  authentication
* apply changes introduced with #2546 for negotiate rewind logic

Fixes #1261
Closes #1975
2019-03-14 09:26:03 +01:00
Daniel Stenberg
e1be825453
alt-svc: the libcurl bits 2019-03-03 11:17:52 +01:00
Daniel Stenberg
65eb65fde6
urldata: simplify bytecounters
- no need to have them protocol specific

- no need to set pointers to them with the Curl_setup_transfer() call

- make Curl_setup_transfer() operate on a transfer pointer, not
  connection

- switch some counters from long to the more proper curl_off_t type

Closes #3627
2019-03-01 17:30:34 +01:00
Daniel Stenberg
4ff055530d
Curl_easy: remove req.maxfd - never used!
Introduced in 8b6314ccfb, but not used anymore in current code. Unclear
since when.

Closes #3626
2019-02-28 15:27:32 +01:00
Daniel Stenberg
62a2534e4c
urldata: convert bools to bitfields and move to end
This allows the compiler to pack and align the structs better in
memory. For a rather feature-complete build on x86_64 Linux, gcc 8.1.2
makes the Curl_easy struct 4.9% smaller. From 6312 bytes to 6000.

Removed an unused struct field.

No functionality changes.

Closes #3610
2019-02-27 08:37:56 +01:00
Daniel Stenberg
880cd5dd20
strerror: make the strerror function use local buffers
Instead of using a fixed 256 byte buffer in the connectdata struct.

In my build, this reduces the size of the connectdata struct by 11.8%,
from 2160 to 1904 bytes with no functionality or performance loss.

This also fixes a bug in schannel's Curl_verify_certificate where it
called Curl_sspi_strerror when it should have called Curl_strerror for
string from GetLastError. the only effect would have been no text or the
wrong text being shown for the error.

Co-authored-by: Jay Satiro

Closes #3612
2019-02-26 10:20:21 +01:00
Daniel Stenberg
b08898fb29
connection: never reuse CONNECT_ONLY conections
and make CONNECT_ONLY conections never reuse any existing ones either.

Reported-by: Pavel Löbl
Bug: https://curl.haxx.se/mail/lib-2019-02/0064.html
Closes #3586
2019-02-19 22:50:02 +01:00
Daniel Stenberg
28177def43
gssapi: fix deprecated header warnings
Heimdal includes on FreeBSD spewed out lots of them. Less so now.

Closes #3566
2019-02-14 08:38:43 +01:00
Daniel Stenberg
a6d134e17a
pretransfer: don't strlen() POSTFIELDS set for GET requests
... since that data won't be used in the request anyway.

Fixes #3548
Reported-by: Renaud Allard
Close #3549
2019-02-12 14:36:54 +01:00
georgeok
a730432e59
spnego_sspi: add support for channel binding
Attempt to add support for Secure Channel binding when negotiate
authentication is used. The problem to solve is that by default IIS
accepts channel binding and curl doesn't utilise them. The result was a
401 response. Scope affects only the Schannel(winssl)-SSPI combination.

Fixes https://github.com/curl/curl/issues/3503
Closes https://github.com/curl/curl/pull/3509
2019-02-01 09:56:27 +01:00
georgeok
0966233744 ntlm_sspi: add support for channel binding
Windows extended potection (aka ssl channel binding) is required
to login to ntlm IIS endpoint, otherwise the server returns 401
responses.

Fixes #3280
Closes #3321
2019-01-19 13:00:53 +01:00
Daniel Stenberg
ba243235ec
urldata: rename easy_conn to just conn
We use "conn" everywhere to be a pointer to the connection.

Introduces two functions that "attaches" and "detaches" the connection
to and from the transfer.

Going forward, we should favour using "data->conn" (since a transfer
always only has a single connection or none at all) to "conn->data"
(since a connection can have none, one or many transfers associated with
it and updating conn->data to be correct is error prone and a frequent
reason for internal issues).

Closes #3442
2019-01-11 15:35:13 +01:00
Daniel Stenberg
4c35574bb7
multi: multiplexing improvements
Fixes #3436
Closes #3448

 Problem 1

After LOTS of scratching my head, I eventually realized that even when doing
10 uploads in parallel, sometimes the socket callback to the application that
tells it what to wait for on the socket, looked like it would reflect the
status of just the single transfer that just changed state.

Digging into the code revealed that this was indeed the truth. When multiple
transfers are using the same connection, the application did not correctly get
the *combined* flags for all transfers which then could make it switch to READ
(only) when in fact most transfers wanted to get told when the socket was
WRITEABLE.

 Problem 1b

A separate but related regression had also been introduced by me when I
cleared connection/transfer association better a while ago, as now the logic
couldn't find the connection and see if that was marked as used by more
transfers and then it would also prematurely remove the socket from the socket
hash table even in times other transfers were still using it!

 Fix 1

Make sure that each socket stored in the socket hash has a "combined" action
field of what to ask the application to wait for, that is potentially the ORed
action of multiple parallel transfers. And remove that socket hash entry only
if there are no transfers left using it.

 Problem 2

The socket hash entry stored an association to a single transfer using that
socket - and when curl_multi_socket_action() was called to tell libcurl about
activities on that specific socket only that transfer was "handled".

This was WRONG, as a single socket/connection can be used by numerous parallel
transfers and not necessarily a single one.

 Fix 2

We now store a list of handles in the socket hashtable entry and when libcurl
is told there's traffic for a particular socket, it now iterates over all
known transfers using that single socket.
2019-01-10 09:49:09 +01:00
Claes Jakobsson
d8cae791f4 hostip: support wildcard hosts
This adds support for wildcard hosts in CURLOPT_RESOLVE. These are
try-last so any non-wildcard entry is resolved first. If specified,
any host not matched by another CURLOPT_RESOLVE config will use this
as fallback.

Example send a.com to 10.0.0.1 and everything else to 10.0.0.2:
  curl --resolve *:443:10.0.0.2 --resolve a.com:443:10.0.0.1 \
       https://a.com https://b.com

This is probably quite similar to using:
  --connect-to a.com:443:10.0.0.1:443 --connect-to :443:10.0.0.2:443

Closes #3406
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2018-12-27 14:23:53 +01:00
Daniel Stenberg
006ff62d8c
http: added options for allowing HTTP/0.9 responses
Added CURLOPT_HTTP09_ALLOWED and --http0.9 for this purpose.

For now, both the tool and library allow HTTP/0.9 by default.
docs/DEPRECATE.md lays out the plan for when to reverse that default: 6
months after the 7.64.0 release. The options are added already now so
that applications/scripts can start using them already now.

Fixes #2873
Closes #3383
2018-12-21 10:49:30 +01:00
Daniel Stenberg
d8a9de6203
pingpong: change default response timeout to 120 seconds
Previously it was 30 minutes
2018-12-17 12:33:18 +01:00
Ayoub Boudhar
f464535bfd
http: Implement trailing headers for chunked transfers
This adds the CURLOPT_TRAILERDATA and CURLOPT_TRAILERFUNCTION
options that allow a callback based approach to sending trailing headers
with chunked transfers.

The test server (sws) was updated to take into account the detection of the
end of transfer in the case of trailing headers presence.

Test 1591 checks that trailing headers can be sent using libcurl.

Closes #3350
2018-12-14 10:10:48 +01:00
Tobias Hintze
5b4cce2e36
host names: allow trailing dot in name resolve, then strip it
Delays stripping of trailing dots to after resolving the hostname.

Fixes #3022
Closes #3222
2018-11-22 13:40:51 +01:00