1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-17 15:05:02 -05:00
Commit Graph

866 Commits

Author SHA1 Message Date
Daniel Stenberg
0b0269341b
c-hyper: add support for transfer-encoding in the request
Closes #7348
2021-07-05 22:46:01 +02:00
Jay Satiro
ca8893468f http: fix crash in rate-limited upload
- Don't set the size of the piece of data to send to the rate limit if
  that limit is larger than the buffer size that will hold the piece.

Prior to this change if CURLOPT_MAX_SEND_SPEED_LARGE
(curl tool: --limit-rate) was set then it was possible that a temporary
buffer used for uploading could be written to out of bounds. A likely
scenario for this would be a non-trivial amount of post data combined
with a rate limit larger than CURLOPT_UPLOAD_BUFFERSIZE (default 64k).

The bug was introduced in 24e469f which is in releases since 7.76.0.

perl -e "print '0' x 200000" > tmp
curl --limit-rate 128k -d @tmp httpbin.org/post

Reported-by: Richard Marion

Fixes https://github.com/curl/curl/issues/7308
Closes https://github.com/curl/curl/pull/7315
2021-06-29 15:18:08 -04:00
Daniel Stenberg
a629506d42
http: make the haproxy support work with unix domain sockets
... it should then pass on "PROXY UNKNOWN" since it doesn't know the
involved IP addresses.

Reported-by: Valentín Gutiérrez
Fixes #7290
Closes #7291
2021-06-24 09:01:49 +02:00
Daniel Stenberg
a6da296867
vtls: only store TIMER_APPCONNECT for non-proxy connect
Introducing a 'isproxy' argument to the connect function so that it
knows wether to store the time stamp or not.

Reported-by: Yongkang Huang
Fixes #7274
Closes #7274
2021-06-19 23:02:16 +02:00
Daniel Stenberg
0c55fbab45
conn: add 'attach' to protocol handler, make libssh2 use it
The libssh2 backend has SSH session associated with the connection but
the callback context is the easy handle, so when a connection gets
attached to a transfer, the protocol handler now allows for a custom
function to get used to set things up correctly.

Reported-by: Michael O'Farrell
Fixes #6898
Closes #7078
2021-05-17 17:57:50 +02:00
Peng-Yu Chen
dbb88523ab
http: use calculated offsets inst of integer literals for header parsing
Assumed to be a minor coding style improvement with no behavior change.

A modern compiler is expected to have the calculation optimized during
compilation. It may be deemed okay even if that's not the case, since
the added overhead is considered very low.

Closes #7032
2021-05-08 17:00:32 +02:00
Daniel Stenberg
1763aceb0c
http: limit the initial send amount to used upload buffer size
Previously this logic would cap the send to CURL_MAX_WRITE_SIZE bytes,
but for the situations where a larger upload buffer has been set, this
function can benefit from sending more bytes. With default size used,
this does the same as before.

Also changed the storage of the size to an 'unsigned int' as it is not
allowed to be set larger than 2M.

Also added cautions to the man pages about changing buffer sizes in
run-time.

Closes #7022
2021-05-07 08:51:39 +02:00
Daniel Stenberg
04cc27460e
http: reset the header buffer when sending the request
A reused transfer handle could otherwise reuse the previous leftover
buffer and havoc would ensue.

Reported-by: sergio-nsk on github
Fixes #7018
Closes #7021
2021-05-06 23:28:45 +02:00
Harry Sintonen
a60294cf43
Curl_http_header: check for colon when matching Persistent-Auth
Closes #6993
2021-05-03 16:35:22 +02:00
Harry Sintonen
b75620b9a0
Curl_http_input_auth: require valid separator after negotiation type
Closes #6993
2021-05-03 16:35:14 +02:00
Daniel Stenberg
8b9de77cd2
http: fix the check for 'Authorization' with Bearer
The code would wrongly check for it using an additional colon.

Reported-by: Blake Burkhart
Closes #6988
2021-05-02 12:08:20 +02:00
Daniel Stenberg
d71ff2b9db
hsts: enable by default
No longer considered experimental.

Closes #6700
2021-04-19 08:22:16 +02:00
Daniel Stenberg
b5726e5549
send_speed: simplify the checks for if a speed limit is set
... as we know the value cannot be set to negative: enforced by
setopt()
2021-03-27 12:38:28 +01:00
Daniel Stenberg
24e469f6d6
http: cap body data amount during send speed limiting
By making sure never to send off more than the allowed number of bytes
per second the speed limit logic is given more room to actually work.

Reported-by: Fabian Keil
Bug: https://curl.se/mail/lib-2021-03/0042.html
Closes #6797
2021-03-27 12:38:15 +01:00
Daniel Stenberg
95cbcec8f9
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.

Closes #6798
2021-03-26 23:19:20 +01:00
Daniel Stenberg
3bbf62b5a4
http: strip default port from URL sent to proxy
To make sure the Host: header and the URL provide the same authority
portion when sent to the proxy, strip the default port number from the
URL if one was provided.

Reported-by: Michael Brown
Fixes #6769
Closes #6778
2021-03-23 13:33:49 +01:00
Daniel Stenberg
6d176bee59
http: make 416 not fail with resume + CURLOPT_FAILONERRROR
When asked to resume a download, libcurl will convert that to HTTP logic
and if then the entire file is already transferred it will result in a
416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that
scenario, it should *not* lead to an error return.

Updated test 1156, added test 1273

Reported-by: Jonathan Watt
Fixes #6740
Closes #6753
2021-03-17 08:26:46 +01:00
Daniel Stenberg
4c0206f90b
http: remove superfluous NULL assign
Closes #6727
2021-03-11 22:45:23 +01:00
Daniel Stenberg
723c0e75fa
http2: remove conn->data use
... but instead use a private alternative that points to the "driving
transfer" from the connection. We set the "user data" associated with
the connection to be the connectdata struct, but when we drive transfers
the code still needs to know the pointer to the transfer. We can change
the user data to become the Curl_easy handle, but with older nghttp2
version we cannot dynamically update that pointer properly when
different transfers are used over the same connection.

Closes #6520
2021-02-15 16:33:53 +01:00
Daniel Stenberg
46620b9743
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545
2021-02-13 22:36:15 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Daniel Stenberg
115c9e27f5
ftp: add 'prefer_ascii' to the transfer state struct
... and make sure the code never updates 'set.prefer_ascii' as it breaks
handle reuse which should use the setting as the user specified it.

Added test 1569 to verify: it first makes an FTP transfer with ';type=A'
and then another without type on the same handle and the second should
then use binary. Previously, curl failed this.

Closes #6578
2021-02-09 14:06:28 +01:00
Jacob Hoffman-Andrews
246399a874
vtls: initial implementation of rustls backend
This adds a new TLS backend, rustls. It uses the C-to-rustls bindings
from https://github.com/abetterinternet/crustls.

Rustls is at https://github.com/ctz/rustls/.

There is still a fair bit to be done, like sending CloseNotify on
connection shutdown, respecting CAPATH, and properly indicating features
like "supports TLS 1.3 ciphersuites." But it works well enough to make
requests and receive responses.

Blog post for context:
https://www.abetterinternet.org/post/memory-safe-curl/

Closes #6350
2021-02-09 11:06:18 +01:00
Jay Satiro
cb2dc1ba89 lib: use int type for more port variables
This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553
2021-02-09 02:53:31 -05:00
Daniel Stenberg
d836d362ec
urldata: remove the 'rtspversion' field
from struct connectdata and the corresponding code in http.c that set
it. It was never used for anything!

Closes #6581
2021-02-08 22:39:34 +01:00
Jay Satiro
98577e4029 build: fix --disable-http-auth
Broken since 215db08 (precedes 7.75.0).

Reported-by: Benbuck Nason

Fixes https://github.com/curl/curl/issues/6567
2021-02-04 14:49:56 -05:00
Jay Satiro
145848e2d1 build: fix --disable-dateparse
Broken since 215db08 (precedes 7.75.0).

Bug: https://curl.se/mail/lib-2021-02/0008.html
Reported-by: Firefox OS
2021-02-04 14:13:00 -05:00
Daniel Stenberg
1dc8aa870e
hostip/proxy: remove conn->data use
Closes #6513
2021-01-26 10:04:47 +01:00
Patrick Monnerat
ecb13416e3
lib: remove conn->data uses
Closes #6499
2021-01-24 18:15:03 +01:00
Daniel Stenberg
e8710633ed
http: empty reply connection are not left intact
... so mark the connection as closed in this condition to prevent that
verbose message to wrongly appear.

Reported-by: Matt Holt
Bug: https://twitter.com/mholt6/status/1352130240265375744
Closes #6503
2021-01-21 16:39:35 +01:00
Daniel Stenberg
c977a6d0dc
chunk/encoding: remove conn->data references
... by anchoring more functions on Curl_easy instead of connectdata

Closes #6498
2021-01-21 13:19:58 +01:00
Fabian Keil
0015e06151
http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy
Added test 1613 to verify.

Closes #6490
2021-01-20 15:21:13 +01:00
Daniel Stenberg
219d9f8620
transfer: remove conn->data use
Closes #6486
2021-01-19 17:07:02 +01:00
Daniel Stenberg
ea6ce0afad
http: make providing Proxy-Connection header not cause duplicated headers
Fixes test 1180

Bug: https://curl.se/mail/lib-2021-01/0095.html
Reported-by: Fabian Keil
Closes #6472
2021-01-19 11:40:15 +01:00
Daniel Stenberg
a304051620
lib: more conn->data cleanups
Closes #6479
2021-01-19 09:14:51 +01:00
Patrick Monnerat
bbe3aa9f88
vtls: reduce conn->data use
Closes #6474
2021-01-19 09:14:40 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Fabian Keil
4f61fd87b2
misc: fix typos
Bug: https://curl.se/mail/lib-2021-01/0063.html
Closes #6434
2021-01-11 15:12:25 +01:00
Daniel Stenberg
8b2dec6ab7
http: make 'authneg' also work for Hyper
When doing a request with a request body expecting a 401/407 back, that
initial request is sent with a zero content-length. Test 177 and more.

Closes #6424
2021-01-10 14:51:33 +01:00
Daniel Stenberg
ab525c059e
http: have CURLOPT_FAILONERROR fail after all headers
... so that Retry-After and other meta-content can still be used.

Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.

Fixes #6408
Closes #6409
2021-01-06 15:14:40 +01:00
Daniel Stenberg
fce912139a
c-hyper: add timecondition to the request
Test 77-78

Closes #6391
2021-01-01 00:49:37 +01:00
Daniel Stenberg
aba01da639
failf: remove newline from formatting strings
... as failf adds one itself.

Also: add an assert() to failf() that triggers on a newline in the
format string!

Closes #6365
2020-12-25 23:42:16 +01:00
Matthias Gatto
eb69797766
http: Make the call to v4 signature
This patch allow to call the v4 signature introduce in previous commit

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 16:28:03 +01:00
Daniel Stenberg
58974d25d8
lib: introduce c-hyper for using Hyper
... as an alternative HTTP backend within libcurl.
2020-12-18 09:58:04 +01:00
Daniel Stenberg
a5bc272223
http: show the request as headers even when split-sending
When the initial request isn't possible to send in its entirety, the
remainder of request would be delivered to the debug callback as data
and would wrongly be counted internally as body-bytes sent.

Extended test 1295 to verify.

Closes #6328
2020-12-17 14:00:27 +01:00
Rikard Falkeborn
920f49a20b
infof/failf calls: fix format specifiers
Update a few format specifiers to match what is being printed.

Closes #6241
2020-11-24 13:18:41 +01:00
Daniel Stenberg
a95a6ce6b8
urldata: remove 'void *protop' and create the union 'p'
... to avoid the use of 'void *' for the protocol specific structs done
per transfer.

Closes #6238
2020-11-23 16:16:16 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Daniel Stenberg
d70a5b5a0f
sendf: move the verbose-check into Curl_debug
Saves us from having the same check done everywhere.

Closes #6159
2020-11-02 22:49:36 +01:00