1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-05 09:05:04 -05:00
Commit Graph

808 Commits

Author SHA1 Message Date
Daniel Stenberg
e15e51384a
http: move header storage to Curl_easy from connectdata
Since the connection can be used by many independent requests (using
HTTP/2 or HTTP/3), things like user-agent and other transfer-specific
data MUST NOT be kept connection oriented as it could lead to requests
getting the wrong string for their requests. This struct data was
lingering like this due to old HTTP1 legacy thinking where it didn't
mattered..

Fixes #5566
Closes #5567
2020-06-15 22:56:25 +02:00
Daniel Stenberg
eab2f95c0d
wording: avoid blacklist/whitelist stereotypes
Instead of discussing if there's value or meaning (implied or not) in
the colors, let's use words without the same possibly negative
associations.

Closes #5546
2020-06-10 08:49:17 +02:00
Daniel Stenberg
9c845be279
urldata: let the HTTP method be in the set.* struct
When the method is updated inside libcurl we must still not change the
method as set by the user as then repeated transfers with that same
handle might not execute the same operation anymore!

This fixes the libcurl part of #5462

Test 1633 added to verify.

Closes #5499
2020-06-02 16:30:36 +02:00
Daniel Stenberg
f3d501dc67
build: disable more code/data when built without proxy support
Added build to travis to verify

Closes #5466
2020-05-30 23:18:16 +02:00
Daniel Stenberg
7414fb25a2
urldata: connect related booleans live in struct ConnectBits
And remove a few unused booleans!

Closes #5461
2020-05-27 08:02:47 +02:00
Daniel Stenberg
ed35d6590e
dynbuf: introduce internal generic dynamic buffer functions
A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to them. Existing
functionality has been ported over.

In my early basic testing, the total number of allocations seem at
roughly the same amount as before, possibly a few less.

See docs/DYNBUF.md for a description of the API.

Closes #5300
2020-05-04 10:40:39 +02:00
Daniel Stenberg
df28ed6116
http: free memory when Alt-Used header creation fails due to OOM
Reported-by: James Fuller
Fixes #5268
Closes #5269
2020-04-20 14:24:56 +02:00
Daniel Stenberg
0ef54abf52
http: don't consider upload done if the request isn't completely sent off
Fixes #4919
Closes #5197
2020-04-08 08:14:23 +02:00
Daniel Stenberg
3e376059bb
http: allow Curl_add_buffer_send() to do a short first send by force
In a debug build, settting the environment variable "CURL_SMALLREQSEND"
will make the first HTTP request send not send more bytes than the set
amount, thus ending up verifying that the logic for handling a split
HTTP request send works correctly.
2020-04-08 08:12:59 +02:00
Daniel Stenberg
0caae3c4be
http: mark POSTs with no body as "upload done" from the start
As we have logic that checks if we get a >= 400 reponse code back before
the upload is done, which then got confused since it wasn't "done" but
yet there was no data to send!

Reported-by: IvanoG on github
Fixes #4996
Closes #5002
2020-03-02 15:43:04 +01:00
Daniel Stenberg
6375b205a9
http: added 417 response treatment
When doing a request with a body + Expect: 100-continue and the server
responds with a 417, the same request will be retried immediately
without the Expect: header.

Added test 357 to verify.

Also added a control instruction to tell the sws test server to not read
the request body if Expect: is present, which the new test 357 uses.

Reported-by: bramus on github
Fixes #4949
Closes #4964
2020-02-26 22:48:14 +01:00
Pedro Monreal
4b6fd29f1a cleanup: fix typos and wording in docs and comments
Closes #4869
Reviewed-by: Emil Engler and Daniel Gustafsson
2020-02-02 18:43:01 +01:00
nao
dea17b519d
http: move "oauth_bearer" from connectdata to Curl_easy
Fixes the bug where oauth_bearer gets deallocated when we re-use a
connection.

Closes #4824
2020-01-21 10:32:43 +01:00
Marcel Raad
291ed52122
lib: fix compiler warnings with CURL_DISABLE_VERBOSE_STRINGS
Closes https://github.com/curl/curl/pull/4775
2020-01-03 17:02:30 +01:00
Daniel Stenberg
ee263de7a3
conncache: fix multi-thread use of shared connection cache
It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-by: Christopher Reid
Fixes #4544
Closes #4557
2019-12-09 15:30:09 +01:00
Daniel Stenberg
0044443a02
parsedate: offer a getdate_capped() alternative
... and use internally. This function will return TIME_T_MAX instead of
failure if the parsed data is found to be larger than what can be
represented. TIME_T_MAX being the largest value curl can represent.

Reviewed-by: Daniel Gustafsson
Reported-by: JanB on github
Fixes #4152
Closes #4651
2019-11-29 11:01:24 +01:00
Javier Blazquez
e0363a47de
ngtcp2: use overflow buffer for extra HTTP/3 data
Fixes #4525
Closes #4603
2019-11-18 16:40:08 +01:00
Daniel Stenberg
d0a7ee3f61
cookies: using a share with cookies shouldn't enable the cookie engine
The 'share object' only sets the storage area for cookies. The "cookie
engine" still needs to be enabled or activated using the normal cookie
options.

This caused the curl command line tool to accidentally use cookies
without having been told to, since curl switched to using shared cookies
in 7.66.0.

Test 1166 verifies

Updated test 506

Fixes #4429
Closes #4434
2019-09-28 18:10:43 +02:00
Daniel Stenberg
fe514ad9ae
http: fix warning on conversion from int to bit
Follow-up from 03ebe66d70
2019-09-21 23:09:59 +02:00
Daniel Stenberg
69ea985d4c
http: fix Expression 'http->postdata' is always false
Fixes warning detected by PVS-Studio
Fixes #4374
Reported-by: Valerii Zapodovnikov
2019-09-20 08:06:47 +02:00
Michael Kaufmann
2a2404153b http: merge two "case" statements 2019-09-18 13:45:41 +02:00
Daniel Stenberg
c4c9e070f3
Curl_fillreadbuffer: avoid double-free trailer buf on error
Reviewed-by: Jay Satiro
Reported-by: Thomas Vegas

Closes #4307
2019-09-09 09:36:27 +02:00
Daniel Stenberg
99775438d6
http: remove chunked-encoding and expect header use for HTTP/3 2019-08-20 16:53:40 +02:00
Daniel Stenberg
68fab35c73
http: fix use of credentials from URL when using HTTP proxy
When a username and password are provided in the URL, they were wrongly
removed from the stored URL so that subsequent uses of the same URL
wouldn't find the crendentials. This made doing HTTP auth with multiple
connections (like Digest) mishave.

Regression from 46e164069d (7.62.0)

Test case 335 added to verify.

Reported-by: Mike Crowe

Fixes #4228
Closes #4229
2019-08-16 10:11:22 +02:00
Daniel Stenberg
f2cc26456b
ngtcp2: do QUIC connections happy-eyeballs friendly 2019-08-14 23:14:12 +02:00
Daniel Stenberg
a93b43cde8
http3: fix the HTTP/3 in the request, make alt-svc set right versions
Closes #4200
2019-08-08 08:28:43 +02:00
Daniel Stenberg
98c3f14871
alt-svc: send Alt-Used: in redirected requests
RFC 7838 section 5:

   When using an alternative service, clients SHOULD include an Alt-Used
   header field in all requests.

Removed CURLALTSVC_ALTUSED again (feature is still EXPERIMENTAL thus
this is deemed ok).

You can disable sending this header just like you disable any other HTTP
header in libcurl.

Closes #4199
2019-08-08 08:24:37 +02:00
Daniel Stenberg
026840e35c
CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly
Even though it cannot fall-back to a lower HTTP version automatically. The
safer way to upgrade remains via CURLOPT_ALTSVC.

CURLOPT_H3 no longer has any bits that do anything and might be removed
before we remove the experimental label.

Updated the curl tool accordingly to use "--http3".

Closes #4197
2019-08-08 08:17:24 +02:00
Daniel Stenberg
f933449d3b
CURLINFO_RETRY_AFTER: parse the Retry-After header value
This is only the libcurl part that provides the information. There's no
user of the parsed value. This change includes three new tests for the
parser.

Ref: #3794
2019-08-07 14:45:39 +02:00
Daniel Stenberg
dc35631ef7
quiche: first working HTTP/3 request
- enable debug log
 - fix use of quiche API
 - use download buffer
 - separate header/body

Closes #4193
2019-08-06 07:39:21 +02:00
Daniel Stenberg
a55faf33d4
cleanup: remove the 'numsocks' argument used in many places
It was used (intended) to pass in the size of the 'socks' array that is
also passed to these functions, but was rarely actually checked/used and
the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries
that should be used instead.

Closes #4169
2019-07-30 23:16:44 +02:00
Balazs Kovacsics
802aa5ae27
HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown
If using the read callback for HTTP_POST, and POSTFIELDSIZE is not set,
automatically add a Transfer-Encoding: chunked header, same as it is
already done for HTTP_PUT, HTTP_POST_FORM and HTTP_POST_MIME.  Update
test 1514 according to the new behaviour.

Closes #4138
2019-07-29 13:18:53 +02:00
Daniel Stenberg
3af0e76d1e
HTTP3: initial (experimental) support
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes #3500
2019-07-21 23:49:03 +02:00
Peter Simonyi
855887af79
http: allow overriding timecond with custom header
With CURLOPT_TIMECONDITION set, a header is automatically added (e.g.
If-Modified-Since).  Allow this to be replaced or suppressed with
CURLOPT_HTTPHEADER.

Fixes #4103
Closes #4109
2019-07-14 16:17:15 +02:00
Daniel Gustafsson
11d7fe118d headers: Remove no longer exported functions
There were a leftover few prototypes of Curl_ functions that we used to
export but no longer do, this removes those prototypes and cleans up any
comments still referring to them.

Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free()
Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn()
were made static in 05b100aee2.
Curl_http_perhapsrewind() made static in 574aecee20.

For the remainder, I didn't trawl the Git logs hard enough to capture
their exact time of deletion, but they were all gone: Curl_splayprint(),
Curl_http2_send_request(), Curl_global_host_cache_dtor(),
Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(),
Curl_http_auth_stage() and Curl_close_connections().

Closes #4096
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2019-07-10 19:26:40 +02:00
Daniel Gustafsson
b96282010e http: clarify header buffer size calculation
The header buffer size calculation can from static analysis seem to
overlow as it performs an addition between two size_t variables and
stores the result in a size_t variable. Overflow is however guarded
against elsewhere since the input to the addition is regulated by
the maximum read buffer size. Clarify this with a comment since the
question was asked.

Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2019-06-24 23:35:06 +02:00
Daniel Stenberg
265f7f42f6
http2: call done_sending on end of upload
To make sure a HTTP/2 stream registers the end of stream.

Bug #4043 made me find this problem but this fix doesn't correct the
reported issue.

Closes #4068
2019-06-24 23:11:37 +02:00
Michael Kaufmann
2e5ceb3934
http: don't parse body-related headers bodyless responses
Responses with status codes 1xx, 204 or 304 don't have a response body. For
these, don't parse these headers:

- Content-Encoding
- Content-Length
- Content-Range
- Last-Modified
- Transfer-Encoding

This change ensures that HTTP/2 upgrades work even if a
"Content-Length: 0" or a "Transfer-Encoding: chunked" header is present.

Co-authored-by: Daniel Stenberg
Closes #3702
Fixes #3968
Closes #3977
2019-06-02 22:58:04 +02:00
Daniel Stenberg
5908009e31
http: fix "error: equality comparison with extraneous parentheses" 2019-05-29 08:17:08 +02:00
Daniel Stenberg
8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00
Steve Holme
fe20826b58
http_ntlm_wb: Move the type-2 message processing into a dedicated function
This brings the code inline with the other HTTP authentication mechanisms.

Closes #3890
2019-05-16 00:03:30 +01:00
Steve Holme
e832d1ef74
http_negotiate: Move the Negotiate state out of the negotiatedata structure
Given that this member variable is not used by the SASL based protocols
there is no need to have it here.

Closes #3882
2019-05-15 00:32:42 +01:00
Steve Holme
85bef18ca1
http_ntlm: Move the NTLM state out of the ntlmdata structure
Given that this member variable is not used by the SASL based protocols
there is no need to have it here.
2019-05-15 00:31:45 +01:00
Daniel Stenberg
6d32b72822
proxy: acknowledge DISABLE_PROXY more 2019-05-13 08:17:10 +02:00
Daniel Stenberg
b084616210
parsedate: CURL_DISABLE_PARSEDATE 2019-05-13 08:17:10 +02:00
Daniel Stenberg
b95456f4e2
mime: acknowledge CURL_DISABLE_MIME 2019-05-13 08:17:09 +02:00
Daniel Stenberg
7c70a3b1ad
http: CURL_DISABLE_HTTP_AUTH 2019-05-13 08:17:09 +02:00
Jay Satiro
b5901cab57 http: Ignore HTTP/2 prior knowledge setting for HTTP proxies
- Do not switch to HTTP/2 for an HTTP proxy that is not tunnelling to
  the destination host.

We already do something similar for HTTPS proxies by not sending h2. [1]

Prior to this change setting CURL_HTTP_VERSION_2_PRIOR_KNOWLEDGE would
incorrectly use HTTP/2 to talk to the proxy, which is not something we
support (yet?). Also it's debatable whether or not that setting should
apply to HTTP/2 proxies.

[1]: https://github.com/curl/curl/commit/17c5d05

Bug: https://github.com/curl/curl/issues/3570
Bug: https://github.com/curl/curl/issues/3832

Closes https://github.com/curl/curl/pull/3853
2019-05-08 22:50:41 -04:00
Daniel Stenberg
c32248f17d
http: mark bundle as not for multiuse on < HTTP/2 response
Fixes #3813
Closes #3815
2019-05-01 22:51:23 +02:00
Daniel Stenberg
2f44e94efb
pipelining: removed
As previously planned and documented in DEPRECATE.md, all pipelining
code is removed.

Closes #3651
2019-04-06 22:49:50 +02:00