Commit Graph

757 Commits

Author SHA1 Message Date
Marc Schlatter dd8a19f8a0
http: send payload when (proxy) authentication is done
The check that prevents payload from sending in case of authentication
doesn't check properly if the authentication is done or not.

They're cases where the proxy respond "200 OK" before sending
authentication challenge. This change takes care of that.

Fixes #2431
Closes #3669
2019-03-13 10:58:52 +01:00
Daniel Stenberg 5345b04a43
alt-svc: add test 355 and 356 to verify with command line curl 2019-03-03 11:17:52 +01:00
Daniel Stenberg e1be825453
alt-svc: the libcurl bits 2019-03-03 11:17:52 +01:00
Daniel Stenberg 65eb65fde6
urldata: simplify bytecounters
- no need to have them protocol specific

- no need to set pointers to them with the Curl_setup_transfer() call

- make Curl_setup_transfer() operate on a transfer pointer, not
  connection

- switch some counters from long to the more proper curl_off_t type

Closes #3627
2019-03-01 17:30:34 +01:00
Daniel Stenberg f1d915ea49
http: set state.infilesize when sending formposts
Without it set, we would unwillingly triger the "HTTP error before end
of send, stop sending" condition even if the entire POST body had been
sent (since it wouldn't know the expected size) which would
unnecessarily log that message and close the connection when it didn't
have to.

Reported-by: Matt McClure
Bug: https://curl.haxx.se/mail/archive-2019-02/0023.html
Closes #3624
2019-02-28 15:24:03 +01:00
Daniel Stenberg 62a2534e4c
urldata: convert bools to bitfields and move to end
This allows the compiler to pack and align the structs better in
memory. For a rather feature-complete build on x86_64 Linux, gcc 8.1.2
makes the Curl_easy struct 4.9% smaller. From 6312 bytes to 6000.

Removed an unused struct field.

No functionality changes.

Closes #3610
2019-02-27 08:37:56 +01:00
Daniel Stenberg 942eb09e8a
http: make adding a blank header thread-safe
Previously the function would edit the provided header in-place when a
semicolon is used to signify an empty header. This made it impossible to
use the same set of custom headers in multiple threads simultaneously.

This approach now makes a local copy when it needs to edit the string.

Reported-by: d912e3 on github
Fixes #3578
Closes #3579
2019-02-19 10:18:47 +01:00
Daniel Stenberg 05b100aee2
cleanup: make local functions static
urlapi: turn three local-only functions into statics

conncache: make conncache_find_first_connection static

multi: make detach_connnection static

connect: make getaddressinfo static

curl_ntlm_core: make hmac_md5 static

http2: make two functions static

http: make http_setup_conn static

connect: make tcpnodelay static

tests: make UNITTEST a thing to mark functions with, so they can be static for
normal builds and non-static for unit test builds

... and mark Curl_shuffle_addr accordingly.

url: make up_free static

setopt: make vsetopt static

curl_endian: make write32_le static

rtsp: make rtsp_connisdead static

warnless: remove unused functions

memdebug: remove one unused function, made another static
2019-02-10 18:38:57 +01:00
Katsuhiko YOSHIDA 1f30dc886d
cookies: skip custom cookies when redirecting cross-site
Closes #3417
2019-01-09 15:18:08 +01:00
Daniel Stenberg ebe658c1e5
Revert "http_negotiate: do not close connection until negotiation is completed"
This reverts commit 07ebaf8378.

This also reopens PR #3275 which brought the change now reverted.

Fixes #3384
Closes #3439
2019-01-07 09:36:36 +01:00
Daniel Stenberg 006ff62d8c
http: added options for allowing HTTP/0.9 responses
Added CURLOPT_HTTP09_ALLOWED and --http0.9 for this purpose.

For now, both the tool and library allow HTTP/0.9 by default.
docs/DEPRECATE.md lays out the plan for when to reverse that default: 6
months after the 7.64.0 release. The options are added already now so
that applications/scripts can start using them already now.

Fixes #2873
Closes #3383
2018-12-21 10:49:30 +01:00
Ayoub Boudhar f464535bfd
http: Implement trailing headers for chunked transfers
This adds the CURLOPT_TRAILERDATA and CURLOPT_TRAILERFUNCTION
options that allow a callback based approach to sending trailing headers
with chunked transfers.

The test server (sws) was updated to take into account the detection of the
end of transfer in the case of trailing headers presence.

Test 1591 checks that trailing headers can be sent using libcurl.

Closes #3350
2018-12-14 10:10:48 +01:00
Daniel Gustafsson 7a09b52c98 cookies: leave secure cookies alone
Only allow secure origins to be able to write cookies with the
'secure' flag set. This reduces the risk of non-secure origins
to influence the state of secure origins. This implements IETF
Internet-Draft draft-ietf-httpbis-cookie-alone-01 which updates
RFC6265.

Closes #2956
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2018-12-13 09:57:58 +01:00
Jay Satiro 552f0205e6
http: fix HTTP auth to include query in URI
- Include query in the path passed to generate HTTP auth.

Recent changes to use the URL API internally (46e1640, 7.62.0)
inadvertently broke authentication URIs by omitting the query.

Fixes https://github.com/curl/curl/issues/3353
Closes #3356
2018-12-11 13:28:20 +01:00
Michael Kaufmann c8bf8cc1e4
http: don't set CURLINFO_CONDITION_UNMET for http status code 204
The http status code 204 (No Content) should not change the "condition
unmet" flag. Only the http status code 304 (Not Modified) should do
this.

Closes #359
2018-12-11 13:22:42 +01:00
Johannes Schindelin cbea2fd2c7
NTLM: force the connection to HTTP/1.1
Since v7.62.0, cURL tries to use HTTP/2 whenever the server announces
the capability. However, NTLM authentication only works with HTTP/1.1,
and will likely remain in that boat (for details, see
https://docs.microsoft.com/en-us/iis/get-started/whats-new-in-iis-10/http2-on-iis#when-is-http2-not-supported).

When we just found out that we want to use NTLM, and when the current
connection runs in HTTP/2 mode, let's force the connection to be closed
and to be re-opened using HTTP/1.1.

Fixes https://github.com/curl/curl/issues/3341.
Closes #3345

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2018-12-07 13:03:21 +01:00
Daniel Stenberg dcd6f81025
snprintf: renamed and we now only use msnprintf()
The function does not return the same value as snprintf() normally does,
so readers may be mislead into thinking the code works differently than
it actually does. A different function name makes this easier to detect.

Reported-by: Tomas Hoger
Assisted-by: Daniel Gustafsson
Fixes #3296
Closes #3297
2018-11-23 08:26:51 +01:00
Elia Tufarolo 07ebaf8378
http_negotiate: do not close connection until negotiation is completed
Fix HTTP POST using CURLAUTH_NEGOTIATE.

Closes #3275
2018-11-16 23:38:47 +01:00
Viktor Szakats ff9d7f4447 spelling fixes [ci skip]
as detected by codespell 1.14.0

Closes https://github.com/curl/curl/pull/3114
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
2018-10-08 19:37:40 +00:00
Daniel Stenberg 4058cf2a7f
http: fix memleak in rewind error path
If the rewind would fail, a strdup() would not get freed.

Detected by OSS-Fuzz

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10665
Closes #3044
2018-09-25 10:30:08 +02:00
Daniel Stenberg e407e79c29
http: add missing return code check
Detected by Coverity. CID 1439610.

Follow-up from 46e164069d

Closes #3034
2018-09-23 23:08:39 +02:00
Daniel Stenberg 46e164069d
url: use the URL API internally as well
... to make it a truly unified URL parser.

Closes #3017
2018-09-22 11:58:10 +02:00
Daniel Stenberg 55dbcb061d
http: made Curl_add_buffer functions take a pointer-pointer
... so that they can clear the original pointer on failure, which makes
the error-paths and their cleanups easier.

Closes #2992
2018-09-16 23:22:37 +02:00
Daniel Stenberg e6e9b006f7
upload: allocate upload buffer on-demand
Saves 16KB on the easy handle for operations that don't need that
buffer.

Part 1 of #2888
2018-08-18 15:48:08 +02:00
Daniel Stenberg 8440616f53
http: fix for tiny "HTTP/0.9" response
Deal with tiny "HTTP/0.9" (header-less) responses by checking the
status-line early, even before a full "HTTP/" is received to allow
detecting 0.9 properly.

Test 1266 and 1267 added to verify.

Fixes #2420
Closes #2872
2018-08-13 23:16:01 +02:00
Daniel Stenberg 2a278fd735
CURLINFO_SIZE_UPLOAD: fix missing counter update
Adds test 1522 for verification.

Reported-by: cjmsoregan
Fixes #2847
Closes #2864
2018-08-11 14:00:59 +02:00
Johannes Schindelin 6f5ef24f06
auth: pick Bearer authentication whenever a token is available
So far, the code tries to pick an authentication method only if
user/password credentials are available, which is not the case for
Bearer authentictation...

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes #2754
2018-07-24 10:11:49 +02:00
Johannes Schindelin df57b439f4
auth: only ever pick CURLAUTH_BEARER if we *have* a Bearer token
The Bearer authentication was added to cURL 7.61.0, but there is a
problem: if CURLAUTH_ANY is selected, and the server supports multiple
authentication methods including the Bearer method, we strongly prefer
that latter method (only CURLAUTH_NEGOTIATE beats it), and if the Bearer
authentication fails, we will never even try to attempt any other
method.

This is particularly unfortunate when we already know that we do not
have any Bearer token to work with.

Such a scenario happens e.g. when using Git to push to Visual Studio
Team Services (which supports Basic and Bearer authentication among
other methods) and specifying the Personal Access Token directly in the
URL (this aproach is frequently taken by automated builds).

Let's make sure that we have a Bearer token to work with before we
select the Bearer authentication among the available authentication
methods.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes #2754
2018-07-24 10:11:25 +02:00
Daniel Stenberg 7b9bc96c77
http2: several cleanups
- separate easy handle from connections better
- added asserts on a number of places
- added sanity check of pipelines for debug builds

Closes #2751
2018-07-20 22:58:42 +02:00
Daniel Stenberg 5ae22272d1
Curl_debug: remove dead printhost code
The struct field is never set (since 5e0d9aea3) so remove the use of it
and remove the connectdata pointer from the prototype.

Reported-by: Tejas
Bug: https://curl.haxx.se/mail/lib-2018-06/0054.html
Closes #2647
2018-06-12 16:00:27 +02:00
Marian Klymov c45360d463
cppcheck: fix warnings
- Get rid of variable that was generating false positive warning
(unitialized)

- Fix issues in tests

- Reduce scope of several variables all over

etc

Closes #2631
2018-06-11 11:14:48 +02:00
Rikard Falkeborn 8ea5d41fe0
strictness: correct {infof, failf} format specifiers
Closes #2623
2018-05-31 11:34:32 +02:00
Linus Lewandowski 239a7061f8
httpauth: add support for Bearer tokens
Closes #2102
2018-05-24 20:39:49 +02:00
Patrick Monnerat 09d16af49a http resume: skip body if http code 416 (range error) is ignored.
This avoids appending error data to already existing good data.

Test 92 is updated to match this change.
New test 1156 checks all combinations of --range/--resume, --fail,
Content-Range header and http status code 200/416.

Fixes #1163
Reported-By: Ithubg on github
Closes #2578
2018-05-22 13:23:02 +02:00
Daniel Stenberg cb529b713f
checksrc: make sure sizeof() is used *with* parentheses
... and unify the source code to adhere.

Closes #2563
2018-05-21 23:21:47 +02:00
Rikard Falkeborn eb49683e55
lib: Fix format specifiers 2018-05-14 09:42:27 +02:00
Daniel Stenberg 8c7b3737d2
http: restore buffer pointer when bad response-line is parsed
... leaving the k->str could lead to buffer over-reads later on.

CVE: CVE-2018-1000301
Assisted-by: Max Dymond

Detected by OSS-Fuzz.
Bug: https://curl.haxx.se/docs/adv_2018-b138.html
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7105
2018-05-14 07:40:31 +02:00
Daniel Stenberg e9d9d1af8a
http: don't set the "rewind" flag when not uploading anything
It triggers an assert.

Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8144
Closes #2546
2018-05-04 13:51:25 +02:00
Marcel Raad 0f31647cf7
lib: silence null-dereference warnings
In debug mode, MingGW-w64's GCC 7.3 issues null-dereference warnings
when dereferencing pointers after DEBUGASSERT-ing that they are not
NULL.
Fix this by removing the DEBUGASSERTs.

Suggested-by: Daniel Stenberg
Ref: https://github.com/curl/curl/pull/2463
2018-04-09 15:54:52 +02:00
Lauri Kasanen 4073cd83b2
cookies: when reading from a file, only remove_expired once
This drops the cookie load time for 8k cookies from 178ms to 15ms.

Closes #2441
2018-04-02 10:40:32 +02:00
Lawrence Matthews 6baeb6df35
CURLOPT_HAPROXYPROTOCOL: support the HAProxy PROXY protocol
Add --haproxy-protocol for the command line tool

Closes #2162
2018-03-17 11:50:06 +01:00
Daniel Stenberg 8123560d44
HTTP: allow "header;" to replace an internal header with a blank one
Reported-by: Michael Kaufmann
Fixes #2357
Closes #2362
2018-03-11 11:46:10 +01:00
Daniel Stenberg 03370fa5a0
http: fix the max header length detection logic
Previously, it would only check for max length if the existing alloc
buffer was to small to fit it, which often would make the header still
get used.

Reported-by: Guido Berhoerster
Bug: https://curl.haxx.se/mail/lib-2018-02/0056.html

Closes #2315
2018-02-16 22:51:07 +01:00
Björn Stenberg b46cfbc068
TODO fixed: Detect when called from within callbacks
Closes #2302
2018-02-15 09:36:03 +01:00
Daniel Stenberg ddd31dc5dd
time_t-fixes: remove typecasts to 'long' for info.filetime
They're now wrong.

Reported-by: Michael Kaufmann

Closes #2277
2018-02-01 07:50:59 +01:00
Daniel Stenberg af32cd3859
http: prevent custom Authorization headers in redirects
... unless CURLOPT_UNRESTRICTED_AUTH is set to allow them. This matches how
curl already handles Authorization headers created internally.

Note: this changes behavior slightly, for the sake of reducing mistakes.

Added test 317 and 318 to verify.

Reported-by: Craig de Stigter
Bug: https://curl.haxx.se/docs/adv_2018-b3bf.html
2018-01-22 10:00:00 +01:00
Daniel Stenberg f68e672715
HTTP: bail out on negative Content-Length: values
... and make the max filesize check trigger if the value is too big.

Updates test 178.

Reported-by: Brad Spencer
Fixes #2212
Closes #2223
2018-01-13 22:49:04 +01:00
Patrick Monnerat dbcced8e32 HTTP: support multiple Content-Encodings
This is implemented as an output streaming stack of unencoders, the last
calling the client write procedure.

New test 230 checks this feature.

Bug: https://github.com/curl/curl/pull/2002
Reported-By: Daniel Bankhead
2017-11-05 15:09:48 +01:00
Daniel Stenberg cda89c8b58 include: remove conncache.h inclusion from where its not needed 2017-11-01 10:06:32 +01:00
Daniel Stenberg 232dffcf24
RTSP: avoid integer overflow on funny RTSP response
... like a very large non-existing RTSP version number.

Added test 577 to verify.

Detected by OSS-fuzz.
Closes #1969
2017-10-09 00:41:48 +02:00