1
0
mirror of https://github.com/moparisthebest/curl synced 2024-12-23 16:48:49 -05:00
Commit Graph

11426 Commits

Author SHA1 Message Date
Daniel Stenberg
6cb803ecb1
connect: mark intentional ignores of setsockopt return values
Pointed out by Coverity

Closes #6431
2021-01-11 10:21:51 +01:00
Jay Satiro
70af179118 http_proxy: Fix CONNECT chunked encoding race condition
- During the end-of-headers response phase do not mark the tunnel
  complete unless the response body was completely parsed/ignored.

Prior to this change if the entirety of a CONNECT response with chunked
encoding was not received by the time the final header was parsed then
the connection would be marked done prematurely, before all the chunked
data could be read in and ignored (since this is what we do with any
CONNECT response body) and the connection could not be used.

Bug: https://curl.se/mail/lib-2021-01/0033.html
Reported-by: Fabian Keil

Closes https://github.com/curl/curl/pull/6432
2021-01-11 03:28:01 -05:00
Daniel Stenberg
229930762f
url: if IDNA conversion fails, fallback to Transitional
This improves IDNA2003 compatiblity.

Reported-by: Bubu on github
Fixes #6423
Closes #6428
2021-01-11 00:21:28 +01:00
Daniel Stenberg
8b2dec6ab7
http: make 'authneg' also work for Hyper
When doing a request with a request body expecting a 401/407 back, that
initial request is sent with a zero content-length. Test 177 and more.

Closes #6424
2021-01-10 14:51:33 +01:00
Daniel Stenberg
1e5cec3297
file: don't provide content-length for directories
... as it is misleading.

Ref #6379
Closes #6421
2021-01-07 23:31:28 +01:00
Daniel Stenberg
f52c6981c5
curl.h: add CURLPROTO_GOPHERS as own protocol identifier
Follow-up to a1f06f32b8, to make sure it can be handled separately
from plain gopher.

Closes #6418
2021-01-06 15:23:41 +01:00
Daniel Stenberg
ab525c059e
http: have CURLOPT_FAILONERROR fail after all headers
... so that Retry-After and other meta-content can still be used.

Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.

Fixes #6408
Closes #6409
2021-01-06 15:14:40 +01:00
Daniel Stenberg
9d7281ff17
global_init: debug builds allocates a byte in init
... to make build tools/valgrind warn if no curl_global_cleanup is
called.

This is conditionally only done for debug builds with the env variable
CURL_GLOBAL_INIT set.

Closes #6410
2021-01-06 15:13:45 +01:00
Daniel Stenberg
0936ecd0ee
pretransfer: setup the User-Agent header here
... and not in the connection setup, as for multiplexed transfers the
connection setup might be skipped and then the transfer would end up
without the set user-agent!

Reported-by: Flameborn on github
Assisted-by: Andrey Gursky
Assisted-by: Jay Satiro
Assisted-by: Mike Gelfand
Fixes #6312
Closes #6417
2021-01-05 23:46:32 +01:00
Daniel Stenberg
25c1d5e6a8
c-hyper: poll the tasks until end correctly
... makes test 36 work.

Closes #6412
2021-01-05 15:33:47 +01:00
Gergely Nagy
56b32b5d75
mk-ca-bundle.pl: deterministic output when using -t
Printing trust purposes are now sorted, making the output deterministic
when running on the same input certdata.txt.

Closes #6413
2021-01-05 13:03:41 +01:00
Himanshu Gupta
f927f38f1e
wolfssl: add SECURE_RENEGOTIATION support
Closes #6411
2021-01-05 10:05:46 +01:00
Daniel Stenberg
4eb908cd87
wolfssl: update copyright year range
Follow-up to 7de2e96535
2021-01-05 09:40:27 +01:00
Daniel Stenberg
4f95ce8dfe
c-hyper: make CURLE_GOT_NOTHING work
Test 30

Closes #6407
2021-01-05 09:38:25 +01:00
Daniel Stenberg
bcce220367
http_proxy: make CONNECT work with the Hyper backend
Makes test 80 run

Closes #6406
2021-01-05 09:37:08 +01:00
Jay Satiro
7de2e96535 wolfssl: Support wolfSSL builds missing TLS 1.1
The wolfSSL TLS library defines NO_OLD_TLS in some of their build
configurations and that causes the library to be built without TLS 1.1.
For example if MD5 is explicitly disabled when building wolfSSL then
that defines NO_OLD_TLS and the library is built without TLS 1.1 [1].

Prior to this change attempting to build curl with a wolfSSL that was
built with NO_OLD_TLS would cause a build link error undefined reference
to wolfTLSv1_client_method.

[1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366

Bug: https://curl.se/mail/lib-2020-12/0121.html
Reported-by: Julian Montes

Closes https://github.com/curl/curl/pull/6388
2021-01-04 18:00:37 -05:00
Daniel Stenberg
648712eec1
httpauth: make multi-request auth work with custom port
When doing HTTP authentication and a port number set with CURLOPT_PORT,
the code would previously have the URL's port number override as if it
had been a redirect to an absolute URL.

Added test 1568 to verify.

Reported-by: UrsusArctos on github
Fixes #6397
Closes #6400
2021-01-02 23:40:47 +01:00
Emil Engler
725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00
Daniel Stenberg
fce912139a
c-hyper: add timecondition to the request
Test 77-78

Closes #6391
2021-01-01 00:49:37 +01:00
Daniel Stenberg
3c7483efb4
c-hyper: make Digest and NTLM work
Test 64, 65, 67, 68, 69, 70, 72

Closes #6390
2020-12-30 22:40:29 +01:00
Daniel Stenberg
a6d20b89db
cleanup: fix two empty expression statement has no effect
Follow-up to 26e46617b9
2020-12-30 16:58:19 +01:00
Paul Groke
8324dc8b1a
dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries
Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix
(similar to the existing '-' prefix for removing entries) to add
DNS cache entries that will time out just like entries that are added
by libcurl itself.

Append " (non-permanent)" to info log message in case a non-permanent
entry is added.

Adjust relevant comments to reflect the new behavior.

Adjust documentation.

Extend unit1607 to test the new functionality.

Closes #6294
2020-12-29 17:49:36 +01:00
Daniel Stenberg
68dde8e330
schannel: fix "empty expression statement has no effect"
Bug: 8ab78f720a (commitcomment-45445950)
Reported-by: Gisle Vanem
Closes #6381
2020-12-29 17:10:47 +01:00
XhmikosR
aadc75626b
misc: assorted typo fixes
Closes #6375
2020-12-26 23:54:25 +01:00
Daniel Stenberg
8ab78f720a
misc: fix "warning: empty expression statement has no effect"
Turned several macros into do-while(0) style to allow their use to work
find with semicolon.

Bug: 08e8455ddd (commitcomment-45433279)
Follow-up to 08e8455ddd
Reported-by: Gisle Vanem
Closes #6376
2020-12-26 23:44:17 +01:00
Daniel Stenberg
aba01da639
failf: remove newline from formatting strings
... as failf adds one itself.

Also: add an assert() to failf() that triggers on a newline in the
format string!

Closes #6365
2020-12-25 23:42:16 +01:00
Daniel Stenberg
ac9a7245fc
speedcheck: exclude paused transfers
Paused transfers should not be stopped due to slow speed even when
CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is
now reset when the transfer is unpaused - as otherwise it would easily
just trigger immediately after unpausing.

Reported-by: Harry Sintonen
Fixes #6358
Closes #6359
2020-12-22 13:51:07 +01:00
Daniel Stenberg
c7f95fa0cc
h2: do not wait for RECV on paused transfers
... as the socket might be readable all the time when paused and thus
causing a busy-loop.

Reported-by: Harry Sintonen
Reviewed-by: Jay Satiro
Fixes #6356
Closes #6357
2020-12-22 12:44:18 +01:00
Matthias Gatto
eb69797766
http: Make the call to v4 signature
This patch allow to call the v4 signature introduce in previous commit

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 16:28:03 +01:00
Matthias Gatto
08e8455ddd
http: introduce AWS HTTP v4 Signature
It is a security process for HTTP.

It doesn't seems to be standard, but it is used by some cloud providers.

Aws:
https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
Outscale:
https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request
GCP (I didn't test that this code work with GCP though):
https://cloud.google.com/storage/docs/access-control/signing-urls-manually

most of the code is in lib/http_v4_signature.c

Information require by the algorithm:
- The URL
- Current time
-  some prefix that are append to some of the signature parameters.

The data extracted from the URL are: the URI, the region,
the host and the API type

example:
https://api.eu-west-2.outscale.com/api/latest/ReadNets
        ~~~ ~~~~~~~~               ~~~~~~~~~~~~~~~~~~~
        ^       ^                          ^
       /         \                        URI
   API type     region

Small description of the algorithm:
- make canonical header using content type, the host, and the date
- hash the post data
- make canonical_request using custom request, the URI,
  the get data, the canonical header, the signed header
  and post data hash
- hash canonical_request
- make str_to_sign using one of the prefix pass in parameter,
  the date, the credential scope and the canonical_request hash
- compute hmac from date, using secret key as key.
- compute hmac from region, using above hmac as key
- compute hmac from api_type, using above hmac as key
- compute hmac from request_type, using above hmac as key
- compute hmac from str_to_sign using above hmac as key
- create Authorization header using above hmac, prefix pass in parameter,
  the date, and above hash

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>

Closes #5703
2020-12-21 16:27:50 +01:00
Matthias Gatto
d52564bacb
http: add hmac support for sha256
It seems current hmac implementation use md5 for the hash,
V4 signature require sha256, so I've added the needed struct in
this commit.

I've added the functions that do the hmac in v4 signature file
as a static function ,in the next patch of the serie,
because it's used only by this file.

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 15:26:41 +01:00
Cristian Rodríguez
d13179db3e
connect: on linux, enable reporting of all ICMP errors on UDP sockets
The linux kernel does not report all ICMP errors back to userspace due
to historical reasons.

IP*_RECVERR sockopt must be turned on to have the correct behaviour
which is to pass all ICMP errors to userspace.

See https://bugzilla.kernel.org/show_bug.cgi?id=202355

Closes #6341
2020-12-21 15:24:08 +01:00
Daniel Stenberg
e1a4647a42
c-hyper: fix compiler warnings
Identified by clang on windows.

Reported-by: Gisle Vanem
Bug: 58974d25d8

Closes #6351
2020-12-20 23:06:46 +01:00
Daniel Stenberg
abb68c192a
build: repair http disabled but mqtt enabled build
... as the mqtt code reuses the "method" originally used for HTTP.

Closes #6344
2020-12-18 16:09:13 +01:00
Jon Wilkes
e2075f2337
cookie: avoid the C1001 internal compiler error with MSVC 14
Fixes #6112
Closes #6135
2020-12-18 14:02:27 +01:00
Daniel Stenberg
debf23eead
mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735

Added test 1916 and 1917 to verify.

Closes #6338
2020-12-18 12:54:05 +01:00
Daniel Stenberg
58974d25d8
lib: introduce c-hyper for using Hyper
... as an alternative HTTP backend within libcurl.
2020-12-18 09:58:04 +01:00
Daniel Stenberg
edc89839be
setopt: adjust to Hyper and disabled HTTP builds 2020-12-18 09:58:04 +01:00
Daniel Stenberg
4daba06762
rtsp: disable if Hyper is used 2020-12-18 09:58:04 +01:00
Daniel Stenberg
34021b804d
getinfo: build with disabled HTTP support 2020-12-18 09:58:03 +01:00
Daniel Stenberg
9211cb2034
version: include hyper version 2020-12-18 09:58:03 +01:00
Daniel Stenberg
a5bc272223
http: show the request as headers even when split-sending
When the initial request isn't possible to send in its entirety, the
remainder of request would be delivered to the debug callback as data
and would wrongly be counted internally as body-bytes sent.

Extended test 1295 to verify.

Closes #6328
2020-12-17 14:00:27 +01:00
Daniel Stenberg
3e17c8ab72
multi: when erroring in TOOFAST state, act as for PERFORM
When failing in TOOFAST, the multi_done() wasn't called so the same
cleanup and handling wasn't done like when it fails in PERFORM, which in
the case of FTP could mean that the control connection wouldn't be
marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused
ftp_disconnect() to use it to send "QUIT", which could end up waiting
for a response a long time before giving up!

Reported-by: Tomas Berger
Fixes #6333
Closes #6337
2020-12-17 13:57:27 +01:00
parazyd
a1f06f32b8
gopher: Implement secure gopher protocol.
This commit introduces a "gophers" handler inside the gopher protocol if
USE_SSL is defined. This protocol is no different than the usual gopher
prococol, with the added TLS encapsulation upon connecting. The protocol
has been adopted in the gopher community, and many people have enabled
TLS in their gopher daemons like geomyidae(8), and clients, like clic(1)
and hurl(1).

I have not implemented test units for this protocol because my knowledge
of Perl is sub-par. However, for someone more knowledgeable it might be
fairly trivial, because the same test that tests the plain gopher
protocol can be used for "gophers" just by adding a TLS listener.

Signed-off-by: parazyd <parazyd@dyne.org>

Closes #6208
2020-12-15 12:58:12 +01:00
Daniel Stenberg
a259eee99f
mqtt: deal with 0 byte reads correctly
OSS-Fuzz found it
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28676

Closes #6327
2020-12-15 11:38:29 +01:00
Daniel Stenberg
796c068895
quiche: remove fprintf() leftover 2020-12-14 11:19:48 +01:00
Jay Satiro
c54565915f digest_sspi: Show InitializeSecurityContext errors in verbose mode
The error is shown with infof rather than failf so that the user will
see the extended error message information only in verbose mode, and
will still see the standard CURLE_AUTH_ERROR message. For example:

---

* schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED
(0x8009030A) - The per-message Quality of Protection is not supported by
the security package
* multi_done
* Connection #1 to host 127.0.0.1 left intact
curl: (94) An authentication function returned an error

---

Ref: https://github.com/curl/curl/issues/6302

Closes https://github.com/curl/curl/pull/6315
2020-12-14 00:25:10 -05:00
Daniel Stenberg
769a4687df
tests/server/disabled: add "wakeup"
To allow the test suite to know if wakeup support is disabled in the
build.
2020-12-11 17:42:17 +01:00
Daniel Stenberg
32793b2c4f
ngtcp2: make it build it current master again
Closes #6296
2020-12-10 08:57:07 +01:00
Cristian Rodríguez
25b4e158e9
connect: defer port selection until connect() time
If supported, defer port selection until connect() time
if --interface is given and source port is 0.

Reproducer:

* start fast webserver on port 80
* starve system of ephemeral ports
$  sysctl net.ipv4.ip_local_port_range="60990 60999"

* start a curl/libcurl "crawler"
$curl --keepalive --parallel --parallel-immediate --head --interface
127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt"

current result:
(possible some successful data)
curl: (45) bind failed with errno 98: Address already in use

result after patch:
(complete success or few connections failing, higlhy depending on load)

Fail only when all the possible 4-tuple combinations are exhausted,
which is impossible to do when port is selected at bind() time becuse
the kernel does not know if socket will be listen()'ed on or connect'ed
yet.

Closes #6295
2020-12-10 08:55:42 +01:00