1
0
mirror of https://github.com/moparisthebest/curl synced 2024-08-13 17:03:50 -04:00
Commit Graph

11712 Commits

Author SHA1 Message Date
Alessandro Ghedini
a0c5781cb6
quiche: don't use primary_ip / primary_port
Closes #6555
2021-02-01 16:12:45 +01:00
Dmitry Wagin
796ce293de
http: improve AWS HTTP v4 Signature auth
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.

Closes #6524
2021-01-30 23:48:22 +01:00
Daniel Stenberg
3c2210713e
hyper: fix CONNECT to set 'data' as userdata
Follow-up to 14e075d1a7
2021-01-30 17:51:47 +01:00
Layla
4fc5e7eda5
connect: fix compile errors in Curl_conninfo_local
.. for the `#else` (`!HAVE_GETSOCKNAME`) case

Fixes https://github.com/curl/curl/issues/6548
Closes #6549

Signed-off-by: Layla <layla@insightfulvr.com>
2021-01-29 23:30:50 +01:00
Michał Antoniak
1c1158a9dd
transfer: fix GCC 10 warning with flag '-Wint-in-bool-context'
... and return the error code from the Curl_mime_rewind call.

Closes #6537
2021-01-29 14:47:28 +01:00
Michał Antoniak
0cf5670c54
avoid warning: enum constant in boolean context 2021-01-29 14:47:26 +01:00
Daniel Stenberg
f6d13090d4
copyright: fix missing year (range) updates 2021-01-29 14:35:13 +01:00
Daniel Stenberg
60de76e2ad
openssl: lowercase the hostname before using it for SNI
... because it turns out several servers out there don't actually behave
correctly otherwise in spite of the fact that the SNI field is
specifically said to be case insensitive in RFC 6066 section 3.

Reported-by: David Earl
Fixes #6540
Closes #6543
2021-01-29 10:40:01 +01:00
Gisle Vanem
3611f6a41e build: fix djgpp builds
- Update build instructions in packages/DOS/README

- Extend 'VPATH' with 'vquic' and 'vssh'.

- Allow 'Makefile.dist' to build both 'lib' and 'src'.

- Allow using the Windows hosted djgpp cross compiler to build for MSDOS
  under Windows.

- 'USE_SSL' -> 'USE_OPENSSL'

- Added a 'link_EXE' macro. Etc, etc.

- Linking 'curl.exe' needs '$(CURLX_CFILES)' too.

- Do not pick-up '../lib/djgpp/*.o' files. Recompile locally.

- Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'.

- Remove 'djgpp-clean'

- Adapt to new C-ares directory structure

- Use conditional variable assignments

Clarify the 'conditional variable assignment' in 'common.dj'.

Closes https://github.com/curl/curl/pull/6382
2021-01-27 22:08:57 -05:00
Ikko Ashimine
73d743cfad
hyper: fix typo in c-hyper.c
settting -> setting

Closes #6538
2021-01-27 16:27:34 +01:00
Daniel Stenberg
9571802190
libssh2: fix CURL_LIBSSH2_DEBUG-enabled build
Follow-up to 2dcc940959

Reported-by: Gisle Vanem
Bug: 2dcc940959 (commitcomment-46420088)
2021-01-27 11:43:21 +01:00
Jay Satiro
44c9e18e5f asyn-thread: fix build for when getaddrinfo missing
This is a follow-up to 8315343 which several days ago moved the resolver
pointer into the async struct but did not update the code that uses it
when getaddrinfo is not present.

Closes https://github.com/curl/curl/pull/6536
2021-01-27 03:26:55 -05:00
Daniel Stenberg
62e0fe2156
urldata: move 'ints' to the end of 'connectdata'
To optimize storage slightly.

Closes #6534
2021-01-27 09:19:15 +01:00
Daniel Stenberg
e76b058f3c
urldata: store ip version in a single byte
Closes #6534
2021-01-27 09:19:12 +01:00
Daniel Stenberg
1c3def53c7
urldata: remove duplicate 'upkeep_interval_ms' from connectdata
... and rely only on the value already set in Curl_easy.

Closes #6534
2021-01-27 09:19:08 +01:00
Daniel Stenberg
d6a37c23a3
urldata: remove 'local_ip' from the connectdata struct
As the info is already stored in the transfer handle anyway, there's no
need to carry around a duplicate buffer for the life-time of the handle.

Closes #6534
2021-01-27 09:19:05 +01:00
Daniel Stenberg
764c6bd3bf
urldata: remove duplicate port number storage
... and use 'int' for ports. We don't use 'unsigned short' since -1 is
still often used internally to signify "unknown value" and 0 - 65535 are
all valid port numbers.

Closes #6534
2021-01-27 09:19:01 +01:00
Daniel Stenberg
642d78026f
urldata: remove the duplicate 'ip_addr_str' field
... as the numerical IP address is already stored and kept in 'primary_ip'.

Closes #6534
2021-01-27 09:18:51 +01:00
Daniel Stenberg
32766cb4db
select: convert Curl_select() to private static function
The old function should not be used anywhere anymore (the only remaining
gskit use has to be fixed to instead use Curl_poll or none at all).

The static function version is now called our_select() and is only built
if necessary.

Closes #6531
2021-01-27 09:05:31 +01:00
Daniel Stenberg
5ad377718d
Curl_chunker: shrink the struct
... by removing a field, converting the hex index into a byte and
rearranging the order. Cuts it down from 48 bytes to 32 on x86_64.

Closes #6527
2021-01-27 08:48:08 +01:00
Tatsuhiro Tsujikawa
d3ca0771d2
ngtcp2: Fix http3 upload stall
Closes #6521
2021-01-26 17:13:04 +01:00
Tatsuhiro Tsujikawa
a1338d4297
ngtcp2: Fix stack buffer overflow
Closes #6521
2021-01-26 17:12:47 +01:00
Daniel Stenberg
ad1c7154b3
warnless.h: remove the prototype for curlx_ultosi
Follow-up to 217552503f
2021-01-26 16:32:27 +01:00
Daniel Stenberg
217552503f
warnless: remove curlx_ultosi
... not used anywhere

Closes #6530
2021-01-26 16:30:29 +01:00
Patrick Monnerat
234638ea63
lib: remove conn->data uses
Closes #6515
2021-01-26 15:19:41 +01:00
Daniel Stenberg
7173fe98ba
pingpong: remove the 'conn' struct member
... as it's superfluous now when Curl_easy is passed in and we can
derive the connection from that instead and avoid the duplicate copy.

Closes #6525
2021-01-26 10:12:59 +01:00
Daniel Stenberg
1dc8aa870e
hostip/proxy: remove conn->data use
Closes #6513
2021-01-26 10:04:47 +01:00
Daniel Stenberg
560fc170ec
url: reduce conn->data references
... there are a few left but let's keep them to last

Closes #6512
2021-01-26 10:00:59 +01:00
Daniel Stenberg
2dcc940959
libssh2: move data from connection object to transfer object
Readdir data, filenames and attributes are strictly related to the
transfer and not the connection. This also reduces the total size of the
fixed connectdata struct.

Closes #6519
2021-01-25 17:07:57 +01:00
Patrick Monnerat
ecb13416e3
lib: remove conn->data uses
Closes #6499
2021-01-24 18:15:03 +01:00
Daniel Stenberg
14e075d1a7
hyper: remove the conn->data references
Closes #6508
2021-01-22 16:07:17 +01:00
Daniel Stenberg
8335c6417e
hostip: remove conn->data from resolver functions
This also moves the 'async' struct from the connectdata struct into the
Curl_easy struct, which seems like a better home for it.

Closes #6497
2021-01-22 08:25:09 +01:00
Jay Satiro
d3a3bdb5ef strerror: skip errnum >= 0 assertion on windows
On Windows an error number may be greater than INT_MAX and negative once
cast to int.

The assertion is checked only in debug builds.

Closes https://github.com/curl/curl/pull/6504
2021-01-22 01:47:04 -05:00
Daniel Stenberg
b97a60f5f4
doh: make Curl_doh_is_resolved survive a NULL pointer
... if Curl_doh() returned a NULL, this function gets called anyway as
in a asynch procedure. Then the doh struct pointer is NULL and signifies
an OOM situation.

Follow-up to 6246a1d8c6
2021-01-21 23:31:19 +01:00
Daniel Stenberg
5a19cb5a3c
wolfssh: remove conn->data references
... and repair recent build breakage

Closes #6507
2021-01-21 16:43:51 +01:00
Daniel Stenberg
e8710633ed
http: empty reply connection are not left intact
... so mark the connection as closed in this condition to prevent that
verbose message to wrongly appear.

Reported-by: Matt Holt
Bug: https://twitter.com/mholt6/status/1352130240265375744
Closes #6503
2021-01-21 16:39:35 +01:00
Daniel Stenberg
c977a6d0dc
chunk/encoding: remove conn->data references
... by anchoring more functions on Curl_easy instead of connectdata

Closes #6498
2021-01-21 13:19:58 +01:00
Erik Olsson
0a5827571f lib: save a bit of space with some structure packing
- Reorder some internal struct members so that less padding is used.

This is an attempt at saving a bit of space by packing some structs
(using pahole to find the holes) where it might make sense to do
so without losing readability.

I.e., I tried to avoid separating fields that seem grouped
together (like the cwd... fields in struct ftp_conn for instance).
Also abstained from touching fields behind conditional macros as
that quickly can get complicated.

Closes https://github.com/curl/curl/pull/6483
2021-01-20 22:52:34 -05:00
Fabian Keil
0015e06151
http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy
Added test 1613 to verify.

Closes #6490
2021-01-20 15:21:13 +01:00
Daniel Stenberg
ba4808d53c
ftp: remove conn->data leftover 2021-01-20 14:59:17 +01:00
Daniel Stenberg
56d1891db6
curl_range: remove conn->data
Closes #6496
2021-01-20 14:59:11 +01:00
Daniel Stenberg
fb4c5d55b5
quiche: fix unused parameter ‘conn’
Follow-up to 2bdec0b3
2021-01-20 09:17:19 +01:00
Daniel Stenberg
4907431903
transfer: fix ‘conn’ undeclared mistake for iconv build
Follow-up to 219d9f8620
2021-01-20 09:13:14 +01:00
Daniel Stenberg
6246a1d8c6
doh: allocate state struct on demand
... instead of having it static within the Curl_easy struct. This takes
away 1176 bytes (18%) from the Curl_easy struct that aren't used very
often and instead makes the code allocate it when needed.

Closes #6492
2021-01-20 09:02:24 +01:00
Daniel Stenberg
d0688dcbdf
socks: use the download buffer instead
The SOCKS code now uses the generic download buffer for temporary
storage during the connection procedure, instead of having its own
private 600 byte buffer that adds to the connectdata struct size. This
works fine because this point the buffer is allocated but is not use for
download yet since the connection hasn't completed.

This reduces the connection struct size by 22% on a 64bit arch!

The SOCKS buffer needs to be at least 600 bytes, and the download buffer
is guaranteed to never be smaller than 1000 bytes.

Closes #6491
2021-01-20 08:54:19 +01:00
Daniel Stenberg
942cf12c2f
urldata: make magic be the first struct field
By making the `magic` identifier the same size and at the same place
within the structs (easy, multi, share), libcurl will be able to more
reliably detect and safely error out if an application passes in the
wrong handle to APIs. Easier to detect and less likely to cause crashes
if done.

Such mixups can't be detected at compile-time due to them being
typedefed void pointers - unless `CURL_STRICTER` is defined.

Closes #6484
2021-01-20 08:51:01 +01:00
Daniel Stenberg
13bc1ea9bc
http_chunks: correct and clarify a comment on hexnumber length
... and also rename the define for max length.

Closes #6489
2021-01-19 23:43:10 +01:00
Daniel Stenberg
e71542a9d9
curl_path: remove conn->data use
Closes #6487
2021-01-19 17:08:49 +01:00
Daniel Stenberg
219d9f8620
transfer: remove conn->data use
Closes #6486
2021-01-19 17:07:02 +01:00
Daniel Stenberg
2bdec0b363
quic: remove conn->data use
Closes #6485
2021-01-19 16:56:35 +01:00
Daniel Stenberg
3acf6fe539
c-hyper: avoid duplicated Proxy-Connection headers 2021-01-19 11:41:23 +01:00
Daniel Stenberg
ea6ce0afad
http: make providing Proxy-Connection header not cause duplicated headers
Fixes test 1180

Bug: https://curl.se/mail/lib-2021-01/0095.html
Reported-by: Fabian Keil
Closes #6472
2021-01-19 11:40:15 +01:00
Daniel Stenberg
7542ec5b32
lib/Makefile.inc: convert to listing each file on its own line
... to make it diff friendlier and easier to read.

Closes #6448
2021-01-19 10:03:07 +01:00
Daniel Stenberg
f86fcd0080
ftplistparser: remove use of conn->data
Closes #6482
2021-01-19 09:22:38 +01:00
Daniel Stenberg
a304051620
lib: more conn->data cleanups
Closes #6479
2021-01-19 09:14:51 +01:00
Patrick Monnerat
bbe3aa9f88
vtls: reduce conn->data use
Closes #6474
2021-01-19 09:14:40 +01:00
Daniel Stenberg
8399d89360
hyper: deliver data to application with Curl_client_write
... just as the native code path does. Avoids sending too large data
chunks in the callback and more.

Reported-by: Gisle Vanem
Fixes #6462
Closes #6473
2021-01-18 23:06:03 +01:00
Daniel Stenberg
f98f219c35
gopher: remove accidental conn->data leftover 2021-01-18 22:57:56 +01:00
Daniel Stenberg
a3efced012
libssh: avoid plain free() of libssh-memory
Since curl's own memory debugging system redefines free() calls to track
and fiddle with memory, it cannot be used on memory allocated by 3rd
party libraries.

Third party libraries SHOULD NOT require free() to release allocated
resources for this reason - and libs can use separate healp allocators
on some systems (like Windows) so free() doesn't necessarily work
anyway.

Filed as an issue with libssh: https://bugs.libssh.org/T268

Closes #6481
2021-01-18 22:50:56 +01:00
Daniel Stenberg
2bf77b3cba
send: assert that Curl_write_plain() has a ->conn when called
To help catch bad invokes.

Closes #6476
2021-01-18 16:12:20 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Razvan Cojocaru
13fe0b6ade cmake: expose CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG
This does for cmake builds what --disable-openssl-auto-load-config
does for autoconf builds.

Closes https://github.com/curl/curl/pull/6435
2021-01-16 02:18:12 -05:00
Gisle Vanem
cbf0446973
easyoptions: add the missing AWS_SIGV4
Follow-up from AWS_SIGV4
2021-01-15 13:13:32 +01:00
Daniel Stenberg
3d0b62fdbe
schannel_verify: fix safefree call typo
Follow-up from e87ad71d1b

Closes #6459
2021-01-15 10:16:17 +01:00
Daniel Stenberg
1123a0eb24
mime: make sure setting MIMEPOST to NULL resets properly
... so that a function can first use MIMEPOST and then set it to NULL to
reset it back to a blank POST.

Added test 584 to verify the fix.

Reported-by: Christoph M. Becker

Fixes #6455
Closes #6456
2021-01-15 08:33:27 +01:00
Daniel Stenberg
b68dc34af3
multi: set the PRETRANSFER time-stamp when we switch to PERFORM
... instead of at end of the DO state. This makes the timer more
accurate for the protocols that use the DOING state (such as FTP), and
simplifies how the function (now called init_perform) is called.

The timer will then include the entire procedure up to PERFORM -
including all instructions for getting the transfer started.

Closes #6454
2021-01-14 23:16:39 +01:00
Gisle Vanem
e87ad71d1b
schannel: plug a memory-leak
... when built without -DUNICODE.

Closes #6457
2021-01-14 22:59:56 +01:00
Daniel Stenberg
ab6eb2ec07
http_chunks: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
e38f4b7e80
transfer: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
8072a882b3
tftp: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
78af707caf
multi: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
a7591fbb73
ldap: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
476e1339ef
doh: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
afb150499f
asyn-ares: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
c2c9e44a4d
vtls: remove Curl_ prefix from static functions 2021-01-14 08:09:09 +01:00
Daniel Stenberg
7e72b9b547
bearssl: remove Curl_ prefix from static functions 2021-01-14 08:09:08 +01:00
Daniel Stenberg
4b55d0468a
mbedtls: remove Curl_ prefix from static functions 2021-01-14 08:09:08 +01:00
Daniel Stenberg
5beed3e742
wolfssl: remove Curl_ prefix from static functions 2021-01-14 08:09:08 +01:00
Daniel Stenberg
39df93a875
nss: remove Curl_ prefix from static functions 2021-01-14 08:09:08 +01:00
Daniel Stenberg
a3e1e6b3b4
gnutls: remove Curl_ prefix from static functions 2021-01-14 08:09:08 +01:00
Daniel Stenberg
3112ede57c
openssl: remove Curl_ prefix from static functions
... as we reserve this prefix to library-wide functions.

Closes #6443
2021-01-14 08:08:54 +01:00
Daniel Stenberg
a79555291e
nss: get the run-time version instead of build-time
Closes #6445
2021-01-13 23:22:07 +01:00
Fabian Keil
4f61fd87b2
misc: fix typos
Bug: https://curl.se/mail/lib-2021-01/0063.html
Closes #6434
2021-01-11 15:12:25 +01:00
Daniel Stenberg
06f1db57ca
multi_runsingle: bail out early on data->conn == NULL
As that's a significant error condition and scan-build warns for NULL
pointer dereferences if we don't.

Closes #6433
2021-01-11 12:33:23 +01:00
Daniel Stenberg
48220f7e3d
multi: skip DONE state if there's no connection left for ftp wildcard
... to avoid running in that state with data->conn being NULL.
2021-01-11 12:33:11 +01:00
Daniel Stenberg
057cb05b4b
libssh2: fix "Value stored to 'readdir_len' is never read"
Detected by scan-build
2021-01-11 12:32:58 +01:00
Daniel Stenberg
6cb803ecb1
connect: mark intentional ignores of setsockopt return values
Pointed out by Coverity

Closes #6431
2021-01-11 10:21:51 +01:00
Jay Satiro
70af179118 http_proxy: Fix CONNECT chunked encoding race condition
- During the end-of-headers response phase do not mark the tunnel
  complete unless the response body was completely parsed/ignored.

Prior to this change if the entirety of a CONNECT response with chunked
encoding was not received by the time the final header was parsed then
the connection would be marked done prematurely, before all the chunked
data could be read in and ignored (since this is what we do with any
CONNECT response body) and the connection could not be used.

Bug: https://curl.se/mail/lib-2021-01/0033.html
Reported-by: Fabian Keil

Closes https://github.com/curl/curl/pull/6432
2021-01-11 03:28:01 -05:00
Daniel Stenberg
229930762f
url: if IDNA conversion fails, fallback to Transitional
This improves IDNA2003 compatiblity.

Reported-by: Bubu on github
Fixes #6423
Closes #6428
2021-01-11 00:21:28 +01:00
Daniel Stenberg
8b2dec6ab7
http: make 'authneg' also work for Hyper
When doing a request with a request body expecting a 401/407 back, that
initial request is sent with a zero content-length. Test 177 and more.

Closes #6424
2021-01-10 14:51:33 +01:00
Daniel Stenberg
1e5cec3297
file: don't provide content-length for directories
... as it is misleading.

Ref #6379
Closes #6421
2021-01-07 23:31:28 +01:00
Daniel Stenberg
f52c6981c5
curl.h: add CURLPROTO_GOPHERS as own protocol identifier
Follow-up to a1f06f32b8, to make sure it can be handled separately
from plain gopher.

Closes #6418
2021-01-06 15:23:41 +01:00
Daniel Stenberg
ab525c059e
http: have CURLOPT_FAILONERROR fail after all headers
... so that Retry-After and other meta-content can still be used.

Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.

Fixes #6408
Closes #6409
2021-01-06 15:14:40 +01:00
Daniel Stenberg
9d7281ff17
global_init: debug builds allocates a byte in init
... to make build tools/valgrind warn if no curl_global_cleanup is
called.

This is conditionally only done for debug builds with the env variable
CURL_GLOBAL_INIT set.

Closes #6410
2021-01-06 15:13:45 +01:00
Daniel Stenberg
0936ecd0ee
pretransfer: setup the User-Agent header here
... and not in the connection setup, as for multiplexed transfers the
connection setup might be skipped and then the transfer would end up
without the set user-agent!

Reported-by: Flameborn on github
Assisted-by: Andrey Gursky
Assisted-by: Jay Satiro
Assisted-by: Mike Gelfand
Fixes #6312
Closes #6417
2021-01-05 23:46:32 +01:00
Daniel Stenberg
25c1d5e6a8
c-hyper: poll the tasks until end correctly
... makes test 36 work.

Closes #6412
2021-01-05 15:33:47 +01:00
Gergely Nagy
56b32b5d75
mk-ca-bundle.pl: deterministic output when using -t
Printing trust purposes are now sorted, making the output deterministic
when running on the same input certdata.txt.

Closes #6413
2021-01-05 13:03:41 +01:00
Himanshu Gupta
f927f38f1e
wolfssl: add SECURE_RENEGOTIATION support
Closes #6411
2021-01-05 10:05:46 +01:00
Daniel Stenberg
4eb908cd87
wolfssl: update copyright year range
Follow-up to 7de2e96535
2021-01-05 09:40:27 +01:00
Daniel Stenberg
4f95ce8dfe
c-hyper: make CURLE_GOT_NOTHING work
Test 30

Closes #6407
2021-01-05 09:38:25 +01:00
Daniel Stenberg
bcce220367
http_proxy: make CONNECT work with the Hyper backend
Makes test 80 run

Closes #6406
2021-01-05 09:37:08 +01:00
Jay Satiro
7de2e96535 wolfssl: Support wolfSSL builds missing TLS 1.1
The wolfSSL TLS library defines NO_OLD_TLS in some of their build
configurations and that causes the library to be built without TLS 1.1.
For example if MD5 is explicitly disabled when building wolfSSL then
that defines NO_OLD_TLS and the library is built without TLS 1.1 [1].

Prior to this change attempting to build curl with a wolfSSL that was
built with NO_OLD_TLS would cause a build link error undefined reference
to wolfTLSv1_client_method.

[1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366

Bug: https://curl.se/mail/lib-2020-12/0121.html
Reported-by: Julian Montes

Closes https://github.com/curl/curl/pull/6388
2021-01-04 18:00:37 -05:00
Daniel Stenberg
648712eec1
httpauth: make multi-request auth work with custom port
When doing HTTP authentication and a port number set with CURLOPT_PORT,
the code would previously have the URL's port number override as if it
had been a redirect to an absolute URL.

Added test 1568 to verify.

Reported-by: UrsusArctos on github
Fixes #6397
Closes #6400
2021-01-02 23:40:47 +01:00
Emil Engler
725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00
Daniel Stenberg
fce912139a
c-hyper: add timecondition to the request
Test 77-78

Closes #6391
2021-01-01 00:49:37 +01:00
Daniel Stenberg
3c7483efb4
c-hyper: make Digest and NTLM work
Test 64, 65, 67, 68, 69, 70, 72

Closes #6390
2020-12-30 22:40:29 +01:00
Daniel Stenberg
a6d20b89db
cleanup: fix two empty expression statement has no effect
Follow-up to 26e46617b9
2020-12-30 16:58:19 +01:00
Paul Groke
8324dc8b1a
dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries
Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix
(similar to the existing '-' prefix for removing entries) to add
DNS cache entries that will time out just like entries that are added
by libcurl itself.

Append " (non-permanent)" to info log message in case a non-permanent
entry is added.

Adjust relevant comments to reflect the new behavior.

Adjust documentation.

Extend unit1607 to test the new functionality.

Closes #6294
2020-12-29 17:49:36 +01:00
Daniel Stenberg
68dde8e330
schannel: fix "empty expression statement has no effect"
Bug: 8ab78f720a (commitcomment-45445950)
Reported-by: Gisle Vanem
Closes #6381
2020-12-29 17:10:47 +01:00
XhmikosR
aadc75626b
misc: assorted typo fixes
Closes #6375
2020-12-26 23:54:25 +01:00
Daniel Stenberg
8ab78f720a
misc: fix "warning: empty expression statement has no effect"
Turned several macros into do-while(0) style to allow their use to work
find with semicolon.

Bug: 08e8455ddd (commitcomment-45433279)
Follow-up to 08e8455ddd
Reported-by: Gisle Vanem
Closes #6376
2020-12-26 23:44:17 +01:00
Daniel Stenberg
aba01da639
failf: remove newline from formatting strings
... as failf adds one itself.

Also: add an assert() to failf() that triggers on a newline in the
format string!

Closes #6365
2020-12-25 23:42:16 +01:00
Daniel Stenberg
ac9a7245fc
speedcheck: exclude paused transfers
Paused transfers should not be stopped due to slow speed even when
CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is
now reset when the transfer is unpaused - as otherwise it would easily
just trigger immediately after unpausing.

Reported-by: Harry Sintonen
Fixes #6358
Closes #6359
2020-12-22 13:51:07 +01:00
Daniel Stenberg
c7f95fa0cc
h2: do not wait for RECV on paused transfers
... as the socket might be readable all the time when paused and thus
causing a busy-loop.

Reported-by: Harry Sintonen
Reviewed-by: Jay Satiro
Fixes #6356
Closes #6357
2020-12-22 12:44:18 +01:00
Matthias Gatto
eb69797766
http: Make the call to v4 signature
This patch allow to call the v4 signature introduce in previous commit

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 16:28:03 +01:00
Matthias Gatto
08e8455ddd
http: introduce AWS HTTP v4 Signature
It is a security process for HTTP.

It doesn't seems to be standard, but it is used by some cloud providers.

Aws:
https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
Outscale:
https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request
GCP (I didn't test that this code work with GCP though):
https://cloud.google.com/storage/docs/access-control/signing-urls-manually

most of the code is in lib/http_v4_signature.c

Information require by the algorithm:
- The URL
- Current time
-  some prefix that are append to some of the signature parameters.

The data extracted from the URL are: the URI, the region,
the host and the API type

example:
https://api.eu-west-2.outscale.com/api/latest/ReadNets
        ~~~ ~~~~~~~~               ~~~~~~~~~~~~~~~~~~~
        ^       ^                          ^
       /         \                        URI
   API type     region

Small description of the algorithm:
- make canonical header using content type, the host, and the date
- hash the post data
- make canonical_request using custom request, the URI,
  the get data, the canonical header, the signed header
  and post data hash
- hash canonical_request
- make str_to_sign using one of the prefix pass in parameter,
  the date, the credential scope and the canonical_request hash
- compute hmac from date, using secret key as key.
- compute hmac from region, using above hmac as key
- compute hmac from api_type, using above hmac as key
- compute hmac from request_type, using above hmac as key
- compute hmac from str_to_sign using above hmac as key
- create Authorization header using above hmac, prefix pass in parameter,
  the date, and above hash

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>

Closes #5703
2020-12-21 16:27:50 +01:00
Matthias Gatto
d52564bacb
http: add hmac support for sha256
It seems current hmac implementation use md5 for the hash,
V4 signature require sha256, so I've added the needed struct in
this commit.

I've added the functions that do the hmac in v4 signature file
as a static function ,in the next patch of the serie,
because it's used only by this file.

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 15:26:41 +01:00
Cristian Rodríguez
d13179db3e
connect: on linux, enable reporting of all ICMP errors on UDP sockets
The linux kernel does not report all ICMP errors back to userspace due
to historical reasons.

IP*_RECVERR sockopt must be turned on to have the correct behaviour
which is to pass all ICMP errors to userspace.

See https://bugzilla.kernel.org/show_bug.cgi?id=202355

Closes #6341
2020-12-21 15:24:08 +01:00
Daniel Stenberg
e1a4647a42
c-hyper: fix compiler warnings
Identified by clang on windows.

Reported-by: Gisle Vanem
Bug: 58974d25d8

Closes #6351
2020-12-20 23:06:46 +01:00
Daniel Stenberg
abb68c192a
build: repair http disabled but mqtt enabled build
... as the mqtt code reuses the "method" originally used for HTTP.

Closes #6344
2020-12-18 16:09:13 +01:00
Jon Wilkes
e2075f2337
cookie: avoid the C1001 internal compiler error with MSVC 14
Fixes #6112
Closes #6135
2020-12-18 14:02:27 +01:00
Daniel Stenberg
debf23eead
mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735

Added test 1916 and 1917 to verify.

Closes #6338
2020-12-18 12:54:05 +01:00
Daniel Stenberg
58974d25d8
lib: introduce c-hyper for using Hyper
... as an alternative HTTP backend within libcurl.
2020-12-18 09:58:04 +01:00
Daniel Stenberg
edc89839be
setopt: adjust to Hyper and disabled HTTP builds 2020-12-18 09:58:04 +01:00
Daniel Stenberg
4daba06762
rtsp: disable if Hyper is used 2020-12-18 09:58:04 +01:00
Daniel Stenberg
34021b804d
getinfo: build with disabled HTTP support 2020-12-18 09:58:03 +01:00
Daniel Stenberg
9211cb2034
version: include hyper version 2020-12-18 09:58:03 +01:00
Daniel Stenberg
a5bc272223
http: show the request as headers even when split-sending
When the initial request isn't possible to send in its entirety, the
remainder of request would be delivered to the debug callback as data
and would wrongly be counted internally as body-bytes sent.

Extended test 1295 to verify.

Closes #6328
2020-12-17 14:00:27 +01:00
Daniel Stenberg
3e17c8ab72
multi: when erroring in TOOFAST state, act as for PERFORM
When failing in TOOFAST, the multi_done() wasn't called so the same
cleanup and handling wasn't done like when it fails in PERFORM, which in
the case of FTP could mean that the control connection wouldn't be
marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused
ftp_disconnect() to use it to send "QUIT", which could end up waiting
for a response a long time before giving up!

Reported-by: Tomas Berger
Fixes #6333
Closes #6337
2020-12-17 13:57:27 +01:00
parazyd
a1f06f32b8
gopher: Implement secure gopher protocol.
This commit introduces a "gophers" handler inside the gopher protocol if
USE_SSL is defined. This protocol is no different than the usual gopher
prococol, with the added TLS encapsulation upon connecting. The protocol
has been adopted in the gopher community, and many people have enabled
TLS in their gopher daemons like geomyidae(8), and clients, like clic(1)
and hurl(1).

I have not implemented test units for this protocol because my knowledge
of Perl is sub-par. However, for someone more knowledgeable it might be
fairly trivial, because the same test that tests the plain gopher
protocol can be used for "gophers" just by adding a TLS listener.

Signed-off-by: parazyd <parazyd@dyne.org>

Closes #6208
2020-12-15 12:58:12 +01:00
Daniel Stenberg
a259eee99f
mqtt: deal with 0 byte reads correctly
OSS-Fuzz found it
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28676

Closes #6327
2020-12-15 11:38:29 +01:00
Daniel Stenberg
796c068895
quiche: remove fprintf() leftover 2020-12-14 11:19:48 +01:00
Jay Satiro
c54565915f digest_sspi: Show InitializeSecurityContext errors in verbose mode
The error is shown with infof rather than failf so that the user will
see the extended error message information only in verbose mode, and
will still see the standard CURLE_AUTH_ERROR message. For example:

---

* schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED
(0x8009030A) - The per-message Quality of Protection is not supported by
the security package
* multi_done
* Connection #1 to host 127.0.0.1 left intact
curl: (94) An authentication function returned an error

---

Ref: https://github.com/curl/curl/issues/6302

Closes https://github.com/curl/curl/pull/6315
2020-12-14 00:25:10 -05:00
Daniel Stenberg
769a4687df
tests/server/disabled: add "wakeup"
To allow the test suite to know if wakeup support is disabled in the
build.
2020-12-11 17:42:17 +01:00
Daniel Stenberg
32793b2c4f
ngtcp2: make it build it current master again
Closes #6296
2020-12-10 08:57:07 +01:00
Cristian Rodríguez
25b4e158e9
connect: defer port selection until connect() time
If supported, defer port selection until connect() time
if --interface is given and source port is 0.

Reproducer:

* start fast webserver on port 80
* starve system of ephemeral ports
$  sysctl net.ipv4.ip_local_port_range="60990 60999"

* start a curl/libcurl "crawler"
$curl --keepalive --parallel --parallel-immediate --head --interface
127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt"

current result:
(possible some successful data)
curl: (45) bind failed with errno 98: Address already in use

result after patch:
(complete success or few connections failing, higlhy depending on load)

Fail only when all the possible 4-tuple combinations are exhausted,
which is impossible to do when port is selected at bind() time becuse
the kernel does not know if socket will be listen()'ed on or connect'ed
yet.

Closes #6295
2020-12-10 08:55:42 +01:00
Hans-Christian Noren Egtvedt
8a10abaf85
connect: zero variable on stack to silence valgrind complaint
Valgrind will complain that ssrem buffer usage if not explicit
initialized, hence initialize it to zero.

This completes the change intially started in commit 2c0d721215 ('ftp:
retry getpeername for FTP with TCP_FASTOPEN') where the ssloc buffer has
a similar memset to zero.

Signed-off-by: Hans-Christian Noren Egtvedt <hegtvedt@cisco.com>
Closes #6289
2020-12-09 23:36:15 +01:00
Jacob Hoffman-Andrews
6d338a87d6 urldata: restore comment on ssl_connect_data.use
This comment was originally on the `use` field, but was separated from
its field in 62a2534.

Closes https://github.com/curl/curl/pull/6287
2020-12-07 18:45:47 -05:00
Daniel Stenberg
d9d0167278
openssl: make the OCSP verification verify the certificate id
CVE-2020-8286

Reported by anonymous

Bug: https://curl.se/docs/CVE-2020-8286.html
2020-12-07 09:27:10 +01:00
Daniel Stenberg
69a358f218
ftp: make wc_statemach loop instead of recurse
CVE-2020-8285

Fixes #6255
Bug: https://curl.se/docs/CVE-2020-8285.html
Reported-by: xnynx on github
2020-12-07 09:25:48 +01:00
Daniel Stenberg
ec9cc725d5
ftp: CURLOPT_FTP_SKIP_PASV_IP by default
The command line tool also independently sets --ftp-skip-pasv-ip by
default.

Ten test cases updated to adapt the modified --libcurl output.

Bug: https://curl.se/docs/CVE-2020-8284.html
CVE-2020-8284

Reported-by: Varnavas Papaioannou
2020-12-07 08:38:05 +01:00
Daniel Stenberg
abd846c374
urlapi: don't accept blank port number field without scheme
... as it makes the URL parser accept "very-long-hostname://" as a valid
host name and we don't want that. The parser now only accepts a blank
(no digits) after the colon if the URL starts with a scheme.

Reported-by: d4d on hackerone

Closes #6283
2020-12-07 00:50:49 +01:00
Daniel Stenberg
2260e0ebe6
Revert "multi: implement wait using winsock events"
This reverts commit d2a7d7c185.

This commit also reverts the subsequent follow-ups to that commit, which
were all done within windows #ifdefs that are removed in this
change. Marc helped me verify this.

Fixes #6146
Closes #6281
2020-12-06 22:40:38 +01:00
Klaus Crusius
2c0d721215
ftp: retry getpeername for FTP with TCP_FASTOPEN
In the case of TFO, the remote host name is not resolved at the
connetion time.

For FTP that has lead to missing hostname for the secondary connection.
Therefore the name resolution is done at the time, when FTP requires it.

Fixes #6252
Closes #6265
Closes #6282
2020-12-06 11:18:28 +01:00
Daniel Stenberg
1835cb916e
openssl: use OPENSSL_init_ssl() with >= 1.1.0
Reported-by: Kovalkov Dmitrii and Per Nilsson
Fixes #6254
Fixes #6256
Closes #6260
2020-12-03 22:30:38 +01:00
Daniel Stenberg
26f682bcc4
quiche: close the connection
Reported-by: Junho Choi
Fixes #6213
Closes #6217
2020-12-02 22:50:39 +01:00
Jay Satiro
2d1df660bc ngtcp2: Fix build error due to symbol name change
- NGTCP2_CRYPTO_LEVEL_APP -> NGTCP2_CRYPTO_LEVEL_APPLICATION

ngtcp2/ngtcp2@76232e9 changed the name.

ngtcp2 master is required to build curl with http3 support.

Closes https://github.com/curl/curl/pull/6271
2020-12-02 16:06:57 -05:00
Klaus Crusius
d6bfbfadd3
cmake: check for linux/tcp.h
The HAVE_LINUX_TCP_H define was not set by cmake.

Closes #6252
2020-12-01 12:32:55 +01:00
Daniel Stenberg
b6b535994e
splay: rename Curl_splayremovebyaddr to Curl_splayremove
... and remove the old unused proto for the old Curl_splayremove
version.

Closes #6269
2020-12-01 08:09:51 +01:00
Daniel Stenberg
2d4d012a49
openssl: free mem_buf in error path
To fix a memory-leak.

Closes #6267
2020-12-01 08:03:47 +01:00
Daniel Stenberg
0d75bf9ae9
openssl: remove #if 0 leftover
Follow-up to 4c9768565e (from Sep 2008)

Closes #6268
2020-11-30 19:59:12 +01:00
Daniel Stenberg
65d2f563fd
ntlm: avoid malloc(0) on zero length user and domain
... and simplify the too-long checks somewhat.

Detected by OSS-Fuzz

Closes #6264
2020-11-29 11:24:54 +01:00
Daiki Ueno
c7b02c5d68
ngtcp2: use the minimal version of QUIC supported by ngtcp2
Closes #6250
2020-11-26 23:31:56 +01:00
Daiki Ueno
ddd3eb99f8
ngtcp2: advertise h3 ALPN unconditionally
Closes #6250
2020-11-26 23:31:53 +01:00
Daiki Ueno
0cbd5d5c4f
vquic/ngtcp2.h: define local_addr as sockaddr_storage
This field needs to be wide enough to hold sockaddr_in6 when
connecting via IPv6.  Otherwise, ngtcp2_conn_read_pkt will drop the
packets because of the address mismatch:
  I00000022 [...] con ignore packet from unknown path

We can safely assume that struct sockaddr_storage is available, as it
is used in the public interface of ngtcp2.

Closes #6250
2020-11-26 23:31:42 +01:00
Daniel Stenberg
0b60d3685e
socks: check for DNS entries with the right port number
The resolve call is done with the right port number, but the subsequent
check used the wrong one, which then could find a previous resolve which
would return and leave the fresh resolve "incomplete" and leaking
memory.

Fixes #6247
Closes #6253
2020-11-26 22:29:34 +01:00
Daniel Stenberg
d6ced230fe curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use
... so don't define it when instructed to use c-ares!
2020-11-26 17:26:59 +01:00
Emil Engler
12cb7a1fe0
file: avoid duplicated code sequence
file_disconnect() is identical with file_do() except the function header
but as the arguments are unused anyway so why not just return file_do()
directly!

Reviewed-by: Daniel Stenberg
Closes #6249
2020-11-25 23:39:28 +01:00
Rikard Falkeborn
920f49a20b
infof/failf calls: fix format specifiers
Update a few format specifiers to match what is being printed.

Closes #6241
2020-11-24 13:18:41 +01:00
Daniel Stenberg
020aa0131b
docs/INTERNALS: remove reference to Curl_sendf()
The function has been removed from common usage. Also removed comment in
gopher.c that still referenced it.

Reported-by: Rikard Falkeborn
Fixes #6242
Closes #6243
2020-11-24 13:17:25 +01:00
Daniel Stenberg
37cdc2a05c
asyn: use 'struct thread_data *' instead of 'void *'
To reduce use of types that can't be checked at compile time. Also
removes several typecasts.

... and rename the struct field from 'os_specific' to 'tdata'.

Closes #6239
Reviewed-by: Jay Satiro
2020-11-23 22:54:18 +01:00
Viktor Szakats
3e092adf67
Makefile.m32: add support for UNICODE builds
It requires the linker to support the `-municode` option.
This is available in more recent mingw-w64 releases.

Ref: https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html
Ref: https://stackoverflow.com/questions/3571250/wwinmain-unicode-and-mingw/11706847#11706847

Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad

Closes #6228
2020-11-23 16:56:27 +00:00
Daniel Stenberg
a95a6ce6b8
urldata: remove 'void *protop' and create the union 'p'
... to avoid the use of 'void *' for the protocol specific structs done
per transfer.

Closes #6238
2020-11-23 16:16:16 +01:00
Oliver Urbann
0d16a49c16
curl: add compatibility for Amiga and GCC 6.5
Changes are mainly reordering and adding of includes required
to compile with a more recent version of GCC.

Closes #6220
2020-11-20 23:36:51 +01:00
Daniel Gustafsson
3a8cdc82dc openssl: guard against OOM on context creation
EVP_MD_CTX_create will allocate memory for the context and returns
NULL in case the allocation fails. Make sure to catch any allocation
failures and exit early if so.

In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex
as the latter is intended for ENGINE selection which we don't do.

Closes #6224
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
2020-11-19 01:40:24 +01:00
Vincent Torri
ee38a725b9
cmake: use libcurl.rc in all Windows builds
Reviewed-by: Marcel Raad
Closes #6215
2020-11-19 14:43:31 +01:00
Daniel Stenberg
73b63876b1
quiche: remove 'static' from local buffer
For thread-safety

Closes #6223
2020-11-19 13:34:44 +01:00
Tobias Hieta
760e854348
cmake: correctly handle linker flags for static libs
curl CMake was setting the the EXE flags for static libraries which made
the /manifest:no flag ended up when linking the static library, which is
not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit
with an error.

The better way to handle this is to make sure that we pass the correct
linker flags to CMAKE_STATIC_LINKER_FLAGS instead.

Reviewed-by: Jakub Zakrzewski
Closes #6195
2020-11-10 23:41:29 +01:00
Daniel Stenberg
c49d205ae0
http_proxy: use enum with state names for 'keepon'
To make the code clearer, change the 'keepon' from an int to an enum
with better state names.

Reported-by: Niranjan Hasabnis
Bug: https://curl.se/mail/lib-2020-11/0026.html
Closes #6193
2020-11-10 01:01:56 +01:00
Daniel Stenberg
7ae59838f0
curl_easy_escape: limit output string length to 3 * max input
... instead of the limiting it to just the max input size. As every
input byte can be expanded to 3 output bytes, this could limit the input
string to 2.66 MB instead of the intended 8 MB.

Reported-by: Marc Schlatter
Closes #6192
2020-11-09 23:01:06 +01:00
Daniel Stenberg
a570f7cd09
Curl_pgrsStartNow: init speed limit time stamps at start
By setting the speed limit time stamps unconditionally at transfer
start, we can start off a transfer without speed limits and yet allow
them to get set during transfer and have an effect.

Reported-by: Kael1117 on github
Fixes #6162
Closes #6184
2020-11-09 08:19:09 +01:00
Daniel Stenberg
5d57a79929
ngtcp2: adapt to recent nghttp3 updates
'reset_stream' was added to the nghttp3_conn_callbacks struct

Closes #6185
2020-11-09 08:17:38 +01:00
Daniel Stenberg
5d903ce3a3
altsvc: minimize variable scope and avoid "DEAD_STORE"
Closes #6182
2020-11-07 18:10:21 +01:00
Daniel Stenberg
20558ff269
url: make sure an HSTS upgrade updates URL and scheme correctly
Closes #6175
2020-11-06 08:22:31 +01:00
Daniel Stenberg
6ee57c8bdd
hsts: remove debug code leftovers
Closes #6175
2020-11-06 08:22:14 +01:00
Daniel Gustafsson
085f91036c hsts: Remove pointless call to free in errorpath
The line variable will always be NULL in the error path, so remove
the free call since it's pointless.

Closes #6170
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2020-11-05 09:42:38 +01:00
Daniel Stenberg
ac0a88fd25
copyright: fix year ranges
Follow-up from 4d2f800677
2020-11-05 08:22:10 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Harry Sintonen
8bdee98187
rtsp: error out on empty Session ID, unified the code 2020-11-04 08:14:36 +01:00
Harry Sintonen
adb0fcfab2
rtsp: fixed Session ID comparison to refuse prefix
Closes #6161
2020-11-04 08:14:01 +01:00
Daniel Stenberg
2cfc4ed983
hsts: add read/write callbacks
- read/write callback options
- man pages for the 4 new setopts
- test 1915 verifies the callbacks

Closes #5896
2020-11-03 16:08:48 +01:00
Daniel Stenberg
7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Daniel Stenberg
d70a5b5a0f
sendf: move the verbose-check into Curl_debug
Saves us from having the same check done everywhere.

Closes #6159
2020-11-02 22:49:36 +01:00
Harry Sintonen
8bcb23593e
gnutls: fix memory leaks (certfields memory wasn't released)
Closes #6153
2020-11-02 09:59:48 +01:00
Daniel Stenberg
584ccb5ef2
http: pass correct header size to debug callback for chunked post
... when the chunked framing was added, the size of the "body part" of
the data was calculated wrongly so the debug callback would get told a
header chunk a few bytes too big that would also contain the first few
bytes of the request body.

Reported-by: Dirk Wetter
Ref: #6144
Closes #6147
2020-10-31 23:46:27 +01:00
Daniel Stenberg
9a844625c4
libssh2: fix transport over HTTPS proxy
The fix in #6021 was not enough. This fix makes sure SCP/SFTP content
can also be transfered over a HTTPS proxy.

Fixes #6113
Closes #6128
2020-10-29 09:31:31 +01:00
Baruch Siach
ddcc110bfc
libssh2: fix build with disabled proxy support
Build breaks because the http_proxy field is missing:

vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy'

Regression from #6021, shipped in curl 7.73.0

Closes #6125
2020-10-26 09:18:35 +01:00
Daniel Stenberg
96450a1a33
alt-svc: enable by default
Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported.

alt-svc support in curl is no longer considered experimental

Closes #5868
2020-10-25 23:08:54 +01:00
Daniel Stenberg
4bfca0a807
libssh2: require version 1.0 or later
... and simplify the code accordingly. libssh2 version 1.0 was released
in April 2009.

Closes #6116
2020-10-22 16:45:40 +02:00
Daniel Stenberg
6dc563cdaa
CMake: store IDN2 information in curl_config.h
This allows the build to enable IDN properly and it makes test 1014
happier.

Ref: #6074
Closes #6108
2020-10-20 14:51:33 +02:00
Daniel Stenberg
a3d5b199f9
openssl: acknowledge SRP disabling in configure properly
Follow-up to 68a5132474

Use a new separate define that is the combination of both
HAVE_OPENSSL_SRP and USE_TLS_SRP: USE_OPENSSL_SRP

Bug: https://curl.haxx.se/mail/lib-2020-10/0037.html

Closes #6094
2020-10-16 14:29:02 +02:00
Viktor Szakats
e34959c0b9
http3: fix two build errors, silence warnings
* fix two build errors due to mismatch between function
  declarations and their definitions
* silence two mismatched signs warnings via casts

Approved-by: Daniel Stenberg
Closes #6093
2020-10-16 07:15:48 +00:00
Viktor Szakats
769c9a4eec
Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3
Approved-by: Daniel Stenberg
Closes #6092
2020-10-16 07:15:19 +00:00
Daniel Stenberg
475c1aba68
checksrc: warn on empty line before open brace
... and fix a few occurances

Closes #6088
2020-10-15 23:32:26 +02:00
Daniel Stenberg
b7ea3d2c22
urlapi: URL encode a '+' in the query part
... when asked to with CURLU_URLENCODE.

Extended test 1560 to verify.
Reported-by: Dietmar Hauser
Fixes #6086
Closes #6087
2020-10-15 23:21:53 +02:00
Philipp Klaus Krause
5168e5ad1d
strerror: use 'const' as the string should never be modified
Closes #6068
2020-10-15 15:43:32 +02:00
Jay Satiro
8b1a10cdf7
connect: repair build without ipv6 availability
Assisted-by: Daniel Stenberg
Reported-by: Tom G. Christensen

Fixes https://github.com/curl/curl/issues/6069
Closes https://github.com/curl/curl/pull/6071
2020-10-15 12:56:57 +02:00
Daniel Stenberg
15997f6db5
test122[12]: remove these two tests
... and remove the objnames scripts they tested. They're not used for
anything anymore so testing them serves no purpose!

Reported-by: Marc Hörsken
Fixes #6080
Closes #6081
2020-10-15 07:56:19 +02:00
Jay Satiro
a326877981
strerror: Revert to local codepage for Windows error string
- Change get_winapi_error() to return the error string in the local
  codepage instead of UTF-8 encoding.

Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it
also changed the error string's encoding from local codepage to UTF-8.

We return the local codepage version of the error string because if it
is output to the user's terminal it will likely be with functions which
expect the local codepage (eg fprintf, failf, infof).

This is essentially a partial revert of bed5f84. The support for xbox
remains but the error string is reverted back to local codepage.

Ref: https://github.com/curl/curl/pull/6005

Reviewed-by: Marcel Raad
Closes #6065
2020-10-13 14:17:27 +02:00
Viktor Szakats
d707a9fa64
windows: fix comparison of mismatched types warning
clang 10, mingw-w64:
```
vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long')
      [-Wsign-compare]
              if(GetLastError() != CRYPT_E_NOT_FOUND)
                 ~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~
```

Approved-by: Daniel Stenberg
Closes #6062
2020-10-12 10:18:53 +00:00
Daniel Stenberg
7d8c89d47b
checksrc: detect // comments on column 0
Spotted while working on #6045

Closes #6048
2020-10-07 10:14:50 +02:00
Frederik Wedel-Heinen
bc5455fa74
mbedtls: add missing header when defining MBEDTLS_DEBUG
Closes #6045
2020-10-07 09:15:39 +02:00
Daniel Stenberg
6497ed45bd
ldap: reduce the amount of #ifdefs needed
Closes #6035
2020-10-03 22:49:39 +02:00
Daniel Stenberg
26a7d51c21
checksrc: warn on space after exclamation mark
Closes #6034
2020-10-02 16:35:53 +02:00
Jay Satiro
9a13f7c2a7 strerror: fix null deref on winapi out-of-memory
Follow-up to bed5f84 from several days ago.

Ref: https://github.com/curl/curl/pull/6005
2020-10-01 13:34:59 -04:00
Kamil Dudka
7920be9473
vtls: deduplicate some DISABLE_PROXY ifdefs
... in the code of gtls, nss, and openssl

Closes #5735
2020-10-01 16:36:24 +02:00
Daniel Stenberg
c8204ed6a2
configure: don't say HTTPS-proxy is enabled when disabled!
Reported-by: Kamil Dudka
Reviewed-by: Kamil Dudka
Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388
Closes #6029
2020-09-30 22:43:43 +02:00
Daniel Gustafsson
2aac895fb6 src: Consistently spell whitespace without whitespace
Whitespace is spelled without a space between white and space, so
make sure to consistently spell it that way across the codebase.

Closes #6023
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
2020-09-30 21:10:14 +02:00
Daniel Stenberg
b312e3238f
sectransp: make it build with --disable-proxy
Follow-up from #5466 and f3d501dc67
Reported-by: Javier Navarro
Fixes #6025
Closes #6026
2020-09-30 10:32:17 +02:00
Daniel Stenberg
edfb6168e9
libssh2: handle the SSH protocols done over HTTPS proxy
Reported-by: Robin Douine
Fixes #4295
Closes #6021
2020-09-29 07:50:07 +02:00
Emil Engler
82d66f1582
memdebug: remove 9 year old unused debug function
There used to be a way to have memdebug fill allocated memory. 9 years
later this has no value there (valgrind and ASAN etc are way better). If
people need to know about it they can have a look at VCS logs.

Closes #5973
2020-09-28 22:15:21 +02:00
Daniel Stenberg
a87cca7b1c
sendf: move Curl_sendf to dict.c and make it static
... as the only remaining user of that function. Also fix gopher.c to
instead use Curl_write()

Closes #6020
2020-09-28 17:05:53 +02:00
Daniel Stenberg
abeeffb11c
schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root
This matches what is returned in other TLS backends in the same
situation.

Reviewed-by: Jay Satiro
Reviewed-by: Emil Engler
Follow-up to 5a3efb1
Reported-by: iammrtau on github
Fixes #6003
Closes #6018
2020-09-28 10:41:51 +02:00
Daniel Stenberg
4a4c724599
ftp: make a 552 response return CURLE_REMOTE_DISK_FULL
Added test 348 to verify. Added a 'STOR' command to the test FTP
server to enable test 348. Documented the command in FILEFORMAT.md

Reported-by: Duncan Wilcox
Fixes #6016
Closes #6017
2020-09-26 13:37:15 +02:00
Daniel Stenberg
7772344e17
pause: only trigger a reread if the unpause sticks
As an unpause might itself get paused again and then triggering another
reread doesn't help.

Follow-up from e040146f22 (shipped since 7.69.1)

Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html
Patch-by: Kunal Chandarana
Fixes #5988
Closes #6013
2020-09-25 15:18:10 +02:00
Daniel Stenberg
3d60a22390
ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define
Closes #6012
2020-09-25 08:59:43 +02:00
Javier Blazquez
bed5f8454a
strerror: honor Unicode API choice on Windows
Closes #6005
2020-09-25 08:55:35 +02:00
Daniel Stenberg
c4693adc62
imap: make imap_send use dynbuf for the send buffer management
Reuses the buffer and thereby reduces number of mallocs over a transfer.

Closes #6010
2020-09-25 08:35:01 +02:00
Daniel Stenberg
92a9b88ebf
Curl_send: return error when pre_receive_plain can't malloc
... will probably trigger some false DEAD CODE positives on non-windows
code analyzers for the conditional code.

Closes #6011
2020-09-25 08:31:52 +02:00
Daniel Stenberg
1397a7de6e
ftp: separate FTPS from FTP over "HTTPS proxy"
When using HTTPS proxy, SSL is used but not in the view of the FTP
protocol handler itself so separate the connection's use of SSL from the
FTP control connection's sue.

Reported-by: Mingtao Yang
Fixes #5523
Closes #6006
2020-09-24 14:09:20 +02:00
Daniel Stenberg
675eeb1c94
pingpong: use a dynbuf for the *_pp_sendf() function
... reuses the same dynamic buffer instead of doing repeated malloc/free
cycles.

Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls
after this change in my test setup (132 => 125), curl 7.72.0 needed 140
calls for this.

Test case 103 makes 9 less allocations now (130). Down from 149 in
7.72.0.

Closes #6004
2020-09-23 15:14:09 +02:00
Daniel Stenberg
f74afa40f8
dynbuf: add Curl_dyn_vaddf
Closes #6004
2020-09-23 15:13:46 +02:00
Daniel Stenberg
7e8561e030
dynbuf: make *addf() not require extra mallocs
... by introducing a printf() function that appends directly into a
dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if
the buffer is already big enough it can just printf directly into it.

Since this less-malloc version requires tthe use of a library internal
printf function, we only provide this version when building libcurl and
not for the dynbuf code that is used when building the curl tool.

Closes #5998
2020-09-23 08:54:42 +02:00
Daniel Stenberg
0548ecaf6a
pingpong: remove a malloc per Curl_pp_vsendf call
This typically makes 7-9 fewer mallocs per FTP transfer.

Closes #5997
2020-09-22 23:13:26 +02:00
Daniel Stenberg
3d64031fa7
symbian: drop support
The OS is deprecated. I see no traces of anyone having actually built
curl for Symbian after 2012.

The public headers are unmodified.

Closes #5989
2020-09-22 15:14:12 +02:00
Daniel Stenberg
1ee289f7b5
curl_krb5.h: rename from krb5.h
Follow-up from f4873ebd0b

Turns out some older openssl installations go bananas otherwise.
Reported-by: Tom van der Woerdt
Fixes #5995
Closes #5996
2020-09-22 13:46:22 +02:00
Daniel Stenberg
e5803089a9
http_proxy: do not count proxy headers in the header bytecount
... as that counter is subsequently used to detect if nothing was
returned from the peer. This made curl return CURLE_OK when it should
have returned CURLE_GOT_NOTHING.

Fixes #5992
Reported-by: Tom van der Woerdt
Closes #5994
2020-09-22 10:54:34 +02:00
Daniel Stenberg
a8e08a87df
setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument
Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the
option is, yeah, not known. Clarified this in the setopt man page too.

Closes #5993
2020-09-22 09:04:13 +02:00
Daniel Stenberg
f4873ebd0b
krb5: merged security.c and krb specific FTP functions in here
These two files were always tightly connected and it was hard to
understand what went into which. This also allows us to make the
ftpsend() function static (moved from ftp.c).

Removed security.c
Renamed curl_sec.h to krb5.h

Closes #5987
2020-09-21 23:31:39 +02:00
Daniel Stenberg
6434a73984
Curl_handler: add 'family' to each protocol
Makes get_protocol_family() faster and it moves the knowledge about the
"families" to each protocol handler, where it belongs.

Closes #5986
2020-09-21 23:30:16 +02:00
Daniel Stenberg
2e645e21de
parsedate: tune the date to epoch conversion
By avoiding an unnecessary error check and the temp use of the tm
struct, the time2epoch conversion function gets a little bit faster.
When repeating test 517, the updated version is perhaps 1% faster (on
one particular build on one particular architecture).

Closes #5985
2020-09-21 16:46:41 +02:00
Daniel Stenberg
3d8731c8f5
ftp: avoid risk of reading uninitialized integers
If the received PASV response doesn't match the expected pattern, we
could end up reading uninitialized integers for IP address and port
number.

Issue pointed out by muse.dev
Closes #5972
2020-09-18 08:26:49 +02:00
Quentin Balland
7921b41ce1
easy_reset: clear retry counter
Closes #5975
Fixes #5974
2020-09-18 07:59:24 +02:00
Daniel Stenberg
cc372af1e9
ftp: get rid of the PPSENDF macro
The use of such a macro hides some of what's actually going on to the
reader and is generally disapproved of in the project.

Closes #5971
2020-09-18 07:56:16 +02:00
Emil Engler
c0f0e400e0
urlapi: use more Curl_safefree
Closes #5968
2020-09-17 09:44:36 +02:00
Marc Hoersken
1060955a61
multi: align WinSock mask variables in Curl_multi_wait
Also skip pre-checking sockets to set timeout_ms to 0
after the first socket has been detected to be ready.

Reviewed-by: rcombs on github
Reviewed-by: Daniel Stenberg

Follow up to #5886
2020-09-17 07:41:14 +02:00
Marc Hoersken
40d67da7ca
multi: reuse WinSock events variable in Curl_multi_wait
Since the struct is quite large (1 long and 10 ints) we
declare it once at the beginning of the function instead
of multiple times inside loops to avoid stack movements.

Reviewed-by: Viktor Szakats
Reviewed-by: Daniel Stenberg

Closes #5886
2020-09-17 07:40:49 +02:00
Gergely Nagy
182ff2d63c
vtls: deduplicate client certificates in ssl_config_data
Closes #5629
2020-09-14 12:56:47 +02:00
Daniel Stenberg
7ea2e1d0c5
ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND
This is primarily interesting for cases where CURLOPT_NOBODY is set as
previously curl would not return an error for this case.

MDTM getting 550 now also returns this error (it returned
CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for
missing files across protocols and specific FTP commands.

libcurl already returns error on a 550 as a MDTM response (when
CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would
happen subsequently anyway since the RETR command would fail.

Add test 1913 and 1914 to verify. Updated several tests accordingly due
to the updated SIZE behavior.

Reported-by: Tomas Berger
Fixes #5953
Closes #5957
2020-09-14 12:53:12 +02:00
Antarpreet Singh
da5ae38db0
imap: set cselect_bits to CURL_CSELECT_IN initially
... when continuing a transfer from a FETCH response.

When the size of the file was small enough that the entirety of the
transfer happens in a single go and schannel buffers holds the entire
data. However, it wasn't completely read in Curl_pp_readresp since a
line break was found before that could happen. So, by the time we are in
imap_state_fetch_resp - there's data in buffers that needs to be read
via Curl_read but nothing to read from the socket. After we setup a
transfer (Curl_setup_transfer), curl just waits on the socket state to
change - which doesn't happen since no new data ever comes.

Closes #5961
2020-09-14 12:29:44 +02:00
Daniel Stenberg
c4ea71ae32
dynbuf: provide curlx_ names for reuse by the curl tool
Closes #5946
2020-09-14 08:32:35 +02:00
Daniel Stenberg
9fffe925d2
dynbuf: make sure Curl_dyn_tail() zero terminates
Closes #5959
2020-09-14 08:29:46 +02:00
Laramie Leavitt
25a25f45ae
http: consolidate nghttp2_session_mem_recv() call paths
Previously there were several locations that called
nghttp2_session_mem_recv and handled responses slightly differently.
Those have been converted to call the existing
h2_process_pending_input() function.

Moved the end-of-session check to h2_process_pending_input() since the
only place the end-of-session state can change is after nghttp2
processes additional input frames.

This will likely fix the fuzzing error. While I don't have a root cause
the out-of-bounds read seems like a use after free, so moving the
nghttp2_session_check_request_allowed() call to a location with a
guaranteed nghttp2 session seems reasonable.

Also updated a few nghttp2 callsites to include error messages and added
a few additional error checks.

Closes #5648
2020-09-10 17:43:47 +02:00
Daniel Stenberg
11ab0ad60f
base64: also build for pop3 and imap
Follow-up to the fix in 20417a13fb

Reported-by: Michael Olbrich
Fixes #5937
Closes #5948
2020-09-10 08:50:04 +02:00
Daniel Stenberg
20417a13fb
base64: enable in build with SMTP
The oauth2 support is used with SMTP and it uses base64 functions.

Reported-by: Michael Olbrich
Fixes #5937
Closes #5938
2020-09-09 09:23:06 +02:00
Daniel Stenberg
17fcdf6a31
lib: fix -Wassign-enum warnings
configure --enable-debug now enables -Wassign-enum with clang,
identifying several enum "abuses" also fixed.

Reported-by: Gisle Vanem
Bug: 879007f811 (commitcomment-42087553)

Closes #5929
2020-09-08 13:53:02 +02:00
Diven Qi
3532262edd
url: use blank credentials when using proxy w/o username and password
Fixes proxy regression brought in commit ad829b21ae (7.71.0)

Fixed #5911
Closes #5914
2020-09-08 00:48:09 +02:00
Daniel Stenberg
6d946ad9fe
openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification
If the error reason from the lib is
SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return
CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR.

This unifies the libcurl return code and makes libressl run test 313
(CRL testing) fine.

Closes #5934
2020-09-07 17:26:27 +02:00
Daniel Stenberg
f93455eb04
altsvc: clone setting in curl_easy_duphandle
The cache content is not duplicated, like other caches, but the setting
and specified file name are.

Test 1908 is extended to verify this somewhat. Since the duplicated
handle gets the same file name, the test unfortunately overwrites the
same file twice (with different contents) which makes it hard to check
automatically.

Closes #5923
2020-09-06 09:54:54 +02:00
Daniel Stenberg
bbf8cae44d
build: drop support for building with Watcom
These files are not maintained, they seem to have no users, Watcom
compilers look like not having users nor releases anymore.

Closes #5918
2020-09-05 21:51:37 +02:00