Commit Graph

25833 Commits

Author SHA1 Message Date
Daniel Stenberg c4e6968127
url: alloc the download buffer at transfer start
... and free it as soon as the transfer is done. It removes the extra
alloc when a new size is set with setopt() and reduces memory for unused
easy handles.

In addition: the closure_handle now doesn't use an allocated buffer at
all but the smallest supported size as a stack based one.

Closes #5472
2020-05-30 23:14:33 +02:00
Daniel Stenberg 842f73de58
timeouts: change millisecond timeouts to timediff_t from time_t
For millisecond timers we like timediff_t better. Also, time_t can be
unsigned so returning a negative value doesn't work then.

Closes #5479
2020-05-30 23:10:57 +02:00
Marc Hoersken fc55c723c4
select: add overflow checks for timeval conversions
Using time_t and suseconds_t if suseconds_t is available,
long on Windows (maybe others in the future) and int elsewhere.

Also handle case of ULONG_MAX being greater or equal to INFINITE.

Assisted-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Part of #5343
2020-05-30 10:21:23 +02:00
Marc Hoersken 4a8f459837
select: use timediff_t instead of time_t and int for timeout_ms
Make all functions in select.[ch] take timeout_ms as timediff_t
which should always be large enough and signed on all platforms
to take all possible timeout values and avoid type conversions.

Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg

Replaces #5107 and partially #5262
Related to #5240 and #5286
Closes #5343
2020-05-30 10:20:40 +02:00
Marc Hoersken 62314d6b71
unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode'
GCC 10 warns about this with warning: implicit conversion
  from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion]

Since 'expected_result' is not really of type 'CURLcode' and
it is not exposed in any way, we can just use 'SANITIZEcode'.

Reviewed-by: Daniel Stenberg
Reviewed-by: Marcel Raad

Closes #5476
2020-05-30 10:07:26 +02:00
Marc Hoersken c0d4faf8e5
tests/libtest: fix undefined reference to 'curlx_win32_fopen'
Since curl_setup.h now makes use of curlx_win32_fopen for Win32
builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined,
we need to include the relevant files for tests using fopen,
because the libtest sources are also including curl_setup.h

Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg

Follow up to #3784 (ffdddb45d9)
Closes #5475
2020-05-30 10:03:00 +02:00
Marc Hoersken bbaef1a042
appveyor: add non-debug plain autotools-based build
This should enable us to catch linking issues with the
testsuite early, like the one described/fixed in #5475.

Reviewed-by: Daniel Stenberg
Reviewed-by: Marcel Raad

Closes #5477
2020-05-30 09:46:04 +02:00
Daniel Stenberg 5ec3226e06
RELEASE-NOTES: synced 2020-05-29 23:59:44 +02:00
Daniel Stenberg d618986a1c
Revert "buildconf: use find -execdir"
This partially reverts commit c712009838.

Keep the ares_ files removed but bring back the older way to run find,
to make it work with busybox's find, as apparently that's being used.

Reported-by: Max Peal
Fixes #5483
Closes #5484
2020-05-29 17:37:24 +02:00
Daniel Stenberg 6531d089e5
server/sws: fix asan warning on use of uninitialized variable 2020-05-29 13:05:09 +02:00
Daniel Stenberg 733a39a907
libssh2: improved error output for wrong quote syntax
Reported-by: Werner Stolz

Closes #5474
2020-05-29 09:35:52 +02:00
Daniel Stenberg 32d4d6b969
mk-lib1521: generate code for testing BLOB options as well
Follow-up to cac5374298

Closes #5478
2020-05-29 09:22:50 +02:00
Daniel Stenberg 2110b996a6
configure: repair the check if argv can be written to
Due to bad escaping of the test code, the test wouldn't build and thus
result in a negative test result, which would lead to the unconditional
assumption that overwriting the arguments doesn't work and thus curl
would never hide credentials given in the command line, even when it
would otherwise be possible.

Regression from commit 2d4c2152c (7.60.0)

Reported-by: huzunhao on github
Fixes #5470
Closes #5471
2020-05-28 23:33:56 +02:00
Peter Wu 3a2ca60d6a CMake: rebuild Makefile.inc.cmake when Makefile.inc changes
Otherwise the build might fail due to missing source files, as
demonstrated by the recent keylog.c addition on an existing build dir.

Closes #5469
2020-05-28 10:46:07 +02:00
Daniel Stenberg 155551c446
urldata: fix comments: Curl_done() is called multi_done() now
... since 575e885db
2020-05-28 10:13:39 +02:00
Peter Wu 697f984ec9 ngtcp2: use common key log routine for better thread-safety
Tested with ngtcp2 built against the OpenSSL library. Additionally
tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC).

The TLS backend (independent of QUIC) may or may not already have opened
the keylog file before. Therefore Curl_tls_keylog_open is always called
to ensure the file is open.
2020-05-27 21:19:51 +02:00
Peter Wu 7be7c56be9 wolfssl: add SSLKEYLOGFILE support
Tested following the same curl and tshark commands as in commit
"vtls: Extract and simplify key log file handling from OpenSSL" using
WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with
`./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`.

Full support for this feature requires certain wolfSSL build options,
see "Availability note" in lib/vtls/wolfssl.c for details.

Closes #5327
2020-05-27 21:19:51 +02:00
Peter Wu 6011a986ca vtls: Extract and simplify key log file handling from OpenSSL
Create a set of routines for TLS key log file handling to enable reuse
with other TLS backends. Simplify the OpenSSL backend as follows:

 - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled.
 - Do not perform dynamic memory allocation when preparing a log entry.
   Unless the TLS specifications change we can suffice with a reasonable
   fixed-size buffer.
 - Simplify state tracking when SSL_CTX_set_keylog_callback is
   unavailable. My original sslkeylog.c code included this tracking in
   order to handle multiple calls to SSL_connect and detect new keys
   after renegotiation (via SSL_read/SSL_write). For curl however we can
   be sure that a single master secret eventually becomes available
   after SSL_connect, so a simple flag is sufficient. An alternative to
   the flag is examining SSL_state(), but this seems more complex and is
   not pursued. Capturing keys after server renegotiation was already
   unsupported in curl and remains unsupported.

Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f
(`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`)
against an OpenSSL 1.1.1f server configured with:

    # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2
    openssl s_server -www -tls1
    # Likewise, but fail the server handshake.
    openssl s_server -www -tls1 -Verify 2
    # TLS 1.3 test. No need to test the failing server handshake.
    openssl s_server -www -tls1_3

Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly
written using Wireshark. For the first and third case, expect four
matches per connection (decrypted Server Finished, Client Finished, HTTP
Request, HTTP Response). For the second case where the handshake fails,
expect a decrypted Server Finished only.

    tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \
        -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \
        -dtls.port==4433,http -ohttp.desegment_body:FALSE \
        -Y 'tls.handshake.verify_data or http'

A single connection can easily be identified via the `tcp.stream` field.
2020-05-27 21:19:51 +02:00
Daniel Stenberg d528d97563
FILEFORMAT: add more features that tests can depend on 2020-05-27 14:33:49 +02:00
Michael Kaufmann 3d44bfb08d
transfer: close connection after excess data has been read
For HTTP 1.x, it's a protocol error when the server sends more bytes
than announced. If this happens, don't reuse the connection, because the
start position of the next response is undefined.

Closes #5440
2020-05-27 08:16:35 +02:00
Estanislau Augé-Pujadas 18e63b1679
Revert "ssh: ignore timeouts during disconnect"
This reverts commit f31760e63b. Shipped in
curl 7.54.1.

Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html
Closes #5465
2020-05-27 08:13:13 +02:00
Daniel Stenberg 7414fb25a2
urldata: connect related booleans live in struct ConnectBits
And remove a few unused booleans!

Closes #5461
2020-05-27 08:02:47 +02:00
Daniel Stenberg 67d2802dea
hostip: on macOS avoid DoH when given a numerical IP address
When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that
numerical IP addresses still need to get "resolved" - but not with DoH.

Reported-by: Viktor Szakats
Fixes #5454
Closes #5459
2020-05-26 17:37:39 +02:00
Daniel Stenberg 96a822f6e2
ngtcp2: cleanup memory when failing to connect
Reported-by: Peter Wu
Fixes #5447 (the ngtcp2 side of it)
Closes #5451
2020-05-26 10:16:32 +02:00
Daniel Stenberg 3ff207f7e3
quiche: clean up memory properly when failing to connect
Addresses the quiche side of #5447
Reported-by: Peter Wu
Closes #5450
2020-05-26 10:15:18 +02:00
Daniel Stenberg d23cc224e6
cleanup: use a single space after equals sign in assignments 2020-05-26 08:28:38 +02:00
Daniel Stenberg ad829b21ae
url: accept "any length" credentials for proxy auth
They're only limited to the maximum string input restrictions, not to
256 bytes.

Added test 1178 to verify

Reported-by: Will Roberts
Fixes #5448
Closes #5449
2020-05-25 23:08:57 +02:00
Maksim Stsepanenka 96f52abf80
test1167: fixes in badsymbols.pl
Closes #5442
2020-05-25 23:07:46 +02:00
Daniel Stenberg d844f2b9ff
altsvc: fix parser for lines ending with CRLF
Fixed the alt-svc parser to treat a newline as end of line.

The unit tests in test 1654 were done without CRLF and thus didn't quite
match the real world. Now they use CRLF as well.

Reported-by: Peter Wu
Assisted-by: Peter Wu
Assisted-by: Jay Satiro
Fixes #5445
Closes #5446
2020-05-25 23:05:27 +02:00
Viktor Szakats 308c243db5
all: fix codespell errors
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/5452
2020-05-25 19:44:04 +00:00
Peter Wu 54d122287c ngtcp2: fix build with current ngtcp2 master implementing draft 28
Based on client.cc changes from ngtcp2. Tested with current git master,
ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380.

Fixes #5444
Closes #5443
2020-05-25 17:34:00 +02:00
Daniel Stenberg 82a33efb33
RELEASE-NOTES: synced
moved the new setopts up to a "change"
2020-05-25 09:40:25 +02:00
Daniel Stenberg b955250b78
RELEASE-NOTES: synced 2020-05-25 09:38:28 +02:00
Daniel Stenberg d75e6ce85a
copyright: updated year ranges out of sync
... and whitelisted a few more files in the the copyright.pl script.
2020-05-24 00:02:33 +02:00
Gilles Vollant 7f40633422
setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency
Closes #5431
2020-05-23 23:19:13 +02:00
Daniel Stenberg 006d2991bd
curl: remove -J "informational" written on stdout
curl would previously show "curl: Saved to filename 'name from header'"
if -J was used and a name was picked from the Content-Disposition
header. That output could interfer with other stdout output, such as -w.

This commit removes that output line.
Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html
Reported-by: Коваленко Анатолий Викторович
Closes #5435
2020-05-23 23:12:53 +02:00
Peter Wu 421171a8d4 travis: simplify quiche build instructions wrt boringssl
quiche builds boringssl as static library, reuse that instead of
building another shared library.

Closes #5438
2020-05-22 21:10:07 +02:00
Peter Wu 06e266e0a5 configure: fix pthread check with static boringssl
A shared boringssl/OpenSSL library requires -lcrypto only for linking.
A static build additionally requires `-ldl -lpthread`. In the latter
case `-lpthread` is added to LIBS which prevented `-pthread` from being
added to CFLAGS. Clear LIBS to fix linking failures for libtest tests.
2020-05-22 21:10:07 +02:00
Daniel Stenberg f6c6a159f5
Revert "sendf: make failf() use the mvsnprintf() return code"
This reverts commit 74623551f3.

Instead mark the function call with (void). Getting the return code and
using it instead triggered Coverity warning CID 1463596 because
snprintf() can return a negative value...

Closes #5441
2020-05-22 12:34:51 +02:00
Daniel Stenberg 5d965b4898
typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *'
Reported-by: Billyzou0741326 on github
Fixes #5432
Closes #5436
2020-05-21 00:05:42 +02:00
Daniel Stenberg bb79320af7
tests/server/util.h: add extern to silence compiler warning
Follow-up from a3b0699d5c
2020-05-20 15:37:09 +02:00
Daniel Stenberg c9fddf64c6
typecheck-gcc.h: fix the OFF_T check
The option number also needs to be less than CURLOPTTYPE_BLOB.

Follow-up to cac5374298
Reported-by: Jeroen Ooms
Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114
2020-05-20 10:08:54 +02:00
Daniel Stenberg 6c889a0dd1
TODO: --dry-run
Closes #5426
2020-05-19 09:46:50 +02:00
Daniel Stenberg 39177b8ef9
TODO: Ratelimit or wait between serial requests
Closes #5406
2020-05-19 09:42:11 +02:00
Daniel Stenberg 33d60d09f3
tool_paramhlp: fixup C89 mistake
Follow-up to c5f0a9db22.
2020-05-19 08:58:33 +02:00
Siva Sivaraman c5f0a9db22
tool_paramhlp: fixed potentially uninitialized strtol() variable
Seems highly unlikely to actually be possible, but better safe than
sorry.

Closes #5417
2020-05-19 08:55:27 +02:00
Siva Sivaraman c4df1f75ad
tool_operate: fixed potentially uninitialized variables
... in curl_easy_getinfo() calls. They're harmless but clearing the
variables makes the code safer and comforts the reader.

Closes #5416
2020-05-19 08:53:41 +02:00
Daniel Stenberg 557dde201c
sha256: move assign to the declaration line
Follow-up to fae30656. Should've been squashed with that commit...
2020-05-19 08:52:38 +02:00
Siva Sivaraman fae3065676
sha256: fixed potentially uninitialized variable
Closes #5414
2020-05-19 08:47:12 +02:00
Daniel Stenberg 74623551f3
sendf: make failf() use the mvsnprintf() return code
... and avoid a strlen() call. Fixes a MonocleAI warning.

Reported-by: MonocleAI
Fixes #5413
Closes #5420
2020-05-19 08:13:31 +02:00