Commit Graph

25772 Commits

Author SHA1 Message Date
Gilles Vollant cac5374298
setopt: support certificate options in memory with struct curl_blob
This change introduces a generic way to provide binary data in setopt
options, called BLOBs.

This change introduces these new setopts:

CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB,
CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB.

Reviewed-by: Daniel Stenberg
Closes #5357
2020-05-15 13:03:59 +02:00
Daniel Stenberg 8df455479f
source cleanup: remove all custom typedef structs
- Stick to a single unified way to use structs
 - Make checksrc complain on 'typedef struct {'
 - Allow them in tests, public headers and examples

 - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually
   typedef different types/structs depending on build conditions.

Closes #5338
2020-05-15 08:54:42 +02:00
Daniel Stenberg 5d54b5e697
travis: remove the .checksrc fiddling 2020-05-15 08:54:26 +02:00
Daniel Stenberg 449e54b483
ftp: make domore_getsock() return the secondary socket properly
Previously, after PASV and immediately after the data connection has
connected, the function would only return the control socket to wait for
which then made the data connection simply timeout and not get polled
correctly. This become obvious when running test 1631 and 1632 event-
based.
2020-05-15 08:28:56 +02:00
Daniel Stenberg 9e95ca10b5
test1632: verify FTP through HTTPS-proxy with connection re-use 2020-05-15 08:28:56 +02:00
Daniel Stenberg c6ce51169e
test1631: verify FTP download through HTTPS-proxy 2020-05-15 08:28:56 +02:00
Daniel Stenberg a3b0699d5c
sws: as last resort, get test number from server cmd file
If it can't be found in the request. Also support --cmdfile to set it to
a custom file name.

runtests.pl always writes this file with the test number in it since a
while back.
2020-05-15 08:28:56 +02:00
Daniel Stenberg f002c850d9
ftp: shut down the secondary connection properly when SSL is used
Reported-by: Neal Poole
Fixes #5340
Closes #5385
2020-05-15 08:28:55 +02:00
Marcel Raad aa6cf7f3b6
KNOWN_BUGS: adapt 5.5 to recent changes
It only applies to non-Unicode builds now.
Also merge 5.10 into it as it's effectively a duplicate.

Closes https://github.com/curl/curl/pull/3784
2020-05-14 18:13:38 +02:00
Marcel Raad ffdddb45d9
curl_setup: support Unicode functions to open files on Windows
Use them only if `_UNICODE` is defined, in which case command-line
arguments have been converted to UTF-8.

Closes https://github.com/curl/curl/pull/3784
2020-05-14 18:13:37 +02:00
Marcel Raad 9e5669f388
tool: support UTF-16 command line on Windows
- use `wmain` instead of `main` when `_UNICODE` is defined [0]
- define `argv_item_t` as `wchar_t *` in this case
- use the curl_multibyte gear to convert the command-line arguments to
  UTF-8

This makes it possible to pass parameters with characters outside of
the current locale on Windows, which is required for some tests, e.g.
the IDN tests. Out of the box, this currently only works with the
Visual Studio project files, which default to Unicode, and winbuild
with the `ENABLE_UNICODE` option.

[0] https://devblogs.microsoft.com/oldnewthing/?p=40643

Ref: https://github.com/curl/curl/issues/3747
Closes https://github.com/curl/curl/pull/3784
2020-05-14 18:13:36 +02:00
Marcel Raad a55c835e6b
curl_multibyte: add to curlx
This will also be needed in the tool and tests.

Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512
Closes https://github.com/curl/curl/pull/3784
2020-05-14 18:13:27 +02:00
Daniel Stenberg 600a8cded4
url: make the updated credentials URL-encoded in the URL
Found-by: Gregory Jefferis
Reported-by: Jeroen Ooms
Added test 1168 to verify. Bug spotted when doing a redirect.
Bug: https://github.com/jeroen/curl/issues/224
Closes #5400
2020-05-14 17:59:47 +02:00
Daniel Stenberg e2a7a6bb9e
tests: add https-proxy support to the test suite
Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like
HTTP proxy but with a full TLS connection to the proxy.

Closes #5399
2020-05-14 16:21:31 +02:00
Daniel Stenberg 61504b5ef9
mailmap: James Fuller 2020-05-14 08:55:00 +02:00
Major_Tom 8e762199b0
vauth/cleartext: fix theoretical integer overflow
Fix theoretical integer overflow in Curl_auth_create_plain_message.

The security impact of the overflow was discussed on hackerone. We
agreed this is more of a theoretical vulnerability, as the integer
overflow would only be triggerable on systems using 32-bits size_t with
over 4GB of available memory space for the process.

Closes #5391
2020-05-14 08:36:35 +02:00
Jay Satiro f9983a6f9e curl.1: Quote globbed URLs
- Quote the globbing example URLs that contain characters [] {} since
  otherwise they may be interpreted as shell metacharacters.

Bug: https://github.com/curl/curl/issues/5388
Reported-by: John Simpson

Closes https://github.com/curl/curl/pull/5394
2020-05-13 18:24:34 -04:00
Daniel Stenberg 66b0775763
checksrc: enhance the ASTERISKSPACE and update code accordingly
Fine: "struct hello *world"

Not fine: "struct hello* world" (and variations)

Closes #5386
2020-05-14 00:02:05 +02:00
Daniel Stenberg f3f5d82e28
docs/options-in-versions: which version added each cmdline option
Added test 971 to verify that the list is in sync with the files in
cmdline-opts. The check also verifies that .d-files that uses Added:
specify the same version number as the options-in-versions file does.

Closes #5381
2020-05-13 23:58:36 +02:00
Daniel Stenberg 72b1aaf9da
docs: unify protocol lists
We boast support for 25 transfer protocols. Make sure the lists are
consistent

Closes #5384
2020-05-13 16:51:19 +02:00
Daniel Stenberg 81a54b12c6
OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN
... to avoid an OpenSSL bug that otherwise makes the CRL check to fail.

Reported-by: Michael Kaufmann
Fixes #5374
Closes #5376
2020-05-13 16:39:36 +02:00
Daniel Stenberg 9242bee3fe
tls13-ciphers.d: shorten the Arg 2020-05-13 16:37:29 +02:00
Daniel Stenberg 63b524c4ec
sasl-authzid.d: add Arg: and shorten the desc 2020-05-13 16:37:28 +02:00
Daniel Stenberg 16d8ff77a1
cert-type.d: mention the available types in the desc 2020-05-13 16:37:28 +02:00
Daniel Stenberg cd35371163
tool: shorten 3 --help descriptions
--happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort

gen.pl already warned about these lines but we didn't listen

Closes #5379
2020-05-13 16:37:22 +02:00
Daniel Stenberg 32aade5c91
configure: the wolfssh backend does not provide SCP
Closes #5387
2020-05-13 16:35:33 +02:00
Daniel Stenberg 5eb546cd80
RELEASE-NOTES: synced 2020-05-13 10:05:49 +02:00
Daniel Stenberg e1f3f3a14f
url: reject too long input when parsing credentials
Since input passed to libcurl with CURLOPT_USERPWD and
CURLOPT_PROXYUSERPWD circumvents the regular string length check we have
in Curl_setstropt(), the input length limit is enforced in
Curl_parse_login_details too, separately.

Reported-by: Thomas Bouzerar
Closes #5383
2020-05-13 08:02:42 +02:00
Daniel Stenberg 3ff89286a9
list-only.d: this option existed already in 4.0 2020-05-12 13:25:47 +02:00
Jay Satiro 79d60a9282 retry-all-errors.d: Shorten the summary line
Follow-up to b995bb5 from a few moments ago.

Reported-by: Daniel Stenberg

Ref: https://github.com/curl/curl/commit/b995bb5#r39108929
2020-05-12 03:12:55 -04:00
denzor a9021716cc easy: fix dangling pointer on easy_perform fail
Closes https://github.com/curl/curl/pull/5363
2020-05-12 03:00:27 -04:00
Jay Satiro b995bb58cb tool: Add option --retry-all-errors to retry on any error
The "sledgehammer" of retrying.

Closes https://github.com/curl/curl/pull/5185
2020-05-12 03:00:15 -04:00
James Le Cuirot 98e5904165
libcurl.pc: Merge Libs.private into Libs for static-only builds
A project being built entirely statically will call pkg-config with
--static, which utilises the Libs.private field. Conversely it will
not use --static when not being built entirely statically, even if
there is only a static build of libcurl available. This will most
likely cause the build to fail due to underlinking unless we merge the
Libs fields.

Consider that this is what the Meson build system does when it
generates pkg-config files.

I have also reflected this in the --libs argument of curl-config even
though REQUIRE_LIB_DEPS always seems to be "yes" anyway.

Closes #5373
2020-05-12 08:53:12 +02:00
Peter Wu ad64169867
CMake: fix runtests.pl with CMake, add new test targets
* runtests.pl:
    - Fix out-of-tree build under CMake when srcdir is not set. Default
      srcdir to the location of runtests.pl.
    - Add a hack to allow CMake to use the TFLAGS option as documented
      in tests/README and used in scripts/travis/script.sh.
  * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie
    support (no one should care, it is already EOL.).
  * Remove CTest since it defines its own 'test' target with no tests
    since all unittests are already broken and not built by default.
  * Add new test targets based on the options from Makefile.am. Since
    new test targets are rarely added, I opted for duplicating the
    runtests.pl options as opposed to creating a new Makefile.inc file.
    Use top-level target names (test-x) instead of x-test since that is
    used by CI and others.

Closes #5358
2020-05-12 08:50:17 +02:00
Peter Wu c2ab2494ef
CMake: do not build test programs by default
The default target should only build libcurl and curl. Add a dedicated
'testdeps' target which will be used later when running tests. Note that
unittests are currently broken in CMake and already excluded.

Closes #5368
2020-05-12 08:50:07 +02:00
Daniel Stenberg 90bbfb5136
FILEFORMAT: moved up the variables section and further polished 2020-05-12 08:23:14 +02:00
Daniel Stenberg c67592fed7
runtests: remove ftp2 support, not used
We once supported two separate ftp instances in the test suite. Has not
been used the last decade.

Closes #5375
2020-05-12 08:23:06 +02:00
Daniel Stenberg db8866fad9
url: sort the protocol schemes in rough popularity order
When looking for a protocol match among supported schemes, check the
most "popular" schemes first. It has zero functionality difference and
for all practical purposes a speed difference will not be measureable
but it still think it makes sense to put the least likely matches last.

"Popularity" based on the 2019 user survey.

Closes #5377
2020-05-12 08:22:04 +02:00
Marc Hoersken cffbcc3110
test1238: avoid tftpd being busy for tests shortly following
The tftpd server may still be busy if the total timeout of
25 seconds has not been reached or no sread error was received
during or after the execution of the timeout test 1238.

Once the next TFTP test comes around (eg. 1242 or 1243),
those will fail because the tftpd server is still waiting
on data from curl due to the UDP protocol being stateless
and having no connection close. On Linux this error may not
happen, because ICMP errors generated due to a swrite error
can also be returned async on the next sread call instead.

Therefore we will now just kill the tftpd server after test
1238 to make sure that the following tests are not affected.

This enables us to no longer ignore tests 1242, 1243, 2002
and 2003 on the CI platforms CirrusCI and AppVeyor.

Assisted-by: Peter Wu
Closes #5364
2020-05-11 19:59:19 +02:00
Daniel Stenberg 743e9c234b write-out.d: added "response_code" 2020-05-11 17:47:01 +02:00
Daniel Stenberg 4b88ac71f2
KNOWN_BUGS: Build with staticly built dependency
I rewrote the item 5.4 to be more generic about static dependencies.
2020-05-11 11:24:33 +02:00
Daniel Stenberg eed30a3101
ROADMAP: remove old entries
MQTT - the start has already landed

tiny-curl - also mostly landed and is a continuous work

make menuconfig - basically no interest from users, not pushing there
2020-05-11 10:12:04 +02:00
Peter Wu ac26be86f3
travis: Add ngtcp2 and quiche tests for CMake
To avoid an explosion of jobs, extend the existing CMake tests with
ngtcp2 and quiche support. macOS was previously moved to GitHub actions,
so the non-Linux case can be dropped.
2020-05-10 23:36:54 +02:00
Peter Wu 5d8c53d320
CMake: add ENABLE_ALT_SVC option
Tested alt-svc with quiche. While at it, add missing MultiSSL reporting
(not tested).
2020-05-10 23:36:54 +02:00
Peter Wu 5bfc874a35
CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche)
Add three new CMake Find modules (using the curl license, but I grant
others the right to apply the CMake BSD license instead).

This CMake config is simpler than the autotools one because it assumes
ngtcp2 and nghttp3 to be used together. Another difference is that this
CMake config checks whether QUIC is actually supported by the TLS
library (patched OpenSSL or boringssl) since this can be a common
configuration mistake that could result in build errors later.

Unlike autotools, CMake does not warn you that the features are
experimental. The user is supposed to already know that and read the
documentation. It requires a very special build environment anyway.

Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from
current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure
out which features (libldap-2.4, libssh2) to disable due to conflicts
with boringssl.

Closes #5359
2020-05-10 23:36:41 +02:00
Marc Hoersken 000f721690
tests/server/tftpd.c: fix include and enhance debug logging
setjmp.h should only be included if HAVE_SETJMP_H is defined.

Add additional log statements to see wether reads and writes
are blocking or finishing before an alarm signal is received.

Assisted-by: Peter Wu
Part of #5364
2020-05-10 17:58:29 +02:00
Daniel Stenberg 6f63ab41b2
tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present
Reported-by: Marcel Raad
Follow-up to 148534db5
Fixes #5367
Closes #5369
2020-05-10 15:33:51 +02:00
Marc Hoersken 54fa68bf44
appveyor: update comments to be clear about toolchain
- CMake-based MSYS builds use mingw-w64 to cross-compile.
- autotools-based builds are compiled using msys2-devel.

The difference is that the later ones are not cross-compiled
to Windows and instead require the msys2 runtime to be present.

At the moment only the Azure Pipelines CI builds actually
run autotools-based cross-compilation builds for Windows.
2020-05-09 18:04:38 +02:00
Marc Hoersken 8329775abb
TODO: update regarding missing Schannel features
Some aspects have already been implemented over the years.

15.1 Client certificates are now supported:

- System stores via e35b0256eb
- PKCS#12 files via 0fdf965126

15.2 Ciphers can now be specified through:

- Algorithms via 9aefbff30d

Reviewed-by: Daniel Stenberg and Marcel Raad
Closes #5358
2020-05-09 12:23:59 +02:00
Daniel Stenberg 92501a12a2
checksrc: close the .checksrc file handle when done reading 2020-05-08 17:00:29 +02:00