1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-14 21:45:13 -05:00
Commit Graph

26941 Commits

Author SHA1 Message Date
Jacob Hoffman-Andrews
246399a874
vtls: initial implementation of rustls backend
This adds a new TLS backend, rustls. It uses the C-to-rustls bindings
from https://github.com/abetterinternet/crustls.

Rustls is at https://github.com/ctz/rustls/.

There is still a fair bit to be done, like sending CloseNotify on
connection shutdown, respecting CAPATH, and properly indicating features
like "supports TLS 1.3 ciphersuites." But it works well enough to make
requests and receive responses.

Blog post for context:
https://www.abetterinternet.org/post/memory-safe-curl/

Closes #6350
2021-02-09 11:06:18 +01:00
Simon Josefsson
3eebbfe8f3
sasl: support SCRAM-SHA-1 and SCRAM-SHA-256 via libgsasl
Closes #6372
2021-02-09 10:53:25 +01:00
Jay Satiro
cb2dc1ba89 lib: use int type for more port variables
This is a follow-up to 764c6bd. Prior to that change port variables
were usually type long.

Closes https://github.com/curl/curl/pull/6553
2021-02-09 02:53:31 -05:00
Jay Satiro
65ca229461 tool_writeout: refactor write-out and write-out json
- Deduplicate the logic used by write-out and write-out json.

Rather than have separate writeLong, writeString, etc, logic for
each of write-out and write-out json instead have respective shared
functions that can output either format and a 'use_json' parameter to
indicate whether it is json that is output.

This will make it easier to maintain. Rather than have to go through
two sets of logic now we only have to go through one.

- Support write-out %{errormsg} and %{exitcode} in json.

- Clarify in the doc that %{exitcode} is the exit code of the transfer.

Prior to this change it just said "The numerical exitcode" which
implies it's the exit code of the tool, and it's not necessarily that.

Closes https://github.com/curl/curl/pull/6544
2021-02-09 02:48:27 -05:00
Jay Satiro
b9f11ae875 lib: drop USE_SOCKETPAIR in favor of CURL_DISABLE_SOCKETPAIR
.. since the former is undocumented and they both do the same thing.

Closes https://github.com/curl/curl/pull/6517
2021-02-09 02:47:51 -05:00
Jay Satiro
b48db908e0 curl_multibyte: fall back to local code page stat/access on Windows
If libcurl is built with Unicode support for Windows then it is assumed
the filename string is Unicode in UTF-8 encoding and it is converted to
UTF-16 to be passed to the wide character version of the respective
function (eg wstat). However the filename string may actually be in the
local encoding so, even if it successfully converted to UTF-16, if it
could not be stat/accessed then try again using the local code page
version of the function (eg wstat fails try stat).

We already do this with fopen (ie wfopen fails try fopen), so I think it
makes sense to extend it to stat and access functions.

Closes https://github.com/curl/curl/pull/6514
2021-02-09 02:47:05 -05:00
Stephan Szabo
1269c80af1 file: Support unicode urls on windows
Closes https://github.com/curl/curl/pull/6501
2021-02-09 02:45:51 -05:00
Vincent Torri
d4a3b87c13 cmake: fix import library name for non-MS compiler on Windows
- Use _imp.lib suffix only for Microsoft's compiler (MSVC).

Prior to this change library suffix _imp.lib was used for the import
library on Windows regardless of compiler.

With this change the other compilers should now use their default
suffix which should be .dll.a.

This change is motivated by the usage of pkg-config on MSYS2.
Indeed, when 'pkg-config --libs libcurl' is used, -lcurl is
passed to ld. The documentation of ld on Windows :

https://sourceware.org/binutils/docs/ld/WIN32.html

lists, in the 'direct linking to a dll' section, the pattern
of the searched import library, and libcurl_imp.lib is not there.

Closes https://github.com/curl/curl/pull/6225
2021-02-09 02:43:04 -05:00
Daniel Stenberg
f1e5e49879
urldata: move 'followlocation' to UrlState
As this is a state variable it does not belong in UserDefined which is
used to store values set by the user.

Closes #6582
2021-02-09 08:22:45 +01:00
Ikko Ashimine
13d8a56408
http_proxy: fix typo in http_proxy.c
settting -> setting

Closes #6583
2021-02-09 08:21:56 +01:00
Fabian Keil
461e3ef680
tests/server: Bump MAX_TAG_LEN to 200
This is useful for tests containing HTML inside of <data> sections.
For <img> tags it's not uncommon to be longer than the previous
limit of 79 bytes.

An example of a previously problem-causing tag is:
<img src="http://config.privoxy.org/send-banner?type=auto" border="0" title="Killed-http://www.privoxy.org/images/privoxy.png-by-size" width="88" height="31">
which is needed for a Privoxy test for the banners-by-size filter.

Previously it caused server failures like:
12:29:05.786961 ====> Client connect
12:29:05.787116 accept_connection 3 returned 4
12:29:05.787194 accept_connection 3 returned 0
12:29:05.787285 Read 119 bytes
12:29:05.787345 Process 119 bytes request
12:29:05.787407 Got request: GET /banners-by-size/9 HTTP/1.1
12:29:05.787464 Requested test number 9 part 0
12:29:05.787686 getpart() failed with error: -2
12:29:05.787744 - request found to be complete (9)
12:29:05.787912 getpart() failed with error: -2
12:29:05.788048 Wrote request (119 bytes) input to log/server.input
12:29:05.788157 Send response test9 section <data>
12:29:05.788443 getpart() failed with error: -2
12:29:05.788498 instructed to close connection after server-reply
12:29:05.788550 ====> Client disconnect 0
12:29:05.871448 exit_signal_handler: 15
12:29:05.871714 signalled to die
12:29:05.872040 ========> IPv4 sws (port 21108 pid: 51758) exits with signal (15)
2021-02-09 08:16:40 +01:00
Fabian Keil
e11e5b2d46
tests/badsymbols.pl: when opening '$incdir' fails include it in the error message 2021-02-09 08:16:40 +01:00
Fabian Keil
e6f4ab3477
runtests.1: document -o, -P, -L, and -E 2021-02-09 08:16:40 +01:00
Fabian Keil
203dc00718
runtests.pl: add %TESTNUMBER variable to make copying tests more convenient 2021-02-09 08:16:40 +01:00
Fabian Keil
8ad31a8c6e
runtests.pl: add an -o option to change internal variables
runtests.pl has lots of internal variables one might want to
change in certain situations, but adding a dedicated option
for every single one of them isn't practical.

Usage:
./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ...
2021-02-09 00:28:29 +01:00
Fabian Keil
e6d55f647d
runtests.pl: cleanups
- show the summarized test result in the last line of the report
- do not use $_ after mapping it to a named variable
  Doing that makes the code harder to follow.
- log the restraints sorted by the number of their occurrences
- fix language when logging restraints that only occured once
- let runhttpserver() use $TESTDIR instead of $srcdir
  ... so it works if a non-default $TESTDIR is being used.
2021-02-09 00:28:29 +01:00
Fabian Keil
3f0bef2b53
runtests.pl: add an -E option to specify an exclude file
It can contain additional restraints for test numbers,
keywords and tools.

The idea is to let third parties like the Privoxy project
distribute an exclude file with their tarballs that specifies
which curl tests are not expected to work when using Privoxy
as a proxy, without having to fork the whole curl test suite.

The syntax could be changed to be extendable and maybe
more closely reflect the "curl test" syntax. Currently
it's a bunch of lines like these:

test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped
keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped
tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped

To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs
on a single line, split them with commas.
2021-02-09 00:28:28 +01:00
Fabian Keil
b47ee58fb7
runtests.pl: add -L parameter to require additional perl libraries
This is useful to change the behaviour of the script without
having to modify the file itself, for example to use a custom
compareparts() function that ignores header differences that
are expected to occur when an external proxy is being used.

Such differences are proxy-specific and thus the modifications
should be maintained together with the proxy.
2021-02-09 00:28:28 +01:00
Fabian Keil
4421359e77
runtests.pl: add a -P option to specify an external proxy
... that should be used when executing the tests.

The assumption is that the proxy is an HTTP proxy.

This option should be used together with -L to provide
a customized compareparts() version that knows which
proxy-specific header differences should be ignored.

This option doesn't work for all test types yet.
2021-02-09 00:28:28 +01:00
Fabian Keil
124198631b
tests: fixup several tests
missing CRs and modified %hostip

lib556/test556: use a real HTTP version to make test reuse more convenient

make sure the weekday in Date headers matches the date

test61: replace stray "^M" (5e 4d) at the end of a cookie with a '^M' (0d)

Gets the test working with external proxies like Privoxy again.

Closes #6463
2021-02-09 00:28:19 +01:00
Daniel Stenberg
0c51036eb6
ftp: never set data->set.ftp_append outside setopt
Since the set value then risks getting used like that when the easy
handle is reused by the application.

Also: renamed the struct field from 'ftp_append' to 'remote_append'
since it is also used for SSH protocols.

Closes #6579
2021-02-08 22:44:37 +01:00
Daniel Stenberg
d836d362ec
urldata: remove the 'rtspversion' field
from struct connectdata and the corresponding code in http.c that set
it. It was never used for anything!

Closes #6581
2021-02-08 22:39:34 +01:00
Daniel Stenberg
ff9ec4e4a0
CURLOPT_QUOTE.3: clarify that libcurl doesn't parse what's sent
... so passed in commands may confuse libcurl's knowledge of state.

Reported-by: Bodo Bergmann
Fixes #6577
Closes #6580
2021-02-08 22:35:50 +01:00
Jacob Hoffman-Andrews
c30bf22f1c
vtls: factor out Curl_ssl_getsock to field of Curl_ssl
Closes #6558
2021-02-08 14:10:55 +01:00
Daniel Stenberg
8a644f3355
RELEASE-PROCEDURE: remove old release dates, add new 2021-02-08 14:04:05 +01:00
Daniel Stenberg
82551c1308
docs/SSL-PROBLEMS: enhanced
Elaborate on the intermediate cert issue, and mention that anything
below TLS 1.2 is generally considered insecure these days.

Closes #6572
2021-02-05 23:56:47 +01:00
Daniel Stenberg
666743a204
THANKS: remove a Jon Rumsey dupe 2021-02-05 13:43:26 +01:00
nimaje
373c9155a0 docs: fix FILE example url in --metalink documentation
In a url after <scheme>:// follows the possibly empty authority part
till the next /, so that url missed a /.

Closes #6573
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
2021-02-05 13:34:12 +01:00
Daniel Stenberg
6cd167a0dc
hostip: fix build with sync resolver
Reported-by: David Goerger
Follow-up from 8335c6417
Fixes #6566
Closes #6568
2021-02-05 08:49:22 +01:00
Daniel Stenberg
62d91e0078
mailmap: Jon Rumsey 2021-02-05 08:46:11 +01:00
Jon Rumsey
4a6a5f0b4a
gskit: correct the gskit_send() prototype
gskit_send() first paramater is a pointer to Curl_easy not connectdata
struct.

Closes #6570
Fixes #6569
2021-02-05 08:43:32 +01:00
Daniel Stenberg
457e864f5e
urldata: fix build without HTTP and MQTT
Reported-by: Joseph Chen
Fixes #6562
Closes #6563
2021-02-04 23:08:07 +01:00
Daniel Stenberg
e32601e13a
ftp: avoid SIZE when asking for a TYPE A file
... as we ignore it anyway because servers don't report the correct size
and proftpd even blatantly returns a 550.

Updates a set of tests accordingly.

Reported-by: awesomenode on github
Fixes #6564
Closes #6565
2021-02-04 23:06:03 +01:00
Daniel Stenberg
5f08a5acd5
pingpong: rename the curl_pp_transfer enum to use PP prefix
Using an FTP prefix for PP provided functionality was misleading.
2021-02-04 23:06:03 +01:00
Daniel Stenberg
bcba216f9b
RELEASE-NOTES: synced
... and bump pending version to 7.75.1 (for now)
2021-02-04 22:51:26 +01:00
Jay Satiro
98577e4029 build: fix --disable-http-auth
Broken since 215db08 (precedes 7.75.0).

Reported-by: Benbuck Nason

Fixes https://github.com/curl/curl/issues/6567
2021-02-04 14:49:56 -05:00
Jay Satiro
145848e2d1 build: fix --disable-dateparse
Broken since 215db08 (precedes 7.75.0).

Bug: https://curl.se/mail/lib-2021-02/0008.html
Reported-by: Firefox OS
2021-02-04 14:13:00 -05:00
jonrumsey
20faeee6f3
OS400: update for CURLOPT_AWS_SIGV4
chkstrings fails because a new string option that could require codepage
conversion has been added.

Closes #6561
Fixes #6560
2021-02-04 08:39:15 +01:00
Daniel Stenberg
a030c59c6d
BUG-BOUNTY: removed the cooperation mention 2021-02-03 14:24:25 +01:00
Daniel Stenberg
2f33be817c
RELEASE-NOTES: synced 2021-02-03 08:02:36 +01:00
Daniel Stenberg
0397557624
THANKS: added contributors from 7.75.0 2021-02-03 08:02:36 +01:00
Daniel Stenberg
2605a6fabb
copyright: fix year ranges in need of updates 2021-02-03 08:02:36 +01:00
Daniel Stenberg
fc9dd2d198
TODO: remove items for next SONAME bump etc
We want to avoid that completely, so we don't plan for things after such
an event.
2021-02-02 08:46:37 +01:00
Jay Satiro
fc9d3444d1
ngtcp2: Fix build error due to change in ngtcp2_settings
- Separate ngtcp2_transport_params.

ngtcp2/ngtcp2@05d7adc made ngtcp2_transport_params separate from
ngtcp2_settings.

ngtcp2 master is required to build curl with http3 support.

Closes #6554
2021-02-02 08:19:12 +01:00
Daniel Stenberg
2c4b338f7e
vtls: remove md5sum
As it is not used anymore.

Reported-by: Jacob Hoffman-Andrews
Bug: https://curl.se/mail/lib-2021-02/0000.html

Closes #6557
2021-02-02 08:13:09 +01:00
Alessandro Ghedini
a0c5781cb6
quiche: don't use primary_ip / primary_port
Closes #6555
2021-02-01 16:12:45 +01:00
Alessandro Ghedini
e58560bdaf travis: enable quiche's FFI feature 2021-02-01 13:13:12 +00:00
Dmitry Wagin
796ce293de
http: improve AWS HTTP v4 Signature auth
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.

Closes #6524
2021-01-30 23:48:22 +01:00
Daniel Stenberg
3c2210713e
hyper: fix CONNECT to set 'data' as userdata
Follow-up to 14e075d1a7
2021-01-30 17:51:47 +01:00
Layla
4fc5e7eda5
connect: fix compile errors in Curl_conninfo_local
.. for the `#else` (`!HAVE_GETSOCKNAME`) case

Fixes https://github.com/curl/curl/issues/6548
Closes #6549

Signed-off-by: Layla <layla@insightfulvr.com>
2021-01-29 23:30:50 +01:00