1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-13 21:15:08 -05:00
Commit Graph

2122 Commits

Author SHA1 Message Date
Dmitry Wagin
796ce293de
http: improve AWS HTTP v4 Signature auth
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.

Closes #6524
2021-01-30 23:48:22 +01:00
Daniel Stenberg
553f286a8f
tool_writeout: make %{errormsg} blank for no errors
Closes #6539
2021-01-28 07:55:25 +01:00
Jay Satiro
782143ee81 test410: fix for windows
- Pass the very long request header via file instead of command line.

Prior to this change the 49k very long request header string was passed
via command line and on Windows that is too long so it was truncated and
the test would fail (specifically msys CI).

Closes https://github.com/curl/curl/pull/6516
2021-01-25 23:28:02 -05:00
Daniel Stenberg
8335c6417e
hostip: remove conn->data from resolver functions
This also moves the 'async' struct from the connectdata struct into the
Curl_easy struct, which seems like a better home for it.

Closes #6497
2021-01-22 08:25:09 +01:00
Fabian Keil
0015e06151
http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy
Added test 1613 to verify.

Closes #6490
2021-01-20 15:21:13 +01:00
Fabian Keil
6fcf22654e
Add test1181: Proxy request with --proxy-header "Connection: Keep-Alive" 2021-01-19 11:41:23 +01:00
Fabian Keil
3bccc182a2
Add test1180: Proxy request with -H "Proxy-Connection: Keep-Alive"
At the moment the test fails as curl sends two Proxy-Connection
headers.
2021-01-19 11:41:23 +01:00
Daniel Stenberg
11b2dd86d3
test410: verify HTTPS GET with a 49K request header
skip test 410 for mesalink in the CI as it otherwise hangs "forever"
2021-01-18 16:10:34 +01:00
Daniel Stenberg
215db086e0
lib: pass in 'struct Curl_easy *' to most functions
... in most cases instead of 'struct connectdata *' but in some cases in
addition to.

- We mostly operate on transfers and not connections.

- We need the transfer handle to log, store data and more. Everything in
  libcurl is driven by a transfer (the CURL * in the public API).

- This work clarifies and separates the transfers from the connections
  better.

- We should avoid "conn->data". Since individual connections can be used
  by many transfers when multiplexing, making sure that conn->data
  points to the current and correct transfer at all times is difficult
  and has been notoriously error-prone over the years. The goal is to
  ultimately remove the conn->data pointer for this reason.

Closes #6425
2021-01-17 23:56:09 +01:00
Daniel Stenberg
30e7641d7d
test1918: verify curl_easy_option_by_name() and curl_easy_option_by_id()
... and as a practical side-effect, make sure that the
Curl_easyopts_check() function is asserted in debug builds, which we
want to detect mismatches between the options list in easyoptions.c and
the options in curl.h

Found-by: Gisle Vanem
Bug: 08e8455ddd (commitcomment-45991815)

Closes #6461
2021-01-15 17:03:29 +01:00
Daniel Stenberg
1123a0eb24
mime: make sure setting MIMEPOST to NULL resets properly
... so that a function can first use MIMEPOST and then set it to NULL to
reset it back to a blank POST.

Added test 584 to verify the fix.

Reported-by: Christoph M. Becker

Fixes #6455
Closes #6456
2021-01-15 08:33:27 +01:00
Daniel Stenberg
060658f634
test179: use consistent header line endings
... to make "Hyper mode" work better.
2021-01-08 10:57:48 +01:00
Daniel Stenberg
ab525c059e
http: have CURLOPT_FAILONERROR fail after all headers
... so that Retry-After and other meta-content can still be used.

Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.

Fixes #6408
Closes #6409
2021-01-06 15:14:40 +01:00
Daniel Stenberg
75f7ab28d2
lib/unit tests: add missing curl_global_cleanup() calls 2021-01-06 15:13:45 +01:00
Daniel Stenberg
80c5a838ce
test66: disable with Hyper
...as Hyper doesn't support HTTP/0.9
2021-01-05 17:27:05 +01:00
Daniel Stenberg
82e6d34803
test1633: set appropriate name
"--retry with a 429 response and Retry-After:"
2021-01-04 17:58:05 +01:00
Daniel Stenberg
648712eec1
httpauth: make multi-request auth work with custom port
When doing HTTP authentication and a port number set with CURLOPT_PORT,
the code would previously have the URL's port number override as if it
had been a redirect to an absolute URL.

Added test 1568 to verify.

Reported-by: UrsusArctos on github
Fixes #6397
Closes #6400
2021-01-02 23:40:47 +01:00
XhmikosR
aadc75626b
misc: assorted typo fixes
Closes #6375
2020-12-26 23:54:25 +01:00
Daniel Stenberg
58c4b8d636
test: verify new --write-out variables
Extended test 1029 and added 1188
2020-12-21 16:38:53 +01:00
Daniel Stenberg
f7a65979f3
test970: adapted to the new internal order of variables 2020-12-21 16:38:52 +01:00
Matthias Gatto
ebdb5f23cc
tests: add very simple AWS HTTP v4 Signature test
Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 16:28:03 +01:00
Daniel Stenberg
a7696c7343
curl: add --create-file-mode [mode]
This option sets the (octal) mode to use for the remote file when one is
created, using the SFTP, SCP or FILE protocols. When not set, the
default is 0644.

Closes #6244
2020-12-21 10:52:41 +01:00
Daniel Stenberg
debf23eead
mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735

Added test 1916 and 1917 to verify.

Closes #6338
2020-12-18 12:54:05 +01:00
Daniel Stenberg
c7dd08ce58
tests: updated tests for Hyper 2020-12-18 09:58:04 +01:00
Daniel Stenberg
a5bc272223
http: show the request as headers even when split-sending
When the initial request isn't possible to send in its entirety, the
remainder of request would be delivered to the debug callback as data
and would wrongly be counted internally as body-bytes sent.

Extended test 1295 to verify.

Closes #6328
2020-12-17 14:00:27 +01:00
Daniel Stenberg
3abfadfc19
test1198/9: add two mqtt publish tests without payload lengths
Closes #6335
2020-12-17 13:55:02 +01:00
Daniel Stenberg
354bc95d96
test1272: fix line ending
Follow-up to f24784f914
2020-12-15 17:38:42 +01:00
Daniel Stenberg
f24784f914
test1272: test gophers 2020-12-15 12:58:19 +01:00
Daniel Stenberg
eddae97406
test1564/1565: require the 'wakeup' feature to run
Fixes #6299
Fixes #6300
Closes #6301
2020-12-11 17:42:18 +01:00
Daniel Stenberg
3a63c190bb
tests: make --libcurl tests only test FTP options if ftp enabled
Adjust six --libcurl tests to only check the FTP option if FTP is
actually present in the build.

Fixes #6303
Closes #6305
2020-12-11 17:28:35 +01:00
Daniel Stenberg
ec9cc725d5
ftp: CURLOPT_FTP_SKIP_PASV_IP by default
The command line tool also independently sets --ftp-skip-pasv-ip by
default.

Ten test cases updated to adapt the modified --libcurl output.

Bug: https://curl.se/docs/CVE-2020-8284.html
CVE-2020-8284

Reported-by: Varnavas Papaioannou
2020-12-07 08:38:05 +01:00
Daniel Stenberg
abd846c374
urlapi: don't accept blank port number field without scheme
... as it makes the URL parser accept "very-long-hostname://" as a valid
host name and we don't want that. The parser now only accepts a blank
(no digits) after the colon if the URL starts with a scheme.

Reported-by: d4d on hackerone

Closes #6283
2020-12-07 00:50:49 +01:00
Daniel Stenberg
72ae6737e0 test506: make it not run in c-ares builds
As the asynch nature of it may trigger events in another order. A c-ares
upgrade made it break.

Reported-by: Marc Hörsken
Fixes #6247
2020-11-26 17:24:55 +01:00
Daniel Stenberg
ee644bc8b3
test493: verify --hsts upgrade and that %{url_effective} reflects that
Closes #6175
2020-11-06 08:22:34 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
2cfc4ed983
hsts: add read/write callbacks
- read/write callback options
- man pages for the 4 new setopts
- test 1915 verifies the callbacks

Closes #5896
2020-11-03 16:08:48 +01:00
Daniel Stenberg
7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Jay Satiro
e4d38014cd tests: fix some http/2 tests for older versions of nghttpx
- Add regex that strips http/2 server header name to those http/2 tests
  that don't already have it.

- Improve that regex in all http/2 tests.

Tests 358 and 359 were failing for me before this change on a system
that uses an older version of nghttpx which includes its version number
in the server header.

Closes https://github.com/curl/curl/pull/6139
2020-10-29 23:08:03 -04:00
Emil Engler
499b3b6ff6
tool_help: make "output" description less confusing
Currently the description of "output" is misleading when comparing it
"verbose".

Closes #6118
2020-10-24 23:46:02 +02:00
Daniel Stenberg
15997f6db5
test122[12]: remove these two tests
... and remove the objnames scripts they tested. They're not used for
anything anymore so testing them serves no purpose!

Reported-by: Marc Hörsken
Fixes #6080
Closes #6081
2020-10-15 07:56:19 +02:00
Daniel Stenberg
b8e4d1cb0a
runtests: add %repeat[]% for test files
... and use this new keywords in all the test files larger than 50K to reduce
their sizes and make them a lot easier to read and understand.

Closes #6040
2020-10-04 17:16:53 +02:00
Daniel Stenberg
e6b21d422e
runtests: provide curl's version string as %VERSION for tests
... so that we can check HTTP requests for User-Agent: curl/%VERSION

Update 600+ test cases accordingly.

Closes #6037
2020-10-02 22:54:23 +02:00
Daniel Stenberg
ddecd28268
test1465: verify --libcurl with binary POST data 2020-10-02 09:35:38 +02:00
Daniel Stenberg
4a4c724599
ftp: make a 552 response return CURLE_REMOTE_DISK_FULL
Added test 348 to verify. Added a 'STOR' command to the test FTP
server to enable test 348. Documented the command in FILEFORMAT.md

Reported-by: Duncan Wilcox
Fixes #6016
Closes #6017
2020-09-26 13:37:15 +02:00
Daniel Stenberg
11522d7292
test163[12]: require http to be built-in to run
... as speaking over an HTTPS proxy implies http!

Closes #6014
2020-09-25 14:29:45 +02:00
Daniel Stenberg
1397a7de6e
ftp: separate FTPS from FTP over "HTTPS proxy"
When using HTTPS proxy, SSL is used but not in the view of the FTP
protocol handler itself so separate the connection's use of SSL from the
FTP control connection's sue.

Reported-by: Mingtao Yang
Fixes #5523
Closes #6006
2020-09-24 14:09:20 +02:00
Dan Fandrich
93653ef9e2 tests/data: Fix some mismatched XML tags in test cases
This allows these test files to pass xmllint.
2020-09-23 14:53:26 -07:00
Daniel Stenberg
3e80895da7
test1297: verify GOT_NOTHING with http proxy tunnel 2020-09-22 10:54:42 +02:00
Daniel Stenberg
2cc6857247
test3015: verify stdout "as text"
Follow-up from 0c1e767e83 to please win32 tests

Closes #5962
2020-09-15 22:51:08 +02:00
anio
0c1e767e83
tool_writeout: add new writeout variable, %{num_headers}
This variable gives the number of headers.

Closes #5947
2020-09-15 00:37:08 +02:00