1
0
mirror of https://github.com/moparisthebest/curl synced 2024-12-25 01:28:51 -05:00
Commit Graph

2081 Commits

Author SHA1 Message Date
Marc Hoersken
3c9066fce5
tests: make Python-based servers compatible with Python 2 and 3
Update smbserver.py and negtelnetserver.py to be compatible with
Python 3 while staying backwards-compatible to support Python 2.

Fix string encoding and handling of echoed and transferred data.

Tested with both Python 2.7.17 and Python 3.7.7

Reported-by: Daniel Stenberg
Assisted-by: Kamil Dudka
Reviewed-by: Marcel Raad

Fixes #5104
Closes #5110
2020-03-19 03:26:19 +01:00
Johannes Schindelin
5450428491 schannel: add "best effort" revocation check option
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
  --ssl-revoke-best-effort to allow a "best effort" revocation check.

A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.

---

When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:

	Unknown error (0x80092012) - The revocation function was unable
	to check revocation for the certificate.

Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.

This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.

In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.

Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).

To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.

Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.

The test 2070 was added to verify this behavior, adapted from 310.

Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Closes https://github.com/curl/curl/pull/4981
2020-03-18 03:23:39 -04:00
Daniel Stenberg
7631f2b752
test 970: verify --write-out '%{json}'
Makes curl_easy_getinfo() of "variable" numerical content instead return
the number set in the env variable `CURL_TIME`.

Makes curl_version() of "variable" textual content. This guarantees a
stable version string which can be tested against. Environment variable
`CURL_VERSION` defines the content.

Assisted-by: Mathias Gumz
2020-03-17 15:04:24 +01:00
Daniel Stenberg
8a2b36df5b
DISABLED: disable test 323
The test uses SRP to "a server not supporting it" but modern stunnel
versions will silently accept it and remain happy. The test is therefore
faulty.

I haven't figured out how to make stunnel explicitly reject SRP-using
connects.

Reported-by: Marc Hörsken
Fixes #5105
Closes #5113
2020-03-17 09:55:35 +01:00
Marc Hoersken
dda8babd07
tests: fix verification of stdout in test 1452 due to newline
Fixes test1452:41:1: error: missing </stdout> tag before </verify>
2020-03-15 23:17:01 +01:00
Marc Hoersken
4be2560e01
tests: remove python_dependencies for smbserver from our tree
Users of the SMB tests will have to install impacket manually.

Reasoning: our in-tree version of impacket was quite outdated
and only compatible with Python 2 which is already end-of-life.
Upgrading to Python 3 and a compatible impacket version would
require to import additional Python-only and CPython-extension
dependencies. This would have hindered portability enormously.

Closes #5094
2020-03-15 10:01:38 +01:00
Daniel Stenberg
fe8ba51209
server/getpart: make the "XML-parser" stricter
When extracting a <section> <part> and there's no </part> before
</section>, this now outputs an error and returns a wrong string to
make users spot the mistake.

Ref: #5070
Closes #5071
2020-03-13 11:03:42 +01:00
Dan Fandrich
c0fe3b3260 tests/data: Fix some XML formatting issues in test cases
This allows these test files to pass xmllint.
2020-03-11 10:33:03 +01:00
Marc Hoersken
fb6134427a
test1129: fix invalid case of closing XML-tag and Content-Length
Fixes #5070
Closes #5072
2020-03-10 22:38:56 +01:00
Marc Hoersken
f11b2cb62b
tests/data: fix static ip instead of dynamic value being used
Follow up to 94ced8e
2020-03-10 13:31:19 +01:00
Marc Hoersken
94ced8e368
tests/data: fix static ip:port instead of dynamic values being used
Closes #5065
2020-03-10 03:56:50 +01:00
Jim Fuller
e96fe70cab
sftp: fix segfault regression introduced by #4747
This fix adds a defensive check for the case where the char *name in
struct libssh2_knownhost is NULL

Fixes #5041
Closes #5062
2020-03-09 15:01:40 +01:00
Patrick Monnerat
e5b366c0b5
mime: fix the binary encoder to handle large data properly
New test 666 checks this is effective.
As upload buffer size is significant in this kind of tests, shorten it
in similar test 652.

Fixes #4860
Closes #4833
Reported-by: RuurdBeerstra on github
2020-03-07 23:26:15 +01:00
Patrick Monnerat
1e4cb333ef
mime: do not perform more than one read in a row
Input buffer filling may delay the data sending if data reads are slow.
To overcome this problem, file and callback data reads do not accumulate
in buffer anymore. All other data (memory data and mime framing) are
considered as fast and still concatenated in buffer.
As this may highly impact performance in terms of data overhead, an early
end of part data check is added to spare a read call.
When encoding a part's data, an encoder may require more bytes than made
available by a single read. In this case, the above rule does not apply
and reads are performed until the encoder is able to deliver some data.

Tests 643, 644, 645, 650 and 654 have been adapted to the output data
changes, with test data size reduced to avoid the boredom of long lists of
1-byte chunks in verification data.
New test 667 checks mimepost using single-byte read callback with encoder.
New test 668 checks the end of part data early detection.

Fixes #4826
Reported-by: MrdUkk on github
2020-03-07 23:26:00 +01:00
Patrick Monnerat
96972ec1c0
mime: latch last read callback status.
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.

Fixes #4813
Reported-by: MrdUkk on github
2020-03-07 23:26:00 +01:00
Jay Satiro
09aa807240 libssh: Fix matching user-specified MD5 hex key
Prior to this change a match would never be successful because it
was mistakenly coded to compare binary data from libssh to a
user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5).

Reported-by: fds242@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/4971
Closes https://github.com/curl/curl/pull/4974
2020-03-07 03:06:11 -05:00
Daniel Stenberg
5494afac8f
tests: disable 962, 963 and 964 on Windows
These tests are also doing UTF-8 SMTP.

Follow-up to df207d2dd9
2020-03-02 14:44:46 +01:00
Steve Holme
df207d2dd9
tests: disable SMTP UTF-8 tests on Windows
Fixes #4988
Closes #4992
2020-03-02 08:29:11 +01:00
Daniel Stenberg
8cd4e6d81f
Revert "mime: latch last read callback status."
This reverts commit 87869e38d7.

Fixes #5014
Closes #5015
Reopens #4833
2020-03-02 08:07:49 +01:00
Daniel Stenberg
d7242f4757
Revert "mime: do not perform more than one read in a row"
This reverts commit ed0f357f7d.
2020-03-02 07:42:54 +01:00
Daniel Stenberg
e002f6c0cb
Revert "mime: fix the binary encoder to handle large data properly"
This reverts commit b2caaa0681.
2020-03-02 07:42:48 +01:00
Patrick Monnerat
b2caaa0681
mime: fix the binary encoder to handle large data properly
New test 666 checks this is effective.
As upload buffer size is significant in this kind of tests, shorten it
in similar test 652.

Fixes #4860
Reported-by: RuurdBeerstra on github
2020-03-02 00:00:59 +01:00
Patrick Monnerat
ed0f357f7d
mime: do not perform more than one read in a row
Input buffer filling may delay the data sending if data reads are slow.
To overcome this problem, file and callback data reads do not accumulate
in buffer anymore. All other data (memory data and mime framing) are
considered as fast and still concatenated in buffer.
As this may highly impact performance in terms of data overhead, an early
end of part data check is added to spare a read call.
When encoding a part's data, an encoder may require more bytes than made
available by a single read. In this case, the above rule does not apply
and reads are performed until the encoder is able to deliver some data.

Tests 643, 644, 645, 650 and 654 have been adapted to the output data
changes, with test data size reduced to avoid the boredom of long lists of
1-byte chunks in verification data.
New test 664 checks mimepost using single-byte read callback with encoder.
New test 665 checks the end of part data early detection.

Fixes #4826
Reported-by: MrdUkk on github
2020-03-02 00:00:58 +01:00
Patrick Monnerat
87869e38d7
mime: latch last read callback status.
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.

Fixes #4813
Reported-by: MrdUkk on github
Closes #4833
2020-03-02 00:00:50 +01:00
Daniel Stenberg
15f51474c8
http2: make pausing/unpausing set/clear local stream window
This reduces the HTTP/2 window size to 32 MB since libcurl might have to
buffer up to this amount of data in memory and yet we don't want it set
lower to potentially impact tranfer performance on high speed networks.

Requires nghttp2 commit b3f85e2daa629
(https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end
up in the next release after 1.40.0.

Fixes #4939
Closes #4940
2020-02-27 10:35:51 +01:00
Steve Holme
0922f76240
tests: Automatically deduce the tool name from the test case for unit tests
It is still possible to override the executable to run during the test,
using the <tool> tag, but this patch removes the requirement that the
tag must be present for unit tests.

It also removes the possibility of human error when existing test cases
are used as the basis for new tests, as recently witnessed in 81c37124.

Reviewed-by: Daniel Stenberg
Closes #4976
2020-02-27 00:51:49 +00:00
Steve Holme
d9e40f57de
test1323: Added the missing 'unit test' feature requirement in the test case 2020-02-27 00:51:49 +00:00
Daniel Stenberg
6375b205a9
http: added 417 response treatment
When doing a request with a body + Expect: 100-continue and the server
responds with a 417, the same request will be retried immediately
without the Expect: header.

Added test 357 to verify.

Also added a control instruction to tell the sws test server to not read
the request body if Expect: is present, which the new test 357 uses.

Reported-by: bramus on github
Fixes #4949
Closes #4964
2020-02-26 22:48:14 +01:00
Steve Holme
34a1ffb05f
smtp: Support the SMTPUTF8 extension for the EXPN command
Simply notify the server we support the SMTPUTF8 extension if it does.
2020-02-26 14:54:51 +00:00
Steve Holme
efce3ea5a8
smtp: Support the SMTPUTF8 extension in the VRFY command 2020-02-26 14:54:51 +00:00
Steve Holme
483edeb8dd
smtp: Support the SMTPUTF8 extension in the RCPT TO command
Note: The RCPT TO command isn't required to advertise to the server that
it contains UTF-8 characters, instead the server is told that a mail may
contain UTF-8 in any envelope command via the MAIL command.
2020-02-26 14:54:51 +00:00
Steve Holme
aba1bf630f
smtp: Support the SMTPUTF8 extension in the MAIL command
Support the SMTPUTF8 extension when sending mailbox information in the
MAIL command (FROM and AUTH parameters). Non-ASCII domain names will
be ACE encoded, if IDN is supported, whilst non-ASCII characters in
the local address part are passed to the server.

Reported-by: ygthien on github
Fixes #4828
2020-02-26 14:49:40 +00:00
Steve Holme
2aa6c1735a
smtp: Support UTF-8 based host names in the VRFY command 2020-02-26 11:02:38 +00:00
Steve Holme
e7959c280c
smtp: Support UTF-8 based host names in the RCPT TO command 2020-02-26 11:02:38 +00:00
Steve Holme
68fb25fa3f
smtp: Support UTF-8 based host names in the MAIL command
Non-ASCII host names will be ACE encoded if IDN is supported.
2020-02-26 11:02:38 +00:00
Steve Holme
4c140a5628
smtp: Added UTF-8 mailbox tests to verify existing behaviour 2020-02-25 04:15:26 +00:00
Steve Holme
6995b77a52
ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case 2020-02-24 23:01:05 +00:00
Steve Holme
3f74e5e604
hmac: Added a unit test for the HMAC hash generation
Closes #4973
2020-02-24 06:57:03 +00:00
Steve Holme
fa009cc798
tests: Added a unit test for MD4 digest generation
Closes #4970
2020-02-23 18:47:32 +00:00
Steve Holme
1f4911bc26
test1610: Fixed the link to the unit test
Typo from 81c37124.
2020-02-23 08:13:09 +00:00
Steve Holme
81c3712434 tests: Added a unit test for SHA256 digest generation
Follow up to 2b5b37c.

Closes #4968
2020-02-22 22:12:27 +00:00
Daniel Stenberg
02f8de6516
altsvc: keep a copy of the file name to survive handle reset
The alt-svc cache survives a call to curl_easy_reset fine, but the file
name to use for saving the cache was cleared. Now the alt-svc cache has
a copy of the file name to survive handle resets.

Added test 1908 to verify.

Reported-by: Craig Andrews
Fixes #4898
Closes #4902
2020-02-09 22:41:49 +01:00
Pierre-Yves Bigourdan
feba3f0549
digest: Do not quote algorithm in HTTP authorisation
RFC 7616 section 3.4 (The Authorization Header Field) states that "For
historical reasons, a sender MUST NOT generate the quoted string syntax
for the following parameters: algorithm, qop, and nc". This removes the
quoting for the algorithm parameter.

Reviewed-by: Steve Holme
Closes #4890
2020-02-07 22:46:21 +01:00
Daniel Stenberg
950b53da0d
ftp: remove superfluous checking for crlf in user or pwd
... as this is already done much earlier in the URL parser.

Also add test case 894 that verifies that pop3 with an encodedd CR in
the user name is rejected.

Closes #4887
2020-02-07 08:18:23 +01:00
Daniel Stenberg
0b030a5b23
global_init: move the IPv6 works status bool to multi handle
Previously it was stored in a global state which contributed to
curl_global_init's thread unsafety. This boolean is now instead figured
out in curl_multi_init() and stored in the multi handle. Less effective,
but thread safe.

Closes #4851
2020-01-28 08:03:22 +01:00
Pavel Volgarev
4a4609bf3c
smtp: Allow RCPT TO command to fail for some recipients
Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS.

Verified with the new tests 3002-3007

Closes #4816
2020-01-21 10:40:19 +01:00
Emil Engler
1774dbd74c
curl: Let -D merge headers in one file again
Closes #4762
Fixes #4753
2020-01-21 09:23:30 +01:00
加藤郁之
7ff9222ced
HTTP: increase EXPECT_100_THRESHOLD to 1Mb
Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html

Closes #4814
2020-01-20 08:33:44 +01:00
Daniel Stenberg
4431ed2484
curl: make #0 not output the full URL
It was not intended nor documented!

Added test 1176 to verify.

Reported-by: vshmuk on hackerone

Closes #4812
2020-01-13 15:37:46 +01:00
Emil Engler
cbb5429001 ngtcp2: Add an error code for QUIC connection errors
- Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection
  errors.

Prior to this change CURLE_FAILED_INIT was used, but that was not
correct.

Closes https://github.com/curl/curl/pull/4754
2020-01-11 18:19:32 -05:00
Jay Satiro
b700662b1c multi: Change curl_multi_wait/poll to error on negative timeout
- Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when
  curl_multi_wait/poll is passed timeout param < 0.

Prior to this change passing a negative value to curl_multi_wait/poll
such as -1 could cause the function to wait forever.

Reported-by: hamstergene@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/4763

Closes https://github.com/curl/curl/pull/4765
2020-01-11 18:16:28 -05:00
Daniel Stenberg
fb723fd9d1
test1167: verify global symbols in public headers are curl prefixed
... using the new badsymbols.pl perl script

Fixes #4793
Closes #4794
2020-01-09 22:57:33 +01:00
Marcel Raad
8cf875ed3b
TrackMemory tests: always remove CR before LF
It was removed for output containing ' =' via `s/ =.*//`. With classic
MinGW, this made lines with `free()` end with CRLF, but lines with e.g.
`malloc()` end with only LF. The tests expect LF only.

Closes https://github.com/curl/curl/pull/4788
2020-01-06 20:56:27 +01:00
Daniel Stenberg
4c34af4c99
curl -w: handle a blank input file correctly
Previously it would end up with an uninitialized memory buffer that
would lead to a crash or junk getting output.

Added test 1271 to verify.

Reported-by: Brian Carpenter
Closes #4786
2020-01-06 10:10:48 +01:00
Daniel Stenberg
392bff4abf
curl: properly free mimepost data
... as it could otherwise leak memory when a transfer failed.

Added test 1293 to verify.

Reported-by: Brian Carpenter
Fixes #4781
Closes #4782
2020-01-04 23:06:44 +01:00
Jay Satiro
9603c8299f tests: Change NTLM tests to require SSL
Prior to this change tests that required NTLM feature did not require
SSL feature.

There are pending changes to cmake builds that will allow enabling NTLM
in non-SSL builds in Windows. In that case the NTLM auth strings created
are different from what is expected by the NTLM tests and they fail:

"The issue with NTLM is that previous non-SSL builds would not enable
NTLM and so the NTLM tests would be skipped."

Assisted-by: marc-groundctl@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729

Closes https://github.com/curl/curl/pull/4768
2019-12-31 02:47:53 -05:00
Daniel Stenberg
25b69c482f
sws: search for "Testno:" header uncondtionally if no testno
Even if the initial request line wasn't found. With the fix to 1455, the
test number is now detected correctly.

(Problem found when running tests in random order.)

Closes #4744
2019-12-19 23:03:33 +01:00
Daniel Stenberg
b4c9982382
tests: set LC_ALL in more tests
Follow-up to 23208e330a

Closes #4743
2019-12-19 23:01:14 +01:00
Daniel Stenberg
23208e330a
test165: set LC_ALL=en_US.UTF-8 too
On my current Debian Unstable with libidn2 2.2.0, I get an error if
LC_ALL is set to blank. Then curl errors out with:

curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8

Closes #4738
2019-12-19 14:26:26 +01:00
Daniel Stenberg
fba8301338
test1456: remove the use of a fixed local port
Fixup the test to instead not compare the port number. It sometimes
caused problems like this:

"curl: (45) bind failed with errno 98: Address already in use"

Closes #4733
2019-12-19 08:59:10 +01:00
Daniel Stenberg
9973dabeb6
test1270: a basic -w redirect_url test
Closes #4728
2019-12-18 08:26:03 +01:00
Marcel Raad
38ad9ea1b8
tests: use DoH feature for DoH tests
Previously, http/2 was used instead.

Assisted-by: Jay Satiro
Closes https://github.com/curl/curl/pull/4692
2019-12-13 20:55:52 +01:00
Daniel Stenberg
ee263de7a3
conncache: fix multi-thread use of shared connection cache
It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-by: Christopher Reid
Fixes #4544
Closes #4557
2019-12-09 15:30:09 +01:00
Daniel Stenberg
9ea769e15f
etag: allow both --etag-compare and --etag-save in same cmdline
Fixes #4669
Closes #4678
2019-12-05 22:57:24 +01:00
Daniel Stenberg
914975fe6e
test342: make it return a 304 as the tag matches 2019-12-05 16:58:50 +01:00
Maros Priputen
18e5cb77e9
curl: two new command line options for etags
--etag-compare and --etag-save

Suggested-by: Paul Hoffman
Fixes #4277
Closes #4543
2019-11-28 13:05:20 +01:00
Gergely Nagy
f3c35e371c
multi: add curl_multi_wakeup()
This commit adds curl_multi_wakeup() which was previously in the TODO
list under the curl_multi_unblock name.

On some platforms and with some configurations this feature might not be
available or can fail, in these cases a new error code
(CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup().

Fixes #4418
Closes #4608
2019-11-25 15:45:56 +01:00
Daniel Stenberg
74f441c6d3
test1175: verify symbols-in-versions and libcurl-errors.3 in sync
Closes #4628
2019-11-22 09:11:53 +01:00
Daniel Stenberg
f70da9c17e
include: make CURLE_HTTP3 use a new error code
To avoid potential issues with error code reuse.

Reported-by: Christoph M. Becker
Assisted-by: Dan Fandrich
Fixes #4601
Closes #4627
2019-11-21 23:16:29 +01:00
Daniel Stenberg
7a46aeb0be
curl: fix -T globbing
Regression from e59371a493 (7.67.0)

Added test 490, 491 and 492 to verify the functionality.

Reported-by: Kamil Dudka
Reported-by: Anderson Sasaki

Fixes #4588
Closes #4591
2019-11-14 13:44:50 +01:00
Daniel Stenberg
b3eb7d172a
quiche: reject headers in the wrong order
Pseudo header MUST come before regular headers or cause an error.

Reported-by: Cynthia Coan
Fixes #4571
Closes #4584
2019-11-12 09:44:24 +01:00
Marcel Raad
86e26996c7
test1558: use double slash after file:
Classic MinGW / MSYS 1 doesn't support `MSYS2_ARG_CONV_EXCL`, so this
test unnecessarily failed when using `file:/` instead of `file:///`.

Closes https://github.com/curl/curl/pull/4554
2019-11-10 20:34:40 +01:00
Daniel Stenberg
5d5bd4460e
test1560: require IPv6 for IPv6 aware URL parsing
The URL parser function can't reject a bad IPv6 address properly when
curl was built without IPv6 support.

Reported-by: Marcel Raad
Fixes #4556
Closes #4572
2019-11-08 23:09:53 +01:00
Marcel Raad
b3378a793c
test1591: fix spelling of http feature
The test never got run because the feature name is `http` in lowercase.

Closes https://github.com/curl/curl/pull/4520
2019-10-24 14:59:26 +02:00
Daniel Stenberg
807c056c08
conn-reuse: requests wanting NTLM can reuse non-NTLM connections
Added test case 338 to verify.

Reported-by: Daniel Silverstone
Fixes #4499
Closes #4514
2019-10-23 10:15:39 +02:00
Marcel Raad
2838fd91be
tests: add missing proxy features 2019-10-23 08:28:03 +02:00
Marcel Raad
d81dbae19f
tests: use %FILE_PWD for file:// URLs
This way, we always have exactly one slash after the host name, making
the tests pass when curl is compiled with the MSYS GCC.

Closes https://github.com/curl/curl/pull/4512
2019-10-21 17:28:48 +02:00
Marcel Raad
2e4405d29c
tests: add connect to non-listen keywords
These tests try to connect to ports nothing is listening on.

Closes https://github.com/curl/curl/pull/4511
2019-10-21 10:19:22 +02:00
Marcel Raad
fff1ba7a6b
test1162: disable MSYS2's POSIX path conversion
This avoids MSYS2 converting the backslasb in the URL to a slash,
causing the test to fail.
2019-10-17 20:42:07 +02:00
Daniel Stenberg
e062043433
url: normalize CURLINFO_EFFECTIVE_URL
The URL extracted with CURLINFO_EFFECTIVE_URL was returned as given as
input in most cases, which made it not get a scheme prefixed like before
if the URL was given without one, and it didn't remove dotdot sequences
etc.

Added test case 1907 to verify that this now works as intended and as
before 7.62.0.

Regression introduced in 7.62.0

Reported-by: Christophe Dervieux
Fixes #4491
Closes #4493
2019-10-16 14:10:55 +02:00
Marcel Raad
347075bc17
tests: line ending fixes for Windows
Mark some files as text.

Closes https://github.com/curl/curl/pull/4490
2019-10-16 10:14:07 +02:00
Marcel Raad
476eb88171
tests: use proxy feature
This makes the tests succeed when using --disable-proxy.

Closes https://github.com/curl/curl/pull/4488
2019-10-15 07:56:55 +02:00
Daniel Stenberg
b59f0626b5
tests: use port 2 instead of 60000 for a safer non-listening port
... when the tests want "connection refused".
2019-10-09 14:16:28 +02:00
Stian Soiland-Reyes
f7f0b0012d
curl: ensure HTTP 429 triggers --retry
This completes #3794.

Also make sure the new tests from #4195 are enabled

Closes #4465
2019-10-05 11:41:06 +02:00
Daniel Stenberg
02c6b984cb
urlapi: fix use-after-free bug
Follow-up from 2c20109a9b

Added test 663 to verify.

Reported by OSS-Fuzz
Bug: https://crbug.com/oss-fuzz/17954

Closes #4453
2019-10-03 22:54:26 +02:00
Daniel Stenberg
c6f250c4d6
redirect: when following redirects to an absolute URL, URL encode it
... to make it handle for example (RFC violating) embeded spaces.

Reported-by: momala454 on github
Fixes #4445
Closes #4447
2019-10-02 07:53:32 +02:00
Daniel Stenberg
e59371a493
curl: create easy handles on-demand and not ahead of time
This should again enable crazy-large download ranges of the style
[1-10000000] that otherwise easily ran out of memory starting in 7.66.0
when this new handle allocating scheme was introduced.

Reported-by: Peter Sumatra
Fixes #4393
Closes #4438
2019-10-02 07:50:32 +02:00
Daniel Stenberg
c24cf6c64c
altsvc: accept quoted ma and persist values
As mandated by the spec. Test 1654 is extended to verify.

Closes #4443
2019-09-30 16:35:12 +02:00
Daniel Stenberg
d0a7ee3f61
cookies: using a share with cookies shouldn't enable the cookie engine
The 'share object' only sets the storage area for cookies. The "cookie
engine" still needs to be enabled or activated using the normal cookie
options.

This caused the curl command line tool to accidentally use cookies
without having been told to, since curl switched to using shared cookies
in 7.66.0.

Test 1166 verifies

Updated test 506

Fixes #4429
Closes #4434
2019-09-28 18:10:43 +02:00
Zenju
a167ab6a1e
FTP: add test for FTPFILE_NOCWD: Avoid redundant CWDs
Add libtest 661

Closes #4417
2019-09-27 08:32:32 +02:00
Zenju
500fb0e4cb
FTP: url-decode path before evaluation
Closes #4428
2019-09-27 08:29:54 +02:00
Daniel Stenberg
ea7744a07e
Revert "FTP: url-decode path before evaluation"
This reverts commit 2f036a72d5.
2019-09-26 14:53:19 +02:00
Zenju
2f036a72d5
FTP: url-decode path before evaluation
Closes #4423
2019-09-26 13:57:44 +02:00
Zenju
1c02a4e874
FTP: remove trailing slash from path for LIST/MLSD
Closes #4348
2019-09-18 09:27:21 +02:00
Paul Dreik
b766602729
doh: fix (harmless) buffer overrun
Added unit test case 1655 to verify.
Close #4352

the code correctly finds the flaws in the old code,
if one temporarily restores doh.c to the old version.
2019-09-15 23:25:24 +02:00
Daniel Stenberg
65f5b958c9
FTP: allow "rubbish" prepended to the SIZE response
This is a protocol violation but apparently there are legacy proprietary
servers doing this.

Added test 336 and 337 to verify.

Reported-by: Philippe Marguinaud
Closes #4339
2019-09-13 14:21:00 +02:00
Zenju
acf1d2acd1
FTP: skip CWD to entry dir when target is absolute
Closes #4332
2019-09-13 14:05:34 +02:00
Daniel Stenberg
68fab35c73
http: fix use of credentials from URL when using HTTP proxy
When a username and password are provided in the URL, they were wrongly
removed from the stored URL so that subsequent uses of the same URL
wouldn't find the crendentials. This made doing HTTP auth with multiple
connections (like Digest) mishave.

Regression from 46e164069d (7.62.0)

Test case 335 added to verify.

Reported-by: Mike Crowe

Fixes #4228
Closes #4229
2019-08-16 10:11:22 +02:00
Jay Satiro
dca6f73613 vauth: Use CURLE_AUTH_ERROR for auth function errors
- Add new error code CURLE_AUTH_ERROR.

Prior to this change auth function errors were signaled by
CURLE_OUT_OF_MEMORY and CURLE_RECV_ERROR, and neither one was
technically correct.

Ref: https://github.com/curl/curl/pull/3848

Co-authored-by: Dominik Hölzl

Closes https://github.com/curl/curl/pull/3864
2019-08-14 03:14:01 -04:00
Daniel Stenberg
69b3ff5118
alt-svc: add protocol version selection masking
So that users can mask in/out specific HTTP versions when Alt-Svc is
used.

 - Removed "h2c" and updated test case accordingly
 - Changed how the altsvc struct is laid out
 - Added ifdefs to make the unittest run even in a quiche-tree

Closes #4201
2019-08-08 09:10:29 +02:00
Daniel Stenberg
f933449d3b
CURLINFO_RETRY_AFTER: parse the Retry-After header value
This is only the libcurl part that provides the information. There's no
user of the parsed value. This change includes three new tests for the
parser.

Ref: #3794
2019-08-07 14:45:39 +02:00
Jay Satiro
a44394eac5 tests: Fix the line endings for the SASL alt-auth tests
- Change data and protocol sections to CRLF line endings.

Prior to this change the tests would fail or hang, which is because
certain sections such as protocol require CRLF line endings.

Follow-up to grandparent commit which added the tests.

Ref: https://github.com/curl/curl/issues/3653
Ref: https://github.com/curl/curl/pull/3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:58 -04:00
Steve Holme
1e08a79258 curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool
Ref: https://github.com/curl/curl/issues/3653
Ref: https://github.com/curl/curl/pull/3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:41 -04:00
Daniel Stenberg
02346abc32
curl_multi_poll: a sister to curl_multi_wait() that waits more
Repeatedly we see problems where using curl_multi_wait() is difficult or
just awkward because if it has no file descriptor to wait for
internally, it returns immediately and leaves it to the caller to wait
for a small amount of time in order to avoid occasional busy-looping.

This is often missed or misunderstood, leading to underperforming
applications.

This change introduces curl_multi_poll() as a replacement drop-in
function that accepts the exact same set of arguments. This function
works identically to curl_multi_wait() - EXCEPT - for the case when
there's nothing to wait for internally, as then this function will by
itself wait for a "suitable" short time before it returns. This
effectiely avoids all risks of busy-looping and should also make it less
likely that apps "over-wait".

This also changes the curl tool to use this funtion internally when
doing parallel transfers and changes curl_easy_perform() to use it
internally.

Closes #4163
2019-08-06 09:33:29 +02:00
Daniel Stenberg
a42b0957ab
http09: disable HTTP/0.9 by default in both tool and library
As the plan has been laid out in DEPRECATED. Update docs accordingly and
verify in test 1174. Now requires the option to be set to allow HTTP/0.9
responses.

Closes #4191
2019-08-06 00:08:14 +02:00
Daniel Stenberg
1132f43012
alt-svc: more liberal ALPN name parsing
Allow pretty much anything to be part of the ALPN identifier. In
particular minus, which is used for "h3-20" (in-progress HTTP/3
versions) etc.

Updated test 356.
Closes #4182
2019-08-03 21:54:00 +02:00
Kamil Dudka
4c187043c5 http_negotiate: improve handling of gss_init_sec_context() failures
If HTTPAUTH_GSSNEGOTIATE was used for a POST request and
gss_init_sec_context() failed, the POST request was sent
with empty body.  This commit also restores the original
behavior of `curl --fail --negotiate`, which was changed
by commit 6c60355323.

Add regression tests 2077 and 2078 to cover this.

Fixes #3992
Closes #4171
2019-08-01 16:21:41 +02:00
Daniel Stenberg
db0a0dfb0e
curl: cap the maximum allowed values for retry time arguments
... to avoid integer overflows later when multiplying with 1000 to
convert seconds to milliseconds.

Added test 1269 to verify.

Reported-by: Jason Lee
Closes #4166
2019-07-30 08:17:01 +02:00
Balazs Kovacsics
802aa5ae27
HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown
If using the read callback for HTTP_POST, and POSTFIELDSIZE is not set,
automatically add a Transfer-Encoding: chunked header, same as it is
already done for HTTP_PUT, HTTP_POST_FORM and HTTP_POST_MIME.  Update
test 1514 according to the new behaviour.

Closes #4138
2019-07-29 13:18:53 +02:00
Daniel Stenberg
b889408500
curl: support parallel transfers
This is done by making sure each individual transfer is first added to a
linked list as then they can be performed serially, or at will, in
parallel.

Closes #3804
2019-07-20 19:14:16 +02:00
Daniel Stenberg
647e726d78
test1173: make it also check all libcurl option man pages
... and adjust those that cause errors

Closes #4116
2019-07-16 12:38:42 +02:00
Daniel Stenberg
7fb66c4034
test1173: detect some basic man page format mistakes
Triggered by PR #4111

Closes #4113
2019-07-15 15:23:24 +02:00
Linos Giannopoulos
797e549d0d
libcurl: Add testcase for gopher redirects
The testcase ensures that redirects to CURLPROTO_GOPHER won't be
allowed, by default, in the future. Also, curl is being used
for convenience while keeping the testcases DRY.

The expected error code is CURLE_UNSUPPORTED_PROTOCOL when the client is
redirected to CURLPROTO_GOPHER

Signed-off-by: Linos Giannopoulos <lgian@skroutz.gr>
2019-07-14 16:30:28 +02:00
Peter Simonyi
855887af79
http: allow overriding timecond with custom header
With CURLOPT_TIMECONDITION set, a header is automatically added (e.g.
If-Modified-Since).  Allow this to be replaced or suppressed with
CURLOPT_HTTPHEADER.

Fixes #4103
Closes #4109
2019-07-14 16:17:15 +02:00
Stephan Szabo
35b8bea20f
tests: update fixed IP for hostip/clientip split
These tests give differences for me on linux when using a hostip
pointing to the external ip address for the local machine.

Closes #4070
2019-06-25 08:04:48 +02:00
Daniel Stenberg
1e9769639b
test153: fix content-length to avoid occasional hang
Closes #4065
2019-06-24 08:33:32 +02:00
Daniel Stenberg
cf35bd79bd
test1523: basic test of CURLOPT_LOW_SPEED_LIMIT 2019-06-18 22:34:05 +02:00
Daniel Stenberg
40259ca655
tests: have runtests figure out disabled features
... so that runtests can skip individual test cases that test features
that are explicitly disabled in this build. This new logic is intended
for disabled features that aren't otherwise easily visible through the
curl_version_info() or other API calls.

tests/server/disabled is a newly built executable that will output a
list of disabled features. Outputs nothing for a default build.

Closes #3950
2019-06-17 16:08:00 +02:00
Daniel Stenberg
13d9bb8bad
test188/189: fix Content-Length
This cures the flaky test results

Closes #4034
2019-06-17 16:01:55 +02:00
Daniel Stenberg
29177f422a
test1165: verify that CURL_DISABLE_ symbols are in sync
between configure.ac and source code. They should be possible to switch
on/off in configure AND be used in source code.
2019-06-11 15:33:58 +02:00
Daniel Stenberg
b6b8f3a384
test334: verify HTTP 204 response with chunked coding header
Verifies that a bodyless response don't parse this content-related
header.
2019-06-02 22:58:11 +02:00
Michael Kaufmann
2e5ceb3934
http: don't parse body-related headers bodyless responses
Responses with status codes 1xx, 204 or 304 don't have a response body. For
these, don't parse these headers:

- Content-Encoding
- Content-Length
- Content-Range
- Last-Modified
- Transfer-Encoding

This change ensures that HTTP/2 upgrades work even if a
"Content-Length: 0" or a "Transfer-Encoding: chunked" header is present.

Co-authored-by: Daniel Stenberg
Closes #3702
Fixes #3968
Closes #3977
2019-06-02 22:58:04 +02:00
Marcel Raad
13df26192e
HAProxy tests: add keywords
Add the proxy and haproxy keywords in order to be able to exclude or
run these specific tests.

Closes https://github.com/curl/curl/pull/3949
2019-05-27 14:48:37 +02:00
Maksim Stsepanenka
1b260757b8
tests: make test 1420 and 1406 work with rtsp-disabled libcurl
Closes #3948
2019-05-27 11:34:06 +02:00
Jay Satiro
db8ec1fa38
Revert all SASL authzid (new feature) commits
- Revert all commits related to the SASL authzid feature since the next
  release will be a patch release, 7.65.1.

Prior to this change CURLOPT_SASL_AUTHZID  / --sasl-authzid was destined
for the next release, assuming it would be a feature release 7.66.0.
However instead the next release will be a patch release, 7.65.1 and
will not contain any new features.

After the patch release after the reverted commits can be restored by
using cherry-pick:

git cherry-pick a14d72c a9499ff 8c1cc36 c2a8d52 0edf690

Details for all reverted commits:

Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()."

This reverts commit 0edf6907ae.

Revert "tests: Fix the line endings for the SASL alt-auth tests"

This reverts commit c2a8d52a13.

Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples"

This reverts commit 8c1cc369d0.

Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool"

This reverts commit a9499ff136.

Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID"

This reverts commit a14d72ca2f.
2019-05-25 23:36:11 +02:00
Jay Satiro
c2a8d52a13 tests: Fix the line endings for the SASL alt-auth tests
- Change data and protocol sections to CRLF line endings.

Prior to this change the tests would fail or hang, which is because
certain sections such as protocol require CRLF line endings.

Follow-up to a9499ff from today which added the tests.

Ref: https://github.com/curl/curl/pull/3790
2019-05-23 04:01:09 -04:00
Steve Holme
a9499ff136
curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool 2019-05-22 22:55:11 +01:00
Daniel Stenberg
31b77c1877
curl: report error for "--no-" on non-boolean options
Reported-by: Olen Andoni
Fixes #3906
Closes #3907
2019-05-20 19:04:54 +02:00
Daniel Stenberg
f943347496
test332: verify the blksize fix 2019-05-20 07:59:11 +02:00
Daniel Stenberg
697b1f911b
curl: make code work with protocol-disabled libcurl
Closes #3844
2019-05-18 16:14:10 +02:00
Daniel Stenberg
ee68bbe29c
parse_proxy: use the URL parser API
As we treat a given proxy as a URL we should use the unified URL parser
to extract the parts out of it.

Closes #3878
2019-05-15 12:02:05 +02:00
Jay Satiro
d934059afd Revert "multi: support verbose conncache closure handle"
This reverts commit b0972bc.

- No longer show verbose output for the conncache closure handle.

The offending commit was added so that the conncache closure handle
would inherit verbose mode from the user's easy handle. (Note there is
no way for the user to set options for the closure handle which is why
that was necessary.) Other debug settings such as the debug function
were not also inherited since we determined that could lead to crashes
if the user's per-handle private data was used on an unexpected handle.

The reporter here says he has a debug function to capture the verbose
output, and does not expect or want any output to stderr; however
because the conncache closure handle does not inherit the debug function
the verbose output for that handle does go to stderr.

There are other plausible scenarios as well such as the user redirects
stderr on their handle, which is also not inherited since it could lead
to crashes when used on an unexpected handle.

Short of allowing the user to set options for the conncache closure
handle I don't think there's much we can safely do except no longer
inherit the verbose setting.

Bug: https://curl.haxx.se/mail/lib-2019-05/0021.html
Reported-by: Kristoffer Gleditsch

Ref: https://github.com/curl/curl/pull/3598
Ref: https://github.com/curl/curl/pull/3618

Closes https://github.com/curl/curl/pull/3856
2019-05-10 15:01:57 -04:00
Daniel Gustafsson
d490d207c7 test2100: Fix typos in test description 2019-05-06 20:12:39 +03:00
Daniel Stenberg
2d0e9b40d3
urlapi: add CURLUPART_ZONEID to set and get
The zoneid can be used with IPv6 numerical addresses.

Updated test 1560 to verify.

Closes #3834
2019-05-05 15:52:46 +02:00
Daniel Stenberg
bdb2dbc103
urlapi: strip off scope id from numerical IPv6 addresses
... to make the host name "usable". Store the scope id and put it back
when extracting a URL out of it.

Also makes curl_url_set() syntax check CURLUPART_HOST.

Fixes #3817
Closes #3822
2019-05-03 12:17:22 +02:00
Daniel Stenberg
5fc28510a4
CURL_MAX_INPUT_LENGTH: largest acceptable string input size
This limits all accepted input strings passed to libcurl to be less than
CURL_MAX_INPUT_LENGTH (8000000) bytes, for these API calls:
curl_easy_setopt() and curl_url_set().

The 8000000 number is arbitrary picked and is meant to detect mistakes
or abuse, not to limit actual practical use cases. By limiting the
acceptable string lengths we also reduce the risk of integer overflows
all over.

NOTE: This does not apply to `CURLOPT_POSTFIELDS`.

Test 1559 verifies.

Closes #3805
2019-04-29 08:02:44 +02:00
Steve Holme
eb84ca3ea8 sasl: Don't send authcid as authzid for the PLAIN mechanism as per RFC 4616
RFC 4616 specifies the authzid is optional in the client authentication
message and that the server will derive the authorisation identity
(authzid) from the authentication identity (authcid) when not specified
by the client.
2019-04-22 12:29:49 +01:00
Daniel Stenberg
c8bbfb90ea
test 196,197,198: add 'retry' keyword [skip ci] 2019-04-21 23:28:47 +02:00
Daniel Stenberg
e04e974aed
test1002: correct the name [skip ci] 2019-04-20 12:44:31 +02:00
Daniel Stenberg
4225e5db1b
test660: verify CONNECT_ONLY with IMAP
which basically just makes sure LOGOUT is *not* issued on disconnect
2019-04-20 12:40:53 +02:00
Daniel Stenberg
54c784556e
test: make tests and test scripts use socksd for SOCKS
Make all SOCKS tests use socksd instead of ssh.
2019-04-13 11:21:53 +02:00
Daniel Stenberg
04fd67555c
socksd: new SOCKS 4+5 server for tests
Closes #3752
2019-04-13 11:21:35 +02:00
Daniel Stenberg
1489d1db85
test1906: verify CURLOPT_CURLU + CURLOPT_PORT usage
Based-on-code-by: Poul T Lomholt
2019-04-11 22:45:13 +02:00
Daniel Stenberg
2f44e94efb
pipelining: removed
As previously planned and documented in DEPRECATE.md, all pipelining
code is removed.

Closes #3651
2019-04-06 22:49:50 +02:00
Mert Yazıcıoğlu
6227e2bd07 vauth/oauth2: Fix OAUTHBEARER token generation
OAUTHBEARER tokens were incorrectly generated in a format similar to
XOAUTH2 tokens. These changes make OAUTHBEARER tokens conform to the
RFC7628.

Fixes: #2487
Reported-by: Paolo Mossino

Closes https://github.com/curl/curl/pull/3377
2019-04-02 15:52:12 -04:00
Dan Fandrich
27fb521df5 tests: Fixed XML validation errors in some test files. 2019-03-24 18:35:02 +01:00
Dan Fandrich
48287c9c96 tests: Fix some incorrect precheck error messages.
[ci skip]
2019-03-24 13:02:05 +01:00
Daniel Stenberg
dc5edf9124
test1541: threaded connection sharing
The threaded-shared-conn.c example turned into test case. Only works if
pthread was detected.

An attempt to detect future regressions such as e3a53e3efb

Closes #3687
2019-03-18 13:58:46 +01:00
Dominik Hölzl
6c60355323
Negotiate: fix for HTTP POST with Negotiate
* Adjusted unit tests 2056, 2057
* do not generally close connections with CURLAUTH_NEGOTIATE after every request
* moved negotiatedata from UrlState to connectdata
* Added stream rewind logic for CURLAUTH_NEGOTIATE
* introduced negotiatedata::GSS_AUTHDONE and negotiatedata::GSS_AUTHSUCC
* Consider authproblem state for CURLAUTH_NEGOTIATE
* Consider reuse_forbid for CURLAUTH_NEGOTIATE
* moved and adjusted negotiate authentication state handling from
  output_auth_headers into Curl_output_negotiate
* Curl_output_negotiate: ensure auth done is always set
* Curl_output_negotiate: Set auth done also if result code is
  GSS_S_CONTINUE_NEEDED/SEC_I_CONTINUE_NEEDED as this result code may
  also indicate the last challenge request (only works with disabled
  Expect: 100-continue and CURLOPT_KEEP_SENDING_ON_ERROR -> 1)
* Consider "Persistent-Auth" header, detect if not present;
  Reset/Cleanup negotiate after authentication if no persistent
  authentication
* apply changes introduced with #2546 for negotiate rewind logic

Fixes #1261
Closes #1975
2019-03-14 09:26:03 +01:00
Marc Schlatter
dd8a19f8a0
http: send payload when (proxy) authentication is done
The check that prevents payload from sending in case of authentication
doesn't check properly if the authentication is done or not.

They're cases where the proxy respond "200 OK" before sending
authentication challenge. This change takes care of that.

Fixes #2431
Closes #3669
2019-03-13 10:58:52 +01:00