1
0
mirror of https://github.com/moparisthebest/curl synced 2024-12-25 09:38:54 -05:00
Commit Graph

2032 Commits

Author SHA1 Message Date
Daniel Stenberg
69b3ff5118
alt-svc: add protocol version selection masking
So that users can mask in/out specific HTTP versions when Alt-Svc is
used.

 - Removed "h2c" and updated test case accordingly
 - Changed how the altsvc struct is laid out
 - Added ifdefs to make the unittest run even in a quiche-tree

Closes #4201
2019-08-08 09:10:29 +02:00
Daniel Stenberg
f933449d3b
CURLINFO_RETRY_AFTER: parse the Retry-After header value
This is only the libcurl part that provides the information. There's no
user of the parsed value. This change includes three new tests for the
parser.

Ref: #3794
2019-08-07 14:45:39 +02:00
Jay Satiro
a44394eac5 tests: Fix the line endings for the SASL alt-auth tests
- Change data and protocol sections to CRLF line endings.

Prior to this change the tests would fail or hang, which is because
certain sections such as protocol require CRLF line endings.

Follow-up to grandparent commit which added the tests.

Ref: https://github.com/curl/curl/issues/3653
Ref: https://github.com/curl/curl/pull/3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:58 -04:00
Steve Holme
1e08a79258 curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool
Ref: https://github.com/curl/curl/issues/3653
Ref: https://github.com/curl/curl/pull/3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:41 -04:00
Daniel Stenberg
02346abc32
curl_multi_poll: a sister to curl_multi_wait() that waits more
Repeatedly we see problems where using curl_multi_wait() is difficult or
just awkward because if it has no file descriptor to wait for
internally, it returns immediately and leaves it to the caller to wait
for a small amount of time in order to avoid occasional busy-looping.

This is often missed or misunderstood, leading to underperforming
applications.

This change introduces curl_multi_poll() as a replacement drop-in
function that accepts the exact same set of arguments. This function
works identically to curl_multi_wait() - EXCEPT - for the case when
there's nothing to wait for internally, as then this function will by
itself wait for a "suitable" short time before it returns. This
effectiely avoids all risks of busy-looping and should also make it less
likely that apps "over-wait".

This also changes the curl tool to use this funtion internally when
doing parallel transfers and changes curl_easy_perform() to use it
internally.

Closes #4163
2019-08-06 09:33:29 +02:00
Daniel Stenberg
a42b0957ab
http09: disable HTTP/0.9 by default in both tool and library
As the plan has been laid out in DEPRECATED. Update docs accordingly and
verify in test 1174. Now requires the option to be set to allow HTTP/0.9
responses.

Closes #4191
2019-08-06 00:08:14 +02:00
Daniel Stenberg
1132f43012
alt-svc: more liberal ALPN name parsing
Allow pretty much anything to be part of the ALPN identifier. In
particular minus, which is used for "h3-20" (in-progress HTTP/3
versions) etc.

Updated test 356.
Closes #4182
2019-08-03 21:54:00 +02:00
Kamil Dudka
4c187043c5 http_negotiate: improve handling of gss_init_sec_context() failures
If HTTPAUTH_GSSNEGOTIATE was used for a POST request and
gss_init_sec_context() failed, the POST request was sent
with empty body.  This commit also restores the original
behavior of `curl --fail --negotiate`, which was changed
by commit 6c60355323.

Add regression tests 2077 and 2078 to cover this.

Fixes #3992
Closes #4171
2019-08-01 16:21:41 +02:00
Daniel Stenberg
db0a0dfb0e
curl: cap the maximum allowed values for retry time arguments
... to avoid integer overflows later when multiplying with 1000 to
convert seconds to milliseconds.

Added test 1269 to verify.

Reported-by: Jason Lee
Closes #4166
2019-07-30 08:17:01 +02:00
Balazs Kovacsics
802aa5ae27
HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown
If using the read callback for HTTP_POST, and POSTFIELDSIZE is not set,
automatically add a Transfer-Encoding: chunked header, same as it is
already done for HTTP_PUT, HTTP_POST_FORM and HTTP_POST_MIME.  Update
test 1514 according to the new behaviour.

Closes #4138
2019-07-29 13:18:53 +02:00
Daniel Stenberg
b889408500
curl: support parallel transfers
This is done by making sure each individual transfer is first added to a
linked list as then they can be performed serially, or at will, in
parallel.

Closes #3804
2019-07-20 19:14:16 +02:00
Daniel Stenberg
647e726d78
test1173: make it also check all libcurl option man pages
... and adjust those that cause errors

Closes #4116
2019-07-16 12:38:42 +02:00
Daniel Stenberg
7fb66c4034
test1173: detect some basic man page format mistakes
Triggered by PR #4111

Closes #4113
2019-07-15 15:23:24 +02:00
Linos Giannopoulos
797e549d0d
libcurl: Add testcase for gopher redirects
The testcase ensures that redirects to CURLPROTO_GOPHER won't be
allowed, by default, in the future. Also, curl is being used
for convenience while keeping the testcases DRY.

The expected error code is CURLE_UNSUPPORTED_PROTOCOL when the client is
redirected to CURLPROTO_GOPHER

Signed-off-by: Linos Giannopoulos <lgian@skroutz.gr>
2019-07-14 16:30:28 +02:00
Peter Simonyi
855887af79
http: allow overriding timecond with custom header
With CURLOPT_TIMECONDITION set, a header is automatically added (e.g.
If-Modified-Since).  Allow this to be replaced or suppressed with
CURLOPT_HTTPHEADER.

Fixes #4103
Closes #4109
2019-07-14 16:17:15 +02:00
Stephan Szabo
35b8bea20f
tests: update fixed IP for hostip/clientip split
These tests give differences for me on linux when using a hostip
pointing to the external ip address for the local machine.

Closes #4070
2019-06-25 08:04:48 +02:00
Daniel Stenberg
1e9769639b
test153: fix content-length to avoid occasional hang
Closes #4065
2019-06-24 08:33:32 +02:00
Daniel Stenberg
cf35bd79bd
test1523: basic test of CURLOPT_LOW_SPEED_LIMIT 2019-06-18 22:34:05 +02:00
Daniel Stenberg
40259ca655
tests: have runtests figure out disabled features
... so that runtests can skip individual test cases that test features
that are explicitly disabled in this build. This new logic is intended
for disabled features that aren't otherwise easily visible through the
curl_version_info() or other API calls.

tests/server/disabled is a newly built executable that will output a
list of disabled features. Outputs nothing for a default build.

Closes #3950
2019-06-17 16:08:00 +02:00
Daniel Stenberg
13d9bb8bad
test188/189: fix Content-Length
This cures the flaky test results

Closes #4034
2019-06-17 16:01:55 +02:00
Daniel Stenberg
29177f422a
test1165: verify that CURL_DISABLE_ symbols are in sync
between configure.ac and source code. They should be possible to switch
on/off in configure AND be used in source code.
2019-06-11 15:33:58 +02:00
Daniel Stenberg
b6b8f3a384
test334: verify HTTP 204 response with chunked coding header
Verifies that a bodyless response don't parse this content-related
header.
2019-06-02 22:58:11 +02:00
Michael Kaufmann
2e5ceb3934
http: don't parse body-related headers bodyless responses
Responses with status codes 1xx, 204 or 304 don't have a response body. For
these, don't parse these headers:

- Content-Encoding
- Content-Length
- Content-Range
- Last-Modified
- Transfer-Encoding

This change ensures that HTTP/2 upgrades work even if a
"Content-Length: 0" or a "Transfer-Encoding: chunked" header is present.

Co-authored-by: Daniel Stenberg
Closes #3702
Fixes #3968
Closes #3977
2019-06-02 22:58:04 +02:00
Marcel Raad
13df26192e
HAProxy tests: add keywords
Add the proxy and haproxy keywords in order to be able to exclude or
run these specific tests.

Closes https://github.com/curl/curl/pull/3949
2019-05-27 14:48:37 +02:00
Maksim Stsepanenka
1b260757b8
tests: make test 1420 and 1406 work with rtsp-disabled libcurl
Closes #3948
2019-05-27 11:34:06 +02:00
Jay Satiro
db8ec1fa38
Revert all SASL authzid (new feature) commits
- Revert all commits related to the SASL authzid feature since the next
  release will be a patch release, 7.65.1.

Prior to this change CURLOPT_SASL_AUTHZID  / --sasl-authzid was destined
for the next release, assuming it would be a feature release 7.66.0.
However instead the next release will be a patch release, 7.65.1 and
will not contain any new features.

After the patch release after the reverted commits can be restored by
using cherry-pick:

git cherry-pick a14d72c a9499ff 8c1cc36 c2a8d52 0edf690

Details for all reverted commits:

Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()."

This reverts commit 0edf6907ae.

Revert "tests: Fix the line endings for the SASL alt-auth tests"

This reverts commit c2a8d52a13.

Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples"

This reverts commit 8c1cc369d0.

Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool"

This reverts commit a9499ff136.

Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID"

This reverts commit a14d72ca2f.
2019-05-25 23:36:11 +02:00
Jay Satiro
c2a8d52a13 tests: Fix the line endings for the SASL alt-auth tests
- Change data and protocol sections to CRLF line endings.

Prior to this change the tests would fail or hang, which is because
certain sections such as protocol require CRLF line endings.

Follow-up to a9499ff from today which added the tests.

Ref: https://github.com/curl/curl/pull/3790
2019-05-23 04:01:09 -04:00
Steve Holme
a9499ff136
curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool 2019-05-22 22:55:11 +01:00
Daniel Stenberg
31b77c1877
curl: report error for "--no-" on non-boolean options
Reported-by: Olen Andoni
Fixes #3906
Closes #3907
2019-05-20 19:04:54 +02:00
Daniel Stenberg
f943347496
test332: verify the blksize fix 2019-05-20 07:59:11 +02:00
Daniel Stenberg
697b1f911b
curl: make code work with protocol-disabled libcurl
Closes #3844
2019-05-18 16:14:10 +02:00
Daniel Stenberg
ee68bbe29c
parse_proxy: use the URL parser API
As we treat a given proxy as a URL we should use the unified URL parser
to extract the parts out of it.

Closes #3878
2019-05-15 12:02:05 +02:00
Jay Satiro
d934059afd Revert "multi: support verbose conncache closure handle"
This reverts commit b0972bc.

- No longer show verbose output for the conncache closure handle.

The offending commit was added so that the conncache closure handle
would inherit verbose mode from the user's easy handle. (Note there is
no way for the user to set options for the closure handle which is why
that was necessary.) Other debug settings such as the debug function
were not also inherited since we determined that could lead to crashes
if the user's per-handle private data was used on an unexpected handle.

The reporter here says he has a debug function to capture the verbose
output, and does not expect or want any output to stderr; however
because the conncache closure handle does not inherit the debug function
the verbose output for that handle does go to stderr.

There are other plausible scenarios as well such as the user redirects
stderr on their handle, which is also not inherited since it could lead
to crashes when used on an unexpected handle.

Short of allowing the user to set options for the conncache closure
handle I don't think there's much we can safely do except no longer
inherit the verbose setting.

Bug: https://curl.haxx.se/mail/lib-2019-05/0021.html
Reported-by: Kristoffer Gleditsch

Ref: https://github.com/curl/curl/pull/3598
Ref: https://github.com/curl/curl/pull/3618

Closes https://github.com/curl/curl/pull/3856
2019-05-10 15:01:57 -04:00
Daniel Gustafsson
d490d207c7 test2100: Fix typos in test description 2019-05-06 20:12:39 +03:00
Daniel Stenberg
2d0e9b40d3
urlapi: add CURLUPART_ZONEID to set and get
The zoneid can be used with IPv6 numerical addresses.

Updated test 1560 to verify.

Closes #3834
2019-05-05 15:52:46 +02:00
Daniel Stenberg
bdb2dbc103
urlapi: strip off scope id from numerical IPv6 addresses
... to make the host name "usable". Store the scope id and put it back
when extracting a URL out of it.

Also makes curl_url_set() syntax check CURLUPART_HOST.

Fixes #3817
Closes #3822
2019-05-03 12:17:22 +02:00
Daniel Stenberg
5fc28510a4
CURL_MAX_INPUT_LENGTH: largest acceptable string input size
This limits all accepted input strings passed to libcurl to be less than
CURL_MAX_INPUT_LENGTH (8000000) bytes, for these API calls:
curl_easy_setopt() and curl_url_set().

The 8000000 number is arbitrary picked and is meant to detect mistakes
or abuse, not to limit actual practical use cases. By limiting the
acceptable string lengths we also reduce the risk of integer overflows
all over.

NOTE: This does not apply to `CURLOPT_POSTFIELDS`.

Test 1559 verifies.

Closes #3805
2019-04-29 08:02:44 +02:00
Steve Holme
eb84ca3ea8 sasl: Don't send authcid as authzid for the PLAIN mechanism as per RFC 4616
RFC 4616 specifies the authzid is optional in the client authentication
message and that the server will derive the authorisation identity
(authzid) from the authentication identity (authcid) when not specified
by the client.
2019-04-22 12:29:49 +01:00
Daniel Stenberg
c8bbfb90ea
test 196,197,198: add 'retry' keyword [skip ci] 2019-04-21 23:28:47 +02:00
Daniel Stenberg
e04e974aed
test1002: correct the name [skip ci] 2019-04-20 12:44:31 +02:00
Daniel Stenberg
4225e5db1b
test660: verify CONNECT_ONLY with IMAP
which basically just makes sure LOGOUT is *not* issued on disconnect
2019-04-20 12:40:53 +02:00
Daniel Stenberg
54c784556e
test: make tests and test scripts use socksd for SOCKS
Make all SOCKS tests use socksd instead of ssh.
2019-04-13 11:21:53 +02:00
Daniel Stenberg
04fd67555c
socksd: new SOCKS 4+5 server for tests
Closes #3752
2019-04-13 11:21:35 +02:00
Daniel Stenberg
1489d1db85
test1906: verify CURLOPT_CURLU + CURLOPT_PORT usage
Based-on-code-by: Poul T Lomholt
2019-04-11 22:45:13 +02:00
Daniel Stenberg
2f44e94efb
pipelining: removed
As previously planned and documented in DEPRECATE.md, all pipelining
code is removed.

Closes #3651
2019-04-06 22:49:50 +02:00
Mert Yazıcıoğlu
6227e2bd07 vauth/oauth2: Fix OAUTHBEARER token generation
OAUTHBEARER tokens were incorrectly generated in a format similar to
XOAUTH2 tokens. These changes make OAUTHBEARER tokens conform to the
RFC7628.

Fixes: #2487
Reported-by: Paolo Mossino

Closes https://github.com/curl/curl/pull/3377
2019-04-02 15:52:12 -04:00
Dan Fandrich
27fb521df5 tests: Fixed XML validation errors in some test files. 2019-03-24 18:35:02 +01:00
Dan Fandrich
48287c9c96 tests: Fix some incorrect precheck error messages.
[ci skip]
2019-03-24 13:02:05 +01:00
Daniel Stenberg
dc5edf9124
test1541: threaded connection sharing
The threaded-shared-conn.c example turned into test case. Only works if
pthread was detected.

An attempt to detect future regressions such as e3a53e3efb

Closes #3687
2019-03-18 13:58:46 +01:00
Dominik Hölzl
6c60355323
Negotiate: fix for HTTP POST with Negotiate
* Adjusted unit tests 2056, 2057
* do not generally close connections with CURLAUTH_NEGOTIATE after every request
* moved negotiatedata from UrlState to connectdata
* Added stream rewind logic for CURLAUTH_NEGOTIATE
* introduced negotiatedata::GSS_AUTHDONE and negotiatedata::GSS_AUTHSUCC
* Consider authproblem state for CURLAUTH_NEGOTIATE
* Consider reuse_forbid for CURLAUTH_NEGOTIATE
* moved and adjusted negotiate authentication state handling from
  output_auth_headers into Curl_output_negotiate
* Curl_output_negotiate: ensure auth done is always set
* Curl_output_negotiate: Set auth done also if result code is
  GSS_S_CONTINUE_NEEDED/SEC_I_CONTINUE_NEEDED as this result code may
  also indicate the last challenge request (only works with disabled
  Expect: 100-continue and CURLOPT_KEEP_SENDING_ON_ERROR -> 1)
* Consider "Persistent-Auth" header, detect if not present;
  Reset/Cleanup negotiate after authentication if no persistent
  authentication
* apply changes introduced with #2546 for negotiate rewind logic

Fixes #1261
Closes #1975
2019-03-14 09:26:03 +01:00
Marc Schlatter
dd8a19f8a0
http: send payload when (proxy) authentication is done
The check that prevents payload from sending in case of authentication
doesn't check properly if the authentication is done or not.

They're cases where the proxy respond "200 OK" before sending
authentication challenge. This change takes care of that.

Fixes #2431
Closes #3669
2019-03-13 10:58:52 +01:00
Daniel Stenberg
ff7e5a29c7
test331: verify set-cookie for dotless host name
Reproduced bug #3649
Closes #3659
2019-03-09 12:59:20 +01:00
Daniel Stenberg
299d9660f8
Revert "cookies: extend domain checks to non psl builds"
This reverts commit 3773de378d.

Regression shipped in 7.64.0
Fixes #3649
2019-03-09 12:59:20 +01:00
Jay Satiro
b0972bc8e1 multi: support verbose conncache closure handle
- Change closure handle to receive verbose setting from the easy handle
  most recently added via curl_multi_add_handle.

The closure handle is a special easy handle used for closing cached
connections. It receives limited settings from the easy handle most
recently added to the multi handle. Prior to this change that did not
include verbose which was a problem because on connection shutdown
verbose mode was not acknowledged.

Ref: https://github.com/curl/curl/pull/3598

Co-authored-by: Daniel Stenberg

Closes https://github.com/curl/curl/pull/3618
2019-03-05 00:00:01 -05:00
Daniel Stenberg
15401fa886
CURLU: fix NULL dereference when used over proxy
Test 659 verifies

Also fixed the test 658 name

Closes #3641
2019-03-04 16:24:08 +01:00
Daniel Stenberg
5345b04a43
alt-svc: add test 355 and 356 to verify with command line curl 2019-03-03 11:17:52 +01:00
Daniel Stenberg
e1be825453
alt-svc: the libcurl bits 2019-03-03 11:17:52 +01:00
Daniel Stenberg
64d598d91d
test578: make it read data from the correct test 2019-02-28 20:46:45 +01:00
Daniel Stenberg
eb43338941
cookies: only save the cookie file if the engine is enabled
Follow-up to 8eddb8f425.

If the cookieinfo pointer is NULL there really is nothing to save.

Without this fix, we got a problem when a handle was using shared object
with cookies and is told to "FLUSH" it to file (which worked) and then
the share object was removed and when the easy handle was closed just
afterwards it has no cookieinfo and no cookies so it decided to save an
empty jar (overwriting the file just flushed).

Test 1905 now verifies that this works.

Assisted-by: Michael Wallner
Assisted-by: Marcel Raad

Closes #3621
2019-02-27 21:59:40 +01:00
Dan Fandrich
87ba80a6df tests: Fixed XML validation errors in some test files. 2019-02-21 22:10:48 +01:00
Daniel Stenberg
c543da9a50
curl: remove MANUAL from -M output
... and remove it from the dist tarball. It has served its time, it
barely gets updated anymore and "everything curl" is now convering all
this document once tried to include, and does it more and better.

In the compressed scenario, this removes ~15K data from the binary,
which is 25% of the -M output.

It remains in the git repo for now for as long as the web site builds a
page using that as source. It renders poorly on the site (especially for
mobile users) so its not even good there.

Closes #3587
2019-02-20 08:20:27 +01:00
Daniel Stenberg
b08898fb29
connection: never reuse CONNECT_ONLY conections
and make CONNECT_ONLY conections never reuse any existing ones either.

Reported-by: Pavel Löbl
Bug: https://curl.haxx.se/mail/lib-2019-02/0064.html
Closes #3586
2019-02-19 22:50:02 +01:00
Daniel Gustafsson
e6522522f9 cookie: Add support for cookie prefixes
The draft-ietf-httpbis-rfc6265bis-02 draft, specify a set of prefixes
and how they should affect cookie initialization, which has been
adopted by the major browsers. This adds support for the two prefixes
defined, __Host- and __Secure, and updates the testcase with the
supplied examples from the draft.

Closes #3554
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2019-02-17 00:09:30 +01:00
Marcel Raad
71786c0505
unit1307: require FTP support
This test doesn't link without FTP support after
fc7ab4835b, which made Curl_fnmatch
unavailable without FTP support.

Closes https://github.com/curl/curl/pull/3565
2019-02-13 08:42:11 +01:00
Frank Gevaerts
35a8fd2886
tests: add stderr comparison to the test suite
The code is more or less copied from the stdout comparison code, maybe
some better reuse is possible.

test 1457 is adjusted to make the output actually match (by using --silent)
test 506 used <stderr> without actually needing it, so that <stderr> block is removed

Closes #3536
2019-02-11 23:00:50 +01:00
Patrick Monnerat
cac0e4a6ad cli tool: do not use mime.h private structures.
Option -F generates an intermediate representation of the mime structure
that is used later to create the libcurl mime structure and generate
the --libcurl statements.

Reported-by: Daniel Stenberg
Fixes #3532
Closes #3546
2019-02-11 19:10:41 +01:00
Daniel Stenberg
ea77fec16f
tests: move objnames-* from lib into tests
Since they're used purely for testing purposes, I think they should
rather be stored there.

Closes #3470
2019-01-15 08:09:34 +01:00
Daniel Stenberg
f6bb05ccbd
test1558: verify CURLINFO_PROTOCOL on file:// transfer
Attempt to reproduce issue #3444.

Closes #3447
2019-01-10 11:22:48 +01:00
Daniel Stenberg
98e6629154
xattr: strip credentials from any URL that is stored
Both user and password are cleared uncondtitionally.

Added unit test 1621 to verify.

Fixes #3423
Closes #3433
2019-01-10 10:49:51 +01:00
Daniel Stenberg
afeb8d9902
cookies: allow secure override when done over HTTPS
Added test 1562 to verify.

Reported-by: Jeroen Ooms
Fixes #3445
Closes #3450
2019-01-10 10:31:06 +01:00
Daniel Stenberg
5f5b5afcb7
test1561: improve test name
[skip ci]
2019-01-09 15:34:36 +01:00
Katsuhiko YOSHIDA
1f30dc886d
cookies: skip custom cookies when redirecting cross-site
Closes #3417
2019-01-09 15:18:08 +01:00
Bernhard M. Wiedemann
114a02c75b
tests: allow tests to pass by 2037-02-12
similar to commit f508d29f39

Closes #3443
2019-01-08 10:36:49 +01:00
Brad Spencer
84a30d0a41
curl_multi_remove_handle() don't block terminating c-ares requests
Added Curl_resolver_kill() for all three resolver modes, which only
blocks when necessary, along with test 1592 to confirm
curl_multi_remove_handle() doesn't block unless it must.

Closes #3428
Fixes #3371
2019-01-07 10:05:20 +01:00
Markus Moeller
7bcca93459 ntlm: update selection of type 3 response
NTLM2 did not work i.e. no NTLMv2 response was created. Changing the
check seems to work.

Ref: https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-NLMP/[MS-NLMP].pdf

Fixes https://github.com/curl/curl/issues/3286
Closes https://github.com/curl/curl/pull/3287
Closes https://github.com/curl/curl/pull/3415
2019-01-01 11:47:17 +01:00
Claes Jakobsson
d8cae791f4 hostip: support wildcard hosts
This adds support for wildcard hosts in CURLOPT_RESOLVE. These are
try-last so any non-wildcard entry is resolved first. If specified,
any host not matched by another CURLOPT_RESOLVE config will use this
as fallback.

Example send a.com to 10.0.0.1 and everything else to 10.0.0.2:
  curl --resolve *:443:10.0.0.2 --resolve a.com:443:10.0.0.1 \
       https://a.com https://b.com

This is probably quite similar to using:
  --connect-to a.com:443:10.0.0.1:443 --connect-to :443:10.0.0.2:443

Closes #3406
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2018-12-27 14:23:53 +01:00
Daniel Stenberg
006ff62d8c
http: added options for allowing HTTP/0.9 responses
Added CURLOPT_HTTP09_ALLOWED and --http0.9 for this purpose.

For now, both the tool and library allow HTTP/0.9 by default.
docs/DEPRECATE.md lays out the plan for when to reverse that default: 6
months after the 7.64.0 release. The options are added already now so
that applications/scripts can start using them already now.

Fixes #2873
Closes #3383
2018-12-21 10:49:30 +01:00
Daniel Gustafsson
3773de378d cookies: extend domain checks to non psl builds
Ensure to perform the checks we have to enforce a sane domain in
the cookie request. The check for non-PSL enabled builds is quite
basic but it's better than nothing.

Closes #2964
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2018-12-19 20:59:09 +01:00
Ayoub Boudhar
f464535bfd
http: Implement trailing headers for chunked transfers
This adds the CURLOPT_TRAILERDATA and CURLOPT_TRAILERFUNCTION
options that allow a callback based approach to sending trailing headers
with chunked transfers.

The test server (sws) was updated to take into account the detection of the
end of transfer in the case of trailing headers presence.

Test 1591 checks that trailing headers can be sent using libcurl.

Closes #3350
2018-12-14 10:10:48 +01:00
Leonardo Taccari
3997510eed
test1201: Add a trailing `?' to the selector
This verify that the `?' in the selector is kept as is.

Verifies the fix in #3370
2018-12-13 10:24:01 +01:00
Daniel Gustafsson
7a09b52c98 cookies: leave secure cookies alone
Only allow secure origins to be able to write cookies with the
'secure' flag set. This reduces the risk of non-secure origins
to influence the state of secure origins. This implements IETF
Internet-Draft draft-ietf-httpbis-cookie-alone-01 which updates
RFC6265.

Closes #2956
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2018-12-13 09:57:58 +01:00
Daniel Stenberg
1945460d48
test1519: use lib1518 and test CURLINFO_REDIRECT_URL more 2018-12-12 07:58:54 +01:00
Daniel Stenberg
435402cfe3
Curl_follow: extract the Location: header field unvalidated
... when not actually following the redirect. Otherwise we return error
for this and an application can't extract the value.

Test 1518 added to verify.

Reported-by: Pavel Pavlov
Fixes #3340
Closes #3364
2018-12-12 07:58:47 +01:00
Daniel Gustafsson
e1be2ecba4 tests: add urlapi unittest
This adds a new unittest intended to cover the internal functions in
the urlapi code, starting with parse_port(). In order to avoid name
collisions in debug builds, parse_port() is renamed Curl_parse_port()
since it will be exported.

Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
2018-12-11 15:02:24 +01:00
Jay Satiro
552f0205e6
http: fix HTTP auth to include query in URI
- Include query in the path passed to generate HTTP auth.

Recent changes to use the URL API internally (46e1640, 7.62.0)
inadvertently broke authentication URIs by omitting the query.

Fixes https://github.com/curl/curl/issues/3353
Closes #3356
2018-12-11 13:28:20 +01:00
Daniel Stenberg
4e326f9247
test329: verify cookie max-age=0 immediate expiry 2018-12-09 18:35:02 +01:00
Daniel Stenberg
40ac6f1e50
test328: verify Content-Encoding: none
Because of issue #3315

Closes #3317
2018-11-28 14:13:14 +01:00
Daniel Stenberg
a52e46f390
cookies: create the cookiejar even if no cookies to save
Important for when the file is going to be read again and thus must not
contain old contents!

Adds test 327 to verify.

Reported-by: daboul on github
Fixes #3299
Closes #3300
2018-11-23 14:50:51 +01:00
Tobias Hintze
9944d6ba33
test: update test20/1322 for eglibc bug workaround
The tests 20 and 1322 are using getaddrinfo of libc for resolving. In
eglibc-2.19 there is a memory leakage and invalid free bug which
surfaces in some special circumstances (PF_UNSPEC hint with invalid or
non-existent names). The valgrind runs in testing fail in these
situations.

As the tests 20/1322 are not specific on either protocol (IPv4/IPv6)
this commit changes the hints to IPv4 protocol by passing `--ipv4` flag
on the tests' command line.  This prevents the valgrind failures.
2018-11-22 13:41:35 +01:00
Jim Fuller
5c4fe0d826
setopt: add CURLOPT_CURLU
Allows an application to pass in a pre-parsed URL via a URL handle.

Closes #3227
2018-11-09 15:47:28 +01:00
Frank Gevaerts
e431daf013
curl: add %{stderr} and %{stdout} for --write-out
Closes #3115
2018-11-07 11:09:55 +01:00
Daniel Stenberg
9df8dc101b
url: a short host name + port is not a scheme
The function identifying a leading "scheme" part of the URL considered a
few letters ending with a colon to be a scheme, making something like
"short:80" to become an unknown scheme instead of a short host name and
a port number.

Extended test 1560 to verify.

Also fixed test203 to use file_pwd to make it get the correct path on
windows. Removed test 2070 since it was a duplicate of 203.

Assisted-by: Marcel Raad
Reported-by: Hagai Auro
Fixes #3220
Fixes #3233
Closes #3223
Closes #3235
2018-11-06 19:11:58 +01:00
Daniel Gustafsson
c37b66aaae infof: clearly indicate truncation
The internal buffer in infof() is limited to 2048 bytes of payload plus
an additional byte for NULL termination. Servers with very long error
messages can however cause truncation of the string, which currently
isn't very clear, and leads to badly formatted output.

This appends a "...\n" (or just "..." in case the format didn't with a
newline char) marker to the end of the string to clearly show
that it has been truncated.

Also include a unittest covering infof() to try and catch any bugs
introduced in this quite important function.

Closes #3216
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
2018-11-03 20:54:18 +01:00
Daniel Stenberg
2c5ec339ea
Curl_follow: accept non-supported schemes for "fake" redirects
When not actually following the redirect and the target URL is only
stored for later retrieval, curl always accepted "non-supported"
schemes. This was a regression from 46e164069d.

Reported-by: Brad King
Fixes #3210
Closes #3215
2018-11-02 09:50:44 +01:00
Daniel Stenberg
350306e472
test2080: verify the fix for CVE-2018-16842 2018-10-30 07:47:50 +01:00
Daniel Stenberg
a669e06946
test1651: unit test Curl_extract_certinfo()
The version used for Gskit, NSS, GnuTLS, WolfSSL and schannel.
2018-10-27 15:59:43 +02:00
Daniel Stenberg
e50a2002bd
FILE: fix CURLOPT_NOBODY and CURLOPT_HEADER output
Now FILE transfers send headers to the header callback like HTTP and
other protocols. Also made curl_easy_getinfo(...CURLINFO_PROTOCOL...)
work for FILE in the callbacks.

Makes "curl -i file://.." and "curl -I file://.." work like before
again. Applied the bold header logic to them too.

Regression from c1c2762 (7.61.0)

Reported-by: Shaun Jackman
Fixes #3083
Closes #3101
2018-10-08 08:35:40 +02:00
Daniel Stenberg
791105bc4e
test2100: test DoH using IPv4-only
To make it only send one DoH request and avoid the race condition that
could lead to the requests getting sent in reversed order and thus
making it hard to compare in the test case.

Fixes #3107
Closes #3108
2018-10-06 23:02:26 +02:00
Daniel Stenberg
94ad57b024
test320: strip out more HTML when comparing
To make the test case work with different gnutls-serv versions better.

Reported-by: Kamil Dudka
Fixes #3093
Closes #3094
2018-10-04 23:20:25 +02:00
Marcel Raad
9ba67b5b79
test1299: use single quotes around asterisk
Ref: https://github.com/curl/curl/issues/1751#issuecomment-321522580
2018-10-02 23:35:52 +02:00
Daniel Stenberg
b8ab30d309
test1650: make it depend on http/2
Follow-up to 570008c99d as it gets link errors.

Reported-by: Michael Kaufmann
Closes #3068
2018-10-01 07:56:49 +02:00
Daniel Stenberg
0d850c0dba
test2100: require http2 to run
Reported-by: Marcel Raad
Fixes #3064
Closes #3065
2018-09-29 22:24:04 +02:00
Viktor Szakats
16fefeee1e test320: fix regression in [ci skip]
The value in question is coming directly from `gnutls-serv`, so it cannot
be modified freely.

Reported-by: Marcel Raad
Ref: 6ae6b2a533 (commitcomment-30621004)
2018-09-24 16:27:58 +00:00
Viktor Szakats
6ae6b2a533 update URLs in tests
- and one in docs/MANUAL as well

Closes https://github.com/curl/curl/pull/3038
2018-09-23 22:26:11 +00:00
Viktor Szakats
b801b453af whitespace fixes
- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few indentations/line continuations
  in manual examples

Closes https://github.com/curl/curl/pull/3037
2018-09-23 22:24:02 +00:00
Daniel Stenberg
46e164069d
url: use the URL API internally as well
... to make it a truly unified URL parser.

Closes #3017
2018-09-22 11:58:10 +02:00
Daniel Stenberg
ba782baac3
certs: generate tests certs with sha256 digest algorithm
As OpenSSL 1.1.1 starts to complain and fail on sha1 CAs:

"SSL certificate problem: CA signature digest algorithm too weak"

Closes #3014
2018-09-20 09:06:21 +02:00
Daniel Stenberg
9307c219ad
urlapi: add CURLU_GUESS_SCHEME and fix hostname acceptance
In order for this API to fully work for libcurl itself, it now offers a
CURLU_GUESS_SCHEME flag that makes it "guess" scheme based on the host
name prefix just like libcurl always did. If there's no known prefix, it
will guess "http://".

Separately, it relaxes the check of the host name so that IDN host names
can be passed in as well.

Both these changes are necessary for libcurl itself to use this API.

Assisted-by: Daniel Gustafsson
Closes #3018
2018-09-19 23:21:52 +02:00
Daniel Stenberg
ddb06ffc0f
urlglob: improve error message
to help user understand what the problem is

Reported-by: Daniel Shahaf

Fixes #2763
Closes #2977
2018-09-13 08:04:49 +02:00
Yiming Jing
92f9db1746
tests/certs: rebuild certs with 2048-bit RSA keys
The previous test certificates contained RSA keys of only 1024 bits.
However, RSA claims that 1024-bit RSA keys are likely to become
crackable some time before 2010. The NIST recommends at least 2048-bit
keys for RSA for now.

Better use full 2048 also for testing.

Closes #2973
2018-09-12 16:09:17 +02:00
Daniel Gustafsson
1870fd2832
misc: fix typos in comments
Closes #2963
2018-09-10 08:32:07 +02:00
Marcel Raad
b1fb9e9563
test324: fix after 3f3b26d6fe
The expected error code is now 60. 51 is dead.
2018-09-08 22:30:28 +02:00
Daniel Stenberg
fb30ac5a2d
URL-API
See header file and man pages for API. All documented API details work
and are tested in the 1560 test case.

Closes #2842
2018-09-08 15:36:11 +02:00
Daniel Stenberg
17ca0ccff4
curl_easy_upkeep: removed 'conn' from the name
... including the associated option.

Fixes #2951
Closes #2952
2018-09-07 13:43:26 +02:00
Max Dymond
7b655fcbad
upkeep: add a connection upkeep API: curl_easy_conn_upkeep()
Add functionality so that protocols can do custom keepalive on their
connections, when an external API function is called.

Add docs for the new options in 7.62.0

Closes #1641
2018-09-07 09:45:29 +02:00
Nicklas Avén
6987fcef65
imap: change from "FETCH" to "UID FETCH"
... and add "MAILINDEX".

As described in #2789, this is a suggested solution.  Changing UID=xx to
actually get mail with UID xx and add "MAILINDEX" to get a mail with a
special index in the mail box (old behavior).  So MAILINDEX=1 gives the
first non deleted mail in the mail box.

Fixes #2789
Closes #2815
2018-09-06 10:57:48 +02:00
Daniel Stenberg
f2b1a18975
DOH: add test case 1650 and 2100 2018-09-06 09:17:25 +02:00
Han Han
3f3b26d6fe
ssl: deprecate CURLE_SSL_CACERT in favour of a unified error code
Long live CURLE_PEER_FAILED_VERIFICATION
2018-09-06 08:27:15 +02:00
Jim Fuller
b004a174ea
tests: add unit tests for url.c
Approved-by: Daniel Gustafsson
Closes #2937
2018-09-05 14:30:48 +02:00
Daniel Stenberg
c497472c8b
test1452: mark as flaky
makes it not run in the CI builds

Closes #2941
2018-09-05 11:39:25 +02:00
Daniel Stenberg
f7208df7d9
pipelining: deprecated
Transparently. The related curl_multi_setopt() options all still returns
OK when pipelining is selected.

To re-enable the support, the single line change in lib/multi.c needs to
be reverted.

See docs/DEPRECATE.md

Closes #2705
2018-09-05 11:34:03 +02:00
Marcel Raad
b8b338351c
test1148: fix precheck output
"precheck command error" is not very helpful.
2018-09-02 12:51:00 +02:00
Daniel Gustafsson
e2ef8d6fa1
cookies: support creation-time attribute for cookies
According to RFC6265 section 5.4, cookies with equal path lengths
SHOULD be sorted by creation-time (earlier first). This adds a
creation-time record to the cookie struct in order to make cookie
sorting more deterministic. The creation-time is defined as the
order of the cookies in the jar, the first cookie read fro the
jar being the oldest. The creation-time is thus not serialized
into the jar. Also remove the strcmp() matching in the sorting as
there is no lexicographic ordering in RFC6265. Existing tests are
updated to match.

Closes #2524
2018-08-31 14:11:37 +02:00
Marcel Raad
b842fa3110
Don't use Windows path %PWD for SSH tests
All these tests failed on Windows because something like
sftp://%HOSTIP:%SSHPORT%PWD/
expanded to
sftp://127.0.0.1:1234c:/msys64/home/bla/curl
and then curl complained about the port number ending with a letter.

Use the original POSIX path instead of the Windows path created in
checksystem to fix this.

Closes https://github.com/curl/curl/pull/2920
2018-08-31 09:13:01 +02:00
Leonardo Taccari
8f3c3cd08a
gopher: Do not translate ?' to %09'
Since GOPHER support was added in curl `?' character was automatically
translated to `%09' (`\t').

However, this behaviour does not seems documented in RFC 4266 and for
search selectors it is documented to directly use `%09' in the URL.
Apart that several gopher servers in the current gopherspace have CGI
support where `?' is used as part of the selector and translating it to
`%09' often leads to surprising results.

Closes #2910
2018-08-24 08:57:49 +02:00
Marcel Raad
387e85efd0
cookie tests: treat files as text
Fixes test failures because of wrong line endings on Windows.
2018-08-23 13:11:20 +02:00
Marcel Raad
86b1e94ae7
Tests: fixes for Windows
- test 1268 requires unix sockets
- test 2072 must be disabled also for MSYS/MinGW
2018-08-22 12:26:21 +02:00
Marcel Raad
93b34981fa
test1148: disable if decimal separator is not point
Modifying the locale with environment variables doesn't work for native
Windows applications. Just disable the test in this case if the decimal
separator is something different than a point. Use a precheck with a
small C program to achieve that.

Closes https://github.com/curl/curl/pull/2786
2018-08-21 19:00:48 +02:00
Daniel Stenberg
ba58ce669c
test1268: check the stderr output as "text"
Follow-up to 099f37e9c5

Pointed-out-by: Marcel Raad
2018-08-16 23:06:04 +02:00
Daniel Stenberg
099f37e9c5
curl: warn the user if a given file name looks like an option
... simply because this is usually a sign of the user having omitted the
file name and the next option is instead "eaten" by the parser as a file
name.

Add test1268 to verify

Closes #2885
2018-08-15 23:05:14 +02:00
Kamil Dudka
a4c7911a48 ssh-libssh: fix infinite connect loop on invalid private key
Added test 656 (based on test 604) to verify the fix.

Bug: https://bugzilla.redhat.com/1595135

Closes #2879
2018-08-14 22:08:06 +02:00
Daniel Stenberg
8440616f53
http: fix for tiny "HTTP/0.9" response
Deal with tiny "HTTP/0.9" (header-less) responses by checking the
status-line early, even before a full "HTTP/" is received to allow
detecting 0.9 properly.

Test 1266 and 1267 added to verify.

Fixes #2420
Closes #2872
2018-08-13 23:16:01 +02:00
Marcel Raad
41dabac761
Split non-portable part off test 1133
Split off testing file names with double quotes into new test 1158.
Disable it for MSYS using a precheck as it doesn't support file names
with double quotes (but Cygwin does, for example).

Fixes https://github.com/curl/curl/issues/2796
Closes https://github.com/curl/curl/pull/2854
2018-08-11 23:32:26 +02:00
Daniel Stenberg
2a278fd735
CURLINFO_SIZE_UPLOAD: fix missing counter update
Adds test 1522 for verification.

Reported-by: cjmsoregan
Fixes #2847
Closes #2864
2018-08-11 14:00:59 +02:00
Daniel Stenberg
b7bdf2100e
test1307: disabled
Turns out that since we're using the native fnmatch function now when
available, and they simply disagree on a huge number of test patterns
that make it hard to test this function like this...

Fixes #2825
2018-08-02 00:57:01 +02:00
Daniel Stenberg
ea6f57696c
test1157: follow-up to 35ecffb9
Ignore the user-agent line.
Pointed-out-by: Marcel Raad
2018-07-29 12:31:09 +02:00
Daniel Stenberg
35ecffb9be
test1157: test -H from empty file
Verifies bugfix #2797
2018-07-28 22:49:00 +02:00
Marcel Raad
26e35844e7
test320: treat curl320.out file as binary
Otherwise, LF line endings are converted to CRLF on Windows,
but no conversion is done for the reply, so the test case fails.

Closes https://github.com/curl/curl/pull/2776
2018-07-22 23:17:06 +02:00
Marcel Raad
6d13432bfa
tests: fixes for Windows line endlings
Set mode="text" when line endings depend on the system representation.

Closes https://github.com/curl/curl/pull/2772
2018-07-21 20:47:42 +02:00
Marcel Raad
1550e844b1
test214: disable MSYS2's POSIX path conversion for URL
By default, the MSYS2 bash converts all backslashes to forward slashes
in URLs. Disable this with MSYS2_ARG_CONV_EXCL for the test to pass.

Ref https://github.com/msys2/msys2/wiki/Porting#filesystem-namespaces
2018-07-21 11:31:03 +02:00
Marcel Raad
7bc1180437
test1143: disable MSYS2's POSIX path conversion
By default, the MSYS2 bash interprets http:/%HOSTIP:%HTTPPORT/want/1143
as a POSIX file list and converts it to a Windows file list.
Disable this with MSYS2_ARG_CONV_EXCL for the test to pass.

Ref https://github.com/msys2/msys2/wiki/Porting#filesystem-namespaces
Closes https://github.com/curl/curl/pull/2765
2018-07-20 16:08:58 +02:00
Marcel Raad
8c00412428
test1422: add required file feature
curl configured with --enable-debug --disable-file currently complains
on test1422:
Info: Protocol "file" not supported or disabled in libcurl

Make test1422 dependend on enabled FILE protocol to fix this.

Fixes https://github.com/curl/curl/issues/2741
Closes https://github.com/curl/curl/pull/2742
2018-07-14 12:56:43 +02:00
Daniel Stenberg
1b76c38904
conn: remove the boolean 'inuse' field
... as the usage needs to be counted.
2018-07-11 23:41:24 +02:00
Javier Blazquez
4c901638b4
multi: fix crash due to dangling entry in connect-pending list
Fixes #2677
Closes #2679
2018-06-23 22:31:52 +02:00
Daniel Stenberg
ac9a179fe9
multi: fix memory leak when stopped during name resolve
When the application just started the transfer and then stops it while
the name resolve in the background thread hasn't completed, we need to
wait for the resolve to complete and then cleanup data accordingly.

Enabled test 1553 again and added test 1590 to also check when the host
name resolves successfully.

Detected by OSS-fuzz.
Closes #1968
2018-06-16 12:01:27 +02:00
Daniel Stenberg
bb9a340c73
runtests: support variables in <strippart>
... and make use of that to make 1455 work better without using a fixed
local port number.

Fixes #2649
Closes #2650
2018-06-12 16:11:21 +02:00
Bernhard M. Wiedemann
f508d29f39
test 46: make test pass after 2025
shifting the expiry date to 2037 for now
to be before the possibly problematic year 2038

similar in spirit to commit e6293cf876

Closes #2646
2018-06-12 00:21:20 +02:00
Viktor Szakats
4bd91bc474 spelling fixes
Detected using the `codespell` tool (version 1.13.0).

Also secure and fix an URL.
2018-06-03 12:14:45 +00:00
Alibek.Jorajev
f66d97b677
CURLOPT_RESOLVE: always purge old entry first
If there's an existing entry using the selected name.

Closes #2622
2018-06-01 12:47:52 +02:00
Björn Stenberg
946ce5b61f
option: disallow username in URL
Adds CURLOPT_DISALLOW_USERNAME_IN_URL and --disallow-username-in-url. Makes
libcurl reject URLs with a username in them.

Closes #2340
2018-05-31 11:27:16 +02:00
Linus Lewandowski
239a7061f8
httpauth: add support for Bearer tokens
Closes #2102
2018-05-24 20:39:49 +02:00
Patrick Monnerat
09d16af49a http resume: skip body if http code 416 (range error) is ignored.
This avoids appending error data to already existing good data.

Test 92 is updated to match this change.
New test 1156 checks all combinations of --range/--resume, --fail,
Content-Range header and http status code 200/416.

Fixes #1163
Reported-By: Ithubg on github
Closes #2578
2018-05-22 13:23:02 +02:00
Daniel Stenberg
c1c27625c7
curl: show headers in bold
The feature is only enabled if the output is believed to be a tty.

-J: There's some minor differences and improvements in -J handling, as
now J should work with -i and it actually creates a file first using the
initial name and then *renames* that to the one found in
Content-Disposition (if any).

-i: only shows headers for HTTP transfers now (as documented).
Previously it would also show for pieces of the transfer that were HTTP
(for example when doing FTP over a HTTP proxy).

-i: now shows trailers as well. Previously they were not shown at all.

--libcurl: the CURLOPT_HEADER is no longer set, as the header output is
now done in the header callback.
2018-05-21 20:05:05 +02:00
Patrick Monnerat
1b55d270ad cookies: do not take cookie name as a parameter
RFC 6265 section 4.2.1 does not set restrictions on cookie names.
This is a follow-up to commit 7f7fcd0.
Also explicitly check proper syntax of cookie name/value pair.

New test 1155 checks that cookie names are not reserved words.

Reported-By: anshnd at github
Fixes #2564
Closes #2566
2018-05-13 01:23:10 +02:00
Daniel Stenberg
c39ed80526
tests: provide 'manual' as a feature to optionally require
... and make test 1026 rely on that feature so that --disable-manual
builds don't cause test failures.

Reported-by: Max Dymond and Anders Roxell
Fixes #2533
Closes #2540
2018-04-29 12:49:38 +02:00
Dan McNulty
8996300211 schannel: add support for CURLOPT_CAINFO
- Move verify_certificate functionality in schannel.c into a new
  file called schannel_verify.c. Additionally, some structure defintions
  from schannel.c have been moved to schannel.h to allow them to be
  used in schannel_verify.c.

- Make verify_certificate functionality for Schannel available on
  all versions of Windows instead of just Windows CE. verify_certificate
  will be invoked on Windows CE or when the user specifies
  CURLOPT_CAINFO and CURLOPT_SSL_VERIFYPEER.

- In verify_certificate, create a custom certificate chain engine that
  exclusively trusts the certificate store backed by the CURLOPT_CAINFO
  file.

- doc updates of --cacert/CAINFO support for schannel

- Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
  when available. This implements a TODO in schannel.c to improve
  handling of multiple SANs in a certificate. In particular, all SANs
  will now be searched instead of just the first name.

- Update tool_operate.c to not search for the curl-ca-bundle.crt file
  when using Schannel to maintain backward compatibility. Previously,
  any curl-ca-bundle.crt file found in that search would have been
  ignored by Schannel. But, with CAINFO support, the file found by
  that search would have been used as the certificate store and
  could cause issues for any users that have curl-ca-bundle.crt in
  the search path.

- Update url.c to not set the build time CURL_CA_BUNDLE if the selected
  SSL backend is Schannel. We allow setting CA location for schannel
  only when explicitly specified by the user via CURLOPT_CAINFO /
  --cacert.

- Add new test cases 3000 and 3001. These test cases check that the first
  and last SAN, respectively, matches the connection hostname. New test
  certificates have been added for these cases. For 3000, the certificate
  prefix is Server-localhost-firstSAN and for 3001, the certificate
  prefix is Server-localhost-secondSAN.

- Remove TODO 15.2 (Add support for custom server certificate
  validation), this commit addresses it.

Closes https://github.com/curl/curl/pull/1325
2018-04-18 03:59:47 -04:00
Daniel Stenberg
6cbe96975b
test1148: tolerate progress updates better
Fixes #2446
Closes #2488
2018-04-16 16:53:46 +02:00
Daniel Stenberg
778235ce21
test1136: fix cookie order after commit c990eadd12 2018-04-09 00:10:20 +02:00
Jon DeVree
695e96b3d5
file: restore old behavior for file:////foo/bar URLs
curl 7.57.0 and up interpret this according to Appendix E.3.2 of RFC
8089 but then returns an error saying this is unimplemented. This is
actually a regression in behavior on both Windows and Unix.

Before curl 7.57.0 this URL was treated as a path of "//foo/bar" and
then passed to the relevant OS API. This means that the behavior of this
case is actually OS dependent.

The Unix path resolution rules say that the OS must handle swallowing
the extra "/" and so this path is the same as "/foo/bar"

The Windows path resolution rules say that this is a UNC path and
automatically handles the SMB access for the program. So curl on Windows
was already doing Appendix E.3.2 without any special code in curl.

Regression

Closes #2438
2018-04-06 14:28:42 +02:00
Lauri Kasanen
746479adcb
cookie: case-insensitive hashing for the domains
closes #2458
2018-04-06 14:13:08 +02:00
Patrick Monnerat
82dfdac5f7 cookie: fix and optimize 2nd top level domain name extraction
This fixes a segfault occurring when a name of the (invalid) form "domain..tld"
is processed.

test46 updated to cover this case.

Follow-up to commit c990ead.

Ref: https://github.com/curl/curl/pull/2440
2018-04-04 15:28:28 +02:00
Lauri Kasanen
c990eadd12
cookie: store cookies per top-level-domain-specific hash table
This makes libcurl handle thousands of cookies much better and speedier.

Closes #2440
2018-04-02 10:48:53 +02:00
Daniel Stenberg
28faaacee2
test1148: set a fixed locale for the test
...as otherwise it might use a different decimal sign.

Bug: #2436
Reported-by: Oumph on github
2018-04-02 10:32:19 +02:00
Rick Deist
d95f3dc0b1
resolve: add CURLOPT_DNS_SHUFFLE_ADDRESSES
This patch adds CURLOPT_DNS_SHUFFLE_ADDRESSES to explicitly request
shuffling of IP addresses returned for a hostname when there is more
than one. This is useful when the application knows that a round robin
approach is appropriate and is willing to accept the consequences of
potentially discarding some preference order returned by the system's
implementation.

Closes #1694
2018-03-17 20:44:14 +01:00
Lawrence Matthews
6baeb6df35
CURLOPT_HAPROXYPROTOCOL: support the HAProxy PROXY protocol
Add --haproxy-protocol for the command line tool

Closes #2162
2018-03-17 11:50:06 +01:00
luz.paz
236402fc2d
cleanup: misc typos in strings and comments
Found via `codespell`

Closes #2389
2018-03-16 11:08:31 +01:00
Daniel Stenberg
a9a7b606c2
transfer: make HTTP without headers count correct body size
This is what "HTTP/0.9" basically looks like.

Reported on IRC

Closes #2382
2018-03-15 10:56:27 +01:00
Daniel Stenberg
634f72fed4
test1208: marked flaky
It fails somewhere between every 3rd to 10th travis-CI run
2018-03-15 08:21:26 +01:00
Daniel Stenberg
535432c0ad
FTP: reject path components with control codes
Refuse to operate when given path components featuring byte values lower
than 32.

Previously, inserting a %00 sequence early in the directory part when
using the 'singlecwd' ftp method could make curl write a zero byte
outside of the allocated buffer.

Test case 340 verifies.

CVE-2018-1000120
Reported-by: Duy Phan Thanh
Bug: https://curl.haxx.se/docs/adv_2018-9cd6.html
2018-03-12 07:47:07 +01:00
Daniel Stenberg
8123560d44
HTTP: allow "header;" to replace an internal header with a blank one
Reported-by: Michael Kaufmann
Fixes #2357
Closes #2362
2018-03-11 11:46:10 +01:00
Daniel Stenberg
b7f90470be
NO_PROXY: fix for IPv6 numericals in the URL
Added test 1265 that verifies.

Reported-by: steelman on github
Fixes #2353
Closes #2355
2018-03-04 19:50:48 +01:00
Francisco Sedano
23713645d4 url: Add option CURLOPT_RESOLVER_START_FUNCTION
- Add new option CURLOPT_RESOLVER_START_FUNCTION to set a callback that
  will be called every time before a new resolve request is started
  (ie before a host is resolved) with a pointer to backend-specific
  resolver data. Currently this is only useful for ares.

- Add new option CURLOPT_RESOLVER_START_DATA to set a user pointer to
  pass to the resolver start callback.

Closes https://github.com/curl/curl/pull/2311
2018-02-21 21:29:10 -05:00
Anders Bakken
50d1b3379a CURLOPT_RESOLVE: Add support for multiple IP addresses per entry
This enables users to preresolve but still take advantage of happy
eyeballs and trying multiple addresses if some are not connecting.

Ref: https://github.com/curl/curl/pull/2260
2018-02-20 04:24:28 -05:00
Daniel Stenberg
70cbb92134
test1556: verify >16KB headers to the header callback 2018-02-16 22:54:50 +01:00
Daniel Stenberg
5e17a5f6cf
test1154: verify that long HTTP headers get rejected 2018-02-16 22:51:59 +01:00
Björn Stenberg
b46cfbc068
TODO fixed: Detect when called from within callbacks
Closes #2302
2018-02-15 09:36:03 +01:00
Patrick Monnerat
e551910f8a tests: new tests for http raw mode
Test 319 checks proper raw mode data with non-chunked gzip
transfer-encoded server data.
Test 326 checks raw mode with chunked server data.

Bug: #2303
Closes #2308
2018-02-13 01:44:57 +01:00
Patrick Monnerat
62cf2d180e smtp: fix processing of initial dot in data
RFC 5321 4.1.1.4 specifies the CRLF terminating the DATA command
should be taken into account when chasing the <CRLF>.<CRLF> end marker.
Thus a leading dot character in data is also subject to escaping.

Tests 911 and test server are adapted to this situation.
New tests 951 and 952 check proper handling of initial dot in data.

Closes #2304
2018-02-12 16:43:15 +01:00
Patrick Monnerat
bd5b9e50ce lib555: drop text conversion and encode data as ascii codes
If CURL_DOES_CONVERSION is enabled, uploaded LFs are mapped to CRLFs,
giving a result that is different from what is expected.
This commit avoids using CURLOPT_TRANSFERTEXT and directly encodes data
to upload in ascii.

Bug: https://github.com/curl/curl/pull/1872
2018-01-26 17:45:20 +01:00
Daniel Stenberg
1c39128d97
parsedate: fix date parsing for systems with 32 bit long
Make curl_getdate() handle dates before 1970 as well (returning negative
values).

Make test 517 test dates for 64 bit time_t.

This fixes bug (3) mentioned in #2238

Closes #2250
2018-01-25 22:20:52 +01:00
Daniel Stenberg
65ceb20dfd
test1454: --connect-to with IPv6 address w/o IPv6 support! 2018-01-23 08:01:29 +01:00
Daniel Stenberg
af32cd3859
http: prevent custom Authorization headers in redirects
... unless CURLOPT_UNRESTRICTED_AUTH is set to allow them. This matches how
curl already handles Authorization headers created internally.

Note: this changes behavior slightly, for the sake of reducing mistakes.

Added test 317 and 318 to verify.

Reported-by: Craig de Stigter
Bug: https://curl.haxx.se/docs/adv_2018-b3bf.html
2018-01-22 10:00:00 +01:00
Daniel Stenberg
87ddeee59b
test558: fix for multissl builds
vtls.c:multissl_init() might do a curl_free() call so strip that out to
make this work with more builds. We just want to verify that
memorytracking works so skipping one line is no harm.
2018-01-18 22:21:33 +01:00
Patrick Monnerat
e44ddfd477 mime: clone mime tree upon easy handle duplication.
A mime tree attached to an easy handle using CURLOPT_MIMEPOST is
strongly bound to the handle: there is a pointer to the easy handle in
each item of the mime tree and following the parent pointer list
of mime items ends in a dummy part stored within the handle.

Because of this binding, a mime tree cannot be shared between different
easy handles, thus it needs to be cloned upon easy handle duplication.

There is no way for the caller to get the duplicated mime tree
handle: it is then set to be automatically destroyed upon freeing the
new easy handle.

New test 654 checks proper mime structure duplication/release.

Add a warning note in curl_mime_data_cb() documentation about sharing
user data between duplicated handles.

Closes #2235
2018-01-14 19:43:12 +01:00
Daniel Stenberg
a06311be27
test395: HTTP with overflow Content-Length value 2018-01-13 22:49:31 +01:00
Daniel Stenberg
67595e7d23
test394: verify abort of rubbish in Content-Length: value 2018-01-13 22:49:31 +01:00
Daniel Stenberg
ac17d79473
test393: verify --max-filesize with excessive Content-Length 2018-01-13 22:49:31 +01:00
Daniel Stenberg
f68e672715
HTTP: bail out on negative Content-Length: values
... and make the max filesize check trigger if the value is too big.

Updates test 178.

Reported-by: Brad Spencer
Fixes #2212
Closes #2223
2018-01-13 22:49:04 +01:00
Mikalai Ananenka
58d7cd28a0 brotli: data at the end of content can be lost
Decoding loop implementation did not concern the case when all
received data is consumed by Brotli decoder and the size of decoded
data internally hold by Brotli decoder is greater than CURL_MAX_WRITE_SIZE.
For content with unencoded length greater than CURL_MAX_WRITE_SIZE this
can result in the loss of data at the end of content.

Closes #2194
2017-12-27 13:00:54 +01:00
Patrick Monnerat
f009bbe1f6 curl_easy_reset: release mime-related data.
Move curl_mime_initpart() and curl_mime_cleanpart() calls to lower-level
functions dealing with UserDefined structure contents.
This avoids memory leakages on curl-generated part mime headers.
New test 2073 checks this using the cli tool --next option: it
triggers a valgrind error if bug is present.

Bug: https://curl.haxx.se/mail/lib-2017-12/0060.html
Reported-by: Martin Galvan
2017-12-20 19:33:50 +01:00
Patrick Monnerat
4acc9d3d1a content_encoding: rework zlib_inflate
- When zlib version is < 1.2.0.4, process gzip trailer before considering
extra data as an error.
- Inflate with Z_BLOCK instead of Z_SYNC_FLUSH to maximize correct data
and minimize corrupt data output.
- Do not try to restart deflate decompression in raw mode if output has
started or if the leading data is not available anymore.
- New test 232 checks inflating raw-deflated content.

Closes #2068
2017-12-20 16:02:42 +01:00
Daniel Stenberg
671f0b506c
Revert "curl: don't set CURLOPT_INTERLEAVEDATA"
This reverts commit 9ffad8eb13.

It was actually added rather recently in 8e8afa82cb due to a crash
that would otherwise happen in the RTSP code. As I don't think we've
fixed that behavior yet, we better keep this work-around until we have
fixed it better.
2017-12-11 00:00:17 +01:00
Michael Kaufmann
4b6f3cff7c tests: mark data files as non-executable in git 2017-12-10 22:36:42 +01:00
Daniel Stenberg
e959f16c5f
multi_done: prune DNS cache
Prune the DNS cache immediately after the dns entry is unlocked in
multi_done. Timed out entries will then get discarded in a more orderly
fashion.

Test506 is updated

Reported-by: Oleg Pudeyev

Fixes #2169
Closes #2170
2017-12-10 14:51:47 +01:00
Daniel Stenberg
9ffad8eb13
curl: don't set CURLOPT_INTERLEAVEDATA
That data is only ever used by the CURLOPT_INTERLEAVEFUNCTION callback
and that option isn't set or used by the curl tool!

Updates the 9 tests that verify --libcurl

Closes #2167
2017-12-08 23:40:54 +01:00
Daniel Stenberg
07cb27c98e
conncache: fix several lock issues
If the lock is released before the dealings with the bundle is over, it may
have changed by another thread in the mean time.

Fixes #2132
Fixes #2151
Closes #2139
2017-12-05 23:21:02 +01:00
Daniel Stenberg
615edc1f73
sasl_getmesssage: make sure we have a long enough string to pass
For pop3/imap/smtp, added test 891 to somewhat verify the pop3
case.

For this, I enhanced the pingpong test server to be able to send back
responses with LF-only instead of always using CRLF.

Closes #2150
2017-12-05 17:05:44 +01:00
Nikos Mavrogiannopoulos
38aef6dc45
libssh2: return CURLE_UPLOAD_FAILED on failure to upload
This brings its in sync with the error code returned by the
libssh backend.

Signed-off-by: Nikos Mavrogiannopoulos <nmav@gnutls.org>
2017-12-01 17:35:15 +01:00
Nikos Mavrogiannopoulos
75427291e5
libssh2: send the correct CURLE error code on scp file not found
That also updates tests to expect the right error code

libssh2 back-end returns CURLE_SSH error if the remote file
is not found. Expect instead CURLE_REMOTE_FILE_NOT_FOUND
which is sent by the libssh backend.

Signed-off-by: Nikos Mavrogiannopoulos <nmav@redhat.com>
2017-12-01 17:35:15 +01:00
Daniel Stenberg
0b664ba968
wildcardmatch: fix heap buffer overflow in setcharset
The code would previous read beyond the end of the pattern string if the
match pattern ends with an open bracket when the default pattern
matching function is used.

Detected by OSS-Fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=4161

CVE-2017-8817

Bug: https://curl.haxx.se/docs/adv_2017-ae72.html
2017-11-27 08:19:34 +01:00