1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-16 06:25:03 -05:00
Commit Graph

301 Commits

Author SHA1 Message Date
Gilles Vollant
148534db57
CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl)
Closes #4346
2020-05-08 15:55:04 +02:00
Kwon-Young Choi
a448a4ce26
curl: allow both --etag-compare and --etag-save with same file name
This change inverse the order of processing for the --etag-compare and
--etag-save option to process first --etag-compare. This in turn allows
to use the same file name to compare and save an etag.

The original behavior of not failing if the etag file does not exists is
conserved.

Fixes #5179
Closes #5180
2020-04-05 01:07:52 +02:00
Harry Sintonen
7a71965e97
build: fixed build for systems with select() in unistd.h
Closes #5169
2020-03-31 10:59:06 +02:00
Johannes Schindelin
5450428491 schannel: add "best effort" revocation check option
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
  --ssl-revoke-best-effort to allow a "best effort" revocation check.

A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.

---

When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:

	Unknown error (0x80092012) - The revocation function was unable
	to check revocation for the certificate.

Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.

This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.

In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.

Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).

To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.

Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.

The test 2070 was added to verify this behavior, adapted from 310.

Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Closes https://github.com/curl/curl/pull/4981
2020-03-18 03:23:39 -04:00
Daniel Stenberg
95c36fff75
tool_operate: fix add_parallel_transfers when more are in queue
Trying to return early from the function if no new transfers were added
would break the "morep" argument and cause issues. This could lead to
zero content "transfers" (within quotes since they would never be
started) when parallel-max was reduced.

Reported-by: Gavin Wong
Analyzed-by: Jay Satiro
Fixes #4937
Closes #5112
2020-03-16 09:51:34 +01:00
Daniel Stenberg
1cc97ba6e4
curl: error on --alt-svc use w/o support
Make the tool check for alt-svc support at run-time and return error
accordingly if not present when the option is used.

Reported-by: Harry Sintonen
Closes #4878
2020-02-05 07:57:38 +01:00
Pavel Volgarev
4a4609bf3c
smtp: Allow RCPT TO command to fail for some recipients
Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS.

Verified with the new tests 3002-3007

Closes #4816
2020-01-21 10:40:19 +01:00
Emil Engler
1774dbd74c
curl: Let -D merge headers in one file again
Closes #4762
Fixes #4753
2020-01-21 09:23:30 +01:00
Daniel Stenberg
ad0aa27a9d
curl: remove 'config' field from OutStruct
As it was just unnecessary duplicated information already stored in the
'per_transfer' struct and that's around mostly anyway.

The duplicated pointer caused problems when the code flow was aborted
before the dupe was filled in and could cause a NULL pointer access.

Reported-by: Brian Carpenter
Fixes #4807
Closes #4810
2020-01-12 17:17:44 +01:00
Daniel Stenberg
392bff4abf
curl: properly free mimepost data
... as it could otherwise leak memory when a transfer failed.

Added test 1293 to verify.

Reported-by: Brian Carpenter
Fixes #4781
Closes #4782
2020-01-04 23:06:44 +01:00
Daniel Stenberg
271ec6b9b6
curl: cleanup multi handle on failure
... to fix memory leak in error path.

Fixes #4772
Closes #4780
Reported-by: Brian Carpenter
2020-01-04 23:05:21 +01:00
Jay Satiro
249f7b37e6 tool_operate: fix mem leak when failed config parse
Found by fuzzing the config file.

Reported-by: Geeknik Labs

Fixes https://github.com/curl/curl/issues/4767
2019-12-29 14:18:48 -05:00
Daniel Stenberg
70a6541515
curl: fix memory leak in OOM in etags logic
Detected by torture tests

Closes #4706
2019-12-12 11:03:55 +01:00
Daniel Stenberg
c6deecd7e9
curl: use errorf() better
Change series of error outputs to use errorf().

Only errors that are due to mistakes in command line option usage should
use helpf(), other types of errors in the tool should rather use
errorf().

Closes #4691
2019-12-10 10:03:33 +01:00
Daniel Stenberg
147fa06891
curl: make the etag load logic work without fseek
The fseek()s were unnecessary and caused Coverity warning CID 1456554

Closes #4681
2019-12-06 23:02:24 +01:00
Daniel Stenberg
9ea769e15f
etag: allow both --etag-compare and --etag-save in same cmdline
Fixes #4669
Closes #4678
2019-12-05 22:57:24 +01:00
Daniel Stenberg
7dffc2b46f
curl: show better error message when no homedir is found
Reported-by: Vlastimil Ovčáčík
Fixes #4644
Closes #4665
2019-12-03 16:26:02 +01:00
Maros Priputen
18e5cb77e9
curl: two new command line options for etags
--etag-compare and --etag-save

Suggested-by: Paul Hoffman
Fixes #4277
Closes #4543
2019-11-28 13:05:20 +01:00
John Schroeder
9a2cbf30b8
curl: fix --upload-file . hangs if delay in STDIN
Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION.

When uploading from stdin in non-blocking mode, a delay in reading
the stream (EAGAIN) causes curl to pause sending data
(CURL_READFUNC_PAUSE).  Prior to this change, a busy read was
detected and unpaused only in the CURLOPT_WRITEFUNCTION handler.
This change performs the same busy read handling in a
CURLOPT_XFERINFOFUNCTION handler.

Fixes #2051
Closes #4599
Reported-by: bdry on github
2019-11-26 09:17:52 +01:00
Daniel Stenberg
215baa74f7
curl: add --parallel-immediate
Starting with this change when doing parallel transfers, without this
option set, curl will prefer to create new transfers multiplexed on an
existing connection rather than creating a brand new one.

--parallel-immediate can be set to tell curl to prefer to use new
connections rather than to wait and try to multiplex.

libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default
on parallel transfers.

Suggested-by: Tom van der Woerdt
Closes #4500
2019-11-21 16:36:10 +01:00
Daniel Stenberg
7a46aeb0be
curl: fix -T globbing
Regression from e59371a493 (7.67.0)

Added test 490, 491 and 492 to verify the functionality.

Reported-by: Kamil Dudka
Reported-by: Anderson Sasaki

Fixes #4588
Closes #4591
2019-11-14 13:44:50 +01:00
Stian Soiland-Reyes
fe5c2464db tool_operate: Fix retry sleep time shown to user when Retry-After
- If server header Retry-After is being used for retry sleep time then
  show that value to the user instead of the normal retry sleep time.

This is a follow-up to 640b973 (7.66.0) which changed curl tool so that
the value from Retry-After header overrides other retry timing options.

Closes https://github.com/curl/curl/pull/4498
2019-10-16 13:56:38 -04:00
Stian Soiland-Reyes
f7f0b0012d
curl: ensure HTTP 429 triggers --retry
This completes #3794.

Also make sure the new tests from #4195 are enabled

Closes #4465
2019-10-05 11:41:06 +02:00
Daniel Stenberg
54c622aa8e
tool_operate: rename functions to make more sense 2019-10-02 07:50:38 +02:00
Daniel Stenberg
e59371a493
curl: create easy handles on-demand and not ahead of time
This should again enable crazy-large download ranges of the style
[1-10000000] that otherwise easily ran out of memory starting in 7.66.0
when this new handle allocating scheme was introduced.

Reported-by: Peter Sumatra
Fixes #4393
Closes #4438
2019-10-02 07:50:32 +02:00
Daniel Stenberg
f8a2058533
curl: exit the create_transfers loop on errors
When looping around the ranges and given URLs to create transfers, all
errors should exit the loop and return. Previously it would keep
looping.

Reported-by: SumatraPeter on github
Bug: #4393
Closes #4396
2019-09-22 12:27:58 +02:00
Daniel Stenberg
a89aeb5451
tool_operate: removed unused variable 'done'
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:08:02 +02:00
Daniel Stenberg
2d5f76f22f
tool_operate: Expression 'config->resume_from' is always true
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:58 +02:00
Kamil Dudka
1ca91bcdb5 curl: fix memory leaked by parse_metalink()
This commit fixes a regression introduced by curl-7_65_3-5-gb88940850.
Detected by tests 2005, 2008, 2009, 2010, 2011, and 2012 with valgrind
and libmetalink enabled.

Closes #4326
2019-09-13 10:09:48 +02:00
Daniel Stenberg
bbde0407d1
curl: make sure the parallel transfers do them all
The logic could erroneously break the loop too early before all
transfers had been transferred.

Reported-by: Tom van der Woerdt
Fixes #4316
Closes #4317
2019-09-10 19:45:19 +02:00
lufia
4a90c5b82e
curl: fix include conditions 2019-08-29 19:25:09 +02:00
Daniel Stenberg
e59540139a
cleanup: remove DOT_CHAR completely
Follow-up to f9c7ba9096

The use of DOT_CHAR for ".ssh" was probably a mistake and is removed
now.

Pointed-out-by: Gisle Vanem
Bug: https://github.com/curl/curl/pull/4230#issuecomment-522960638

Closes #4247
2019-08-20 23:07:59 +02:00
Daniel Stenberg
026840e35c
CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly
Even though it cannot fall-back to a lower HTTP version automatically. The
safer way to upgrade remains via CURLOPT_ALTSVC.

CURLOPT_H3 no longer has any bits that do anything and might be removed
before we remove the experimental label.

Updated the curl tool accordingly to use "--http3".

Closes #4197
2019-08-08 08:17:24 +02:00
Daniel Stenberg
640b9733de
curl: make use of CURLINFO_RETRY_AFTER when retrying
If a Retry-After: header was used in the response, that value overrides
other retry timing options.

Fixes #3794
Closes #4195
2019-08-07 14:45:44 +02:00
Daniel Stenberg
d39ae9d493
curl: use CURLINFO_PROTOCOL to check for HTTP(s)
... instead of CURLINFO_EFFECTIVE_URL to avoid string operations.
2019-08-07 14:45:39 +02:00
Steve Holme
1e08a79258 curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool
Ref: https://github.com/curl/curl/issues/3653
Ref: https://github.com/curl/curl/pull/3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:41 -04:00
Daniel Stenberg
02346abc32
curl_multi_poll: a sister to curl_multi_wait() that waits more
Repeatedly we see problems where using curl_multi_wait() is difficult or
just awkward because if it has no file descriptor to wait for
internally, it returns immediately and leaves it to the caller to wait
for a small amount of time in order to avoid occasional busy-looping.

This is often missed or misunderstood, leading to underperforming
applications.

This change introduces curl_multi_poll() as a replacement drop-in
function that accepts the exact same set of arguments. This function
works identically to curl_multi_wait() - EXCEPT - for the case when
there's nothing to wait for internally, as then this function will by
itself wait for a "suitable" short time before it returns. This
effectiely avoids all risks of busy-looping and should also make it less
likely that apps "over-wait".

This also changes the curl tool to use this funtion internally when
doing parallel transfers and changes curl_easy_perform() to use it
internally.

Closes #4163
2019-08-06 09:33:29 +02:00
Daniel Stenberg
62aa0ba91a
curl: remove outdated comment
Turned bad with commit b889408500

Reported-by: niallor on github
Fixes #4172
Closes #4173
2019-07-30 23:19:03 +02:00
Daniel Stenberg
a2ab576768
curl: avoid uncessary libcurl timeouts (in parallel mode)
When curl_multi_wait() returns OK without file descriptors to wait for,
it might already have done a long timeout.

Closes #4159
2019-07-29 14:57:57 +02:00
Jay Satiro
cdf7d13bcb tool_operate: fix implicit call to easysrc_cleanup
easysrc_cleanup is only defined when CURL_DISABLE_LIBCURL_OPTION is not
defined, and prior to this change would be called regardless.

Bug: https://github.com/curl/curl/pull/3804#issuecomment-513922637
Reported-by: Marcel Raad

Closes https://github.com/curl/curl/pull/4142
2019-07-23 10:44:55 -04:00
Daniel Stenberg
23c99f60ba
curl:create_transfers check return code from curl_easy_setopt
From commit b8894085

Pointed out by Coverity CID 1451703

Closes #4134
2019-07-22 00:28:55 +02:00
Daniel Stenberg
3af0e76d1e
HTTP3: initial (experimental) support
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes #3500
2019-07-21 23:49:03 +02:00
Daniel Stenberg
7644abf8e8
curl: remove dead code
The loop never loops (since b889408500), pointed out by Coverity (CID
1451702)

Closes #4133
2019-07-21 23:42:35 +02:00
Daniel Stenberg
b889408500
curl: support parallel transfers
This is done by making sure each individual transfer is first added to a
linked list as then they can be performed serially, or at will, in
parallel.

Closes #3804
2019-07-20 19:14:16 +02:00
Koen Dergent
92963d612b
curl: skip CURLOPT_PROXY_CAPATH for disabled-proxy builds
Closes #4061
2019-06-24 12:45:01 +02:00
Daniel Stenberg
b9e0c6d28c
curl: improved skip-setopt-options when built with disabled features
Reduces #ifdefs in src/tool_operate.c

Follow-up from 4e86f2fc4e
Closes #3936
2019-06-18 22:31:26 +02:00
Jay Satiro
db8ec1fa38
Revert all SASL authzid (new feature) commits
- Revert all commits related to the SASL authzid feature since the next
  release will be a patch release, 7.65.1.

Prior to this change CURLOPT_SASL_AUTHZID  / --sasl-authzid was destined
for the next release, assuming it would be a feature release 7.66.0.
However instead the next release will be a patch release, 7.65.1 and
will not contain any new features.

After the patch release after the reverted commits can be restored by
using cherry-pick:

git cherry-pick a14d72c a9499ff 8c1cc36 c2a8d52 0edf690

Details for all reverted commits:

Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()."

This reverts commit 0edf6907ae.

Revert "tests: Fix the line endings for the SASL alt-auth tests"

This reverts commit c2a8d52a13.

Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples"

This reverts commit 8c1cc369d0.

Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool"

This reverts commit a9499ff136.

Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID"

This reverts commit a14d72ca2f.
2019-05-25 23:36:11 +02:00
Steve Holme
a9499ff136
curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool 2019-05-22 22:55:11 +01:00
Daniel Stenberg
697b1f911b
curl: make code work with protocol-disabled libcurl
Closes #3844
2019-05-18 16:14:10 +02:00
Daniel Stenberg
8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00