1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-14 05:25:06 -05:00
Commit Graph

284 Commits

Author SHA1 Message Date
Maros Priputen
18e5cb77e9
curl: two new command line options for etags
--etag-compare and --etag-save

Suggested-by: Paul Hoffman
Fixes #4277
Closes #4543
2019-11-28 13:05:20 +01:00
John Schroeder
9a2cbf30b8
curl: fix --upload-file . hangs if delay in STDIN
Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION.

When uploading from stdin in non-blocking mode, a delay in reading
the stream (EAGAIN) causes curl to pause sending data
(CURL_READFUNC_PAUSE).  Prior to this change, a busy read was
detected and unpaused only in the CURLOPT_WRITEFUNCTION handler.
This change performs the same busy read handling in a
CURLOPT_XFERINFOFUNCTION handler.

Fixes #2051
Closes #4599
Reported-by: bdry on github
2019-11-26 09:17:52 +01:00
Daniel Stenberg
215baa74f7
curl: add --parallel-immediate
Starting with this change when doing parallel transfers, without this
option set, curl will prefer to create new transfers multiplexed on an
existing connection rather than creating a brand new one.

--parallel-immediate can be set to tell curl to prefer to use new
connections rather than to wait and try to multiplex.

libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default
on parallel transfers.

Suggested-by: Tom van der Woerdt
Closes #4500
2019-11-21 16:36:10 +01:00
Daniel Stenberg
7a46aeb0be
curl: fix -T globbing
Regression from e59371a493 (7.67.0)

Added test 490, 491 and 492 to verify the functionality.

Reported-by: Kamil Dudka
Reported-by: Anderson Sasaki

Fixes #4588
Closes #4591
2019-11-14 13:44:50 +01:00
Stian Soiland-Reyes
fe5c2464db tool_operate: Fix retry sleep time shown to user when Retry-After
- If server header Retry-After is being used for retry sleep time then
  show that value to the user instead of the normal retry sleep time.

This is a follow-up to 640b973 (7.66.0) which changed curl tool so that
the value from Retry-After header overrides other retry timing options.

Closes https://github.com/curl/curl/pull/4498
2019-10-16 13:56:38 -04:00
Stian Soiland-Reyes
f7f0b0012d
curl: ensure HTTP 429 triggers --retry
This completes #3794.

Also make sure the new tests from #4195 are enabled

Closes #4465
2019-10-05 11:41:06 +02:00
Daniel Stenberg
54c622aa8e
tool_operate: rename functions to make more sense 2019-10-02 07:50:38 +02:00
Daniel Stenberg
e59371a493
curl: create easy handles on-demand and not ahead of time
This should again enable crazy-large download ranges of the style
[1-10000000] that otherwise easily ran out of memory starting in 7.66.0
when this new handle allocating scheme was introduced.

Reported-by: Peter Sumatra
Fixes #4393
Closes #4438
2019-10-02 07:50:32 +02:00
Daniel Stenberg
f8a2058533
curl: exit the create_transfers loop on errors
When looping around the ranges and given URLs to create transfers, all
errors should exit the loop and return. Previously it would keep
looping.

Reported-by: SumatraPeter on github
Bug: #4393
Closes #4396
2019-09-22 12:27:58 +02:00
Daniel Stenberg
a89aeb5451
tool_operate: removed unused variable 'done'
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:08:02 +02:00
Daniel Stenberg
2d5f76f22f
tool_operate: Expression 'config->resume_from' is always true
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:58 +02:00
Kamil Dudka
1ca91bcdb5 curl: fix memory leaked by parse_metalink()
This commit fixes a regression introduced by curl-7_65_3-5-gb88940850.
Detected by tests 2005, 2008, 2009, 2010, 2011, and 2012 with valgrind
and libmetalink enabled.

Closes #4326
2019-09-13 10:09:48 +02:00
Daniel Stenberg
bbde0407d1
curl: make sure the parallel transfers do them all
The logic could erroneously break the loop too early before all
transfers had been transferred.

Reported-by: Tom van der Woerdt
Fixes #4316
Closes #4317
2019-09-10 19:45:19 +02:00
lufia
4a90c5b82e
curl: fix include conditions 2019-08-29 19:25:09 +02:00
Daniel Stenberg
e59540139a
cleanup: remove DOT_CHAR completely
Follow-up to f9c7ba9096

The use of DOT_CHAR for ".ssh" was probably a mistake and is removed
now.

Pointed-out-by: Gisle Vanem
Bug: https://github.com/curl/curl/pull/4230#issuecomment-522960638

Closes #4247
2019-08-20 23:07:59 +02:00
Daniel Stenberg
026840e35c
CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly
Even though it cannot fall-back to a lower HTTP version automatically. The
safer way to upgrade remains via CURLOPT_ALTSVC.

CURLOPT_H3 no longer has any bits that do anything and might be removed
before we remove the experimental label.

Updated the curl tool accordingly to use "--http3".

Closes #4197
2019-08-08 08:17:24 +02:00
Daniel Stenberg
640b9733de
curl: make use of CURLINFO_RETRY_AFTER when retrying
If a Retry-After: header was used in the response, that value overrides
other retry timing options.

Fixes #3794
Closes #4195
2019-08-07 14:45:44 +02:00
Daniel Stenberg
d39ae9d493
curl: use CURLINFO_PROTOCOL to check for HTTP(s)
... instead of CURLINFO_EFFECTIVE_URL to avoid string operations.
2019-08-07 14:45:39 +02:00
Steve Holme
1e08a79258 curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool
Ref: https://github.com/curl/curl/issues/3653
Ref: https://github.com/curl/curl/pull/3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:41 -04:00
Daniel Stenberg
02346abc32
curl_multi_poll: a sister to curl_multi_wait() that waits more
Repeatedly we see problems where using curl_multi_wait() is difficult or
just awkward because if it has no file descriptor to wait for
internally, it returns immediately and leaves it to the caller to wait
for a small amount of time in order to avoid occasional busy-looping.

This is often missed or misunderstood, leading to underperforming
applications.

This change introduces curl_multi_poll() as a replacement drop-in
function that accepts the exact same set of arguments. This function
works identically to curl_multi_wait() - EXCEPT - for the case when
there's nothing to wait for internally, as then this function will by
itself wait for a "suitable" short time before it returns. This
effectiely avoids all risks of busy-looping and should also make it less
likely that apps "over-wait".

This also changes the curl tool to use this funtion internally when
doing parallel transfers and changes curl_easy_perform() to use it
internally.

Closes #4163
2019-08-06 09:33:29 +02:00
Daniel Stenberg
62aa0ba91a
curl: remove outdated comment
Turned bad with commit b889408500

Reported-by: niallor on github
Fixes #4172
Closes #4173
2019-07-30 23:19:03 +02:00
Daniel Stenberg
a2ab576768
curl: avoid uncessary libcurl timeouts (in parallel mode)
When curl_multi_wait() returns OK without file descriptors to wait for,
it might already have done a long timeout.

Closes #4159
2019-07-29 14:57:57 +02:00
Jay Satiro
cdf7d13bcb tool_operate: fix implicit call to easysrc_cleanup
easysrc_cleanup is only defined when CURL_DISABLE_LIBCURL_OPTION is not
defined, and prior to this change would be called regardless.

Bug: https://github.com/curl/curl/pull/3804#issuecomment-513922637
Reported-by: Marcel Raad

Closes https://github.com/curl/curl/pull/4142
2019-07-23 10:44:55 -04:00
Daniel Stenberg
23c99f60ba
curl:create_transfers check return code from curl_easy_setopt
From commit b8894085

Pointed out by Coverity CID 1451703

Closes #4134
2019-07-22 00:28:55 +02:00
Daniel Stenberg
3af0e76d1e
HTTP3: initial (experimental) support
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes #3500
2019-07-21 23:49:03 +02:00
Daniel Stenberg
7644abf8e8
curl: remove dead code
The loop never loops (since b889408500), pointed out by Coverity (CID
1451702)

Closes #4133
2019-07-21 23:42:35 +02:00
Daniel Stenberg
b889408500
curl: support parallel transfers
This is done by making sure each individual transfer is first added to a
linked list as then they can be performed serially, or at will, in
parallel.

Closes #3804
2019-07-20 19:14:16 +02:00
Koen Dergent
92963d612b
curl: skip CURLOPT_PROXY_CAPATH for disabled-proxy builds
Closes #4061
2019-06-24 12:45:01 +02:00
Daniel Stenberg
b9e0c6d28c
curl: improved skip-setopt-options when built with disabled features
Reduces #ifdefs in src/tool_operate.c

Follow-up from 4e86f2fc4e
Closes #3936
2019-06-18 22:31:26 +02:00
Jay Satiro
db8ec1fa38
Revert all SASL authzid (new feature) commits
- Revert all commits related to the SASL authzid feature since the next
  release will be a patch release, 7.65.1.

Prior to this change CURLOPT_SASL_AUTHZID  / --sasl-authzid was destined
for the next release, assuming it would be a feature release 7.66.0.
However instead the next release will be a patch release, 7.65.1 and
will not contain any new features.

After the patch release after the reverted commits can be restored by
using cherry-pick:

git cherry-pick a14d72c a9499ff 8c1cc36 c2a8d52 0edf690

Details for all reverted commits:

Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()."

This reverts commit 0edf6907ae.

Revert "tests: Fix the line endings for the SASL alt-auth tests"

This reverts commit c2a8d52a13.

Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples"

This reverts commit 8c1cc369d0.

Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool"

This reverts commit a9499ff136.

Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID"

This reverts commit a14d72ca2f.
2019-05-25 23:36:11 +02:00
Steve Holme
a9499ff136
curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool 2019-05-22 22:55:11 +01:00
Daniel Stenberg
697b1f911b
curl: make code work with protocol-disabled libcurl
Closes #3844
2019-05-18 16:14:10 +02:00
Daniel Stenberg
8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00
Daniel Stenberg
f506ce099f
curl: don't set FTP options for FTP-disabled builds
... since libcurl has started to be totally unaware of options for
disabled protocols they now return error.

Bug: c9c5304dd4 (commitcomment-33533937)

Reported-by: Marcel Raad
Closes #3886
2019-05-16 08:11:37 +02:00
Chris Young
5c215bdbdf
tool_operate: build on AmigaOS 2019-03-15 10:16:33 +01:00
Daniel Stenberg
c51c78dd8d
alt-svc: the curl command line bits 2019-03-03 11:17:52 +01:00
Patrick Monnerat
e49e5eaa10 cli tool: fix mime post with --disable-libcurl-option configure option
Reported-by: Marcel Raad
Fixes #3576
Closes #3583
2019-02-19 17:08:58 +01:00
Marcel Raad
91e397b242
tool_operate: fix typecheck warning
Use long for CURLOPT_HTTP09_ALLOWED to fix the following warning:
tool_operate.c: In function 'operate_do':
../include/curl/typecheck-gcc.h:47:9: error: call to
'_curl_easy_setopt_err_long' declared with attribute warning:
curl_easy_setopt expects a long argument for this option [-Werror]

Closes https://github.com/curl/curl/pull/3534
2019-02-07 18:36:53 +01:00
Daniel Stenberg
180501cb02
schannel: stop calling it "winssl"
Stick to "Schannel" everywhere. The configure option --with-winssl is
kept to allow existing builds to work but --with-schannel is added as an
alias.

Closes #3504
2019-02-01 08:20:38 +01:00
Daniel Stenberg
006ff62d8c
http: added options for allowing HTTP/0.9 responses
Added CURLOPT_HTTP09_ALLOWED and --http0.9 for this purpose.

For now, both the tool and library allow HTTP/0.9 by default.
docs/DEPRECATE.md lays out the plan for when to reverse that default: 6
months after the 7.64.0 release. The options are added already now so
that applications/scripts can start using them already now.

Fixes #2873
Closes #3383
2018-12-21 10:49:30 +01:00
Daniel Stenberg
4849267197
curl -J: do not append to the destination file
Reported-by: Kamil Dudka
Fixes #3380
Closes #3381
2018-12-19 08:44:02 +01:00
Han Han
78ff4e0de3
ssl: replace all internal uses of CURLE_SSL_CACERT
Closes #3291
2018-11-20 14:57:00 +01:00
Daniel Stenberg
52db54869e
curl: fix --local-port integer overflow
The tool's local port command line range parser didn't check for integer
overflows and could pass "weird" data to libcurl for this option.
libcurl however, has a strict range check for the values so it rejects
anything outside of the accepted range.

Reported-by: Brian Carpenter
Closes #3242
2018-11-07 11:48:17 +01:00
Daniel Stenberg
5ffbb63e42
curl: --doh-url added 2018-09-06 09:17:25 +02:00
Jay Satiro
351c0f3a55
tool_operate: Add http code 408 to transient list for --retry
- Treat 408 request timeout as transient so that curl will retry the
  request if --retry was used.

Closes #2925
2018-09-03 13:19:14 +02:00
Jay Satiro
705cc899e3 tool_operate: Fix setting proxy TLS 1.3 ciphers 2018-08-31 19:40:55 -04:00
Anderson Toshiyuki Sasaki
298d2565e2
ssl: set engine implicitly when a PKCS#11 URI is provided
This allows the use of PKCS#11 URI for certificates and keys without
setting the corresponding type as "ENG" and the engine as "pkcs11"
explicitly. If a PKCS#11 URI is provided for certificate, key,
proxy_certificate or proxy_key, the corresponding type is set as "ENG"
if not provided and the engine is set to "pkcs11" if not provided.

Acked-by: Nikos Mavrogiannopoulos
Closes #2333
2018-08-08 09:46:01 +02:00
Björn Stenberg
946ce5b61f
option: disallow username in URL
Adds CURLOPT_DISALLOW_USERNAME_IN_URL and --disallow-username-in-url. Makes
libcurl reject URLs with a username in them.

Closes #2340
2018-05-31 11:27:16 +02:00
Daniel Stenberg
050c93c46f
setopt: add TLS 1.3 ciphersuites
Adds CURLOPT_TLS13_CIPHERS and CURLOPT_PROXY_TLS13_CIPHERS.

curl: added --tls13-ciphers and --proxy-tls13-ciphers

Fixes #2435
Reported-by: zzq1015 on github
Closes #2607
2018-05-29 16:12:52 +02:00
Daniel Stenberg
c1c27625c7
curl: show headers in bold
The feature is only enabled if the output is believed to be a tty.

-J: There's some minor differences and improvements in -J handling, as
now J should work with -i and it actually creates a file first using the
initial name and then *renames* that to the one found in
Content-Disposition (if any).

-i: only shows headers for HTTP transfers now (as documented).
Previously it would also show for pieces of the transfer that were HTTP
(for example when doing FTP over a HTTP proxy).

-i: now shows trailers as well. Previously they were not shown at all.

--libcurl: the CURLOPT_HEADER is no longer set, as the header output is
now done in the header callback.
2018-05-21 20:05:05 +02:00