1
0
mirror of https://github.com/moparisthebest/curl synced 2024-08-13 17:03:50 -04:00
Commit Graph

280 Commits

Author SHA1 Message Date
Stian Soiland-Reyes
fe5c2464db tool_operate: Fix retry sleep time shown to user when Retry-After
- If server header Retry-After is being used for retry sleep time then
  show that value to the user instead of the normal retry sleep time.

This is a follow-up to 640b973 (7.66.0) which changed curl tool so that
the value from Retry-After header overrides other retry timing options.

Closes https://github.com/curl/curl/pull/4498
2019-10-16 13:56:38 -04:00
Stian Soiland-Reyes
f7f0b0012d
curl: ensure HTTP 429 triggers --retry
This completes #3794.

Also make sure the new tests from #4195 are enabled

Closes #4465
2019-10-05 11:41:06 +02:00
Daniel Stenberg
54c622aa8e
tool_operate: rename functions to make more sense 2019-10-02 07:50:38 +02:00
Daniel Stenberg
e59371a493
curl: create easy handles on-demand and not ahead of time
This should again enable crazy-large download ranges of the style
[1-10000000] that otherwise easily ran out of memory starting in 7.66.0
when this new handle allocating scheme was introduced.

Reported-by: Peter Sumatra
Fixes #4393
Closes #4438
2019-10-02 07:50:32 +02:00
Daniel Stenberg
f8a2058533
curl: exit the create_transfers loop on errors
When looping around the ranges and given URLs to create transfers, all
errors should exit the loop and return. Previously it would keep
looping.

Reported-by: SumatraPeter on github
Bug: #4393
Closes #4396
2019-09-22 12:27:58 +02:00
Daniel Stenberg
a89aeb5451
tool_operate: removed unused variable 'done'
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:08:02 +02:00
Daniel Stenberg
2d5f76f22f
tool_operate: Expression 'config->resume_from' is always true
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:58 +02:00
Kamil Dudka
1ca91bcdb5 curl: fix memory leaked by parse_metalink()
This commit fixes a regression introduced by curl-7_65_3-5-gb88940850.
Detected by tests 2005, 2008, 2009, 2010, 2011, and 2012 with valgrind
and libmetalink enabled.

Closes #4326
2019-09-13 10:09:48 +02:00
Daniel Stenberg
bbde0407d1
curl: make sure the parallel transfers do them all
The logic could erroneously break the loop too early before all
transfers had been transferred.

Reported-by: Tom van der Woerdt
Fixes #4316
Closes #4317
2019-09-10 19:45:19 +02:00
lufia
4a90c5b82e
curl: fix include conditions 2019-08-29 19:25:09 +02:00
Daniel Stenberg
e59540139a
cleanup: remove DOT_CHAR completely
Follow-up to f9c7ba9096

The use of DOT_CHAR for ".ssh" was probably a mistake and is removed
now.

Pointed-out-by: Gisle Vanem
Bug: https://github.com/curl/curl/pull/4230#issuecomment-522960638

Closes #4247
2019-08-20 23:07:59 +02:00
Daniel Stenberg
026840e35c
CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly
Even though it cannot fall-back to a lower HTTP version automatically. The
safer way to upgrade remains via CURLOPT_ALTSVC.

CURLOPT_H3 no longer has any bits that do anything and might be removed
before we remove the experimental label.

Updated the curl tool accordingly to use "--http3".

Closes #4197
2019-08-08 08:17:24 +02:00
Daniel Stenberg
640b9733de
curl: make use of CURLINFO_RETRY_AFTER when retrying
If a Retry-After: header was used in the response, that value overrides
other retry timing options.

Fixes #3794
Closes #4195
2019-08-07 14:45:44 +02:00
Daniel Stenberg
d39ae9d493
curl: use CURLINFO_PROTOCOL to check for HTTP(s)
... instead of CURLINFO_EFFECTIVE_URL to avoid string operations.
2019-08-07 14:45:39 +02:00
Steve Holme
1e08a79258 curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool
Ref: https://github.com/curl/curl/issues/3653
Ref: https://github.com/curl/curl/pull/3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:41 -04:00
Daniel Stenberg
02346abc32
curl_multi_poll: a sister to curl_multi_wait() that waits more
Repeatedly we see problems where using curl_multi_wait() is difficult or
just awkward because if it has no file descriptor to wait for
internally, it returns immediately and leaves it to the caller to wait
for a small amount of time in order to avoid occasional busy-looping.

This is often missed or misunderstood, leading to underperforming
applications.

This change introduces curl_multi_poll() as a replacement drop-in
function that accepts the exact same set of arguments. This function
works identically to curl_multi_wait() - EXCEPT - for the case when
there's nothing to wait for internally, as then this function will by
itself wait for a "suitable" short time before it returns. This
effectiely avoids all risks of busy-looping and should also make it less
likely that apps "over-wait".

This also changes the curl tool to use this funtion internally when
doing parallel transfers and changes curl_easy_perform() to use it
internally.

Closes #4163
2019-08-06 09:33:29 +02:00
Daniel Stenberg
62aa0ba91a
curl: remove outdated comment
Turned bad with commit b889408500

Reported-by: niallor on github
Fixes #4172
Closes #4173
2019-07-30 23:19:03 +02:00
Daniel Stenberg
a2ab576768
curl: avoid uncessary libcurl timeouts (in parallel mode)
When curl_multi_wait() returns OK without file descriptors to wait for,
it might already have done a long timeout.

Closes #4159
2019-07-29 14:57:57 +02:00
Jay Satiro
cdf7d13bcb tool_operate: fix implicit call to easysrc_cleanup
easysrc_cleanup is only defined when CURL_DISABLE_LIBCURL_OPTION is not
defined, and prior to this change would be called regardless.

Bug: https://github.com/curl/curl/pull/3804#issuecomment-513922637
Reported-by: Marcel Raad

Closes https://github.com/curl/curl/pull/4142
2019-07-23 10:44:55 -04:00
Daniel Stenberg
23c99f60ba
curl:create_transfers check return code from curl_easy_setopt
From commit b8894085

Pointed out by Coverity CID 1451703

Closes #4134
2019-07-22 00:28:55 +02:00
Daniel Stenberg
3af0e76d1e
HTTP3: initial (experimental) support
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes #3500
2019-07-21 23:49:03 +02:00
Daniel Stenberg
7644abf8e8
curl: remove dead code
The loop never loops (since b889408500), pointed out by Coverity (CID
1451702)

Closes #4133
2019-07-21 23:42:35 +02:00
Daniel Stenberg
b889408500
curl: support parallel transfers
This is done by making sure each individual transfer is first added to a
linked list as then they can be performed serially, or at will, in
parallel.

Closes #3804
2019-07-20 19:14:16 +02:00
Koen Dergent
92963d612b
curl: skip CURLOPT_PROXY_CAPATH for disabled-proxy builds
Closes #4061
2019-06-24 12:45:01 +02:00
Daniel Stenberg
b9e0c6d28c
curl: improved skip-setopt-options when built with disabled features
Reduces #ifdefs in src/tool_operate.c

Follow-up from 4e86f2fc4e
Closes #3936
2019-06-18 22:31:26 +02:00
Jay Satiro
db8ec1fa38
Revert all SASL authzid (new feature) commits
- Revert all commits related to the SASL authzid feature since the next
  release will be a patch release, 7.65.1.

Prior to this change CURLOPT_SASL_AUTHZID  / --sasl-authzid was destined
for the next release, assuming it would be a feature release 7.66.0.
However instead the next release will be a patch release, 7.65.1 and
will not contain any new features.

After the patch release after the reverted commits can be restored by
using cherry-pick:

git cherry-pick a14d72c a9499ff 8c1cc36 c2a8d52 0edf690

Details for all reverted commits:

Revert "os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid()."

This reverts commit 0edf6907ae.

Revert "tests: Fix the line endings for the SASL alt-auth tests"

This reverts commit c2a8d52a13.

Revert "examples: Added SASL PLAIN authorisation identity (authzid) examples"

This reverts commit 8c1cc369d0.

Revert "curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool"

This reverts commit a9499ff136.

Revert "sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID"

This reverts commit a14d72ca2f.
2019-05-25 23:36:11 +02:00
Steve Holme
a9499ff136
curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool 2019-05-22 22:55:11 +01:00
Daniel Stenberg
697b1f911b
curl: make code work with protocol-disabled libcurl
Closes #3844
2019-05-18 16:14:10 +02:00
Daniel Stenberg
8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00
Daniel Stenberg
f506ce099f
curl: don't set FTP options for FTP-disabled builds
... since libcurl has started to be totally unaware of options for
disabled protocols they now return error.

Bug: c9c5304dd4 (commitcomment-33533937)

Reported-by: Marcel Raad
Closes #3886
2019-05-16 08:11:37 +02:00
Chris Young
5c215bdbdf
tool_operate: build on AmigaOS 2019-03-15 10:16:33 +01:00
Daniel Stenberg
c51c78dd8d
alt-svc: the curl command line bits 2019-03-03 11:17:52 +01:00
Patrick Monnerat
e49e5eaa10 cli tool: fix mime post with --disable-libcurl-option configure option
Reported-by: Marcel Raad
Fixes #3576
Closes #3583
2019-02-19 17:08:58 +01:00
Marcel Raad
91e397b242
tool_operate: fix typecheck warning
Use long for CURLOPT_HTTP09_ALLOWED to fix the following warning:
tool_operate.c: In function 'operate_do':
../include/curl/typecheck-gcc.h:47:9: error: call to
'_curl_easy_setopt_err_long' declared with attribute warning:
curl_easy_setopt expects a long argument for this option [-Werror]

Closes https://github.com/curl/curl/pull/3534
2019-02-07 18:36:53 +01:00
Daniel Stenberg
180501cb02
schannel: stop calling it "winssl"
Stick to "Schannel" everywhere. The configure option --with-winssl is
kept to allow existing builds to work but --with-schannel is added as an
alias.

Closes #3504
2019-02-01 08:20:38 +01:00
Daniel Stenberg
006ff62d8c
http: added options for allowing HTTP/0.9 responses
Added CURLOPT_HTTP09_ALLOWED and --http0.9 for this purpose.

For now, both the tool and library allow HTTP/0.9 by default.
docs/DEPRECATE.md lays out the plan for when to reverse that default: 6
months after the 7.64.0 release. The options are added already now so
that applications/scripts can start using them already now.

Fixes #2873
Closes #3383
2018-12-21 10:49:30 +01:00
Daniel Stenberg
4849267197
curl -J: do not append to the destination file
Reported-by: Kamil Dudka
Fixes #3380
Closes #3381
2018-12-19 08:44:02 +01:00
Han Han
78ff4e0de3
ssl: replace all internal uses of CURLE_SSL_CACERT
Closes #3291
2018-11-20 14:57:00 +01:00
Daniel Stenberg
52db54869e
curl: fix --local-port integer overflow
The tool's local port command line range parser didn't check for integer
overflows and could pass "weird" data to libcurl for this option.
libcurl however, has a strict range check for the values so it rejects
anything outside of the accepted range.

Reported-by: Brian Carpenter
Closes #3242
2018-11-07 11:48:17 +01:00
Daniel Stenberg
5ffbb63e42
curl: --doh-url added 2018-09-06 09:17:25 +02:00
Jay Satiro
351c0f3a55
tool_operate: Add http code 408 to transient list for --retry
- Treat 408 request timeout as transient so that curl will retry the
  request if --retry was used.

Closes #2925
2018-09-03 13:19:14 +02:00
Jay Satiro
705cc899e3 tool_operate: Fix setting proxy TLS 1.3 ciphers 2018-08-31 19:40:55 -04:00
Anderson Toshiyuki Sasaki
298d2565e2
ssl: set engine implicitly when a PKCS#11 URI is provided
This allows the use of PKCS#11 URI for certificates and keys without
setting the corresponding type as "ENG" and the engine as "pkcs11"
explicitly. If a PKCS#11 URI is provided for certificate, key,
proxy_certificate or proxy_key, the corresponding type is set as "ENG"
if not provided and the engine is set to "pkcs11" if not provided.

Acked-by: Nikos Mavrogiannopoulos
Closes #2333
2018-08-08 09:46:01 +02:00
Björn Stenberg
946ce5b61f
option: disallow username in URL
Adds CURLOPT_DISALLOW_USERNAME_IN_URL and --disallow-username-in-url. Makes
libcurl reject URLs with a username in them.

Closes #2340
2018-05-31 11:27:16 +02:00
Daniel Stenberg
050c93c46f
setopt: add TLS 1.3 ciphersuites
Adds CURLOPT_TLS13_CIPHERS and CURLOPT_PROXY_TLS13_CIPHERS.

curl: added --tls13-ciphers and --proxy-tls13-ciphers

Fixes #2435
Reported-by: zzq1015 on github
Closes #2607
2018-05-29 16:12:52 +02:00
Daniel Stenberg
c1c27625c7
curl: show headers in bold
The feature is only enabled if the output is believed to be a tty.

-J: There's some minor differences and improvements in -J handling, as
now J should work with -i and it actually creates a file first using the
initial name and then *renames* that to the one found in
Content-Disposition (if any).

-i: only shows headers for HTTP transfers now (as documented).
Previously it would also show for pieces of the transfer that were HTTP
(for example when doing FTP over a HTTP proxy).

-i: now shows trailers as well. Previously they were not shown at all.

--libcurl: the CURLOPT_HEADER is no longer set, as the header output is
now done in the header callback.
2018-05-21 20:05:05 +02:00
Daniel Gustafsson
2f13e3d23d
checksrc: force indentation of lines after an else
This extends the INDENTATION case to also handle 'else' statements
and require proper indentation on the following line. Also fixes the
offending cases found in the codebase.

Closes #2532
2018-04-27 00:51:35 +02:00
Dan McNulty
8996300211 schannel: add support for CURLOPT_CAINFO
- Move verify_certificate functionality in schannel.c into a new
  file called schannel_verify.c. Additionally, some structure defintions
  from schannel.c have been moved to schannel.h to allow them to be
  used in schannel_verify.c.

- Make verify_certificate functionality for Schannel available on
  all versions of Windows instead of just Windows CE. verify_certificate
  will be invoked on Windows CE or when the user specifies
  CURLOPT_CAINFO and CURLOPT_SSL_VERIFYPEER.

- In verify_certificate, create a custom certificate chain engine that
  exclusively trusts the certificate store backed by the CURLOPT_CAINFO
  file.

- doc updates of --cacert/CAINFO support for schannel

- Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
  when available. This implements a TODO in schannel.c to improve
  handling of multiple SANs in a certificate. In particular, all SANs
  will now be searched instead of just the first name.

- Update tool_operate.c to not search for the curl-ca-bundle.crt file
  when using Schannel to maintain backward compatibility. Previously,
  any curl-ca-bundle.crt file found in that search would have been
  ignored by Schannel. But, with CAINFO support, the file found by
  that search would have been used as the certificate store and
  could cause issues for any users that have curl-ca-bundle.crt in
  the search path.

- Update url.c to not set the build time CURL_CA_BUNDLE if the selected
  SSL backend is Schannel. We allow setting CA location for schannel
  only when explicitly specified by the user via CURLOPT_CAINFO /
  --cacert.

- Add new test cases 3000 and 3001. These test cases check that the first
  and last SAN, respectively, matches the connection hostname. New test
  certificates have been added for these cases. For 3000, the certificate
  prefix is Server-localhost-firstSAN and for 3001, the certificate
  prefix is Server-localhost-secondSAN.

- Remove TODO 15.2 (Add support for custom server certificate
  validation), this commit addresses it.

Closes https://github.com/curl/curl/pull/1325
2018-04-18 03:59:47 -04:00
Stefan Agner
464a019cbe tool_operate: Fix retry on FTP 4xx to ignore other protocols
Only treat response code as FTP response codes in case the
protocol type is FTP.

This fixes an issue where an HTTP download was treated as FTP
in case libcurl returned with 33. This happens when the
download has already finished and the server responses 416:
  HTTP/1.1 416 Requested Range Not Satisfiable

This should not be treated as an FTP error.

Fixes #2464
Closes #2465
2018-04-07 02:57:57 -04:00
Lawrence Matthews
6baeb6df35
CURLOPT_HAPROXYPROTOCOL: support the HAProxy PROXY protocol
Add --haproxy-protocol for the command line tool

Closes #2162
2018-03-17 11:50:06 +01:00