1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-10 11:35:07 -05:00
Commit Graph

2453 Commits

Author SHA1 Message Date
Daniel Stenberg
8feeafc61a
glob: pass an 'int' as len when using printf's %*s
Detected by Coverity CID 1486629.

Closes #7324
2021-06-30 14:05:56 +02:00
Daniel Stenberg
2784a585b3
single_transfer: ignore blank --output-dir
... as otherwise it creates a rather unexpected target directory with a
leading slash.

Reported-by: Harry Sintonen
Fixes #7218
Closes #7233
2021-06-11 12:50:46 +02:00
Daniel Stenberg
265b14d6b3
metalink: remove
Warning: this will make existing curl command lines that use metalink to
stop working.

Reasons for removal:

1. We've found several security problems and issues involving the
   metalink support in curl. The issues are not detailed here. When
   working on those, it become apparent to the team that several of the
   problems are due to the system design, metalink library API and what
   the metalink RFC says. They are very hard to fix on the curl side
   only.

2. The metalink usage with curl was only very briefly documented and was
   not following the "normal" curl usage pattern in several ways, making
   it surprising and non-intuitive which could lead to further security
   issues.

3. The metalink library was last updated 6 years ago and wasn't so
   active the years before that either. An unmaintained library means
   there's a security problem waiting to happen. This is probably reason
   enough.

4. Metalink requires an XML parsing library, which is complex code (even
   the smaller alternatives) and to this day often gets security
   updates.

5. Metalink is not a widely used curl feature. In the 2020 curl user
   survey, only 1.4% of the responders said that they'd are using it. In
   2021 that number was 1.2%. Searching the web also show very few
   traces of it being used, even with other tools.

6. The torrent format and associated technology clearly won for
   downloading large files from multiple sources in parallel.

Cloes #7176
2021-06-07 08:14:25 +02:00
Daniel Stenberg
82d334001f
copyright: update copyright year ranges to 2021 2021-05-26 08:18:11 +02:00
Daniel Stenberg
e61d8e36d2
curl: include libmetalink version in --version output
Closes #7112
2021-05-22 11:29:41 +02:00
Ikko Ashimine
b09944b3ae
tool_getparam: fix comment typo in tool_getparam.c
enfore -> enforce

Closes #7074
2021-05-15 23:33:30 +02:00
Kamil Dudka
92953dc387 tool_getparam: handle failure of curlx_convert_tchar_to_UTF8()
Reported by GCC analyzer:

Error: GCC_ANALYZER_WARNING (CWE-476):
src/tool_getparam.c: scope_hint: In function 'parse_args'
src/tool_getparam.c:2318:38: warning[-Wanalyzer-possible-null-dereference]: dereference of possibly-NULL 'orig_opt'
lib/curlx.h:56: included_from: Included from here.
src/tool_getparam.c:28: included_from: Included from here.
lib/curl_multibyte.h:70:51: note: in definition of macro 'curlx_convert_tchar_to_UTF8'
src/tool_getparam.c:2316:16: note: in expansion of macro 'curlx_convert_tchar_to_UTF8'

Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Closes #7023
2021-05-06 15:52:09 +02:00
Jay Satiro
54e7475016 schannel: Disable auto credentials; add an option to enable it
- Disable auto credentials by default. This is a breaking change
  for clients that are using it, wittingly or not.

- New libcurl ssl option value CURLSSLOPT_AUTO_CLIENT_CERT tells libcurl
  to automatically locate and use a client certificate for
  authentication, when requested by the server.

- New curl tool options --ssl-auto-client-cert and
  --proxy-ssl-auto-client-cert map to CURLSSLOPT_AUTO_CLIENT_CERT.

This option is only supported for Schannel (the native Windows SSL
library). Prior to this change Schannel would, with no notification to
the client, attempt to locate a client certificate and send it to the
server, when requested by the server. Since the server can request any
certificate that supports client authentication in the OS certificate
store it could be a privacy violation and unexpected.

Fixes https://github.com/curl/curl/issues/2262
Reported-by: Jeroen Ooms
Assisted-by: Wes Hinsley
Assisted-by: Rich FitzJohn

Ref: https://curl.se/mail/lib-2021-02/0066.html
Reported-by: Morten Minde Neergaard

Closes https://github.com/curl/curl/pull/6673
2021-04-22 16:53:37 -04:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Daniel Stenberg
3057c6cd73
Revert "cmake: make libcurl library output name configurable"
This reverts commit 1cba36d216.

CMake provides properties that can be set on a target to rename the
output artifact without changing the name of a target.

Ref: #6899
2021-04-21 15:03:26 +02:00
Ralph Langendam
1cba36d216
cmake: make libcurl library output name configurable
Closes #6899
2021-04-21 10:03:07 +02:00
Jay Satiro
711879ca7c tool_operate: don't discard failed parallel transfer result
- Save a parallel transfer's result code only when it fails and the
  transfer is not being retried.

Prior to this change the result code was always set which meant that a
failed result could be erroneously discarded if a different transfer
later had a successful result (CURLE_OK).

Before:

> curl --fail -Z https://httpbin.org/status/404 https://httpbin.org/delay/10
> echo %ERRORLEVEL%
0

After:

> curl --fail -Z https://httpbin.org/status/404 https://httpbin.org/delay/10
> echo %ERRORLEVEL%
22

Closes #xxxx
2021-04-20 01:51:56 -04:00
Daniel Stenberg
cf65d4237e
curl: ignore options asking for SSLv2 or SSLv3
Instead output a warning about it and continue with the defaults.

These SSL versions are typically not supported by the TLS libraries since a
long time back already since they are inherently insecure and broken. Asking
for them to be used will just cause an error to be returned slightly later.

In the unlikely event that a user's TLS library actually still supports these
protocol versions, this change might make the request a little less insecure.

Closes #6772
2021-04-19 08:14:05 +02:00
Jay Satiro
2f78be51eb
tool_writeout: fix the HTTP_CODE json output
Update test 970 accordingly.

Reported-by: Michal Rus
Fixes #6905
Closes #6906
2021-04-17 11:03:30 +02:00
Victor Vieux
0d7c55bd57
tool_getparam: replace (in-place) '%20' by '+' according to RFC1866
Signed-off-by: Victor Vieux <victorvieux@gmail.com>

Closes #6895
2021-04-15 13:45:12 +02:00
Daniel Stenberg
95d525a9e1
misc: update copyright year ranges to match latest updates 2021-04-13 14:32:30 +02:00
Jürgen Gmach
c6e213e584
src/tool_vms.c: remove duplicated word in comment
Closes #6881
2021-04-10 11:43:18 +02:00
Cherish98
4b4401e26a tool_progress: Fix progress meter in parallel mode
Make sure the total amount of DL/UL bytes are counted before the
transfer finalizes. Otherwise if a transfer finishes too quick, its
total numbers are not added, and results in a DL%/UL% that goes above
100%.

Detail:

progress_meter() is called periodically, and it may not catch a
transfer's total bytes if the value was unknown during the last call,
and the transfer is finished and deleted (i.e., lost) during the next
call.

Closes https://github.com/curl/curl/pull/6840
2021-04-05 23:43:26 -04:00
Daniel Stenberg
85e6975643
copyright: update copyright year ranges to 2021
Reviewed-by: Emil Engler
Closes #6802
2021-03-27 23:00:14 +01:00
Daniel Stenberg
985c184d5b
tool_operate: bail if set CURLOPT_HTTP09_ALLOWED returns error
Closes #6727
2021-03-11 22:45:20 +01:00
Daniel Stenberg
78f642ffab
config: remove CURL_SIZEOF_CURL_OFF_T use only SIZEOF_CURL_OFF_T
Make the code consistently use a single name for the size of the
"curl_off_t" type.

Closes #6702
2021-03-11 10:11:56 +01:00
Vladimir Varlamov
e1655b2477 docs: add missing Arg tag to --stderr
Prior to this change the required argument was not shown.

curl.1 before: --stderr
curl.1 after: --stderr <file>

curl --help before:
     --stderr        Where to redirect stderr

curl --help after:
     --stderr <file>  Where to redirect stderr

Closes https://github.com/curl/curl/pull/6692
2021-03-04 18:01:55 -05:00
Jay Satiro
5743d66b91 tool_help: Increase space between option and description
- Increase the minimum number of spaces between the option and the
  description from 1 to 2.

Before:
~~~
 -u, --user <user:password> Server user and password
 -A, --user-agent <name> Send User-Agent <name> to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit
 -w, --write-out <format> Use output FORMAT after completion
     --xattr         Store metadata in extended file attributes
~~~

After:
~~~
 -u, --user <user:password>  Server user and password
 -A, --user-agent <name>  Send User-Agent <name> to server
 -v, --verbose       Make the operation more talkative
 -V, --version       Show version number and quit
 -w, --write-out <format>  Use output FORMAT after completion
     --xattr         Store metadata in extended file attributes
~~~

Closes https://github.com/curl/curl/pull/6674
2021-02-28 13:41:15 -05:00
Daniel Stenberg
40f3c18e6a
curl: set CURLOPT_NEW_FILE_PERMS if requested
The --create-file-mode code logic accepted the value but never actually
passed it on to libcurl!

Follow-up to a7696c7343 (shipped in 7.75.0)
Reported-by: Johannes Lesr
Fixes #6657
Closes #6666
2021-02-27 00:30:36 +01:00
Daniel Stenberg
15bc86df33
tool_operate: check argc before accessing argv[1]
Follow-up to 09363500b
Reported-by: Emil Engler
Reviewed-by: Daniel Gustafsson
Closes #6668
2021-02-27 00:25:47 +01:00
Daniel Gustafsson
09a5bff1c9 gnutls: Fix nettle discovery
Commit e06fa7462a removed support for libgcrypt leaving only
support for nettle which has been the default crypto library in
GnuTLS for a long time. There were however a few conditionals on
USE_GNUTLS_NETTLE which cause compilation errors in the metalink
code (as it used the gcrypt fallback instead as a result). See the
below autobuild for an example of the error:

  https://curl.se/dev/log.cgi?id=20210225123226-30704#prob1

This removes all uses of USE_GNUTLS_NETTLE and also removes the
gcrypt support from the metalink code while at it.

Closes #6656
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-02-25 18:20:33 +01:00
Daniel Gustafsson
82c583dcf0 cookies: Support multiple -b parameters
Previously only a single -b cookie parameter was supported with the last
one winning.  This adds support for supplying multiple -b params to have
them serialized semicolon separated.  Both cookiefiles and cookies can be
entered multiple times.

Closes #6649
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2021-02-25 18:12:28 +01:00
Viktor Szakats
44872aefc2
http: add support to read and store the referrer header
- add CURLINFO_REFERER libcurl option
- add --write-out '%{referer}' command-line option
- extend --xattr command-line option to fill user.xdg.referrer.url extended
  attribute with the referrer (if there was any)

Closes #6591
2021-02-19 13:57:19 +00:00
Viktor Szakats
cdb630655d
http: do not add a referrer header with empty value
Previously an empty 'Referer:' header was added to the HTTP request when
passing `--referer ';auto'` or `--referer ''` on the command-line. This
patch makes `--referer` work like `--header 'Referer:'` and will only add
the header if it has a non-zero length value.

Reviewed-by: Jay Satiro
Closes #6610
2021-02-16 10:19:37 +00:00
Jay Satiro
53022e1893 doh: add options to disable ssl verification
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
  CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
  same as their respective counterparts.

- New curl tool options --doh-insecure and --doh-cert-status do the same
  as their respective counterparts.

Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.

Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676

Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
2021-02-14 18:20:48 -05:00
Daniel Stenberg
1560cd1f9f
curl: provide libgsasl version and feature info in -V output
Closes #6592
2021-02-11 16:16:45 +01:00
Daniel Stenberg
8a964cb217
curl: add --fail-with-body
Prevent both --fail and --fail-with-body on the same command line.

Verify with test 349, 360 and 361.

Closes #6449
2021-02-11 08:35:27 +01:00
Jay Satiro
3183217246 tool_help: add missing argument for --create-file-mode
Prior to this change the required argument was not shown in curl --help.

before:
     --create-file-mode File mode for created files

after:
     --create-file-mode <mode> File mode (octal) for created files

Reported-by: ZimCodes@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/6590
2021-02-10 14:10:49 -05:00
Viktor Szakats
2dfe3d7093
Makefile.m32: add support for libgsasl dependency
Reviewed-by: Marcel Raad
Closes #6586
2021-02-10 18:48:29 +00:00
Marcel Raad
fa624f0baa
tool_paramhlp: reduce variable scope
Closes https://github.com/curl/curl/pull/6576
2021-02-10 16:47:15 +01:00
Jay Satiro
65ca229461 tool_writeout: refactor write-out and write-out json
- Deduplicate the logic used by write-out and write-out json.

Rather than have separate writeLong, writeString, etc, logic for
each of write-out and write-out json instead have respective shared
functions that can output either format and a 'use_json' parameter to
indicate whether it is json that is output.

This will make it easier to maintain. Rather than have to go through
two sets of logic now we only have to go through one.

- Support write-out %{errormsg} and %{exitcode} in json.

- Clarify in the doc that %{exitcode} is the exit code of the transfer.

Prior to this change it just said "The numerical exitcode" which
implies it's the exit code of the tool, and it's not necessarily that.

Closes https://github.com/curl/curl/pull/6544
2021-02-09 02:48:27 -05:00
Daniel Stenberg
2605a6fabb
copyright: fix year ranges in need of updates 2021-02-03 08:02:36 +01:00
Dmitry Wagin
796ce293de
http: improve AWS HTTP v4 Signature auth
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.

Closes #6524
2021-01-30 23:48:22 +01:00
Daniel Stenberg
f6d13090d4
copyright: fix missing year (range) updates 2021-01-29 14:35:13 +01:00
Daniel Stenberg
553f286a8f
tool_writeout: make %{errormsg} blank for no errors
Closes #6539
2021-01-28 07:55:25 +01:00
Gisle Vanem
3611f6a41e build: fix djgpp builds
- Update build instructions in packages/DOS/README

- Extend 'VPATH' with 'vquic' and 'vssh'.

- Allow 'Makefile.dist' to build both 'lib' and 'src'.

- Allow using the Windows hosted djgpp cross compiler to build for MSDOS
  under Windows.

- 'USE_SSL' -> 'USE_OPENSSL'

- Added a 'link_EXE' macro. Etc, etc.

- Linking 'curl.exe' needs '$(CURLX_CFILES)' too.

- Do not pick-up '../lib/djgpp/*.o' files. Recompile locally.

- Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'.

- Remove 'djgpp-clean'

- Adapt to new C-ares directory structure

- Use conditional variable assignments

Clarify the 'conditional variable assignment' in 'common.dj'.

Closes https://github.com/curl/curl/pull/6382
2021-01-27 22:08:57 -05:00
Daniel Stenberg
a705f28bb3
curl: include the file name in --xattr/--remote-time error msgs 2021-01-27 08:45:11 +01:00
Daniel Stenberg
f42ee77adb
curl: s/config->global/global/ in single_transfer() 2021-01-27 08:45:11 +01:00
Daniel Stenberg
71acece933
curl: move fprintf outputs to warnf
For setting and getting time of the download. To make the outputs
respect --silent etc.

Reported-by: Viktor Szakats
Fixes #6533
Closes #6535
2021-01-27 08:45:02 +01:00
Daniel Stenberg
8f32ead8bf
tool_operate: spellfix a comment 2021-01-14 13:40:12 +01:00
Jay Satiro
3831043eff tool_doswin: Restore original console settings on CTRL signal
- Move Windows terminal init code from tool_main to tool_doswin.

- Restore the original console settings on CTRL+C and CTRL+BREAK.

Background: On Windows the curl tool changes the console settings to
enable virtual terminal processing (eg color output) if supported
(ie Win 10). The original settings are restored on exit but prior to
this change were not restored in the case of the CTRL signals.

Windows VT behavior varies depending on console/powershell/terminal;
refer to the discussion in #6226.

Assisted-by: Rich Turner

Closes https://github.com/curl/curl/pull/6226
2021-01-12 03:44:48 -05:00
Emil Engler
3d78301106
help: update to current codebase
This commit bumps the help to the current state of the project.

Closes #6437
2021-01-12 08:09:22 +01:00
Daniel Stenberg
f52c6981c5
curl.h: add CURLPROTO_GOPHERS as own protocol identifier
Follow-up to a1f06f32b8, to make sure it can be handled separately
from plain gopher.

Closes #6418
2021-01-06 15:23:41 +01:00
Jay Satiro
df1240ccac tool_operate: fix the suppression logic of some error messages
- Fix the failed truncation and failed writing body error messages to
  not be shown unless error messages are shown. (ie the user has
  specified -sS, or has not specified -s).

- Also prefix same error messages with "curl: ", for example:
  curl: (23) Failed to truncate, exiting

Prior to this change the failed truncation error messages would be shown
if not -s, but did not account for -sS which should show.

Prior to this change the failed writing body error messages would be
shown always.

Ref: https://curl.se/docs/manpage.html#-S

Bug: https://curl.se/mail/archive-2020-12/0017.html
Reported-by: Hongyi Zhao

Closes https://github.com/curl/curl/pull/6402
2021-01-04 18:02:50 -05:00
Emil Engler
725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00