Commit Graph

2341 Commits

Author SHA1 Message Date
Marcel Raad 0c6112a139
WIN32: stop forcing narrow-character API
Except where the results are only used for character output.
getenv is not touched because it's part of the public API, and having
it return UTF-8 instead of ANSI would be a breaking change.

Fixes https://github.com/curl/curl/issues/5658
Fixes https://github.com/curl/curl/issues/5712
Closes https://github.com/curl/curl/pull/5718
2020-07-27 10:42:38 +02:00
Marcel Raad c90e48c005
util: silence conversion warnings
timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might
be a 64-bit integer. This is the case when building for recent macOS
versions, for example. Just treat tv_usec as an int, which should
hopefully always be sufficient on systems with
`HAVE_CLOCK_GETTIME_MONOTONIC`.

Closes https://github.com/curl/curl/pull/5695
2020-07-19 10:34:58 +02:00
Daniel Stenberg 3c4c4ec2ee
strdup: remove the odd strlen check
It confuses code analyzers with its use of -1 for unsigned value. Also,
a check that's not normally used in strdup() code - and not necessary.

Closes #5697
2020-07-18 12:37:25 +02:00
Daniel Stenberg b331a5fa09
file2memory: use a define instead of -1 unsigned value
... to use the maximum value for 'size_t' when detecting integer overflow.
Changed the limit to max/4 as already that seems unreasonably large.

Codacy didn't like the previous approach.

Closes #5683
2020-07-16 00:34:05 +02:00
Daniel Stenberg f5e6907d52
curl: add %{method} to the -w variables
Gets the CURLINFO_EFFECTIVE_METHOD from libcurl.

Added test 1197 to verify.
2020-07-14 17:53:45 +02:00
Viktor Szakats 8fa3f7809a
windows: add unicode to feature list
Reviewed-by: Marcel Raad
Reviewed-by: Marc Hörsken

Closes #5491
2020-07-14 08:30:17 +00:00
Gilles Vollant e13357b14b
content_encoding: add zstd decoding support
include zstd curl patch for Makefile.m32 from vszakats
and include Add CMake support for zstd from Peter Wu

Helped-by: Viktor Szakats
Helped-by: Peter Wu
Closes #5453
2020-07-12 18:11:37 +02:00
Kamil Dudka d2fd845c35 tool_getparam: make --krb option work again
It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301.

Bug: https://bugzilla.redhat.com/1833193
Closes #5640
2020-07-03 00:04:47 +02:00
Daniel Stenberg 032e838b73
terminology: call them null-terminated strings
Updated terminology in docs, comments and phrases to refer to C strings
as "null-terminated". Done to unify with how most other C oriented docs
refer of them and what users in general seem to prefer (based on a
single highly unscientific poll on twitter).

Reported-by: coinhubs on github
Fixes #5598
Closes #5608
2020-06-28 00:31:24 +02:00
Jay Satiro cb85275377 tool_cb_hdr: Fix etag warning output and return code
- Return 'failure' on failure, to follow the existing style.

- Put Warning: and the warning message on the same line.

Ref: https://github.com/curl/curl/issues/5610

Closes https://github.com/curl/curl/pull/5612
2020-06-26 09:48:58 -04:00
Jay Satiro bc052cc878 tool_operate: Don't use Windows CA store as a fallback
Background:

148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate
store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides
CURLOPT_CAINFO if both are set. The curl tool will fall back to
CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set
via CURLOPT_CAINFO.

Problem:

libcurl may be built with hardcoded paths to a certificate bundle or
directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are
ignored.

Solution:

A solution is still being discussed but since there's an impending
release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool.

Ref: https://github.com/curl/curl/issues/5585
2020-06-22 15:26:07 -04:00
Daniel Stenberg 8236aba585
tool_getparam: -i is not OK if -J is used
Reported-by: sn on hackerone
Bug: https://curl.haxx.se/docs/CVE-2020-8177.html
2020-06-22 00:49:58 +02:00
Daniel Stenberg fa4fbc533f
urlglob: treat literal IPv6 addresses with zone IDs as a host name
... and not as a "glob". Now done by passing the supposed host to the
URL parser which supposedly will do a better job at identifying "real"
numerical IPv6 addresses.

Reported-by: puckipedia on github
Fixes #5576
Closes #5579
2020-06-18 16:43:06 +02:00
Daniel Stenberg ada7bd1191
tool_getparam: repair the error message for unknown flag
Follow-up to 9e5669f388
Detected by Coverity CID 1464582 ("Logically dead code")

Closes #5577
2020-06-18 13:59:42 +02:00
Jay Satiro f54b6c4bc2 tool_getparam: fix memory leak in parse_args
Prior to this change in Windows Unicode builds most parsed options would
not be freed.

Found using _CrtDumpMemoryLeaks().

Ref: https://github.com/curl/curl/issues/5545
2020-06-09 11:49:44 -04:00
Daniel Stenberg a3e972313b
tool_cfgable: free login_options at exit
Memory leak
Reported-by: Geeknik Labs
Fixes #5535
Closes #5536
2020-06-08 08:41:11 +02:00
Viktor Szakats 308c243db5
all: fix codespell errors
Reviewed-by: Jay Satiro
Reviewed-by: Daniel Stenberg
Closes https://github.com/curl/curl/pull/5452
2020-05-25 19:44:04 +00:00
Daniel Stenberg d75e6ce85a
copyright: updated year ranges out of sync
... and whitelisted a few more files in the the copyright.pl script.
2020-05-24 00:02:33 +02:00
Daniel Stenberg 006d2991bd
curl: remove -J "informational" written on stdout
curl would previously show "curl: Saved to filename 'name from header'"
if -J was used and a name was picked from the Content-Disposition
header. That output could interfer with other stdout output, such as -w.

This commit removes that output line.
Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html
Reported-by: Коваленко Анатолий Викторович
Closes #5435
2020-05-23 23:12:53 +02:00
Daniel Stenberg 33d60d09f3
tool_paramhlp: fixup C89 mistake
Follow-up to c5f0a9db22.
2020-05-19 08:58:33 +02:00
Siva Sivaraman c5f0a9db22
tool_paramhlp: fixed potentially uninitialized strtol() variable
Seems highly unlikely to actually be possible, but better safe than
sorry.

Closes #5417
2020-05-19 08:55:27 +02:00
Siva Sivaraman c4df1f75ad
tool_operate: fixed potentially uninitialized variables
... in curl_easy_getinfo() calls. They're harmless but clearing the
variables makes the code safer and comforts the reader.

Closes #5416
2020-05-19 08:53:41 +02:00
Gilles Vollant cac5374298
setopt: support certificate options in memory with struct curl_blob
This change introduces a generic way to provide binary data in setopt
options, called BLOBs.

This change introduces these new setopts:

CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB,
CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB.

Reviewed-by: Daniel Stenberg
Closes #5357
2020-05-15 13:03:59 +02:00
Daniel Stenberg 8df455479f
source cleanup: remove all custom typedef structs
- Stick to a single unified way to use structs
 - Make checksrc complain on 'typedef struct {'
 - Allow them in tests, public headers and examples

 - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually
   typedef different types/structs depending on build conditions.

Closes #5338
2020-05-15 08:54:42 +02:00
Marcel Raad 9e5669f388
tool: support UTF-16 command line on Windows
- use `wmain` instead of `main` when `_UNICODE` is defined [0]
- define `argv_item_t` as `wchar_t *` in this case
- use the curl_multibyte gear to convert the command-line arguments to
  UTF-8

This makes it possible to pass parameters with characters outside of
the current locale on Windows, which is required for some tests, e.g.
the IDN tests. Out of the box, this currently only works with the
Visual Studio project files, which default to Unicode, and winbuild
with the `ENABLE_UNICODE` option.

[0] https://devblogs.microsoft.com/oldnewthing/?p=40643

Ref: https://github.com/curl/curl/issues/3747
Closes https://github.com/curl/curl/pull/3784
2020-05-14 18:13:36 +02:00
Marcel Raad a55c835e6b
curl_multibyte: add to curlx
This will also be needed in the tool and tests.

Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512
Closes https://github.com/curl/curl/pull/3784
2020-05-14 18:13:27 +02:00
Daniel Stenberg 66b0775763
checksrc: enhance the ASTERISKSPACE and update code accordingly
Fine: "struct hello *world"

Not fine: "struct hello* world" (and variations)

Closes #5386
2020-05-14 00:02:05 +02:00
Daniel Stenberg cd35371163
tool: shorten 3 --help descriptions
--happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort

gen.pl already warned about these lines but we didn't listen

Closes #5379
2020-05-13 16:37:22 +02:00
Jay Satiro 79d60a9282 retry-all-errors.d: Shorten the summary line
Follow-up to b995bb5 from a few moments ago.

Reported-by: Daniel Stenberg

Ref: https://github.com/curl/curl/commit/b995bb5#r39108929
2020-05-12 03:12:55 -04:00
Jay Satiro b995bb58cb tool: Add option --retry-all-errors to retry on any error
The "sledgehammer" of retrying.

Closes https://github.com/curl/curl/pull/5185
2020-05-12 03:00:15 -04:00
Daniel Stenberg 6f63ab41b2
tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present
Reported-by: Marcel Raad
Follow-up to 148534db5
Fixes #5367
Closes #5369
2020-05-10 15:33:51 +02:00
Gilles Vollant 148534db57
CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl)
Closes #4346
2020-05-08 15:55:04 +02:00
Daniel Stenberg 878214df44
tool: do not declare functions with Curl_ prefix
To avoid collision risks with private libcurl symbols when linked with
static versions (or just versions not hiding internal symbols).

Reported-by: hydra3333 on github
Fixes #5219
Closes #5234
2020-04-14 08:12:27 +02:00
Nathaniel R. Lewis 7fa1578471
cmake: add aliases so exported target names are available in tree
Reviewed-by: Brad King
Closes #5206
2020-04-13 23:27:35 +02:00
Kwon-Young Choi a448a4ce26
curl: allow both --etag-compare and --etag-save with same file name
This change inverse the order of processing for the --etag-compare and
--etag-save option to process first --etag-compare. This in turn allows
to use the same file name to compare and save an etag.

The original behavior of not failing if the etag file does not exists is
conserved.

Fixes #5179
Closes #5180
2020-04-05 01:07:52 +02:00
Daniel Stenberg d1369fe216
misc: copyright year updates
Follow-up to 7a71965e9
2020-03-31 11:02:18 +02:00
Harry Sintonen 7a71965e97
build: fixed build for systems with select() in unistd.h
Closes #5169
2020-03-31 10:59:06 +02:00
Daniel Stenberg 0e607542dc
cleanup: insert newline after if() conditions
Our code style mandates we put the conditional block on a separate
line. These mistakes are now detected by the updated checksrc.
2020-03-30 16:05:30 +02:00
Daniel Stenberg 93fafb93db
curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented
Considered experimental and therefore we can do this.

Closes #5157
2020-03-29 23:28:49 +02:00
Michael Kaufmann 884de1a763 writeout_json: Fix data type issues
Load long values correctly (e.g. for http_code).

Use curl_off_t (not long) for:
- size_download (CURLINFO_SIZE_DOWNLOAD_T)
- size_upload (CURLINFO_SIZE_UPLOAD_T)

The unit for these values is bytes/second, not microseconds:
- speed_download (CURLINFO_SPEED_DOWNLOAD_T)
- speed_upload (CURLINFO_SPEED_UPLOAD_T)

Fixes #5131
Closes #5152
2020-03-27 23:32:25 +01:00
Daniel Stenberg 361d4f3fdc
docs/make: generate curl.1 from listed files only
Previously it rendered the page from files matching "*.d" in the correct
directory, which worked fine in git builds when the files were added but
made it easy to forget adding the files to the dist.

Now, only man page sections listed in DPAGES in Makefile.inc will be
used, thus "forcing" us to update this to get the man page right and get
it included in the dist at the same time.

Ref: #5146
Closes #5149
2020-03-26 13:25:59 +01:00
Daniel Stenberg 9a8b3b3e13
copyright: fix out-of-date copyright ranges and missing headers
Reported by the new script 'scripts/copyright.pl'. The script has a
regex whitelist for the files that don't need copyright headers.

Removed three (mostly usesless) README files from docs/

Closes #5141
2020-03-24 15:05:59 +01:00
Rici Lake 4e0b4fee4a
cmdline: fix handling of OperationConfig linked list (--next)
Ensures that -K/--config inserts new items at the end of the list
instead of overwriting the second item, and that after a -K/--config
option has been parsed, the option parser's view of the current config
is update.

Fixes #5120
Closes #5123
2020-03-20 11:14:12 +01:00
Daniel Stenberg 8d9802b0ae
writeout_json: use curl_off_t printf() option for the time output
Follow-up to: 04c03416e6

Closes #5115
2020-03-18 23:31:37 +01:00
Daniel Stenberg 0ae463ffd4
tool_setopt: correct the copyright year range
Follow-up to 5450428491
2020-03-18 08:33:28 +01:00
Johannes Schindelin 5450428491 schannel: add "best effort" revocation check option
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
  --ssl-revoke-best-effort to allow a "best effort" revocation check.

A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.

---

When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:

	Unknown error (0x80092012) - The revocation function was unable
	to check revocation for the certificate.

Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.

This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.

In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.

Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).

To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.

Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.

The test 2070 was added to verify this behavior, adapted from 310.

Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Closes https://github.com/curl/curl/pull/4981
2020-03-18 03:23:39 -04:00
Viktor Szakats 7284061361 windows: suppress UI in all CryptAcquireContext() calls
Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters
Reviewed-by: Marc Hörsken
Closes https://github.com/curl/curl/pull/5088
2020-03-17 23:08:02 +00:00
Daniel Stenberg ab9dc5ae2a
writeout_json: add missing comma to fix the HTTP version
Follow-up to 04c03416e6
2020-03-17 16:05:05 +01:00
Mathias Gumz 04c03416e6
writeout: support to generate JSON output
This commit adds support to generate JSON via the writeout feature:

    -w "%{json}"

It leverages the existing infrastructure as much as possible. Thus,
generating the JSON on STDERR is possible by:

    -w "%{stderr}%{json}"

This implements a variant of
https://github.com/curl/curl/wiki/JSON#--write-out-json.

Closes #4870
2020-03-17 15:01:28 +01:00
Daniel Stenberg 95c36fff75
tool_operate: fix add_parallel_transfers when more are in queue
Trying to return early from the function if no new transfers were added
would break the "morep" argument and cause issues. This could lead to
zero content "transfers" (within quotes since they would never be
started) when parallel-max was reduced.

Reported-by: Gavin Wong
Analyzed-by: Jay Satiro
Fixes #4937
Closes #5112
2020-03-16 09:51:34 +01:00