1
0
mirror of https://github.com/moparisthebest/curl synced 2024-12-23 08:38:49 -05:00
Commit Graph

108 Commits

Author SHA1 Message Date
Daniel Stenberg
1763aceb0c
http: limit the initial send amount to used upload buffer size
Previously this logic would cap the send to CURL_MAX_WRITE_SIZE bytes,
but for the situations where a larger upload buffer has been set, this
function can benefit from sending more bytes. With default size used,
this does the same as before.

Also changed the storage of the size to an 'unsigned int' as it is not
allowed to be set larger than 2M.

Also added cautions to the man pages about changing buffer sizes in
run-time.

Closes #7022
2021-05-07 08:51:39 +02:00
Gilles Vollant
77fc3859b2 SSL: support in-memory CA certs for some backends
- New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to
  specify in-memory PEM certificates for OpenSSL, Schannel (Windows)
  and Secure Transport (Apple) SSL backends.

Prior to this change PEM certificates could only be imported from a file
and not from memory.

Co-authored-by: moparisthebest@users.noreply.github.com

Ref: https://github.com/curl/curl/pull/4679
Ref: https://github.com/curl/curl/pull/5677
Ref: https://github.com/curl/curl/pull/6109

Closes https://github.com/curl/curl/pull/6662
2021-05-05 02:29:16 -04:00
Jay Satiro
54e7475016 schannel: Disable auto credentials; add an option to enable it
- Disable auto credentials by default. This is a breaking change
  for clients that are using it, wittingly or not.

- New libcurl ssl option value CURLSSLOPT_AUTO_CLIENT_CERT tells libcurl
  to automatically locate and use a client certificate for
  authentication, when requested by the server.

- New curl tool options --ssl-auto-client-cert and
  --proxy-ssl-auto-client-cert map to CURLSSLOPT_AUTO_CLIENT_CERT.

This option is only supported for Schannel (the native Windows SSL
library). Prior to this change Schannel would, with no notification to
the client, attempt to locate a client certificate and send it to the
server, when requested by the server. Since the server can request any
certificate that supports client authentication in the OS certificate
store it could be a privacy violation and unexpected.

Fixes https://github.com/curl/curl/issues/2262
Reported-by: Jeroen Ooms
Assisted-by: Wes Hinsley
Assisted-by: Rich FitzJohn

Ref: https://curl.se/mail/lib-2021-02/0066.html
Reported-by: Morten Minde Neergaard

Closes https://github.com/curl/curl/pull/6673
2021-04-22 16:53:37 -04:00
Daniel Stenberg
063d3f3b96
tidy-up: make conditional checks more consistent
... remove '== NULL' and '!= 0'

Closes #6912
2021-04-22 09:10:17 +02:00
Daniel Stenberg
d71ff2b9db
hsts: enable by default
No longer considered experimental.

Closes #6700
2021-04-19 08:22:16 +02:00
Daniel Stenberg
eff614fb02
vtls: refuse setting any SSL version
... previously they were supported if a TLS library would (unexpectedly)
still support them, but from this change they will be refused already in
curl_easy_setopt(). SSLv2 and SSLv3 have been known to be insecure for
many years now.

Closes #6773
2021-04-19 08:16:02 +02:00
Daniel Stenberg
520bd5225c
cookie: CURLOPT_COOKIEFILE set to NULL switches off cookies
Add test 676 to verify that setting CURLOPT_COOKIEFILE to NULL again clears
the cookiejar from memory.

Reported-by: Stefan Karpinski
Fixes #6889
Closes #6891
2021-04-14 23:09:36 +02:00
Daniel Stenberg
95cbcec8f9
urldata: merge "struct DynamicStatic" into "struct UrlState"
Both were used for the same purposes and there was no logical separation
between them. Combined, this also saves 16 bytes in less holes in my
test build.

Closes #6798
2021-03-26 23:19:20 +01:00
Daniel Stenberg
32a71333d4
setopt: error on CURLOPT_HTTP09_ALLOWED set true with Hyper
Not supported.

Closes #6727
2021-03-11 22:45:12 +01:00
Daniel Stenberg
70472a44de
urldata: remove the _ORIG suffix from string names
It doesn't provide any useful info but only makes the names longer.

Closes #6624
2021-02-19 08:07:19 +01:00
Jay Satiro
53022e1893 doh: add options to disable ssl verification
- New libcurl options CURLOPT_DOH_SSL_VERIFYHOST,
  CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the
  same as their respective counterparts.

- New curl tool options --doh-insecure and --doh-cert-status do the same
  as their respective counterparts.

Prior to this change DOH SSL certificate verification settings for
verifyhost and verifypeer were supposed to be inherited respectively
from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug
were not. As a result DOH verification remained at the default, ie
enabled, and it was not possible to disable. This commit changes
behavior so that the DOH verification settings are independent and not
inherited.

Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676

Fixes https://github.com/curl/curl/issues/4578
Closes https://github.com/curl/curl/pull/6597
2021-02-14 18:20:48 -05:00
Daniel Stenberg
46620b9743
http: use credentials from transfer, not connection
HTTP auth "accidentally" worked before this cleanup since the code would
always overwrite the connection credentials with the credentials from
the most recent transfer and since HTTP auth is typically done first
thing, this has not been an issue. It was still wrong and subject to
possible race conditions or future breakage if the sequence of functions
would change.

The data.set.str[] strings MUST remain unmodified exactly as set by the
user, and the credentials to use internally are instead set/updated in
state.aptr.*

Added test 675 to verify different credentials used in two requests done
over a reused HTTP connection, which previously behaved wrongly.

Fixes #6542
Closes #6545
2021-02-13 22:36:15 +01:00
Daniel Stenberg
88dd1a8a11
urldata: don't touch data->set.httpversion at run-time
Rename it to 'httpwant' and make a cloned field in the state struct as
well for run-time updates.

Also: refuse non-supported HTTP versions. Verified with test 129.

Closes #6585
2021-02-12 08:13:37 +01:00
Daniel Stenberg
528f71c2ec
ftp: add 'list_only' to the transfer state struct
and rename it from 'ftp_list_only' since it is also used for SSH and
POP3. The state is updated internally for 'type=D' FTP URLs.

Added test case 1570 to verify.

Closes #6578
2021-02-09 14:06:28 +01:00
Daniel Stenberg
0c51036eb6
ftp: never set data->set.ftp_append outside setopt
Since the set value then risks getting used like that when the easy
handle is reused by the application.

Also: renamed the struct field from 'ftp_append' to 'remote_append'
since it is also used for SSH protocols.

Closes #6579
2021-02-08 22:44:37 +01:00
Daniel Stenberg
457e864f5e
urldata: fix build without HTTP and MQTT
Reported-by: Joseph Chen
Fixes #6562
Closes #6563
2021-02-04 23:08:07 +01:00
Dmitry Wagin
796ce293de
http: improve AWS HTTP v4 Signature auth
- Add support services without region and service prefixes in
the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
by providing region and service parameters via aws-sigv4 option.
- Add [:region[:service]] suffix to aws-sigv4 option;
- Fix memory allocation errors.
- Refactor memory management.
- Use Curl_http_method instead() STRING_CUSTOMREQUEST.
- Refactor canonical headers generating.
- Remove repeated sha256_to_hex() usage.
- Add some docs fixes.
- Add some codestyle fixes.
- Add overloaded strndup() for debug - curl_dbg_strndup().
- Update tests.

Closes #6524
2021-01-30 23:48:22 +01:00
Daniel Stenberg
e76b058f3c
urldata: store ip version in a single byte
Closes #6534
2021-01-27 09:19:12 +01:00
Daniel Stenberg
942cf12c2f
urldata: make magic be the first struct field
By making the `magic` identifier the same size and at the same place
within the structs (easy, multi, share), libcurl will be able to more
reliably detect and safely error out if an application passes in the
wrong handle to APIs. Easier to detect and less likely to cause crashes
if done.

Such mixups can't be detected at compile-time due to them being
typedefed void pointers - unless `CURL_STRICTER` is defined.

Closes #6484
2021-01-20 08:51:01 +01:00
Emil Engler
725ec470e2
language: s/behaviour/behavior/g
We currently use both spellings the british "behaviour" and the american
"behavior". However "behavior" is more used in the project so I think
it's worth dropping the british name.

Closes #6395
2021-01-02 23:35:59 +01:00
Paul Groke
8324dc8b1a
dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries
Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix
(similar to the existing '-' prefix for removing entries) to add
DNS cache entries that will time out just like entries that are added
by libcurl itself.

Append " (non-permanent)" to info log message in case a non-permanent
entry is added.

Adjust relevant comments to reflect the new behavior.

Adjust documentation.

Extend unit1607 to test the new functionality.

Closes #6294
2020-12-29 17:49:36 +01:00
Matthias Gatto
eb69797766
http: Make the call to v4 signature
This patch allow to call the v4 signature introduce in previous commit

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>
2020-12-21 16:28:03 +01:00
Daniel Stenberg
edc89839be
setopt: adjust to Hyper and disabled HTTP builds 2020-12-18 09:58:04 +01:00
Klaus Crusius
d6bfbfadd3
cmake: check for linux/tcp.h
The HAVE_LINUX_TCP_H define was not set by cmake.

Closes #6252
2020-12-01 12:32:55 +01:00
Daniel Stenberg
4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg
2cfc4ed983
hsts: add read/write callbacks
- read/write callback options
- man pages for the 4 new setopts
- test 1915 verifies the callbacks

Closes #5896
2020-11-03 16:08:48 +01:00
Daniel Stenberg
7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Daniel Stenberg
96450a1a33
alt-svc: enable by default
Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported.

alt-svc support in curl is no longer considered experimental

Closes #5868
2020-10-25 23:08:54 +01:00
Daniel Stenberg
a8e08a87df
setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument
Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the
option is, yeah, not known. Clarified this in the setopt man page too.

Closes #5993
2020-09-22 09:04:13 +02:00
Daniel Stenberg
dd51f04b11
setopt: avoid curl_ on local variable
Closes #5906
2020-09-02 22:41:59 +02:00
Daniel Stenberg
e37e446868
mqtt: enable by default
No longer considered experimental.

Closes #5858
2020-08-31 09:45:09 +02:00
Michael Baentsch
ede125b7b7
tls: add CURLOPT_SSL_EC_CURVES and --curves
Closes #5892
2020-08-30 17:24:04 +02:00
Daniel Stenberg
6ebe63fac2
options: API for meta-data about easy options
const struct curl_easyoption *curl_easy_option_by_name(const char *name);

 const struct curl_easyoption *curl_easy_option_by_id (CURLoption id);

 const struct curl_easyoption *
 curl_easy_option_next(const struct curl_easyoption *prev);

The purpose is to provide detailed enough information to allow for
example libcurl bindings to get option information at run-time about
what easy options that exist and what arguments they expect.

Assisted-by: Jeroen Ooms
Closes #5365
2020-08-27 14:17:36 +02:00
Daniel Stenberg
bc3b5bcf87
setopt: if the buffer exists, refuse the new BUFFERSIZE
The buffer only exists during transfer and then we shouldn't change the
size (the setopt is not documented to work then).

Reported-by: Harry Sintonen
Closes #5842
2020-08-24 09:08:56 +02:00
Daniel Stenberg
91cb16b21f
setopt: unset NOBODY switches to GET if still HEAD
Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented
action but before 7.71.0 that used to switch back to GET and with this
change (assuming the method is still set to HEAD) this behavior is
brought back.

Reported-by: causal-agent on github
Fixes #5725
Closes #5728
2020-07-27 19:33:16 +02:00
Daniel Stenberg
9c845be279
urldata: let the HTTP method be in the set.* struct
When the method is updated inside libcurl we must still not change the
method as set by the user as then repeated transfers with that same
handle might not execute the same operation anymore!

This fixes the libcurl part of #5462

Test 1633 added to verify.

Closes #5499
2020-06-02 16:30:36 +02:00
Daniel Stenberg
f3d501dc67
build: disable more code/data when built without proxy support
Added build to travis to verify

Closes #5466
2020-05-30 23:18:16 +02:00
Daniel Stenberg
c4e6968127
url: alloc the download buffer at transfer start
... and free it as soon as the transfer is done. It removes the extra
alloc when a new size is set with setopt() and reduces memory for unused
easy handles.

In addition: the closure_handle now doesn't use an allocated buffer at
all but the smallest supported size as a stack based one.

Closes #5472
2020-05-30 23:14:33 +02:00
Gilles Vollant
7f40633422
setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency
Closes #5431
2020-05-23 23:19:13 +02:00
Gilles Vollant
cac5374298
setopt: support certificate options in memory with struct curl_blob
This change introduces a generic way to provide binary data in setopt
options, called BLOBs.

This change introduces these new setopts:

CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB,
CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB.

Reviewed-by: Daniel Stenberg
Closes #5357
2020-05-15 13:03:59 +02:00
Gilles Vollant
148534db57
CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl)
Closes #4346
2020-05-08 15:55:04 +02:00
Johannes Schindelin
5450428491 schannel: add "best effort" revocation check option
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
  --ssl-revoke-best-effort to allow a "best effort" revocation check.

A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.

---

When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:

	Unknown error (0x80092012) - The revocation function was unable
	to check revocation for the certificate.

Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.

This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.

In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.

Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).

To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.

Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.

The test 2070 was added to verify this behavior, adapted from 310.

Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Closes https://github.com/curl/curl/pull/4981
2020-03-18 03:23:39 -04:00
Ernst Sjöstrand
c8f086bcc3
ares: store dns parameters for duphandle
With c-ares the dns parameters lives in ares_channel. Store them in the
curl handle and set them again in easy_duphandle.

Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0.

Fixes #4893
Closes #5020
Signed-off-by: Ernst Sjöstrand <ernst.sjostrand@verisure.com>
2020-03-07 13:50:23 +01:00
Daniel Stenberg
0c76795caf
cleanup: comment typos
Spotted by 'codespell'

Closes #4957
2020-02-21 08:38:00 +01:00
Pavel Volgarev
4a4609bf3c
smtp: Allow RCPT TO command to fail for some recipients
Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS.

Verified with the new tests 3002-3007

Closes #4816
2020-01-21 10:40:19 +01:00
Jay Satiro
689443bf42 lib: fix some loose ends for recently added CURLSSLOPT_NO_PARTIALCHAIN
Add support for CURLSSLOPT_NO_PARTIALCHAIN in CURLOPT_PROXY_SSL_OPTIONS
and OS400 package spec.

Also I added the option to the NameValue list in the tool even though it
isn't exposed as a command-line option (...yet?). (NameValue stringizes
the option name for the curl cmd -> libcurl source generator)

Follow-up to 564d88a which added CURLSSLOPT_NO_PARTIALCHAIN.

Ref: https://github.com/curl/curl/pull/4655
2019-12-05 19:17:31 -05:00
Jay Satiro
0edf75865a setopt: Fix ALPN / NPN user option when built without HTTP2
- Stop treating lack of HTTP2 as an unknown option error result for
  CURLOPT_SSL_ENABLE_ALPN and CURLOPT_SSL_ENABLE_NPN.

Prior to this change it was impossible to disable ALPN / NPN if libcurl
was built without HTTP2. Setting either option would result in
CURLE_UNKNOWN_OPTION and the respective internal option would not be
set. That was incorrect since ALPN and NPN are used independent of
HTTP2.

Reported-by: Shailesh Kapse

Fixes https://github.com/curl/curl/issues/4668
Closes https://github.com/curl/curl/pull/4672
2019-12-05 18:38:40 -05:00
Daniel Stenberg
564d88a8bd
openssl: CURLSSLOPT_NO_PARTIALCHAIN can disable partial cert chains
Closes #4655
2019-12-03 16:28:50 +01:00
Daniel Stenberg
249541f12f
cookies: change argument type for Curl_flush_cookies
The second argument is really a 'bool' so use that and pass in TRUE/FALSE
to make it clear.

Closes #4455
2019-10-03 22:56:28 +02:00
Daniel Stenberg
ed73509157
setopt: handle ALTSVC set to NULL 2019-09-27 16:26:50 +02:00