1
0
mirror of https://github.com/moparisthebest/curl synced 2024-11-17 06:55:02 -05:00
Commit Graph

241 Commits

Author SHA1 Message Date
Daniel Stenberg
5ffbb63e42
curl: --doh-url added 2018-09-06 09:17:25 +02:00
Jay Satiro
351c0f3a55
tool_operate: Add http code 408 to transient list for --retry
- Treat 408 request timeout as transient so that curl will retry the
  request if --retry was used.

Closes #2925
2018-09-03 13:19:14 +02:00
Jay Satiro
705cc899e3 tool_operate: Fix setting proxy TLS 1.3 ciphers 2018-08-31 19:40:55 -04:00
Anderson Toshiyuki Sasaki
298d2565e2
ssl: set engine implicitly when a PKCS#11 URI is provided
This allows the use of PKCS#11 URI for certificates and keys without
setting the corresponding type as "ENG" and the engine as "pkcs11"
explicitly. If a PKCS#11 URI is provided for certificate, key,
proxy_certificate or proxy_key, the corresponding type is set as "ENG"
if not provided and the engine is set to "pkcs11" if not provided.

Acked-by: Nikos Mavrogiannopoulos
Closes #2333
2018-08-08 09:46:01 +02:00
Björn Stenberg
946ce5b61f
option: disallow username in URL
Adds CURLOPT_DISALLOW_USERNAME_IN_URL and --disallow-username-in-url. Makes
libcurl reject URLs with a username in them.

Closes #2340
2018-05-31 11:27:16 +02:00
Daniel Stenberg
050c93c46f
setopt: add TLS 1.3 ciphersuites
Adds CURLOPT_TLS13_CIPHERS and CURLOPT_PROXY_TLS13_CIPHERS.

curl: added --tls13-ciphers and --proxy-tls13-ciphers

Fixes #2435
Reported-by: zzq1015 on github
Closes #2607
2018-05-29 16:12:52 +02:00
Daniel Stenberg
c1c27625c7
curl: show headers in bold
The feature is only enabled if the output is believed to be a tty.

-J: There's some minor differences and improvements in -J handling, as
now J should work with -i and it actually creates a file first using the
initial name and then *renames* that to the one found in
Content-Disposition (if any).

-i: only shows headers for HTTP transfers now (as documented).
Previously it would also show for pieces of the transfer that were HTTP
(for example when doing FTP over a HTTP proxy).

-i: now shows trailers as well. Previously they were not shown at all.

--libcurl: the CURLOPT_HEADER is no longer set, as the header output is
now done in the header callback.
2018-05-21 20:05:05 +02:00
Daniel Gustafsson
2f13e3d23d
checksrc: force indentation of lines after an else
This extends the INDENTATION case to also handle 'else' statements
and require proper indentation on the following line. Also fixes the
offending cases found in the codebase.

Closes #2532
2018-04-27 00:51:35 +02:00
Dan McNulty
8996300211 schannel: add support for CURLOPT_CAINFO
- Move verify_certificate functionality in schannel.c into a new
  file called schannel_verify.c. Additionally, some structure defintions
  from schannel.c have been moved to schannel.h to allow them to be
  used in schannel_verify.c.

- Make verify_certificate functionality for Schannel available on
  all versions of Windows instead of just Windows CE. verify_certificate
  will be invoked on Windows CE or when the user specifies
  CURLOPT_CAINFO and CURLOPT_SSL_VERIFYPEER.

- In verify_certificate, create a custom certificate chain engine that
  exclusively trusts the certificate store backed by the CURLOPT_CAINFO
  file.

- doc updates of --cacert/CAINFO support for schannel

- Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
  when available. This implements a TODO in schannel.c to improve
  handling of multiple SANs in a certificate. In particular, all SANs
  will now be searched instead of just the first name.

- Update tool_operate.c to not search for the curl-ca-bundle.crt file
  when using Schannel to maintain backward compatibility. Previously,
  any curl-ca-bundle.crt file found in that search would have been
  ignored by Schannel. But, with CAINFO support, the file found by
  that search would have been used as the certificate store and
  could cause issues for any users that have curl-ca-bundle.crt in
  the search path.

- Update url.c to not set the build time CURL_CA_BUNDLE if the selected
  SSL backend is Schannel. We allow setting CA location for schannel
  only when explicitly specified by the user via CURLOPT_CAINFO /
  --cacert.

- Add new test cases 3000 and 3001. These test cases check that the first
  and last SAN, respectively, matches the connection hostname. New test
  certificates have been added for these cases. For 3000, the certificate
  prefix is Server-localhost-firstSAN and for 3001, the certificate
  prefix is Server-localhost-secondSAN.

- Remove TODO 15.2 (Add support for custom server certificate
  validation), this commit addresses it.

Closes https://github.com/curl/curl/pull/1325
2018-04-18 03:59:47 -04:00
Stefan Agner
464a019cbe tool_operate: Fix retry on FTP 4xx to ignore other protocols
Only treat response code as FTP response codes in case the
protocol type is FTP.

This fixes an issue where an HTTP download was treated as FTP
in case libcurl returned with 33. This happens when the
download has already finished and the server responses 416:
  HTTP/1.1 416 Requested Range Not Satisfiable

This should not be treated as an FTP error.

Fixes #2464
Closes #2465
2018-04-07 02:57:57 -04:00
Lawrence Matthews
6baeb6df35
CURLOPT_HAPROXYPROTOCOL: support the HAProxy PROXY protocol
Add --haproxy-protocol for the command line tool

Closes #2162
2018-03-17 11:50:06 +01:00
luz.paz
236402fc2d
cleanup: misc typos in strings and comments
Found via `codespell`

Closes #2389
2018-03-16 11:08:31 +01:00
Jay Satiro
dd027c80fe lib: CURLOPT_HAPPY_EYEBALLS_TIMEOUT => CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS
- In keeping with the naming of our other connect timeout options rename
  CURLOPT_HAPPY_EYEBALLS_TIMEOUT to CURLOPT_HAPPY_EYEBALLS_TIMEOUT_MS.

This change adds the _MS suffix since the option expects milliseconds.
This is more intuitive for our users since other connect timeout options
that expect milliseconds use _MS such as CURLOPT_TIMEOUT_MS,
CURLOPT_CONNECTTIMEOUT_MS, CURLOPT_ACCEPTTIMEOUT_MS.

The tool option already uses an -ms suffix, --happy-eyeballs-timeout-ms.

Follow-up to 2427d94 which added the lib and tool option yesterday.

Ref: https://github.com/curl/curl/pull/2260
2018-02-21 15:16:50 -05:00
Anders Bakken
2427d94c6d url: Add option CURLOPT_HAPPY_EYEBALLS_TIMEOUT
- Add new option CURLOPT_HAPPY_EYEBALLS_TIMEOUT to set libcurl's happy
  eyeball timeout value.

- Add new optval macro CURL_HET_DEFAULT to represent the default happy
  eyeballs timeout value (currently 200 ms).

- Add new tool option --happy-eyeballs-timeout-ms to expose
  CURLOPT_HAPPY_EYEBALLS_TIMEOUT. The -ms suffix is used because the
  other -timeout options in the tool expect seconds not milliseconds.

Closes https://github.com/curl/curl/pull/2260
2018-02-20 17:51:43 -05:00
Michael Kaufmann
d25b050379 time-cond: fix reading the file modification time on Windows
On Windows, stat() may adjust the unix file time by a daylight saving time
offset. Avoid this by calling GetFileTime() instead.

Fixes #2164
Closes #2204
2018-02-05 22:02:10 +01:00
Daniel Stenberg
671f0b506c
Revert "curl: don't set CURLOPT_INTERLEAVEDATA"
This reverts commit 9ffad8eb13.

It was actually added rather recently in 8e8afa82cb due to a crash
that would otherwise happen in the RTSP code. As I don't think we've
fixed that behavior yet, we better keep this work-around until we have
fixed it better.
2017-12-11 00:00:17 +01:00
Daniel Stenberg
9ffad8eb13
curl: don't set CURLOPT_INTERLEAVEDATA
That data is only ever used by the CURLOPT_INTERLEAVEFUNCTION callback
and that option isn't set or used by the curl tool!

Updates the 9 tests that verify --libcurl

Closes #2167
2017-12-08 23:40:54 +01:00
Dan Fandrich
df7839b68c Set and use more necessary options when some protocols are disabled
When curl and libcurl are built with some protocols disabled, they stop
setting and receiving some options that don't make sense with those
protocols.  In particular, when HTTP is disabled many options aren't set
that are used only by HTTP.  However, some options that appear to be
HTTP-only are actually used by other protocols as well (some despite
having HTTP in the name) and should be set, but weren't. This change now
causes some of these options to be set and used for more (or for all)
protocols. In particular, this fixes tests 646 through 649 in an
HTTP-disabled build, which use the MIME API in the mail protocols.
2017-09-30 02:10:54 +02:00
Daniel Stenberg
299896ca09
curl: check fseek() return code and bail on error
Detected by coverity. CID 1418137.
2017-09-20 12:03:44 +02:00
Daniel Stenberg
e5743f08e7
code style: use spaces around pluses 2017-09-11 09:29:50 +02:00
Daniel Stenberg
6b84438d9a
code style: use spaces around equals signs 2017-09-11 09:29:50 +02:00
Patrick Monnerat
fec7a858b8 mime: use in curl cli tool instead of form API.
Extended -F option syntax to support multipart mail messages.
-F keyword headers= added to include custom headers in parts.
Documentation upgraded.
2017-09-02 18:17:33 +01:00
Daniel Stenberg
f412a5aabb
curl: shorten and clean up CA cert verification error message
The previous message was just too long for ordinary people and it was
encouraging users to use `--insecure` a little too easy.

Based-on-work-by: Frank Denis

Closes #1810
Closes #1817
2017-08-22 23:32:43 +02:00
Daniel Stenberg
6b9bc5a4fe
CURL_SIZEOF_LONG: removed, use only SIZEOF_LONG 2017-08-17 10:27:00 +02:00
Viktor Szakats
b7b4dc0d49 ssh: add the ability to enable compression (for SCP/SFTP)
The required low-level logic was already available as part of
`libssh2` (via `LIBSSH2_FLAG_COMPRESS` `libssh2_session_flag()`[1]
option.)

This patch adds the new `libcurl` option `CURLOPT_SSH_COMPRESSION`
(boolean) and the new `curl` command-line option `--compressed-ssh`
to request this `libssh2` feature. To have compression enabled, it
is required that the SSH server supports a (zlib) compatible
compression method and that `libssh2` was built with `zlib` support
enabled.

[1] https://www.libssh2.org/libssh2_session_flag.html

Ref: https://github.com/curl/curl/issues/1732
Closes https://github.com/curl/curl/pull/1735
2017-08-17 03:32:00 -04:00
Kamil Dudka
ce2c3ebda2 curl --socks5-{basic,gssapi}: control socks5 auth
Closes https://github.com/curl/curl/pull/1454
2017-06-28 08:03:00 +02:00
Daniel Stenberg
9b167fd090 --request-target: instead of --strip-path-slash
... and CURLOPT_REQUEST_TARGET instead of CURLOPT_STRIP_PATH_SLASH.

This option instead provides the full "alternative" target to use in the
request, instead of extracting the path from the URL.

Test 1298 and 1299 updated accordingly.

Idea-by: Evert Pot
Suggestion: https://daniel.haxx.se/blog/2017/06/19/options-with-curl/comment-page-1/#comment-18373

Closes #1593
2017-06-21 23:39:10 +02:00
Daniel Stenberg
b778ae4c5e http: add --strip-path-slash and CURLOPT_STRIP_PATH_SLASH
... to enable sending "OPTIONS *" which wasn't possible previously.

This option currently only works for HTTP.

Added test cases 1298 + 1299 to verify

Fixes #1280
Closes #1462
2017-06-19 16:39:22 +02:00
Daniel Stenberg
5385450afd curl: prevent binary output spewed to terminal
... unless "--output -" is used. Binary detection is done by simply
checking for a binary zero in early data.

Added test 1425 1426 to verify.

Closes #1512
2017-06-16 23:38:13 +02:00
Daniel Stenberg
a8e388dd10 curl: remove tool_writeenv.[ch]
... and USE_ENVIRONMENT and --environment. It was once added for RISC OS
support and its platform specific behavior has been annoying ever
since. Added in commit c3c8bbd3b2, mostly unchanged since
then. Most probably not actually used for years.

Closes #1463
2017-05-06 23:13:08 +02:00
Dan Fandrich
4da846a41e tool_operate: use utimes instead of obsolescent utime when available 2017-04-27 22:34:12 +02:00
Daniel Stenberg
96ece5c0ee curl: set a 100K buffer size by default
Test command 'time curl http://localhost/80GB -so /dev/null' on a Debian
Linux.

Before (middle performing run out 9):

 real    0m28.078s
 user    0m11.240s
 sys     0m12.876s

After (middle performing run out 9)

 real    0m26.356s (93.9%)
 user    0m5.324s  (47.4%)
 sys     0m8.368s  (65.0%)

Also, doing SFTP over a 200 millsecond latency link is now about 6 times
faster.

Closes #1446
2017-04-25 11:02:19 +02:00
Marcel Raad
b547fff566
tool_operate: fix MinGW compiler warning
MinGW complains:
tool_operate.c:197:15: error: comparison is always true due to limited range
of data type [-Werror=type-limits]

Fix this by only doing the comparison if 'long' is large enough to hold the
constant it is compared with.

Closes https://github.com/curl/curl/pull/1378
2017-04-05 13:08:16 +02:00
Marcel Raad
446eaa941d
tool_operate: move filetime code to its own function
Ref: https://github.com/curl/curl/pull/1378
2017-04-05 13:08:15 +02:00
Jay Satiro
98afec033e tool_operate: Fix showing HTTPS-Proxy options on CURLE_SSL_CACERT
- Show the HTTPS-proxy options on CURLE_SSL_CACERT if libcurl was built
  with HTTPS-proxy support.

Prior to this change those options were shown only if an HTTPS-proxy was
specified by --proxy, but that did not take into account environment
variables such as http_proxy, https_proxy, etc. Follow-up to e1187c4.

Bug: https://github.com/curl/curl/issues/1331
Reported-by: Nehal J Wani
2017-03-16 18:23:31 -04:00
Desmond O. Chang
d2bcf1e3e2 url: add option CURLOPT_SUPPRESS_CONNECT_HEADERS
- Add new option CURLOPT_SUPPRESS_CONNECT_HEADERS to allow suppressing
  proxy CONNECT response headers from the user callback functions
  CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION.

- Add new tool option --suppress-connect-headers to expose
  CURLOPT_SUPPRESS_CONNECT_HEADERS and allow suppressing proxy CONNECT
  response headers from --dump-header and --include.

Assisted-by: Jay Satiro
Assisted-by: CarloCannas@users.noreply.github.com
Closes https://github.com/curl/curl/pull/783
2017-03-12 01:32:33 -05:00
Jozef Kralik
6448f98c18 vtls: add options to specify range of enabled TLS versions
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as
the --tls-max option of the curl tool.

Closes https://github.com/curl/curl/pull/1166
2017-03-08 15:54:07 +01:00
Jay Satiro
b259646ea1 url: Improve CURLOPT_PROXY_CAPATH error handling
- Change CURLOPT_PROXY_CAPATH to return CURLE_NOT_BUILT_IN if the option
  is not supported, which is the same as what we already do for
  CURLOPT_CAPATH.

- Change the curl tool to handle CURLOPT_PROXY_CAPATH error
  CURLE_NOT_BUILT_IN as a warning instead of as an error, which is the
  same as what we already do for CURLOPT_CAPATH.

- Fix CAPATH docs to show that CURLE_NOT_BUILT_IN is returned when the
  respective CAPATH option is not supported by the SSL library.

Ref: https://github.com/curl/curl/pull/1257
2017-02-21 22:24:40 -05:00
Michael Kaufmann
ddedc1b3f3 string formatting: fix 4 printf-style format strings 2017-02-19 14:13:42 +01:00
Jay Satiro
e1187c42a4 tool_operate: Show HTTPS-Proxy options on CURLE_SSL_CACERT
When CURLE_SSL_CACERT occurs the tool shows a lengthy error message to
the user explaining possible solutions such as --cacert and --insecure.

This change appends to that message similar options --proxy-cacert and
--proxy-insecure when there's a specified HTTPS proxy.

Closes https://github.com/curl/curl/issues/1258
2017-02-11 16:35:47 -05:00
Isaac Boukris
1d786faee1 unix_socket: add support for abstract unix domain socket
In addition to unix domain sockets, Linux also supports an
abstract namespace which is independent of the filesystem.

In order to support it, add new CURLOPT_ABSTRACT_UNIX_SOCKET
option which uses the same storage as CURLOPT_UNIX_SOCKET_PATH
internally, along with a flag to specify abstract socket.

On non-supporting platforms, the abstract address will be
interpreted as an empty string and fail gracefully.

Also add new --abstract-unix-socket tool parameter.

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reported-by: Chungtsun Li (typeless)
Reviewed-by: Daniel Stenberg
Reviewed-by: Peter Wu
Closes #1197
Fixes #1061
2017-01-13 16:25:20 +01:00
Isaac Boukris
a301cbb655 curl: reset the easy handle at --next
So that only "global" options (verbose mostly) survive into the next
transfer, and the others have to be set again unless default is fine.
2017-01-13 15:33:24 +01:00
Jay Satiro
ee3c83f39c tool_operate: Fix --remote-time incorrect times on Windows
- Use Windows API SetFileTime to set the file time instead of utime.

Avoid utime on Windows if possible because it may apply a daylight
saving time offset to our UTC file time.

Bug: https://curl.haxx.se/mail/archive-2016-11/0033.html
Reported-by: Tim

Closes https://github.com/curl/curl/pull/1121
2016-12-28 21:19:40 -05:00
Daniel Stenberg
aa6db9642e curl: remove superfluous include file
The <netinet/tcp.h> is a leftover from the past when TCP socket options
were set in this file. This include causes build issues on AIX 4.3.

Reported-by: Kim Minjoong

Closes #1178
2016-12-27 22:02:14 +01:00
Daniel Stenberg
845522cadb preproxy: renamed what was added as SOCKS_PROXY
CURLOPT_SOCKS_PROXY -> CURLOPT_PRE_PROXY

Added the corresponding --preroxy command line option. Sets a SOCKS
proxy to connect to _before_ connecting to a HTTP(S) proxy.
2016-12-16 16:04:23 +01:00
Daniel Stenberg
642398c651 curl: normal socks proxies still use CURLOPT_PROXY
... the newly introduced CURLOPT_SOCKS_PROXY is special and should be
asked for specially. (Needs new code.)

Unified proxy type to a single variable in the config struct.
2016-12-16 15:34:14 +01:00
Daniel Stenberg
7907a2bec9 CURLOPT_SOCKS_PROXYTYPE: removed
This was added as part of the SOCKS+HTTPS proxy merge but there's no
need to support this as we prefer to have the protocol specified as a
prefix instead.
2016-12-16 15:10:19 +01:00
Daniel Stenberg
b228d2952b checksrc: stricter no-space-before-paren enforcement
In order to make the code style more uniform everywhere
2016-12-13 23:39:11 +01:00
Daniel Stenberg
dbadaebfc4 checksrc: code style: use 'char *name' style 2016-11-24 23:58:22 +01:00
Alex Rousskov
cb4e2be7c6 proxy: Support HTTPS proxy and SOCKS+HTTP(s)
* HTTPS proxies:

An HTTPS proxy receives all transactions over an SSL/TLS connection.
Once a secure connection with the proxy is established, the user agent
uses the proxy as usual, including sending CONNECT requests to instruct
the proxy to establish a [usually secure] TCP tunnel with an origin
server. HTTPS proxies protect nearly all aspects of user-proxy
communications as opposed to HTTP proxies that receive all requests
(including CONNECT requests) in vulnerable clear text.

With HTTPS proxies, it is possible to have two concurrent _nested_
SSL/TLS sessions: the "outer" one between the user agent and the proxy
and the "inner" one between the user agent and the origin server
(through the proxy). This change adds supports for such nested sessions
as well.

A secure connection with a proxy requires its own set of the usual SSL
options (their actual descriptions differ and need polishing, see TODO):

  --proxy-cacert FILE        CA certificate to verify peer against
  --proxy-capath DIR         CA directory to verify peer against
  --proxy-cert CERT[:PASSWD] Client certificate file and password
  --proxy-cert-type TYPE     Certificate file type (DER/PEM/ENG)
  --proxy-ciphers LIST       SSL ciphers to use
  --proxy-crlfile FILE       Get a CRL list in PEM format from the file
  --proxy-insecure           Allow connections to proxies with bad certs
  --proxy-key KEY            Private key file name
  --proxy-key-type TYPE      Private key file type (DER/PEM/ENG)
  --proxy-pass PASS          Pass phrase for the private key
  --proxy-ssl-allow-beast    Allow security flaw to improve interop
  --proxy-sslv2              Use SSLv2
  --proxy-sslv3              Use SSLv3
  --proxy-tlsv1              Use TLSv1
  --proxy-tlsuser USER       TLS username
  --proxy-tlspassword STRING TLS password
  --proxy-tlsauthtype STRING TLS authentication type (default SRP)

All --proxy-foo options are independent from their --foo counterparts,
except --proxy-crlfile which defaults to --crlfile and --proxy-capath
which defaults to --capath.

Curl now also supports %{proxy_ssl_verify_result} --write-out variable,
similar to the existing %{ssl_verify_result} variable.

Supported backends: OpenSSL, GnuTLS, and NSS.

* A SOCKS proxy + HTTP/HTTPS proxy combination:

If both --socks* and --proxy options are given, Curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS
proxy.

TODO: Update documentation for the new APIs and --proxy-* options.
Look for "Added in 7.XXX" marks.
2016-11-24 23:41:44 +01:00