Commit Graph

1329 Commits

Author SHA1 Message Date
Daniel Stenberg efc83d6d54 http-proxy: only attempt FTP over HTTP proxy
... all other non-HTTP protocol schemes are now defaulting to "tunnel
trough" mode if a HTTP proxy is specified. In reality there are no HTTP
proxies out there that allow those other schemes.

Assisted-by: Ray Satiro, Michael Kaufmann

Closes #1505
2017-06-15 13:46:48 +02:00
Daniel Stenberg 5113ad0424 http-proxy: do the HTTP CONNECT process entirely non-blocking
Mentioned as a problem since 2007 (8f87c15bda) and of course it
existed even before that.

Closes #1547
2017-06-14 23:43:52 +02:00
Daniel Stenberg 5d7952f52e url: fix buffer overwrite with file protocol (CVE-2017-9502)
Bug: https://github.com/curl/curl/issues/1540
Advisory: https://curl.haxx.se/docs/adv_20170614.html

Assisted-by: Ray Satiro
Reported-by: Marcel Raad
2017-06-13 09:34:20 +02:00
Daniel Stenberg 844896d064 setopt: check CURLOPT_ADDRESS_SCOPE option range
... and return error instead of triggering an assert() when being way
out of range.
2017-06-09 01:01:55 +02:00
Daniel Stenberg cccac4fb2b test1521: test *all* curl_easy_setopt options
mk-lib1521.pl generates a test program (lib1521.c) that calls
curl_easy_setopt() for every known option with a few typical values to
make sure they work (ignoring the return codes).

Some small changes were necessary to avoid asserts and NULL accesses
when doing this.

The perl script needs to be manually rerun when we add new options.

Closes #1543
2017-06-05 14:25:18 +02:00
Daniel Stenberg a1b3a95c96 url: fix declaration of 'pipe' shadows a global declaration
follow-up to 4cdb1be824
2017-05-23 07:41:52 +02:00
Dan Fandrich d836c9fee4 url.c: add a compile-time check that CURL_MAX_WRITE_SIZE is large enough
Some code (e.g. Curl_fillreadbuffer) assumes that this buffer is not
exceedingly tiny and will break if it is. This same check is already
done at run time in the CURLOPT_BUFFERSIZE option.
2017-05-13 23:21:27 +02:00
Daniel Stenberg 4cdb1be824 pipeline: fix mistakenly trying to pipeline POSTs
The function IsPipeliningPossible() would return TRUE if either
pipelining OR HTTP/2 were possible on a connection, which would lead to
it returning TRUE even for POSTs on HTTP/1 connections.

It now returns a bitmask so that the caller can differentiate which kind
the connection allows.

Fixes #1481
Closes #1483
Reported-by: stootill at github
2017-05-12 17:01:02 +02:00
Daniel Stenberg 4858c451ee url: let CURLOPT_BUFFERSIZE realloc to smaller sizes too
Closes #1449
2017-05-02 00:04:52 +02:00
Daniel Stenberg e3ed5cb380 BUFSIZE: rename to READBUFFER_*, make separate MASTERBUF_SIZE 2017-05-01 22:55:29 +02:00
Daniel Stenberg 0cab3a394a http-proxy: use a dedicated CONNECT response buffer
To make it suitably independent of the receive buffer and its flexible
size.
2017-05-01 22:55:29 +02:00
Daniel Stenberg f535f4f5fc buffer_size: make sure it always has the correct size
Removes the need for CURL_BUFSIZE
2017-05-01 22:55:29 +02:00
Daniel Stenberg c2ddc12d60 CURLOPT_BUFFERSIZE: 1024 bytes is now the minimum size
The buffer is needed to receive FTP, HTTP CONNECT responses etc so
already at this size things risk breaking and smaller is certainly not
wise.
2017-05-01 22:55:29 +02:00
Dan Fandrich 12e7a8c3d6 url: fixed a memory leak on OOM while setting CURLOPT_BUFFERSIZE 2017-04-29 18:52:51 +02:00
Marcel Raad b851421642
lib: remove unused code
This fixes the following clang warnings:
macro is not used [-Wunused-macros]
will never be executed [-Wunreachable-code]

Closes https://github.com/curl/curl/pull/1448
2017-04-26 10:01:53 +02:00
Martin Kepplinger fb67c977b9 url: declare get_protocol_family() static
get_protocol_family() is not defined static even though there is a
static local forward declaration. Let's simply make the definition match
it's declaration.

Bug: https://curl.haxx.se/mail/lib-2017-04/0127.html
2017-04-26 00:02:51 +02:00
Daniel Stenberg e698b82287 transfer: remove 'uploadbuf' pointer and cleanup readwrite_upload()
The data->req.uploadbuf struct member served no good purpose, instead we
use ->state.uploadbuffer directly. It makes it clearer in the code which
buffer that's being used.

Removed the 'SingleRequest *' argument from the readwrite_upload() proto
as it can be derived from the Curl_easy struct. Also made the code in
the readwrite_upload() function use the 'k->' shortcut to all references
to struct fields in 'data->req', which previously was made with a mix of
both.
2017-04-25 10:55:35 +02:00
Jay Satiro 33cfcfd9f0 TLS: Fix switching off SSL session id when client cert is used
Move the sessionid flag to ssl_primary_config so that ssl and proxy_ssl
will each have their own sessionid flag.

Regression since HTTPS-Proxy support was added in cb4e2be. Prior to that
this issue had been fixed in 247d890, CVE-2016-5419.

Bug: https://github.com/curl/curl/issues/1341
Reported-by: lijian996@users.noreply.github.com

The new incarnation of this bug is called CVE-2017-7468 and is documented
here: https://curl.haxx.se/docs/adv_20170419.html
2017-04-18 07:56:34 +02:00
Antti Hätälä 36e604fc78 url: don't free postponed data on connection reuse
- Don't free postponed data on a connection that will be reused since
  doing so can cause data loss when pipelining.

Only Windows builds are affected by this.

Closes https://github.com/curl/curl/issues/1380
2017-04-04 16:03:26 -04:00
Daniel Stenberg e60fe20fdf llist: replace Curl_llist_alloc with Curl_llist_init
No longer allocate the curl_llist head struct for lists separately.

Removes 17 (15%) tiny allocations in a normal "curl localhost" invoke.

closes #1381
2017-04-04 15:27:45 +02:00
Isaac Boukris 7975d10cf8 http: Fix proxy connection reuse with basic-auth
When using basic-auth, connections and proxy connections
can be re-used with different Authorization headers since
it does not authenticate the connection (like NTLM does).

For instance, the below command should re-use the proxy
connection, but it currently doesn't:
curl -v -U alice:a -x http://localhost:8181 http://localhost/
  --next -U bob:b -x http://localhost:8181 http://localhost/

This is a regression since refactoring of ConnectionExists()
as part of: cb4e2be7c6

Fix the above by removing the username and password compare
when re-using proxy connection at proxy_info_matches().

However, this fix brings back another bug would make curl
to re-print the old proxy-authorization header of previous
proxy basic-auth connection because it wasn't cleared.

For instance, in the below command the second request should
fail if the proxy requires authentication, but would succeed
after the above fix (and before aforementioned commit):
curl -v -U alice:a -x http://localhost:8181 http://localhost/
  --next -x http://localhost:8181 http://localhost/

Fix this by clearing conn->allocptr.proxyuserpwd after use
unconditionally, same as we do for conn->allocptr.userpwd.

Also fix test 540 to not expect digest auth header to be
resent when connection is reused.

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

Closes https://github.com/curl/curl/pull/1350
2017-03-28 03:54:43 -04:00
klemens f7df67cff0 spelling fixes
Closes #1356
2017-03-26 23:56:23 +02:00
Sylvestre Ledru 66de563482 Improve code readbility
... by removing the else branch after a return, break or continue.

Closes #1310
2017-03-13 23:11:45 +01:00
Desmond O. Chang d2bcf1e3e2 url: add option CURLOPT_SUPPRESS_CONNECT_HEADERS
- Add new option CURLOPT_SUPPRESS_CONNECT_HEADERS to allow suppressing
  proxy CONNECT response headers from the user callback functions
  CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION.

- Add new tool option --suppress-connect-headers to expose
  CURLOPT_SUPPRESS_CONNECT_HEADERS and allow suppressing proxy CONNECT
  response headers from --dump-header and --include.

Assisted-by: Jay Satiro
Assisted-by: CarloCannas@users.noreply.github.com
Closes https://github.com/curl/curl/pull/783
2017-03-12 01:32:33 -05:00
Dan Fandrich 381aff0385 url: don't compile detect_proxy if HTTP support is disabled 2017-03-11 18:02:34 +01:00
Jozef Kralik 6448f98c18 vtls: add options to specify range of enabled TLS versions
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as
the --tls-max option of the curl tool.

Closes https://github.com/curl/curl/pull/1166
2017-03-08 15:54:07 +01:00
Daniel Stenberg aadb7c7b62 URL: return error on malformed URLs with junk after port number
... because it causes confusion with users. Example URLs:

"http://[127.0.0.1]:11211:80" which a lot of languages' URL parsers will
parse and claim uses port number 80, while libcurl would use port number
11211.

"http://user@example.com:80@localhost" which by the WHATWG URL spec will
be treated to contain user name 'user@example.com' but according to
RFC3986 is user name 'user' for the host 'example.com' and then port 80
is followed by "@localhost"

Both these formats are now rejected, and verified so in test 1260.

Reported-by: Orange Tsai
2017-03-06 16:08:21 +01:00
Jay Satiro 9f20333443 url: split off proxy init and parsing from create_conn
Move the proxy parse/init into helper create_conn_helper_init_proxy to
mitigate the chances some non-proxy code will be mistakenly added to it.

Ref: https://github.com/curl/curl/issues/1274#issuecomment-281556510
Ref: https://github.com/curl/curl/pull/1293

Closes https://github.com/curl/curl/pull/1298
2017-03-03 03:14:25 -05:00
Isaac Boukris 1f8023ceb5 url: fix unix-socket support for proxy-disabled builds
Prior to this change if curl was built with Unix Socket support
(--enable-unix-sockets) and without Proxy support (--disable-proxy) then
unix socket options would erroneously be ignored.

Regression introduced in:
0b8d682f81

Bug: https://github.com/curl/curl/issues/1274
Reported-by: mccormickt12@users.noreply.github.com

Closes https://github.com/curl/curl/pull/1289
2017-02-25 18:21:15 -05:00
Dan Fandrich ce9ba3ba10 proxy: fixed a memory leak on OOM 2017-02-25 21:36:39 +01:00
Jay Satiro 93cce2489d url: Default the CA proxy bundle location to CURL_CA_BUNDLE
If the compile-time CURL_CA_BUNDLE location is defined use it as the
default value for the proxy CA bundle location, which is the same as
what we already do for the regular CA bundle location.

Ref: https://github.com/curl/curl/pull/1257
2017-02-23 17:06:11 -05:00
Jay Satiro b259646ea1 url: Improve CURLOPT_PROXY_CAPATH error handling
- Change CURLOPT_PROXY_CAPATH to return CURLE_NOT_BUILT_IN if the option
  is not supported, which is the same as what we already do for
  CURLOPT_CAPATH.

- Change the curl tool to handle CURLOPT_PROXY_CAPATH error
  CURLE_NOT_BUILT_IN as a warning instead of as an error, which is the
  same as what we already do for CURLOPT_CAPATH.

- Fix CAPATH docs to show that CURLE_NOT_BUILT_IN is returned when the
  respective CAPATH option is not supported by the SSL library.

Ref: https://github.com/curl/curl/pull/1257
2017-02-21 22:24:40 -05:00
Daniel Stenberg ca6ea6d9be TLS: make SSL_VERIFYSTATUS work again
The CURLOPT_SSL_VERIFYSTATUS option was not properly handled by libcurl
and thus even if the status couldn't be verified, the connection would
be allowed and the user would not be told about the failed verification.

Regression since cb4e2be7c6

CVE-2017-2629
Bug: https://curl.haxx.se/docs/adv_20170222.html

Reported-by: Marcus Hoffmann
2017-02-21 08:33:18 +01:00
Michael Kaufmann 2f8d0df085 proxy: fix hostname resolution and IDN conversion
Properly resolve, convert and log the proxy host names.
Support the "--connect-to" feature for SOCKS proxies and for passive FTP
data transfers.

Follow-up to cb4e2be

Reported-by: Jay Satiro
Fixes https://github.com/curl/curl/issues/1248
2017-02-18 15:04:43 +01:00
Daniel Stenberg c5c4e816b4 URL: only accept ";options" in SMTP/POP3/IMAP URL schemes
Fixes #1252
2017-02-10 14:51:53 +01:00
Richy Kim 6b7616690e CURLOPT_BUFFERSIZE: support enlarging receive buffer
Replace use of fixed macro BUFSIZE to define the size of the receive
buffer.  Reappropriate CURLOPT_BUFFERSIZE to include enlarging receive
buffer size.  Upon setting, resize buffer if larger than the current
default size up to a MAX_BUFSIZE (512KB). This can benefit protocols
like SFTP.

Closes #1222
2017-01-19 23:38:04 +01:00
Michael Kaufmann 2d6831b758 CURLOPT_CONNECT_TO: Fix compile warnings
Fix compile warnings that appeared only when curl has been configured
with '--disable-verbose'.
2017-01-18 13:02:25 +01:00
Daniel Stenberg e9f45aa91b parseurl: move back buffer to function scope
Regression since 1d4202ad, which moved the buffer into a more narrow
scope, but the data in that buffer was used outside of that more narrow
scope.

Reported-by: Dan Fandrich
Bug: https://curl.haxx.se/mail/lib-2017-01/0093.html
2017-01-18 00:01:03 +01:00
Isaac Boukris 1d786faee1 unix_socket: add support for abstract unix domain socket
In addition to unix domain sockets, Linux also supports an
abstract namespace which is independent of the filesystem.

In order to support it, add new CURLOPT_ABSTRACT_UNIX_SOCKET
option which uses the same storage as CURLOPT_UNIX_SOCKET_PATH
internally, along with a flag to specify abstract socket.

On non-supporting platforms, the abstract address will be
interpreted as an empty string and fail gracefully.

Also add new --abstract-unix-socket tool parameter.

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reported-by: Chungtsun Li (typeless)
Reviewed-by: Daniel Stenberg
Reviewed-by: Peter Wu
Closes #1197
Fixes #1061
2017-01-13 16:25:20 +01:00
Daniel Stenberg 7d6e3f8cfa IDN: Use TR46 non-transitional
Assisted-by: Tim Rühsen
2017-01-13 14:53:50 +01:00
Daniel Stenberg ee357664df IDN: revert use of the transitional option
It made the german ß get converted to ss, IDNA2003 style, and we can't
have that for the .de TLD - a primary reason for our switch to IDNA2008.

Test 165 verifies.
2017-01-13 14:31:40 +01:00
Tim Rühsen ba315745f7 IDN: Fix compile time detection of linidn2 TR46
Follow-up to f30cbcac1

Closes #1207
2017-01-13 13:38:12 +01:00
ERAMOTO Masaya 2ac1942c72 url: --noproxy option overrides NO_PROXY environment variable
Under condition using http_proxy env var, noproxy list was the
combination of --noproxy option and NO_PROXY env var previously. Since
this commit, --noproxy option overrides NO_PROXY environment variable
even if use http_proxy env var.

Closes #1140
2017-01-13 11:18:29 +01:00
ERAMOTO Masaya efdbfde7ca url: Refactor detect_proxy()
If defined CURL_DISABLE_HTTP, detect_proxy() returned NULL. If not
defined CURL_DISABLE_HTTP, detect_proxy() checked noproxy list.

Thus refactor to set proxy to NULL instead of calling detect_proxy() if
define CURL_DISABLE_HTTP, and refactor to call detect_proxy() if not
define CURL_DISABLE_HTTP and the host is not in the noproxy list.
2017-01-13 11:18:11 +01:00
ERAMOTO Masaya 1b57557882 url: Fix NO_PROXY env var to work properly with --proxy option.
The combination of --noproxy option and http_proxy env var works well
both for proxied hosts and non-proxied hosts.

However, when combining NO_PROXY env var with --proxy option,
non-proxied hosts are not reachable while proxied host is OK.

This patch allows us to access non-proxied hosts even if using NO_PROXY
env var with --proxy option.
2017-01-13 11:18:11 +01:00
Tim Rühsen f30cbcac11 IDN: Use TR46 'transitional' for toASCII translations
References: http://unicode.org/faq/idn.html
            http://unicode.org/reports/tr46

Closes #1206
2017-01-13 10:21:25 +01:00
Jay Satiro 1d4202ade6 url: Fix parsing for when 'file' is the default protocol
Follow-up to 3463408.

Prior to 3463408 file:// hostnames were silently stripped.

Prior to this commit it did not work when a schemeless url was used with
file as the default protocol.

Ref: https://curl.haxx.se/mail/lib-2016-11/0081.html
Closes https://github.com/curl/curl/pull/1124

Also fix for drive letters:

- Support --proto-default file c:/foo/bar.txt

- Support file://c:/foo/bar.txt

- Fail when a file:// drive letter is detected and not MSDOS/Windows.

Bug: https://github.com/curl/curl/issues/1187
Reported-by: Anatol Belski
Assisted-by: Anatol Belski
2017-01-12 15:37:11 -05:00
Daniel Stenberg 845522cadb preproxy: renamed what was added as SOCKS_PROXY
CURLOPT_SOCKS_PROXY -> CURLOPT_PRE_PROXY

Added the corresponding --preroxy command line option. Sets a SOCKS
proxy to connect to _before_ connecting to a HTTP(S) proxy.
2016-12-16 16:04:23 +01:00
Daniel Stenberg 7907a2bec9 CURLOPT_SOCKS_PROXYTYPE: removed
This was added as part of the SOCKS+HTTPS proxy merge but there's no
need to support this as we prefer to have the protocol specified as a
prefix instead.
2016-12-16 15:10:19 +01:00
Daniel Stenberg 1c3e8bbfed checksrc: warn for assignments within if() expressions
... they're already frowned upon in our source code style guide, this
now enforces the rule harder.
2016-12-14 01:29:44 +01:00