1
0
mirror of https://github.com/moparisthebest/curl synced 2024-12-25 09:38:54 -05:00
Commit Graph

1435 Commits

Author SHA1 Message Date
Jon DeVree
695e96b3d5
file: restore old behavior for file:////foo/bar URLs
curl 7.57.0 and up interpret this according to Appendix E.3.2 of RFC
8089 but then returns an error saying this is unimplemented. This is
actually a regression in behavior on both Windows and Unix.

Before curl 7.57.0 this URL was treated as a path of "//foo/bar" and
then passed to the relevant OS API. This means that the behavior of this
case is actually OS dependent.

The Unix path resolution rules say that the OS must handle swallowing
the extra "/" and so this path is the same as "/foo/bar"

The Windows path resolution rules say that this is a UNC path and
automatically handles the SMB access for the program. So curl on Windows
was already doing Appendix E.3.2 without any special code in curl.

Regression

Closes #2438
2018-04-06 14:28:42 +02:00
Daniel Stenberg
b7f90470be
NO_PROXY: fix for IPv6 numericals in the URL
Added test 1265 that verifies.

Reported-by: steelman on github
Fixes #2353
Closes #2355
2018-03-04 19:50:48 +01:00
Viktor Szakats
7e35eb7729 spelling fixes
Detected using the `codespell` tool.

Also contains one URL protocol upgrade.

Closes https://github.com/curl/curl/pull/2334
2018-02-23 23:29:01 +00:00
Anders Bakken
2427d94c6d url: Add option CURLOPT_HAPPY_EYEBALLS_TIMEOUT
- Add new option CURLOPT_HAPPY_EYEBALLS_TIMEOUT to set libcurl's happy
  eyeball timeout value.

- Add new optval macro CURL_HET_DEFAULT to represent the default happy
  eyeballs timeout value (currently 200 ms).

- Add new tool option --happy-eyeballs-timeout-ms to expose
  CURLOPT_HAPPY_EYEBALLS_TIMEOUT. The -ms suffix is used because the
  other -timeout options in the tool expect seconds not milliseconds.

Closes https://github.com/curl/curl/pull/2260
2018-02-20 17:51:43 -05:00
Daniel Stenberg
eb6e3c4f6e
CONNECT_TO: fail attempt to set an IPv6 numerical without IPv6 support
Bug: https://curl.haxx.se/mail/lib-2018-01/0087.html
Reported-by: John Hascall

Closes #2257
2018-01-23 08:01:04 +01:00
Jay Satiro
908a9a6742 build: remove HAVE_LIMITS_H check
.. because limits.h presence isn't optional, it's required by C89.

Ref: http://port70.net/~nsz/c/c89/c89-draft.html#2.2.4.2

Closes https://github.com/curl/curl/pull/2215
2018-01-05 23:34:30 -05:00
Patrick Monnerat
f009bbe1f6 curl_easy_reset: release mime-related data.
Move curl_mime_initpart() and curl_mime_cleanpart() calls to lower-level
functions dealing with UserDefined structure contents.
This avoids memory leakages on curl-generated part mime headers.
New test 2073 checks this using the cli tool --next option: it
triggers a valgrind error if bug is present.

Bug: https://curl.haxx.se/mail/lib-2017-12/0060.html
Reported-by: Martin Galvan
2017-12-20 19:33:50 +01:00
Daniel Stenberg
179ee78e86
curl: remove __EMX__ #ifdefs
These are OS/2-specific things added to the code in the year 2000. They
were always ugly. If there's any user left, they still don't need it
done this way.

Closes #2166
2017-12-08 23:45:16 +01:00
Jan-E
b261c44e8c
URL: tolerate backslash after drive letter for FILE:
... as in "file://c:\some\path\curl.out"

Reviewed-by: Matthew Kerwin
Closes #2154
2017-12-06 00:53:20 +01:00
Daniel Stenberg
ea3a5d07dc
conncache: only allow multiplexing within same multi handle
Connections that are used for HTTP/1.1 Pipelining or HTTP/2 multiplexing
only get additional transfers added to them if the existing connection
is held by the same multi or easy handle. libcurl does not support doing
HTTP/2 streams in different threads using a shared connection.

Closes #2152
2017-12-05 23:36:08 +01:00
Daniel Stenberg
07cb27c98e
conncache: fix several lock issues
If the lock is released before the dealings with the bundle is over, it may
have changed by another thread in the mean time.

Fixes #2132
Fixes #2151
Closes #2139
2017-12-05 23:21:02 +01:00
Nikos Mavrogiannopoulos
a2f3966800
libssh: added SFTP support
The SFTP back-end supports asynchronous reading only, limited
to 32-bit file length. Writing is synchronous with no other
limitations.

This also brings keyboard-interactive authentication.

Signed-off-by: Nikos Mavrogiannopoulos <nmav@gnutls.org>
2017-12-01 17:38:37 +01:00
Nikos Mavrogiannopoulos
c92d2e14cf
Added support for libssh SSH SCP back-end
libssh is an alternative library to libssh2.
https://www.libssh.org/

That patch set also introduces support for ECDSA
ed25519 keys, as well as gssapi authentication.

Signed-off-by: Nikos Mavrogiannopoulos <nmav@redhat.com>
2017-12-01 17:35:14 +01:00
Johannes Schindelin
9194a9959b
SSL: Avoid magic allocation of SSL backend specific data
Originally, my idea was to allocate the two structures (or more
precisely, the connectdata structure and the four SSL backend-specific
strucutres required for ssl[0..1] and proxy_ssl[0..1]) in one go, so
that they all could be free()d together.

However, getting the alignment right is tricky. Too tricky.

So let's just bite the bullet and allocate the SSL backend-specific
data separately.

As a consequence, we now have to be very careful to release the memory
allocated for the SSL backend-specific data whenever we release any
connectdata.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Closes #2119
2017-12-01 10:28:31 +01:00
Jay Satiro
9b5e12a549
url: fix alignment of ssl_backend_data struct
- Align the array of ssl_backend_data on a max 32 byte boundary.

8 is likely to be ok but I went with 32 for posterity should one of
the ssl_backend_data structs change to contain a larger sized variable
in the future.

Prior to this change (since dev 70f1db3, release 7.56) the connectdata
structure was undersized by 4 bytes in 32-bit builds with ssl enabled
because long long * was mistakenly used for alignment instead of
long long, with the intention being an 8 byte boundary. Also long long
may not be an available type.

The undersized connectdata could lead to oob read/write past the end in
what was expected to be the last 4 bytes of the connection's secondary
socket https proxy ssl_backend_data struct (the secondary socket in a
connection is used by ftp, others?).

Closes https://github.com/curl/curl/issues/2093

CVE-2017-8818

Bug: https://curl.haxx.se/docs/adv_2017-af0a.html
2017-11-27 08:19:25 +01:00
Matthew Kerwin
25634611f9
URL: update "file:" URL handling
* LOTS of comment updates
* explicit error for SMB shares (e.g. "file:////share/path/file")
* more strict handling of authority (i.e. "//localhost/")
* now accepts dodgy old "C:|" drive letters
* more precise handling of drive letters in and out of Windows
  (especially recognising both "file:c:/" and "file:/c:/")

Closes #2110
2017-11-24 23:12:55 +01:00
Daniel Stenberg
fa939220df
url: reject ASCII control characters and space in host names
Host names like "127.0.0.1 moo" would otherwise be accepted by some
getaddrinfo() implementations.

Updated test 1034 and 1035 accordingly.

Fixes #2073
Closes #2092
2017-11-22 11:14:06 +01:00
Daniel Stenberg
9554c3c6e5
Curl_open: fix OOM return error correctly
Closes #2098
2017-11-21 09:02:50 +01:00
Michael Kaufmann
ae7369b6d0 URL: return error on malformed URLs with junk after IPv6 bracket
Follow-up to aadb7c7. Verified by new test 1263.

Closes #2072
2017-11-14 18:20:56 +01:00
Daniel Stenberg
aa7668b948
setopt: split out curl_easy_setopt() to its own file
... to make url.c smaller.

Closes #1944
2017-11-10 23:08:20 +01:00
Daniel Stenberg
67c55a26d5
share: add support for sharing the connection cache 2017-11-09 11:07:44 +01:00
Daniel Stenberg
cbb22cb76d
url: remove unncessary NULL-check
Since 'conn' won't be NULL in there and we also access the pointer in
there without the check.

Coverity CID 1420610
2017-11-06 08:23:11 +01:00
Patrick Monnerat
dbcced8e32 HTTP: support multiple Content-Encodings
This is implemented as an output streaming stack of unencoders, the last
calling the client write procedure.

New test 230 checks this feature.

Bug: https://github.com/curl/curl/pull/2002
Reported-By: Daniel Bankhead
2017-11-05 15:09:48 +01:00
Jay Satiro
462f3cac34 url: remove arg value check from CURLOPT_SSH_AUTH_TYPES
Since CURLSSH_AUTH_ANY (aka CURLSSH_AUTH_DEFAULT) is ~0 an arg value
check on this option is incorrect; we have to accept any value.

Prior to this change since f121575 (7.56.1+) CURLOPT_SSH_AUTH_TYPES
erroneously rejected CURLSSH_AUTH_ANY with CURLE_BAD_FUNCTION_ARGUMENT.

Bug: https://github.com/curl/curl/commit/f121575#commitcomment-25347120
2017-11-04 18:36:07 -04:00
Daniel Stenberg
921bf1de52 CURLOPT_INFILESIZE: accept -1
Regression since f121575

Reported-by: Petr Voytsik
Fixes #2047
2017-11-03 14:40:19 +01:00
Jay Satiro
b51e0742b9 url: fix CURLOPT_DNS_CACHE_TIMEOUT arg value check to allow -1
Prior to this change since f121575 (7.56.1+) CURLOPT_DNS_CACHE_TIMEOUT
erroneously rejected -1 with CURLE_BAD_FUNCTION_ARGUMENT.
2017-11-02 00:30:28 -04:00
Daniel Stenberg
25cb41d35d CURLOPT_MAXREDIRS: allow -1 as a value
... which is valid according to documentation. Regression since
f121575c0b.

Verified now in test 501.

Reported-by: cbartl on github
Fixes #2038
Closes #2039
2017-11-01 11:56:19 +01:00
Jay Satiro
b8bd6dc110 url: fix CURLOPT_POSTFIELDSIZE arg value check to allow -1
.. also add same arg value check to CURLOPT_POSTFIELDSIZE_LARGE.

Prior to this change since f121575 (7.56.1+) CURLOPT_POSTFIELDSIZE
erroneously rejected -1 value with CURLE_BAD_FUNCTION_ARGUMENT.

Bug: https://curl.haxx.se/mail/lib-2017-11/0000.html
Reported-by: Andrew Lambert
2017-11-01 02:11:30 -04:00
Daniel Stenberg
0d85eed3df
Curl_timeleft: change return type to timediff_t
returning 'time_t' is problematic when that type is unsigned and we
return values less than zero to signal "already expired", used in
several places in the code.

Closes #2021
2017-10-28 10:40:51 +02:00
Daniel Stenberg
961c8667d2
setopt: fix CURLOPT_SSH_AUTH_TYPES option read
Regression since f121575c0b

Reported-by: Rob Cotrone
2017-10-27 22:57:51 +02:00
Max Dymond
7b11c5dbe6
wildcards: don't use with non-supported protocols
Fixes timeouts in the fuzzing tests for non-FTP protocols.

Closes #2016
2017-10-26 13:34:45 +02:00
Daniel Stenberg
5d543fe906 time: rename Curl_tvnow to Curl_now
... since the 'tv' stood for timeval and this function does not return a
timeval struct anymore.

Also, cleaned up the Curl_timediff*() functions to avoid typecasts and
clean up the descriptive comments.

Closes #2011
2017-10-25 18:48:05 +02:00
Daniel Stenberg
b9d25f9a6b timediff: return timediff_t from the time diff functions
... to cater for systems with unsigned time_t variables.

- Renamed the functions to curlx_timediff and Curl_timediff_us.

- Added overflow protection for both of them in either direction for
  both 32 bit and 64 bit time_ts

- Reprefixed the curlx_time functions to use Curl_*

Reported-by: Peter Piekarski
Fixes #2004
Closes #2005
2017-10-25 09:54:37 +02:00
Daniel Stenberg
f121575c0b
setopt: range check most long options
... filter early instead of risking "funny values" having to be dealt
with elsewhere.
2017-10-16 09:23:33 +02:00
Daniel Stenberg
172ce9cc19
setopt: avoid integer overflows when setting millsecond values
... that are multiplied by 1000 when stored.

For 32 bit long systems, the max value accepted (2147483 seconds) is >
596 hours which is unlikely to ever be set by a legitimate application -
and previously it didn't work either, it just caused undefined behavior.

Also updated the man pages for these timeout options to mention the
return code.

Closes #1938
2017-10-16 09:23:19 +02:00
Artak Galoyan
5505df7d24 url: Update current connection SSL verify params in setopt
Now VERIFYHOST, VERIFYPEER and VERIFYSTATUS options change during active
connection updates the current connection's (i.e.'connectdata'
structure) appropriate ssl_config (and ssl_proxy_config) structures
variables, making these options effective for ongoing connection.

This functionality was available before and was broken by the
following change:
"proxy: Support HTTPS proxy and SOCKS+HTTP(s)"
CommitId: cb4e2be7c6.

Bug: https://github.com/curl/curl/issues/1941

Closes https://github.com/curl/curl/pull/1951
2017-10-11 03:14:26 -04:00
Patrick Monnerat
b557182db1 mime: improve unbinding top multipart from easy handle.
Also avoid dangling pointers in referencing parts.
2017-10-08 18:38:34 +01:00
Michael Kaufmann
5f1fa5827d idn: fix source code comment 2017-10-03 18:46:50 +02:00
Michael Kaufmann
c4ebd8b46d proxy: read the "no_proxy" variable only if necessary
Reviewed-by: Daniel Stenberg

Closes #1919
2017-10-03 18:04:42 +02:00
Dan Fandrich
df7839b68c Set and use more necessary options when some protocols are disabled
When curl and libcurl are built with some protocols disabled, they stop
setting and receiving some options that don't make sense with those
protocols.  In particular, when HTTP is disabled many options aren't set
that are used only by HTTP.  However, some options that appear to be
HTTP-only are actually used by other protocols as well (some despite
having HTTP in the name) and should be set, but weren't. This change now
causes some of these options to be set and used for more (or for all)
protocols. In particular, this fixes tests 646 through 649 in an
HTTP-disabled build, which use the MIME API in the mail protocols.
2017-09-30 02:10:54 +02:00
Michael Kaufmann
284d06df9e reuse_conn: don't copy flags that are known to be equal
A connection can only be reused if the flags "conn_to_host" and
"conn_to_port" match. Therefore it is not necessary to copy these flags
in reuse_conn().

Closes #1918
2017-09-28 21:18:02 +02:00
Daniel Stenberg
22708eae40
URL: on connection re-use, still pick the new remote port
... as when a proxy connection is being re-used, it can still get a
different remote port.

Fixes #1887
Reported-by: Oli Kingshott
2017-09-14 16:49:40 +02:00
Daniel Stenberg
e5743f08e7
code style: use spaces around pluses 2017-09-11 09:29:50 +02:00
Daniel Stenberg
6b84438d9a
code style: use spaces around equals signs 2017-09-11 09:29:50 +02:00
Patrick Monnerat
ce0881edee mime: new MIME API.
Available in HTTP, SMTP and IMAP.
Deprecates the FORM API.
See CURLOPT_MIMEPOST.
Lib code and associated documentation.
2017-09-02 17:47:10 +01:00
Johannes Schindelin
70f1db321a
vtls: encapsulate SSL backend-specific data
So far, all of the SSL backends' private data has been declared as
part of the ssl_connect_data struct, in one big #if .. #elif .. #endif
block.

This can only work as long as the SSL backend is a compile-time option,
something we want to change in the next commits.

Therefore, let's encapsulate the exact data needed by each SSL backend
into a private struct, and let's avoid bleeding any SSL backend-specific
information into urldata.h. This is also necessary to allow multiple SSL
backends to be compiled in at the same time, as e.g. OpenSSL's and
CyaSSL's headers cannot be included in the same .c file.

To avoid too many malloc() calls, we simply append the private structs
to the connectdata struct in allocate_conn().

This requires us to take extra care of alignment issues: struct fields
often need to be aligned on certain boundaries e.g. 32-bit values need to
be stored at addresses that divide evenly by 4 (= 32 bit / 8
bit-per-byte).

We do that by assuming that no SSL backend's private data contains any
fields that need to be aligned on boundaries larger than `long long`
(typically 64-bit) would need. Under this assumption, we simply add a
dummy field of type `long long` to the `struct connectdata` struct. This
field will never be accessed but acts as a placeholder for the four
instances of ssl_backend_data instead. the size of each ssl_backend_data
struct is stored in the SSL backend-specific metadata, to allow
allocate_conn() to know how much extra space to allocate, and how to
initialize the ssl[sockindex]->backend and proxy_ssl[sockindex]->backend
pointers.

This would appear to be a little complicated at first, but is really
necessary to encapsulate the private data of each SSL backend correctly.
And we need to encapsulate thusly if we ever want to allow selecting
CyaSSL and OpenSSL at runtime, as their headers cannot be included within
the same .c file (there are just too many conflicting definitions and
declarations for that).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:58 +02:00
Johannes Schindelin
f0b4db1ab0
vtls: move the SUPPORT_HTTPS_PROXY flag into the Curl_ssl struct
That will allow us to choose the SSL backend at runtime.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Johannes Schindelin
937899a3b8
vtls: convert the have_curlssl_* constants to runtime flags
The entire idea of introducing the Curl_ssl struct to describe SSL
backends is to prepare for choosing the SSL backend at runtime.

To that end, convert all the #ifdef have_curlssl_* style conditionals
to use bit flags instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2017-08-28 14:56:56 +02:00
Viktor Szakats
b7b4dc0d49 ssh: add the ability to enable compression (for SCP/SFTP)
The required low-level logic was already available as part of
`libssh2` (via `LIBSSH2_FLAG_COMPRESS` `libssh2_session_flag()`[1]
option.)

This patch adds the new `libcurl` option `CURLOPT_SSH_COMPRESSION`
(boolean) and the new `curl` command-line option `--compressed-ssh`
to request this `libssh2` feature. To have compression enabled, it
is required that the SSH server supports a (zlib) compatible
compression method and that `libssh2` was built with `zlib` support
enabled.

[1] https://www.libssh2.org/libssh2_session_flag.html

Ref: https://github.com/curl/curl/issues/1732
Closes https://github.com/curl/curl/pull/1735
2017-08-17 03:32:00 -04:00
Even Rouault
6e0e152ce5
parse_proxy(): fix memory leak in case of invalid proxy server name
Fixes the below leak:

$ valgrind --leak-check=full ~/install-curl-git/bin/curl --proxy "http://a:b@/x" http://127.0.0.1
curl: (5) Couldn't resolve proxy name
==5048==
==5048== HEAP SUMMARY:
==5048==     in use at exit: 532 bytes in 12 blocks
==5048==   total heap usage: 5,288 allocs, 5,276 frees, 445,271 bytes allocated
==5048==
==5048== 2 bytes in 1 blocks are definitely lost in loss record 1 of 12
==5048==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5048==    by 0x4E6CB79: parse_login_details (url.c:5614)
==5048==    by 0x4E6BA82: parse_proxy (url.c:5091)
==5048==    by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346)
==5048==    by 0x4E6EA18: create_conn (url.c:6498)
==5048==    by 0x4E6F9B4: Curl_connect (url.c:6967)
==5048==    by 0x4E86D05: multi_runsingle (multi.c:1436)
==5048==    by 0x4E88432: curl_multi_perform (multi.c:2160)
==5048==    by 0x4E7C515: easy_transfer (easy.c:708)
==5048==    by 0x4E7C74A: easy_perform (easy.c:794)
==5048==    by 0x4E7C7B1: curl_easy_perform (easy.c:813)
==5048==    by 0x414025: operate_do (tool_operate.c:1563)
==5048==
==5048== 2 bytes in 1 blocks are definitely lost in loss record 2 of 12
==5048==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5048==    by 0x4E6CBB6: parse_login_details (url.c:5621)
==5048==    by 0x4E6BA82: parse_proxy (url.c:5091)
==5048==    by 0x4E6C46D: create_conn_helper_init_proxy (url.c:5346)
==5048==    by 0x4E6EA18: create_conn (url.c:6498)
==5048==    by 0x4E6F9B4: Curl_connect (url.c:6967)
==5048==    by 0x4E86D05: multi_runsingle (multi.c:1436)
==5048==    by 0x4E88432: curl_multi_perform (multi.c:2160)
==5048==    by 0x4E7C515: easy_transfer (easy.c:708)
==5048==    by 0x4E7C74A: easy_perform (easy.c:794)
==5048==    by 0x4E7C7B1: curl_easy_perform (easy.c:813)
==5048==    by 0x414025: operate_do (tool_operate.c:1563)

Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=2984
Credit to OSS Fuzz for discovery

Closes #1761
2017-08-11 11:52:56 +02:00
Daniel Stenberg
4dee50b9c8 timeval: struct curltime is a struct timeval replacement
... to make all libcurl internals able to use the same data types for
the struct members. The timeval struct differs subtly on several
platforms so it makes it cumbersome to use everywhere.

Ref: #1652
Closes #1693
2017-07-28 15:51:25 +02:00
Max Dymond
c75f63d7c4 handler: refactor connection checking
Add a new type of callback to Curl_handler which performs checks on
the connection. Alter RTSP so that it uses this callback to do its
own check on connection health.
2017-06-30 10:17:27 +02:00
Kamil Dudka
8924f58c37 CURLOPT_SOCKS5_AUTH: allowed methods for SOCKS5 proxy auth
If libcurl was built with GSS-API support, it unconditionally advertised
GSS-API authentication while connecting to a SOCKS5 proxy.  This caused
problems in environments with improperly configured Kerberos: a stock
libcurl failed to connect, despite libcurl built without GSS-API
connected fine using username and password.

This commit introduces the CURLOPT_SOCKS5_AUTH option to control the
allowed methods for SOCKS5 authentication at run time.

Note that a new option was preferred over reusing CURLOPT_PROXYAUTH
for compatibility reasons because the set of authentication methods
allowed by default was different for HTTP and SOCKS5 proxies.

Bug: https://curl.haxx.se/mail/lib-2017-01/0005.html
Closes https://github.com/curl/curl/pull/1454
2017-06-28 08:02:58 +02:00
Daniel Stenberg
9b167fd090 --request-target: instead of --strip-path-slash
... and CURLOPT_REQUEST_TARGET instead of CURLOPT_STRIP_PATH_SLASH.

This option instead provides the full "alternative" target to use in the
request, instead of extracting the path from the URL.

Test 1298 and 1299 updated accordingly.

Idea-by: Evert Pot
Suggestion: https://daniel.haxx.se/blog/2017/06/19/options-with-curl/comment-page-1/#comment-18373

Closes #1593
2017-06-21 23:39:10 +02:00
Daniel Stenberg
b778ae4c5e http: add --strip-path-slash and CURLOPT_STRIP_PATH_SLASH
... to enable sending "OPTIONS *" which wasn't possible previously.

This option currently only works for HTTP.

Added test cases 1298 + 1299 to verify

Fixes #1280
Closes #1462
2017-06-19 16:39:22 +02:00
Jay Satiro
2a733d6418 url: refactor the check for Windows drive letter in path
- Move the logic to detect a Windows drive letter prefix
  (eg c: in c:foo) into a function-like macro.

Closes https://github.com/curl/curl/pull/1571
2017-06-16 03:03:25 -04:00
Daniel Stenberg
efc83d6d54 http-proxy: only attempt FTP over HTTP proxy
... all other non-HTTP protocol schemes are now defaulting to "tunnel
trough" mode if a HTTP proxy is specified. In reality there are no HTTP
proxies out there that allow those other schemes.

Assisted-by: Ray Satiro, Michael Kaufmann

Closes #1505
2017-06-15 13:46:48 +02:00
Daniel Stenberg
5113ad0424 http-proxy: do the HTTP CONNECT process entirely non-blocking
Mentioned as a problem since 2007 (8f87c15bda) and of course it
existed even before that.

Closes #1547
2017-06-14 23:43:52 +02:00
Daniel Stenberg
5d7952f52e url: fix buffer overwrite with file protocol (CVE-2017-9502)
Bug: https://github.com/curl/curl/issues/1540
Advisory: https://curl.haxx.se/docs/adv_20170614.html

Assisted-by: Ray Satiro
Reported-by: Marcel Raad
2017-06-13 09:34:20 +02:00
Daniel Stenberg
844896d064 setopt: check CURLOPT_ADDRESS_SCOPE option range
... and return error instead of triggering an assert() when being way
out of range.
2017-06-09 01:01:55 +02:00
Daniel Stenberg
cccac4fb2b test1521: test *all* curl_easy_setopt options
mk-lib1521.pl generates a test program (lib1521.c) that calls
curl_easy_setopt() for every known option with a few typical values to
make sure they work (ignoring the return codes).

Some small changes were necessary to avoid asserts and NULL accesses
when doing this.

The perl script needs to be manually rerun when we add new options.

Closes #1543
2017-06-05 14:25:18 +02:00
Daniel Stenberg
a1b3a95c96 url: fix declaration of 'pipe' shadows a global declaration
follow-up to 4cdb1be824
2017-05-23 07:41:52 +02:00
Dan Fandrich
d836c9fee4 url.c: add a compile-time check that CURL_MAX_WRITE_SIZE is large enough
Some code (e.g. Curl_fillreadbuffer) assumes that this buffer is not
exceedingly tiny and will break if it is. This same check is already
done at run time in the CURLOPT_BUFFERSIZE option.
2017-05-13 23:21:27 +02:00
Daniel Stenberg
4cdb1be824 pipeline: fix mistakenly trying to pipeline POSTs
The function IsPipeliningPossible() would return TRUE if either
pipelining OR HTTP/2 were possible on a connection, which would lead to
it returning TRUE even for POSTs on HTTP/1 connections.

It now returns a bitmask so that the caller can differentiate which kind
the connection allows.

Fixes #1481
Closes #1483
Reported-by: stootill at github
2017-05-12 17:01:02 +02:00
Daniel Stenberg
4858c451ee url: let CURLOPT_BUFFERSIZE realloc to smaller sizes too
Closes #1449
2017-05-02 00:04:52 +02:00
Daniel Stenberg
e3ed5cb380 BUFSIZE: rename to READBUFFER_*, make separate MASTERBUF_SIZE 2017-05-01 22:55:29 +02:00
Daniel Stenberg
0cab3a394a http-proxy: use a dedicated CONNECT response buffer
To make it suitably independent of the receive buffer and its flexible
size.
2017-05-01 22:55:29 +02:00
Daniel Stenberg
f535f4f5fc buffer_size: make sure it always has the correct size
Removes the need for CURL_BUFSIZE
2017-05-01 22:55:29 +02:00
Daniel Stenberg
c2ddc12d60 CURLOPT_BUFFERSIZE: 1024 bytes is now the minimum size
The buffer is needed to receive FTP, HTTP CONNECT responses etc so
already at this size things risk breaking and smaller is certainly not
wise.
2017-05-01 22:55:29 +02:00
Dan Fandrich
12e7a8c3d6 url: fixed a memory leak on OOM while setting CURLOPT_BUFFERSIZE 2017-04-29 18:52:51 +02:00
Marcel Raad
b851421642
lib: remove unused code
This fixes the following clang warnings:
macro is not used [-Wunused-macros]
will never be executed [-Wunreachable-code]

Closes https://github.com/curl/curl/pull/1448
2017-04-26 10:01:53 +02:00
Martin Kepplinger
fb67c977b9 url: declare get_protocol_family() static
get_protocol_family() is not defined static even though there is a
static local forward declaration. Let's simply make the definition match
it's declaration.

Bug: https://curl.haxx.se/mail/lib-2017-04/0127.html
2017-04-26 00:02:51 +02:00
Daniel Stenberg
e698b82287 transfer: remove 'uploadbuf' pointer and cleanup readwrite_upload()
The data->req.uploadbuf struct member served no good purpose, instead we
use ->state.uploadbuffer directly. It makes it clearer in the code which
buffer that's being used.

Removed the 'SingleRequest *' argument from the readwrite_upload() proto
as it can be derived from the Curl_easy struct. Also made the code in
the readwrite_upload() function use the 'k->' shortcut to all references
to struct fields in 'data->req', which previously was made with a mix of
both.
2017-04-25 10:55:35 +02:00
Jay Satiro
33cfcfd9f0 TLS: Fix switching off SSL session id when client cert is used
Move the sessionid flag to ssl_primary_config so that ssl and proxy_ssl
will each have their own sessionid flag.

Regression since HTTPS-Proxy support was added in cb4e2be. Prior to that
this issue had been fixed in 247d890, CVE-2016-5419.

Bug: https://github.com/curl/curl/issues/1341
Reported-by: lijian996@users.noreply.github.com

The new incarnation of this bug is called CVE-2017-7468 and is documented
here: https://curl.haxx.se/docs/adv_20170419.html
2017-04-18 07:56:34 +02:00
Antti Hätälä
36e604fc78 url: don't free postponed data on connection reuse
- Don't free postponed data on a connection that will be reused since
  doing so can cause data loss when pipelining.

Only Windows builds are affected by this.

Closes https://github.com/curl/curl/issues/1380
2017-04-04 16:03:26 -04:00
Daniel Stenberg
e60fe20fdf llist: replace Curl_llist_alloc with Curl_llist_init
No longer allocate the curl_llist head struct for lists separately.

Removes 17 (15%) tiny allocations in a normal "curl localhost" invoke.

closes #1381
2017-04-04 15:27:45 +02:00
Isaac Boukris
7975d10cf8 http: Fix proxy connection reuse with basic-auth
When using basic-auth, connections and proxy connections
can be re-used with different Authorization headers since
it does not authenticate the connection (like NTLM does).

For instance, the below command should re-use the proxy
connection, but it currently doesn't:
curl -v -U alice:a -x http://localhost:8181 http://localhost/
  --next -U bob:b -x http://localhost:8181 http://localhost/

This is a regression since refactoring of ConnectionExists()
as part of: cb4e2be7c6

Fix the above by removing the username and password compare
when re-using proxy connection at proxy_info_matches().

However, this fix brings back another bug would make curl
to re-print the old proxy-authorization header of previous
proxy basic-auth connection because it wasn't cleared.

For instance, in the below command the second request should
fail if the proxy requires authentication, but would succeed
after the above fix (and before aforementioned commit):
curl -v -U alice:a -x http://localhost:8181 http://localhost/
  --next -x http://localhost:8181 http://localhost/

Fix this by clearing conn->allocptr.proxyuserpwd after use
unconditionally, same as we do for conn->allocptr.userpwd.

Also fix test 540 to not expect digest auth header to be
resent when connection is reused.

Signed-off-by: Isaac Boukris <iboukris@gmail.com>

Closes https://github.com/curl/curl/pull/1350
2017-03-28 03:54:43 -04:00
klemens
f7df67cff0 spelling fixes
Closes #1356
2017-03-26 23:56:23 +02:00
Sylvestre Ledru
66de563482 Improve code readbility
... by removing the else branch after a return, break or continue.

Closes #1310
2017-03-13 23:11:45 +01:00
Desmond O. Chang
d2bcf1e3e2 url: add option CURLOPT_SUPPRESS_CONNECT_HEADERS
- Add new option CURLOPT_SUPPRESS_CONNECT_HEADERS to allow suppressing
  proxy CONNECT response headers from the user callback functions
  CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION.

- Add new tool option --suppress-connect-headers to expose
  CURLOPT_SUPPRESS_CONNECT_HEADERS and allow suppressing proxy CONNECT
  response headers from --dump-header and --include.

Assisted-by: Jay Satiro
Assisted-by: CarloCannas@users.noreply.github.com
Closes https://github.com/curl/curl/pull/783
2017-03-12 01:32:33 -05:00
Dan Fandrich
381aff0385 url: don't compile detect_proxy if HTTP support is disabled 2017-03-11 18:02:34 +01:00
Jozef Kralik
6448f98c18 vtls: add options to specify range of enabled TLS versions
This commit introduces the CURL_SSLVERSION_MAX_* constants as well as
the --tls-max option of the curl tool.

Closes https://github.com/curl/curl/pull/1166
2017-03-08 15:54:07 +01:00
Daniel Stenberg
aadb7c7b62 URL: return error on malformed URLs with junk after port number
... because it causes confusion with users. Example URLs:

"http://[127.0.0.1]:11211:80" which a lot of languages' URL parsers will
parse and claim uses port number 80, while libcurl would use port number
11211.

"http://user@example.com:80@localhost" which by the WHATWG URL spec will
be treated to contain user name 'user@example.com' but according to
RFC3986 is user name 'user' for the host 'example.com' and then port 80
is followed by "@localhost"

Both these formats are now rejected, and verified so in test 1260.

Reported-by: Orange Tsai
2017-03-06 16:08:21 +01:00
Jay Satiro
9f20333443 url: split off proxy init and parsing from create_conn
Move the proxy parse/init into helper create_conn_helper_init_proxy to
mitigate the chances some non-proxy code will be mistakenly added to it.

Ref: https://github.com/curl/curl/issues/1274#issuecomment-281556510
Ref: https://github.com/curl/curl/pull/1293

Closes https://github.com/curl/curl/pull/1298
2017-03-03 03:14:25 -05:00
Isaac Boukris
1f8023ceb5 url: fix unix-socket support for proxy-disabled builds
Prior to this change if curl was built with Unix Socket support
(--enable-unix-sockets) and without Proxy support (--disable-proxy) then
unix socket options would erroneously be ignored.

Regression introduced in:
0b8d682f81

Bug: https://github.com/curl/curl/issues/1274
Reported-by: mccormickt12@users.noreply.github.com

Closes https://github.com/curl/curl/pull/1289
2017-02-25 18:21:15 -05:00
Dan Fandrich
ce9ba3ba10 proxy: fixed a memory leak on OOM 2017-02-25 21:36:39 +01:00
Jay Satiro
93cce2489d url: Default the CA proxy bundle location to CURL_CA_BUNDLE
If the compile-time CURL_CA_BUNDLE location is defined use it as the
default value for the proxy CA bundle location, which is the same as
what we already do for the regular CA bundle location.

Ref: https://github.com/curl/curl/pull/1257
2017-02-23 17:06:11 -05:00
Jay Satiro
b259646ea1 url: Improve CURLOPT_PROXY_CAPATH error handling
- Change CURLOPT_PROXY_CAPATH to return CURLE_NOT_BUILT_IN if the option
  is not supported, which is the same as what we already do for
  CURLOPT_CAPATH.

- Change the curl tool to handle CURLOPT_PROXY_CAPATH error
  CURLE_NOT_BUILT_IN as a warning instead of as an error, which is the
  same as what we already do for CURLOPT_CAPATH.

- Fix CAPATH docs to show that CURLE_NOT_BUILT_IN is returned when the
  respective CAPATH option is not supported by the SSL library.

Ref: https://github.com/curl/curl/pull/1257
2017-02-21 22:24:40 -05:00
Daniel Stenberg
ca6ea6d9be TLS: make SSL_VERIFYSTATUS work again
The CURLOPT_SSL_VERIFYSTATUS option was not properly handled by libcurl
and thus even if the status couldn't be verified, the connection would
be allowed and the user would not be told about the failed verification.

Regression since cb4e2be7c6

CVE-2017-2629
Bug: https://curl.haxx.se/docs/adv_20170222.html

Reported-by: Marcus Hoffmann
2017-02-21 08:33:18 +01:00
Michael Kaufmann
2f8d0df085 proxy: fix hostname resolution and IDN conversion
Properly resolve, convert and log the proxy host names.
Support the "--connect-to" feature for SOCKS proxies and for passive FTP
data transfers.

Follow-up to cb4e2be

Reported-by: Jay Satiro
Fixes https://github.com/curl/curl/issues/1248
2017-02-18 15:04:43 +01:00
Daniel Stenberg
c5c4e816b4 URL: only accept ";options" in SMTP/POP3/IMAP URL schemes
Fixes #1252
2017-02-10 14:51:53 +01:00
Richy Kim
6b7616690e CURLOPT_BUFFERSIZE: support enlarging receive buffer
Replace use of fixed macro BUFSIZE to define the size of the receive
buffer.  Reappropriate CURLOPT_BUFFERSIZE to include enlarging receive
buffer size.  Upon setting, resize buffer if larger than the current
default size up to a MAX_BUFSIZE (512KB). This can benefit protocols
like SFTP.

Closes #1222
2017-01-19 23:38:04 +01:00
Michael Kaufmann
2d6831b758 CURLOPT_CONNECT_TO: Fix compile warnings
Fix compile warnings that appeared only when curl has been configured
with '--disable-verbose'.
2017-01-18 13:02:25 +01:00
Daniel Stenberg
e9f45aa91b parseurl: move back buffer to function scope
Regression since 1d4202ad, which moved the buffer into a more narrow
scope, but the data in that buffer was used outside of that more narrow
scope.

Reported-by: Dan Fandrich
Bug: https://curl.haxx.se/mail/lib-2017-01/0093.html
2017-01-18 00:01:03 +01:00
Isaac Boukris
1d786faee1 unix_socket: add support for abstract unix domain socket
In addition to unix domain sockets, Linux also supports an
abstract namespace which is independent of the filesystem.

In order to support it, add new CURLOPT_ABSTRACT_UNIX_SOCKET
option which uses the same storage as CURLOPT_UNIX_SOCKET_PATH
internally, along with a flag to specify abstract socket.

On non-supporting platforms, the abstract address will be
interpreted as an empty string and fail gracefully.

Also add new --abstract-unix-socket tool parameter.

Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Reported-by: Chungtsun Li (typeless)
Reviewed-by: Daniel Stenberg
Reviewed-by: Peter Wu
Closes #1197
Fixes #1061
2017-01-13 16:25:20 +01:00
Daniel Stenberg
7d6e3f8cfa IDN: Use TR46 non-transitional
Assisted-by: Tim Rühsen
2017-01-13 14:53:50 +01:00
Daniel Stenberg
ee357664df IDN: revert use of the transitional option
It made the german ß get converted to ss, IDNA2003 style, and we can't
have that for the .de TLD - a primary reason for our switch to IDNA2008.

Test 165 verifies.
2017-01-13 14:31:40 +01:00
Tim Rühsen
ba315745f7 IDN: Fix compile time detection of linidn2 TR46
Follow-up to f30cbcac1

Closes #1207
2017-01-13 13:38:12 +01:00
ERAMOTO Masaya
2ac1942c72 url: --noproxy option overrides NO_PROXY environment variable
Under condition using http_proxy env var, noproxy list was the
combination of --noproxy option and NO_PROXY env var previously. Since
this commit, --noproxy option overrides NO_PROXY environment variable
even if use http_proxy env var.

Closes #1140
2017-01-13 11:18:29 +01:00
ERAMOTO Masaya
efdbfde7ca url: Refactor detect_proxy()
If defined CURL_DISABLE_HTTP, detect_proxy() returned NULL. If not
defined CURL_DISABLE_HTTP, detect_proxy() checked noproxy list.

Thus refactor to set proxy to NULL instead of calling detect_proxy() if
define CURL_DISABLE_HTTP, and refactor to call detect_proxy() if not
define CURL_DISABLE_HTTP and the host is not in the noproxy list.
2017-01-13 11:18:11 +01:00