Commit Graph

1524 Commits

Author SHA1 Message Date
Daniel Stenberg c069027139
conncache: various concept cleanups
More connection cache accesses are protected by locks.

CONNCACHE_* is a beter prefix for the connection cache lock macros.

Curl_attach_connnection: now called as soon as there's a connection
struct available and before the connection is added to the connection
cache.

Curl_disconnect: now assumes that the connection is already removed from
the connection cache.

Ref: #4915
Closes #5009
2020-04-30 14:27:54 +02:00
Bjorn Stenberg 2522903b79
mqtt: add new experimental protocol
Closes #5173
2020-04-14 13:03:40 +02:00
Daniel Stenberg d4afa0fc20
url: allow non-HTTPS altsvc-matching for debug builds
This is already partly supported but this part was missing.
Reported-by: James Fuller

Closes #5205
2020-04-09 17:46:24 +02:00
Daniel Stenberg 1d1e9e8ad7
Curl_is_ASCII_name: handle a NULL argument
Make the function tolerate a NULL pointer input to avoid dereferencing
that pointer.

Follow-up to efce3ea5a8
Detected by OSS-Fuzz
Reviewed-By: Steve Holme
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907
Fixes #4985
Closes #4986
2020-02-27 13:55:29 +01:00
Steve Holme 9352a40779
url: Make the IDN conversion functions available to others 2020-02-26 11:01:47 +00:00
Daniel Stenberg 14916a82e2
altsvc: make saving the cache an atomic operation
... by writing the file to temp name then rename to the final when done.

Assisted-by: Jay Satiro
Fixes #4936
Closes #4942
2020-02-18 07:49:21 +01:00
Steve Holme f8f4a94465
url: Include the failure reason when curl_win32_idn_to_ascii() fails
Provide the failure reason in the failf() info just as we do for the
libidn2 version of code.

Closes #4899
2020-02-09 11:38:54 +00:00
Pedro Monreal 4b6fd29f1a cleanup: fix typos and wording in docs and comments
Closes #4869
Reviewed-by: Emil Engler and Daniel Gustafsson
2020-02-02 18:43:01 +01:00
Steve Holme 6ef123522a ntlm: Move the winbind data into the NTLM data structure
To assist with adding winbind support to the SASL NTLM authentication,
move the winbind specific data out of conndata into ntlmdata.
2020-02-02 12:23:48 +01:00
Daniel Stenberg db9af34634
multi_done: if multiplexed, make conn->data point to another transfer
... since the current transfer is being killed. Setting to NULL is
wrong, leaving it pointing to 'data' is wrong since that handle might be
about to get freed.

Fixes #4845
Closes #4858
Reported-by: dmitrmax on github
2020-01-28 13:00:41 +01:00
Peter Piekarski 34e6bc42b0
conn: do not reuse connection if SOCKS proxy credentials differ
Closes #4835
2020-01-24 17:00:58 +01:00
nao dea17b519d
http: move "oauth_bearer" from connectdata to Curl_easy
Fixes the bug where oauth_bearer gets deallocated when we re-use a
connection.

Closes #4824
2020-01-21 10:32:43 +01:00
Daniel Stenberg 9607532873
ConnectionExists: respect the max_concurrent_streams limits
A regression made the code use 'multiplexed' as a boolean instead of the
counter it is intended to be. This made curl try to "over-populate"
connections with new streams.

This regression came with 41fcdf71a1, shipped in curl 7.65.0.

Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same
check.

Reported-by: Kunal Ekawde
Fixes #4779
Closes #4784
2020-01-13 15:44:58 +01:00
Daniel Stenberg 6773c7ca65
wolfSSH: new SSH backend
Adds support for SFTP (not SCP) using WolfSSH.

Closes #4231
2020-01-12 17:19:12 +01:00
Daniel Stenberg 8c0807aa16
create_conn: prefer multiplexing to using new connections
... as it would previously prefer new connections rather than
multiplexing in most conditions! The (now removed) code was a leftover
from the Pipelining code that was translated wrongly into a
multiplex-only world.

Reported-by: Kunal Ekawde
Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html
Closes #4732
2019-12-19 09:00:56 +01:00
Daniel Stenberg ee263de7a3
conncache: fix multi-thread use of shared connection cache
It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-by: Christopher Reid
Fixes #4544
Closes #4557
2019-12-09 15:30:09 +01:00
Jay Satiro 9c1806ae46 build: Disable Visual Studio warning "conditional expression is constant"
- Disable warning C4127 "conditional expression is constant" globally
  in curl_setup.h for when building with Microsoft's compiler.

This mainly affects building with the Visual Studio project files found
in the projects dir.

Prior to this change the cmake and winbuild build systems already
disabled 4127 globally for when building with Microsoft's compiler.
Also, 4127 was already disabled for all build systems in the limited
circumstance of the WHILE_FALSE macro which disabled the warning
specifically for while(0). This commit removes the WHILE_FALSE macro and
all other cruft in favor of disabling globally in curl_setup.

Background:

We have various macros that cause 0 or 1 to be evaluated, which would
cause warning C4127 in Visual Studio. For example this causes it:

    #define Curl_resolver_asynch() 1

Full behavior is not clearly defined and inconsistent across versions.
However it is documented that since VS 2015 Update 3 Microsoft has
addressed this somewhat but not entirely, not warning on while(true) for
example.

Prior to this change some C4127 warnings occurred when I built with
Visual Studio using the generated projects in the projects dir.

Closes https://github.com/curl/curl/pull/4658
2019-12-01 19:01:02 -05:00
Jay Satiro 1f6a18685e lib: Move lib/ssh.h -> lib/vssh/ssh.h
Follow-up to 5b2d703 which moved ssh source files to vssh.

Closes https://github.com/curl/curl/pull/4609
2019-11-17 23:27:39 -05:00
Wyatt O'Day 93738efe16 build: fix for CURL_DISABLE_DOH
Fixes https://github.com/curl/curl/issues/4565
Closes https://github.com/curl/curl/pull/4566
2019-11-06 23:48:10 -05:00
Daniel Stenberg dcd7e37c3a
url: make Curl_close() NULLify the pointer too
This is the common pattern used in the code and by a unified approach we
avoid mistakes.

Closes #4534
2019-10-30 09:36:21 +01:00
Daniel Stenberg 0cbd6f8df7
url: Curl_free_request_state() should also free doh handles
... or risk DoH memory leaks.

Reported-by: Paul Dreik
Fixes #4463
Closes #4527
2019-10-29 22:48:19 +01:00
Daniel Stenberg 807c056c08
conn-reuse: requests wanting NTLM can reuse non-NTLM connections
Added test case 338 to verify.

Reported-by: Daniel Silverstone
Fixes #4499
Closes #4514
2019-10-23 10:15:39 +02:00
Daniel Stenberg e062043433
url: normalize CURLINFO_EFFECTIVE_URL
The URL extracted with CURLINFO_EFFECTIVE_URL was returned as given as
input in most cases, which made it not get a scheme prefixed like before
if the URL was given without one, and it didn't remove dotdot sequences
etc.

Added test case 1907 to verify that this now works as intended and as
before 7.62.0.

Regression introduced in 7.62.0

Reported-by: Christophe Dervieux
Fixes #4491
Closes #4493
2019-10-16 14:10:55 +02:00
Daniel Stenberg 249541f12f
cookies: change argument type for Curl_flush_cookies
The second argument is really a 'bool' so use that and pass in TRUE/FALSE
to make it clear.

Closes #4455
2019-10-03 22:56:28 +02:00
Daniel Stenberg 8a00560de2
http2: move state-init from creation to pre-transfer
To make sure that the HTTP/2 state is initialized correctly for
duplicated handles. It would otherwise easily generate "spurious"
PRIORITY frames to get sent over HTTP/2 connections when duplicated easy
handles were used.

Reported-by: Daniel Silverstone
Fixes #4303
Closes #4442
2019-10-03 22:55:42 +02:00
Jay Satiro cded993700 url: don't set appconnect time for non-ssl/non-ssh connections
Prior to this change non-ssl/non-ssh connections that were reused set
TIMER_APPCONNECT [1]. Arguably that was incorrect since no SSL/SSH
handshake took place.

[1]: TIMER_APPCONNECT is publicly known as CURLINFO_APPCONNECT_TIME in
libcurl and %{time_appconnect} in the curl tool. It is documented as
"the time until the SSL/SSH handshake is completed".

Reported-by: Marcel Hernandez

Ref: https://github.com/curl/curl/issues/3760

Closes https://github.com/curl/curl/pull/3773
2019-09-26 03:04:26 -04:00
Jay Satiro af3ced3b9c
url: fix the NULL hostname compiler warning case
Closes #4403
2019-09-25 14:35:02 +02:00
Daniel Stenberg 389426e3d0
url: remove dead code
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:13 +02:00
Daniel Stenberg 3ab45650e2
url: part of expression is always true: (bundle->multiuse == 0)
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:11 +02:00
Daniel Stenberg 3c5f9ba899
url: only reuse TLS connections with matching pinning
If the requests have different CURLOPT_PINNEDPUBLICKEY strings set, the
connection should not be reused.

Bug: https://curl.haxx.se/mail/lib-2019-09/0061.html
Reported-by: Sebastian Haglund

Closes #4347
2019-09-19 14:58:24 +02:00
Daniel Stenberg beb4350917
url: cleanup dangling DOH request headers too
Follow-up to 9bc44ff64d

Credit to OSS-Fuzz
Bug: https://crbug.com/oss-fuzz/17269

Closes #4372
2019-09-18 07:48:49 +02:00
Daniel Stenberg 9bc44ff64d
doh: clean up dangling DOH handles and memory on easy close
If you set the same URL for target as for DoH (and it isn't a DoH
server), like "https://example.com" in both, the easy handles used for
the DoH requests could be left "dangling" and end up not getting freed.

Reported-by: Paul Dreik
Closes #4366
2019-09-16 17:31:56 +02:00
Daniel Stenberg 5050edb124
cleanup: move functions out of url.c and make them static
Closes #4289
2019-09-03 13:11:12 +02:00
Daniel Stenberg e59540139a
cleanup: remove DOT_CHAR completely
Follow-up to f9c7ba9096

The use of DOT_CHAR for ".ssh" was probably a mistake and is removed
now.

Pointed-out-by: Gisle Vanem
Bug: https://github.com/curl/curl/pull/4230#issuecomment-522960638

Closes #4247
2019-08-20 23:07:59 +02:00
Daniel Stenberg 69b3ff5118
alt-svc: add protocol version selection masking
So that users can mask in/out specific HTTP versions when Alt-Svc is
used.

 - Removed "h2c" and updated test case accordingly
 - Changed how the altsvc struct is laid out
 - Added ifdefs to make the unittest run even in a quiche-tree

Closes #4201
2019-08-08 09:10:29 +02:00
Daniel Stenberg a93b43cde8
http3: fix the HTTP/3 in the request, make alt-svc set right versions
Closes #4200
2019-08-08 08:28:43 +02:00
Daniel Stenberg 98c3f14871
alt-svc: send Alt-Used: in redirected requests
RFC 7838 section 5:

   When using an alternative service, clients SHOULD include an Alt-Used
   header field in all requests.

Removed CURLALTSVC_ALTUSED again (feature is still EXPERIMENTAL thus
this is deemed ok).

You can disable sending this header just like you disable any other HTTP
header in libcurl.

Closes #4199
2019-08-08 08:24:37 +02:00
Daniel Stenberg fd00e14adb
curl.h: add CURL_HTTP_VERSION_3 to the version enum
It can't be set for CURLOPT_HTTP_VERSION, but it can be extracted with
CURLINFO_HTTP_VERSION.
2019-08-07 12:51:52 +02:00
Steve Holme 7c469fa537 sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID
Added the ability for the calling program to specify the authorisation
identity (authzid), the identity to act as, in addition to the
authentication identity (authcid) and password when using SASL PLAIN
authentication.

Fixes #3653
Closes #3790

NOTE: This commit was cherry-picked and is part of a series of commits
that added the authzid feature for upcoming 7.66.0. The series was
temporarily reverted in db8ec1f so that it would not ship in a 7.65.x
patch release.

Closes https://github.com/curl/curl/pull/4186
2019-08-06 11:38:20 -04:00
Daniel Stenberg a42b0957ab
http09: disable HTTP/0.9 by default in both tool and library
As the plan has been laid out in DEPRECATED. Update docs accordingly and
verify in test 1174. Now requires the option to be set to allow HTTP/0.9
responses.

Closes #4191
2019-08-06 00:08:14 +02:00
Daniel Stenberg cc28a502bc
url: make use of new HTTP version if alt-svc has one 2019-08-03 21:55:51 +02:00
Daniel Stenberg a9a2b26830
url: set conn->transport to default TCP at init time 2019-08-03 21:55:51 +02:00
Daniel Stenberg b1616dad8f
timediff: make it 64 bit (if possible) even with 32 bit time_t
... to make it hold microseconds too.

Fixes #4165
Closes #4168
2019-08-01 07:43:24 +02:00
Daniel Stenberg a55faf33d4
cleanup: remove the 'numsocks' argument used in many places
It was used (intended) to pass in the size of the 'socks' array that is
also passed to these functions, but was rarely actually checked/used and
the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries
that should be used instead.

Closes #4169
2019-07-30 23:16:44 +02:00
Daniel Stenberg 3af0e76d1e
HTTP3: initial (experimental) support
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes #3500
2019-07-21 23:49:03 +02:00
Jay Satiro e8442e4ffc libcurl: Restrict redirect schemes (follow-up)
- Allow FTPS on redirect.

- Update default allowed redirect protocols in documentation.

Follow-up to 6080ea0.

Ref: https://github.com/curl/curl/pull/4094

Closes https://github.com/curl/curl/pull/4115
2019-07-17 00:48:40 -04:00
Linos Giannopoulos 6080ea098d
libcurl: Restrict redirect schemes
All protocols except for CURLPROTO_FILE/CURLPROTO_SMB and their TLS
counterpart were allowed for redirect. This vastly broadens the
exploitation surface in case of a vulnerability such as SSRF [1], where
libcurl-based clients are forced to make requests to arbitrary hosts.

For instance, CURLPROTO_GOPHER can be used to smuggle any TCP-based
protocol by URL-encoding a payload in the URI. Gopher will open a TCP
connection and send the payload.

Only HTTP/HTTPS and FTP are allowed. All other protocols have to be
explicitly enabled for redirects through CURLOPT_REDIR_PROTOCOLS.

[1]: https://www.acunetix.com/blog/articles/server-side-request-forgery-vulnerability/

Signed-off-by: Linos Giannopoulos <lgian@skroutz.gr>

Closes #4094
2019-07-14 16:29:55 +02:00
Daniel Stenberg 755083d00d
conn_maxage: move the check to prune_dead_connections()
... and avoid the locking issue.

Reported-by: Kunal Ekawde
Fixes #4029
Closes #4032
2019-06-17 23:51:26 +02:00
Cliff Crosland f67009dd98 url: Fix CURLOPT_MAXAGE_CONN time comparison
Old connections are meant to expire from the connection cache after
CURLOPT_MAXAGE_CONN seconds. However, they actually expire after 1000x
that value. This occurs because a time value measured in milliseconds is
accidentally divided by 1M instead of by 1,000.

Closes https://github.com/curl/curl/pull/4013
2019-06-12 01:10:22 -04:00
Daniel Stenberg 094b5f3540
multi: make sure 'data' can present in several sockhash entries
Since more than one socket can be used by each transfer at a given time,
each sockhash entry how has its own hash table with transfers using that
socket.

In addition, the sockhash entry can now be marked 'blocked = TRUE'"
which then makes the delete function just set 'removed = TRUE' instead
of removing it "for real", as a way to not rip out the carpet under the
feet of a parent function that iterates over the transfers of that same
sockhash entry.

Reported-by: Tom van der Woerdt
Fixes #3961
Fixes #3986
Fixes #3995
Fixes #4004
Closes #3997
2019-06-10 00:47:48 +02:00