Commit Graph

618 Commits

Author SHA1 Message Date
Daniel Stenberg 06f1db57ca
multi_runsingle: bail out early on data->conn == NULL
As that's a significant error condition and scan-build warns for NULL
pointer dereferences if we don't.

Closes #6433
2021-01-11 12:33:23 +01:00
Daniel Stenberg 48220f7e3d
multi: skip DONE state if there's no connection left for ftp wildcard
... to avoid running in that state with data->conn being NULL.
2021-01-11 12:33:11 +01:00
Daniel Stenberg 648712eec1
httpauth: make multi-request auth work with custom port
When doing HTTP authentication and a port number set with CURLOPT_PORT,
the code would previously have the URL's port number override as if it
had been a redirect to an absolute URL.

Added test 1568 to verify.

Reported-by: UrsusArctos on github
Fixes #6397
Closes #6400
2021-01-02 23:40:47 +01:00
Daniel Stenberg 3e17c8ab72
multi: when erroring in TOOFAST state, act as for PERFORM
When failing in TOOFAST, the multi_done() wasn't called so the same
cleanup and handling wasn't done like when it fails in PERFORM, which in
the case of FTP could mean that the control connection wouldn't be
marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused
ftp_disconnect() to use it to send "QUIT", which could end up waiting
for a response a long time before giving up!

Reported-by: Tomas Berger
Fixes #6333
Closes #6337
2020-12-17 13:57:27 +01:00
Daniel Stenberg 2260e0ebe6
Revert "multi: implement wait using winsock events"
This reverts commit d2a7d7c185.

This commit also reverts the subsequent follow-ups to that commit, which
were all done within windows #ifdefs that are removed in this
change. Marc helped me verify this.

Fixes #6146
Closes #6281
2020-12-06 22:40:38 +01:00
Daniel Stenberg b6b535994e
splay: rename Curl_splayremovebyaddr to Curl_splayremove
... and remove the old unused proto for the old Curl_splayremove
version.

Closes #6269
2020-12-01 08:09:51 +01:00
Daniel Stenberg 4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg 475c1aba68
checksrc: warn on empty line before open brace
... and fix a few occurances

Closes #6088
2020-10-15 23:32:26 +02:00
Marc Hoersken 1060955a61
multi: align WinSock mask variables in Curl_multi_wait
Also skip pre-checking sockets to set timeout_ms to 0
after the first socket has been detected to be ready.

Reviewed-by: rcombs on github
Reviewed-by: Daniel Stenberg

Follow up to #5886
2020-09-17 07:41:14 +02:00
Marc Hoersken 40d67da7ca
multi: reuse WinSock events variable in Curl_multi_wait
Since the struct is quite large (1 long and 10 ints) we
declare it once at the beginning of the function instead
of multiple times inside loops to avoid stack movements.

Reviewed-by: Viktor Szakats
Reviewed-by: Daniel Stenberg

Closes #5886
2020-09-17 07:40:49 +02:00
Daniel Stenberg 9e90ff0839
hash: make it 'struct Curl_hash'
As internal global names should use captical C.

Closes #5906
2020-09-02 22:41:58 +02:00
Daniel Stenberg 9b3f888a00
llist: make it "struct Curl_llist"
As internal global names should use captical C.

Closes #5906
2020-09-02 22:41:58 +02:00
Marc Hoersken 003e81e2a3
multi: handle connection state winsock events
Learn from the way Cygwin handles and maps the WinSock events
to simulate correct and complete poll and select behaviour
according to Richard W. Stevens Network Programming book.

Reviewed-by: Jay Satiro
Reviewed-by: Marcel Raad

Follow up to #5634
Closes #5867
2020-08-28 17:57:52 +02:00
Daniel Stenberg a2c85bb8e4
Curl_pgrsTime - return new time to avoid timeout integer overflow
Setting a timeout to INT_MAX could cause an immediate error to get
returned as timeout because of an overflow when different values of
'now' were used.

This is primarily fixed by having Curl_pgrsTime() return the "now" when
TIMER_STARTSINGLE is set so that the parent function will continue using
that time.

Reported-by: Ionuț-Francisc Oancea
Fixes #5583
Closes #5847
2020-08-28 14:16:41 +02:00
Marc Hoersken 3334ee6bcb
multi: expand pre-check for socket readiness
Check readiness of all sockets before waiting on them
to avoid locking in case the one-time event FD_WRITE
was already consumed by a previous wait operation.

More information about WinSock network events:
https://docs.microsoft.com/en-us/windows/win32/api/
   winsock2/nf-winsock2-wsaeventselect#return-value

Closes #5634
2020-08-25 12:11:45 +02:00
rcombs d2a7d7c185
multi: implement wait using winsock events
This avoids using a pair of TCP ports to provide wakeup functionality
for every multi instance on Windows, where socketpair() is emulated
using a TCP socket on loopback which could in turn lead to socket
resource exhaustion.

A previous version of this patch failed to account for how in WinSock,
FD_WRITE is set only once when writing becomes possible and not again
until after a send has failed due to the buffer filling. This contrasts
to how FD_READ and FD_OOB continue to be set until the conditions they
refer to no longer apply. This meant that if a user wrote some data to
a socket, but not enough data to completely fill its send buffer, then
waited on that socket to become writable, we'd erroneously stall until
their configured timeout rather than returning immediately.

This version of the patch addresses that issue by checking each socket
we're waiting on to become writable with select() before the wait, and
zeroing the timeout if it's already writable.

Assisted-by: Marc Hörsken
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Stenberg
Tested-by: Gergely Nagy
Tested-by: Rasmus Melchior Jacobsen
Tested-by: Tomas Berger

Replaces #5397
Reverts #5632
Closes #5634
2020-08-25 12:11:24 +02:00
Daniel Stenberg 3c9e021f86
Curl_easy: remember last connection by id, not by pointer
CVE-2020-8231

Bug: https://curl.haxx.se/docs/CVE-2020-8231.html

Reported-by: Marc Aldorasi
Closes #5824
2020-08-17 14:33:09 +02:00
Emil Engler da901fefe0
multi: Remove 10-year old out-commented code
The code hasn't been touched since 2010-08-18

Closes #5805
2020-08-12 15:28:46 +02:00
Daniel Stenberg ca567dc5a4
multi: Condition 'extrawait' is always true
Reported by Codacy.

Reviewed-by: Marcel Raad
Closes #5759
2020-08-01 23:21:29 +02:00
Marc Aldorasi d5bb459ccf
multi_remove_handle: close unused connect-only connections
Previously any connect-only connections in a multi handle would be kept
alive until the multi handle was closed.  Since these connections cannot
be re-used, they can be marked for closure when the associated easy
handle is removed from the multi handle.

Closes #5749
2020-08-01 15:26:08 +02:00
Daniel Stenberg abe59221fc
curl_multi_setopt: fix compiler warning "result is always false"
On systems with 32 bit long the expression is always false. Avoid
the warning.

Reported-by: Gisle Vanem
Bug: 61a08508f6 (commitcomment-40941232)
Closes #5736
2020-07-28 23:37:01 +02:00
Daniel Stenberg 61a08508f6
multi: remove two checks always true
Detected by Codacy
Closes #5676
2020-07-14 00:12:08 +02:00
Daniel Stenberg 25222b14ef
Revert "multi: implement wait using winsock events"
This reverts commit 8bc25c590e.

That commit (from #5397) introduced a regression in 7.71.0.

Reported-by: tmkk on github
Fixes #5631
Closes #5632
2020-06-30 12:27:23 +02:00
Daniel Stenberg ff1c37cbe3
url: make sure pushed streams get an allocated download buffer
Follow-up to c4e6968127

When a new transfer is created, as a resuly of an acknowledged push,
that transfer needs a download buffer allocated.

Closes #5590
2020-06-23 15:13:27 +02:00
rcombs 8bc25c590e
multi: implement wait using winsock events
This avoids using a pair of TCP ports to provide wakeup functionality
for every multi instance on Windows, where socketpair() is emulated
using a TCP socket on loopback which could in turn lead to socket
resource exhaustion.

Reviewed-by: Gergely Nagy
Reviewed-by: Marc Hörsken

Closes #5397
2020-06-17 07:18:08 +02:00
xquery 0a35580e21
multi: add defensive check on data->multi->num_alive
Closes #5540
2020-06-08 16:14:07 +02:00
Marc Hoersken 3186f50054
timeouts: move ms timeouts to timediff_t from int and long
Now that all functions in select.[ch] take timediff_t instead
of the limited int or long, we can remove type conversions
and related preprocessor checks to silence compiler warnings.

Avoiding conversions from time_t was already done in 842f73de.

Based upon #5262
Supersedes #5214, #5220 and #5221
Follow up to #5343 and #5479
Closes #5490
2020-06-06 20:05:58 +02:00
Daniel Stenberg f3d501dc67
build: disable more code/data when built without proxy support
Added build to travis to verify

Closes #5466
2020-05-30 23:18:16 +02:00
Daniel Stenberg c4e6968127
url: alloc the download buffer at transfer start
... and free it as soon as the transfer is done. It removes the extra
alloc when a new size is set with setopt() and reduces memory for unused
easy handles.

In addition: the closure_handle now doesn't use an allocated buffer at
all but the smallest supported size as a stack based one.

Closes #5472
2020-05-30 23:14:33 +02:00
Daniel Stenberg ed35d6590e
dynbuf: introduce internal generic dynamic buffer functions
A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to them. Existing
functionality has been ported over.

In my early basic testing, the total number of allocations seem at
roughly the same amount as before, possibly a few less.

See docs/DYNBUF.md for a description of the API.

Closes #5300
2020-05-04 10:40:39 +02:00
Daniel Stenberg c069027139
conncache: various concept cleanups
More connection cache accesses are protected by locks.

CONNCACHE_* is a beter prefix for the connection cache lock macros.

Curl_attach_connnection: now called as soon as there's a connection
struct available and before the connection is added to the connection
cache.

Curl_disconnect: now assumes that the connection is already removed from
the connection cache.

Ref: #4915
Closes #5009
2020-04-30 14:27:54 +02:00
Jay Satiro a268ad5d7e multi: Improve parameter check for curl_multi_remove_handle
- If an easy handle is owned by a multi different from the one specified
  then return CURLM_BAD_EASY_HANDLE.

Prior to this change I assume user error could cause corruption.

Closes https://github.com/curl/curl/pull/5116
2020-03-18 02:58:42 -04:00
Jay Satiro 2258b7bcc2 multi: skip EINTR check on wakeup socket if it was closed
- Don't check errno on wakeup socket if sread returned 0 since sread
  doesn't set errno in that case.

This is a follow-up to cf7760a from several days ago which fixed
Curl_multi_wait to stop busy looping sread on the non-blocking wakeup
socket if it was closed (ie sread returns 0). Due to a logic error it
was still possible to busy loop in that case if errno == EINTR.

Closes https://github.com/curl/curl/pull/5047
2020-03-06 02:56:21 -05:00
Daniel Stenberg cf7760ab52
multi_wait: stop loop when sread() returns zero
It's unclear why it would ever return zero here, but this change fixes
Robert's problem and it shouldn't loop forever...

Reported-by: Robert Dunaj
Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html
Closes #5019
2020-03-02 15:45:05 +01:00
Daniel Stenberg 4a4b63daaa
socks: make the connect phase non-blocking
Removes two entries from KNOWN_BUGS.

Closes #4907
2020-02-17 00:08:48 +01:00
Daniel Stenberg d60b1b37a1
multi: if Curl_readwrite sets 'comeback' use expire, not loop
Otherwise, a very fast single transfer ricks starving out other
concurrent transfers.

Closes #4927
2020-02-16 22:52:41 +01:00
Jay Satiro 4d1aa8d43b multi: fix outdated comment
- Do not say that conn->data is "cleared" by multi_done().

If the connection is in use then multi_done assigns another easy handle
still using the connection to conn->data, therefore in that case it is
not cleared.

Closes https://github.com/curl/curl/pull/4901
2020-02-12 16:55:33 -05:00
Daniel Stenberg db9af34634
multi_done: if multiplexed, make conn->data point to another transfer
... since the current transfer is being killed. Setting to NULL is
wrong, leaving it pointing to 'data' is wrong since that handle might be
about to get freed.

Fixes #4845
Closes #4858
Reported-by: dmitrmax on github
2020-01-28 13:00:41 +01:00
Daniel Stenberg 0b030a5b23
global_init: move the IPv6 works status bool to multi handle
Previously it was stored in a global state which contributed to
curl_global_init's thread unsafety. This boolean is now instead figured
out in curl_multi_init() and stored in the multi handle. Less effective,
but thread safe.

Closes #4851
2020-01-28 08:03:22 +01:00
Daniel Stenberg 9607532873
ConnectionExists: respect the max_concurrent_streams limits
A regression made the code use 'multiplexed' as a boolean instead of the
counter it is intended to be. This made curl try to "over-populate"
connections with new streams.

This regression came with 41fcdf71a1, shipped in curl 7.65.0.

Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same
check.

Reported-by: Kunal Ekawde
Fixes #4779
Closes #4784
2020-01-13 15:44:58 +01:00
Daniel Stenberg 29babeafec
misc: Copyright year out of date, should be 2020
Follow-up to recent commits

[skip ci]
2020-01-12 16:55:50 +01:00
Jay Satiro b700662b1c multi: Change curl_multi_wait/poll to error on negative timeout
- Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when
  curl_multi_wait/poll is passed timeout param < 0.

Prior to this change passing a negative value to curl_multi_wait/poll
such as -1 could cause the function to wait forever.

Reported-by: hamstergene@users.noreply.github.com

Fixes https://github.com/curl/curl/issues/4763

Closes https://github.com/curl/curl/pull/4765
2020-01-11 18:16:28 -05:00
Xiang Xiao 060fb84a5a lib: remove erroneous +x file permission on some c files
Modified by commit eb9a604 accidentally.

Closes https://github.com/curl/curl/pull/4756
2019-12-27 22:55:52 -05:00
Daniel Stenberg 29ca9fc596
multi: free sockhash on OOM
This would otherwise leak memory in the error path.

Detected by torture test 1540.

Closes #4713
2019-12-13 22:46:46 +01:00
Daniel Stenberg c7bc689fc3
conn: always set bits.close with connclose()
Closes #4690
2019-12-09 17:17:42 +01:00
Daniel Stenberg ee263de7a3
conncache: fix multi-thread use of shared connection cache
It could accidentally let the connection get used by more than one
thread, leading to double-free and more.

Reported-by: Christopher Reid
Fixes #4544
Closes #4557
2019-12-09 15:30:09 +01:00
Gergely Nagy f3c35e371c
multi: add curl_multi_wakeup()
This commit adds curl_multi_wakeup() which was previously in the TODO
list under the curl_multi_unblock name.

On some platforms and with some configurations this feature might not be
available or can fail, in these cases a new error code
(CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup().

Fixes #4418
Closes #4608
2019-11-25 15:45:56 +01:00
Gergely Nagy d1476aa11e multi: Fix curl_multi_poll wait when extra_fds && !extra_nfds
Prior to this change:

The check if an extra wait is necessary was based not on the
number of extra fds but on the pointer.

If a non-null pointer was given in extra_fds, but extra_nfds
was zero, then the wait was skipped even though poll was not
called.

Closes https://github.com/curl/curl/pull/4610
2019-11-17 23:35:48 -05:00
Daniel Stenberg 4e1eee1500
multi_poll: avoid busy-loop when called without easy handles attached
Fixes #4594
Closes #4595
Reported-by: 3dyd on github
2019-11-14 23:22:22 +01:00
Daniel Stenberg 13182b33f7
remove_handle: clear expire timers after multi_done()
Since 59041f0, a new timer might be set in multi_done() so the clearing
of the timers need to happen afterwards!

Reported-by: Max Kellermann
Fixes #4575
Closes #4583
2019-11-11 14:49:17 +01:00