Commit Graph

319 Commits

Author SHA1 Message Date
Jay Satiro 3bfda07004 easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares
- Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in
  curl_easy_duphandle.

Prior to this change if c-ares was used as the resolver backend and
either it was too old or libcurl was built without IPv6 support then
some of our resolver functions could return CURLE_NOT_BUILT_IN to
curl_easy_duphandle causing it to fail.

Caused by c8f086b which shipped in 7.69.1.

Reported-by: Karl Chen

Fixes https://github.com/curl/curl/issues/5097
Closes https://github.com/curl/curl/pull/5100
2020-03-14 19:07:05 -04:00
Patrick Monnerat 96972ec1c0
mime: latch last read callback status.
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.

Fixes #4813
Reported-by: MrdUkk on github
2020-03-07 23:26:00 +01:00
Ernst Sjöstrand c8f086bcc3
ares: store dns parameters for duphandle
With c-ares the dns parameters lives in ares_channel. Store them in the
curl handle and set them again in easy_duphandle.

Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0.

Fixes #4893
Closes #5020
Signed-off-by: Ernst Sjöstrand <ernst.sjostrand@verisure.com>
2020-03-07 13:50:23 +01:00
Daniel Stenberg e364546fb3
version: make curl_version* thread-safe without using global context
Closes #5010
2020-03-07 12:10:11 +01:00
Daniel Stenberg e040146f22
pause: force a connection (re-)check after unpausing
There might be data available that was already read off the socket, for
example in the TLS layer.

Reported-by: Anders Berg
Fixes #4966
Closes #5049
2020-03-07 10:49:51 +01:00
Daniel Stenberg e54b1885d1
pause: bail out on bad input
A NULL easy handle or an easy handle without an associated connection
cannot be paused or unpaused.

Closes #5050
2020-03-06 17:31:01 +01:00
Daniel Stenberg 64258bd0aa
pause: return early for calls that don't change pause state
Reviewed-by: Patrick Monnerat
Ref: #4833
Closes #5026
2020-03-06 10:25:40 +01:00
Daniel Stenberg 8aa04e9a24
Revert "pause: force-drain the transfer on unpause"
This reverts commit fa0216b294 (from #5000)

Clearly that didn't solve the problem correctly.

Reported-by: Christopher Reid
Reopens #4966
Fixes #5044
2020-03-05 23:45:41 +01:00
Daniel Stenberg 8cd4e6d81f
Revert "mime: latch last read callback status."
This reverts commit 87869e38d7.

Fixes #5014
Closes #5015
Reopens #4833
2020-03-02 08:07:49 +01:00
Patrick Monnerat 87869e38d7
mime: latch last read callback status.
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.

Fixes #4813
Reported-by: MrdUkk on github
Closes #4833
2020-03-02 00:00:50 +01:00
Daniel Stenberg fa0216b294
pause: force-drain the transfer on unpause
... since the socket might not actually be readable anymore when for
example the data is already buffered in the TLS layer.

Fixes #4966
Reported-by: Anders Berg
Closes #5000
2020-02-29 10:53:09 +01:00
Daniel Stenberg 15f51474c8
http2: make pausing/unpausing set/clear local stream window
This reduces the HTTP/2 window size to 32 MB since libcurl might have to
buffer up to this amount of data in memory and yet we don't want it set
lower to potentially impact tranfer performance on high speed networks.

Requires nghttp2 commit b3f85e2daa629
(https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end
up in the next release after 1.40.0.

Fixes #4939
Closes #4940
2020-02-27 10:35:51 +01:00
Jay Satiro 55bb83c92b easy: remove dead code
multi is already assigned to data->multi by curl_multi_add_handle.

Closes https://github.com/curl/curl/pull/4900
2020-02-12 16:54:51 -05:00
Daniel Stenberg 0b030a5b23
global_init: move the IPv6 works status bool to multi handle
Previously it was stored in a global state which contributed to
curl_global_init's thread unsafety. This boolean is now instead figured
out in curl_multi_init() and stored in the multi handle. Less effective,
but thread safe.

Closes #4851
2020-01-28 08:03:22 +01:00
Daniel Stenberg 1ad49feb71
global_init: assume the EINTR bit by default
- Removed from global_init since it isn't thread-safe. The symbol will
  still remain to not break compiles, it just won't have any effect going
  forward.

- make the internals NOT loop on EINTR (the opposite from previously).
  It only risks returning from the select/poll/wait functions early, and that
  should be risk-free.

Closes #4840
2020-01-26 18:27:18 +01:00
Daniel Stenberg 920af1a664
wolfssh: make it init properly via Curl_ssh_init()
Closes #4846
2020-01-24 10:27:31 +01:00
Daniel Stenberg 6773c7ca65
wolfSSH: new SSH backend
Adds support for SFTP (not SCP) using WolfSSH.

Closes #4231
2020-01-12 17:19:12 +01:00
Daniel Stenberg bc5d22c3de
global_init: undo the "intialized" bump in case of failure
... so that failures in the global init function don't count as a
working init and it can then be called again.

Reported-by: Paul Groke
Fixes #4636
Closes #4653
2019-12-02 00:04:56 +01:00
Jay Satiro 1f6a18685e lib: Move lib/ssh.h -> lib/vssh/ssh.h
Follow-up to 5b2d703 which moved ssh source files to vssh.

Closes https://github.com/curl/curl/pull/4609
2019-11-17 23:27:39 -05:00
Daniel Stenberg 32747aafa0
pause: avoid updating socket if done was already called
... avoids unnecesary recursive risk when the transfer is already done.

Reported-by: Richard Bowker
Fixes #4563
Closes #4574
2019-11-10 16:48:47 +01:00
Daniel Stenberg dcd7e37c3a
url: make Curl_close() NULLify the pointer too
This is the common pattern used in the code and by a unified approach we
avoid mistakes.

Closes #4534
2019-10-30 09:36:21 +01:00
Daniel Stenberg 2e68e5a023
easy: part of conditional expression is always true: !result
Fixes warning detected by PVS-Studio
Fixes #4374
2019-09-20 08:07:20 +02:00
Daniel Stenberg 5050edb124
cleanup: move functions out of url.c and make them static
Closes #4289
2019-09-03 13:11:12 +02:00
Daniel Stenberg d6dea75af7
vssh: move ssh init/cleanup functions into backend code 2019-08-17 16:57:55 +02:00
Daniel Stenberg 02346abc32
curl_multi_poll: a sister to curl_multi_wait() that waits more
Repeatedly we see problems where using curl_multi_wait() is difficult or
just awkward because if it has no file descriptor to wait for
internally, it returns immediately and leaves it to the caller to wait
for a small amount of time in order to avoid occasional busy-looping.

This is often missed or misunderstood, leading to underperforming
applications.

This change introduces curl_multi_poll() as a replacement drop-in
function that accepts the exact same set of arguments. This function
works identically to curl_multi_wait() - EXCEPT - for the case when
there's nothing to wait for internally, as then this function will by
itself wait for a "suitable" short time before it returns. This
effectiely avoids all risks of busy-looping and should also make it less
likely that apps "over-wait".

This also changes the curl tool to use this funtion internally when
doing parallel transfers and changes curl_easy_perform() to use it
internally.

Closes #4163
2019-08-06 09:33:29 +02:00
Jay Satiro 78ed3abe11 easy: resize receive buffer on easy handle reset
- In curl_easy_reset attempt to resize the receive buffer to its default
  size. If realloc fails then continue using the previous size.

Prior to this change curl_easy_reset did not properly handle resetting
the receive buffer (data->state.buffer). It reset the variable holding
its size (data->set.buffer_size) to the default size (READBUFFER_SIZE)
but then did not actually resize the buffer. If a user resized the
buffer by using CURLOPT_BUFFERSIZE to set the size smaller than the
default, later called curl_easy_reset and attempted to reuse the handle
then a heap overflow would very likely occur during that handle's next
transfer.

Reported-by: Felix Hädicke

Fixes https://github.com/curl/curl/issues/4143
Closes https://github.com/curl/curl/pull/4145
2019-07-25 22:24:06 -04:00
Daniel Stenberg 680f141332
unpause: trigger a timeout for event-based transfers
... so that timeouts or other state machine actions get going again
after a changing pause state. For example, if the last delivery was
paused there's no pending socket activity.

Reported-by: sstruchtrup on github
Fixes #3994
Closes #4001
2019-06-09 18:33:59 +02:00
Daniel Stenberg 8ece8177f1
cleanup: remove FIXME and TODO comments
They serve very little purpose and mostly just add noise. Most of them
have been around for a very long time. I read them all before removing
or rephrasing them.

Ref: #3876
Closes #3883
2019-05-16 09:16:56 +02:00
Steve Holme 52dfab65d6
auth: Rename the various authentication clean up functions
For consistency and to a avoid confusion.

Closes #3869
2019-05-12 18:37:00 +01:00
Marcel Raad 15fd9abfea
easy: fix another "clarify calculation precedence" warning
I missed this one in commit 6b3dde7fe6.
2019-05-12 13:36:45 +02:00
Marcel Raad 6b3dde7fe6
build: fix "clarify calculation precedence" warnings
Codacy/CppCheck warns about this. Consistently use parentheses as we
already do in some places to silence the warning.

Closes https://github.com/curl/curl/pull/3866
2019-05-12 09:31:09 +02:00
Steve Holme 5c2aebe308
http_digest: Don't expose functions when HTTP and Crypto Auth are disabled
Closes #3861
2019-05-11 17:23:32 +01:00
Daniel Stenberg f73de32d46
CURLOPT_DNS_USE_GLOBAL_CACHE: remove
Remove the code too. The functionality has been disabled in code since
7.62.0. Setting this option will from now on simply be ignored and have
no function.

Closes #3654
2019-04-11 22:22:39 +02:00
Jay Satiro db374c50db system_win32: move win32_init here from easy.c
.. since system_win32 is a more appropriate location for the functions
and to extern the globals.

Ref: https://github.com/curl/curl/commit/ca597ad#r32446578
Reported-by: Gisle Vanem

Closes https://github.com/curl/curl/pull/3625
2019-03-02 01:49:48 -05:00
Marcel Raad 50f1bb2632 Fix strict-prototypes GCC warning
As seen in the MinGW autobuilds. Caused by commit
f26bc29cfe.
2019-02-22 19:12:30 +01:00
Jay Satiro f26bc29cfe easy: fix win32 init to work without CURL_GLOBAL_WIN32
- Change the behavior of win32_init so that the required initialization
  procedures are not affected by CURL_GLOBAL_WIN32 flag.

libcurl via curl_global_init supports initializing for win32 with an
optional flag CURL_GLOBAL_WIN32, which if omitted was meant to stop
Winsock initialization. It did so internally by skipping win32_init()
when that flag was set. Since then win32_init() has been expanded to
include required initialization routines that are separate from
Winsock and therefore must be called in all cases. This commit fixes
it so that CURL_GLOBAL_WIN32 only controls the optional win32
initialization (which is Winsock initialization, according to our doc).

The only users affected by this change are those that don't pass
CURL_GLOBAL_WIN32 to curl_global_init. For them this commit removes the
risk of a potential crash.

Ref: https://github.com/curl/curl/pull/3573

Fixes https://github.com/curl/curl/issues/3313
Closes https://github.com/curl/curl/pull/3575
2019-02-18 02:12:12 -05:00
Daniel Stenberg ca597ad34a
Curl_now: figure out windows version in win32_init
... and avoid use of static variables that aren't thread safe.

Fixes regression from e9ababd4f5 (present in the 7.64.0 release)

Reported-by: Paul Groke
Fixes #3572
Closes #3573
2019-02-15 23:23:14 +01:00
Daniel Stenberg ba243235ec
urldata: rename easy_conn to just conn
We use "conn" everywhere to be a pointer to the connection.

Introduces two functions that "attaches" and "detaches" the connection
to and from the transfer.

Going forward, we should favour using "data->conn" (since a transfer
always only has a single connection or none at all) to "conn->data"
(since a connection can have none, one or many transfers associated with
it and updating conn->data to be correct is error prone and a frequent
reason for internal issues).

Closes #3442
2019-01-11 15:35:13 +01:00
Daniel Stenberg 34fe0e1622
curl_easy_perform: fix timeout handling
curl_multi_wait() was erroneously used from within
curl_easy_perform(). It could lead to it believing there was no socket
to wait for and then instead sleep for a while instead of monitoring the
socket and then miss acting on that activity as swiftly as it should
(causing an up to 1000 ms delay).

Reported-by: Antoni Villalonga
Fixes #3305
Closes #3306
Closes #3308
2018-11-25 11:25:43 +01:00
Romain Fliedel 6765e6d9e6
ares: remove fd from multi fd set when ares is about to close the fd
When using c-ares for asyn dns, the dns socket fd was silently closed
by c-ares without curl being aware. curl would then 'realize' the fd
has been removed at next call of Curl_resolver_getsock, and only then
notify the CURLMOPT_SOCKETFUNCTION to remove fd from its poll set with
CURL_POLL_REMOVE. At this point the fd is already closed.

By using ares socket state callback (ARES_OPT_SOCK_STATE_CB), this
patch allows curl to be notified that the fd is not longer needed
for neither for write nor read. At this point by calling
Curl_multi_closed we are able to notify multi with CURL_POLL_REMOVE
before the fd is actually closed by ares.

In asyn-ares.c Curl_resolver_duphandle we can't use ares_dup anymore
since it does not allow passing a different sock_state_cb_data

Closes #3238
2018-11-20 19:58:45 +01:00
Viktor Szakats b801b453af whitespace fixes
- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few indentations/line continuations
  in manual examples

Closes https://github.com/curl/curl/pull/3037
2018-09-23 22:24:02 +00:00
Daniel Stenberg 46e164069d
url: use the URL API internally as well
... to make it a truly unified URL parser.

Closes #3017
2018-09-22 11:58:10 +02:00
Daniel Stenberg 17ca0ccff4
curl_easy_upkeep: removed 'conn' from the name
... including the associated option.

Fixes #2951
Closes #2952
2018-09-07 13:43:26 +02:00
Max Dymond 7b655fcbad
upkeep: add a connection upkeep API: curl_easy_conn_upkeep()
Add functionality so that protocols can do custom keepalive on their
connections, when an external API function is called.

Add docs for the new options in 7.62.0

Closes #1641
2018-09-07 09:45:29 +02:00
Laurent Bonnans 4939f36524
vtls: reinstantiate engine on duplicated handles
Handles created with curl_easy_duphandle do not use the SSL engine set
up in the original handle. This fixes the issue by storing the engine
name in the internal url state and setting the engine from its name
inside curl_easy_duphandle.

Reported-by: Anton Gerasimov
Signed-of-by: Laurent Bonnans
Fixes #2829
Closes #2833
2018-08-18 15:46:23 +02:00
Daniel Stenberg 3ef67c6861
easy_perform: use *multi_timeout() to get wait times
... and trim the threaded Curl_resolver_getsock() to return zero
millisecond wait times during the first three milliseconds so that
localhost or names in the OS resolver cache gets detected and used
faster.

Closes #2685
2018-06-27 13:01:00 +02:00
Viktor Szakats 4bd91bc474 spelling fixes
Detected using the `codespell` tool (version 1.13.0).

Also secure and fix an URL.
2018-06-03 12:14:45 +00:00
Daniel Stenberg 2b126cd708
duphandle: make sure CURLOPT_RESOLVE is duplicated fine too
Verified in test 1502 now

Fixes #2485
Closes #2486
Reported-by: Ernst Sjöstrand
2018-04-12 16:21:24 +02:00
Daniel Stenberg fb4f568b1e
add_handle/easy_perform: clear errorbuffer on start if set
To offer applications a more defined behavior, we clear the buffer as
early as possible.

Assisted-by: Jay Satiro

Fixes #2190
Closes #2377
2018-03-17 12:07:37 +01:00
Daniel Stenberg 2404aa080e
pause: when changing pause state, update socket state
Especially unpausing a transfer might have to move the socket back to the
"currently used sockets" hash to get monitored. Otherwise it would never get
any more data and get stuck. Easily triggered with pausing using the
multi_socket API.

Reported-by: Philip Prindeville
Bug: https://curl.haxx.se/mail/lib-2018-03/0048.html
Fixes #2393
Closes #2391
2018-03-16 23:41:55 +01:00