For a single-stream download from localhost, we managed to increase
transfer speed from 1.6MB/sec to around 400MB/sec, mostly because of
this single fix.
connect.c:953:5: warning: initializer element is not computable at load
time
connect.c:953:5: warning: missing initializer for field 'dwMinorVersion'
of 'OSVERSIONINFOEX'
curl_sspi.c:97:5: warning: initializer element is not computable at load
time
curl_sspi.c:97:5: warning: missing initializer for field 'szCSDVersion'
of 'OSVERSIONINFOEX'
When CURL_SOCKET_BAD is returned in the callback, it should be treated
as an error (CURLE_COULDNT_CONNECT) if no other socket is subsequently
created when trying to connect to a server.
Bug: http://curl.haxx.se/mail/lib-2015-06/0047.html
This header file must be included after all header files except
memdebug.h, as it does similar memory function redefinitions and can be
similarly affected by conflicting definitions in system or dependent
library headers.
Bug: https://github.com/bagder/curl/pull/168
(trynextip)
- Don't try the "other" protocol family unless IPv6 is available. In an
IPv4-only build the other family can only be IPv6 which is unavailable.
This change essentially stops IPv4-only builds from attempting the
"happy eyeballs" secondary parallel connection that is supposed to be
used by the "other" address family.
Prior to this change in IPv4-only builds that secondary parallel
connection attempt could be erroneously used by the same family (IPv4)
which caused a bug where every address after the first for a host could
be tried twice, often in parallel. This change fixes that bug. An
example of the bug is shown below.
Assume MTEST resolves to 3 addresses 127.0.0.2, 127.0.0.3 and 127.0.0.4:
* STATE: INIT => CONNECT handle 0x64f4b0; line 1046 (connection #-5000)
* Rebuilt URL to: http://MTEST/
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x64f4b0; line 1083
(connection #0)
* Trying 127.0.0.2...
* STATE: WAITRESOLVE => WAITCONNECT handle 0x64f4b0; line 1163
(connection #0)
* Trying 127.0.0.3...
* connect to 127.0.0.2 port 80 failed: Connection refused
* Trying 127.0.0.3...
* connect to 127.0.0.3 port 80 failed: Connection refused
* Trying 127.0.0.4...
* connect to 127.0.0.3 port 80 failed: Connection refused
* Trying 127.0.0.4...
* connect to 127.0.0.4 port 80 failed: Connection refused
* connect to 127.0.0.4 port 80 failed: Connection refused
* Failed to connect to MTEST port 80: Connection refused
* Closing connection 0
* The cache now contains 0 members
* Expire cleared
curl: (7) Failed to connect to MTEST port 80: Connection refused
The bug was born in commit bagder/curl@2d435c7.
In function Curl_closesocket() in connect.c the call to
Curl_multi_closed() was wrongly omitted if a socket close function
(CURLOPT_CLOSESOCKETFUNCTION) is registered.
That would lead to not removing the socket from the internal hash table
and not calling the multi socket callback appropriately.
Bug: http://curl.haxx.se/bug/view.cgi?id=1493
For consistency, as we seem to have a bit of a mixed bag, changed all
instances of ipv4 and ipv6 in comments and documentations to use the
correct case.
There was a confusion between these: this commit tries to disambiguate them.
- Scope can be computed from the address itself.
- Scope id is scope dependent: it is currently defined as 1-based local
interface index for link-local scoped addresses, and as a site index(?) for
(obsolete) site-local addresses. Linux only supports it for link-local
addresses.
The URL parser properly parses a scope id as an interface index, but stores it
in a field named "scope": confusion. The field has been renamed into "scope_id".
Curl_if2ip() used the scope id as it was a scope. This caused failures
to bind to an interface.
Scope is now computed from the addresses and Curl_if2ip() matches them.
If redundantly specified in the URL, scope id is check for mismatch with
the interface index.
This commit should fix SF bug #1451.
This patch prepares for adding UNIX domain sockets support.
TCP_NODELAY and TCP_KEEPALIVE are specific to TCP/IP sockets, so do not
apply these to other socket types. bindlocal only works for IP sockets
(independent of TCP/UDP), so filter that out too for other types.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
When the connection code decides to close a socket it informs the multi
system via the Curl_multi_closed function. The multi system may, in
turn, invoke the CURLMOPT_SOCKETFUNCTION function with
CURL_POLL_REMOVE. This happens after the socket has already been
closed. Reorder the code so that CURL_POLL_REMOVE is called before the
socket is closed.
Debug output 'typo' fix.
Don't print an extra "0x" in
* Pipe broke: handle 0x0x2546d88, url = /
Add debug output.
Print the number of connections in the connection cache when
adding one, and not only when one is removed.
Fix typos in comments.
... for the local variable name in functions holding the return
code. Using the same name universally makes code easier to read and
follow.
Also, unify code for checking for CURLcode errors with:
if(result) or if(!result)
instead of
if(result == CURLE_OK), if(CURLE_OK == result) or if(result != CURLE_OK)
Introducing Curl_expire_latest(). To be used when we the code flow only
wants to get called at a later time that is "no later than X" so that
something can be checked (and another timeout be added).
The low-speed logic for example could easily be made to set very many
expire timeouts if it would be called faster or sooner than what it had
set its own timer and this goes for a few other timers too that aren't
explictiy checked for timer expiration in the code.
If there's no condition the code that says if(time-passed >= TIME), then
Curl_expire_latest() is preferred to Curl_expire().
If there exists such a condition, it is on the other hand important that
Curl_expire() is used and not the other.
Bug: http://curl.haxx.se/mail/lib-2014-06/0235.html
Reported-by: Florian Weimer
A conditionally compiled block in connect.c references WinSock 2
symbols, but used `#ifdef HAVE_WINSOCK_H` instead of `#ifdef
HAVE_WINSOCK2_H`.
Bug: http://curl.haxx.se/mail/lib-2014-08/0155.html
Make all code use connclose() and connkeep() when changing the "close
state" for a connection. These two macros take a string argument with an
explanation, and debug builds of curl will include that in the debug
output. Helps tracking connection re-use/close issues.
In commit 0b3750b5c2 (released in 7.36.0) we fixed a timeout issue
but instead broke the timings.
To fix this, I introduce a new timestamp to use for the timeouts and
restored the previous timestamp and timestamp position so that the old
timer functionality is restored.
In addition to that, that change also broke connection timeouts for when
more than one connect was used (as it would then count the total time
from the first connect and not for the most recent one). Now
Curl_timeleft() has been modified so that it checks against different
start times depending on which timeout it checks.
Test 1303 is updated accordingly.
Bug: http://curl.haxx.se/mail/lib-2014-05/0147.html
Reported-by: Ryan Braud
Starting with Visual Studio 2013 (VC12) and Windows 8.1 the
GetVersionInfoEx() function has been marked as deprecated and it's
return value atered. Updated connect.c and curl_sspi.c to use
VerifyVersionInfo() where possible, which has been available since
Windows 2000.
With the recently added timeout "reminder" functionality, there's no
reason left for us to execute timeout code before the time is
ripe. Simplifies the handling too.
This will make the *TIMEOUT and *CONNECTTIMEOUT options more accurate
again, which probably is most important when the *_MS versions are used.
In multi_socket, make sure to update 'now' after having handled activity
on a socket.
Fixes a bug when all addresses in the first family fail immediately, due
to "Network unreachable" for example, curl would hang and never try the
next address family.
Iterate through all address families when to trying establish the first
connection attempt.
Bug: http://curl.haxx.se/bug/view.cgi?id=1315
Reported-by: Michal Górny and Anthony G. Basile
This fixes a rare Happy Eyeballs bug where if the first IP family runs
out of addresses before the second-family-timer fires, and the second
IP family's first connect fails immediately, no further IPs of the
second family are attempted.
singleipconnect() could return the file descriptor of an open socket
even though the function returned a CURLE_COULDNT_CONNECT error code
from commit ed1662c374 and 02fbc26d59.
This could cause tests 19, 704 and 1233 to fail on FreeBSD, AIX and
Solaris.
singleipconnect() did not return the open socket descriptor on some
errors, thereby sometimes causing a socket leak. This patch ensures
the socket is always returned.
This patch adds a 200ms delay between the first and second address
family socket connection attempts.
It also iterates over IP addresses in the order returned by the
system, meaning most dual-stack systems will try IPv6 first.
Additionally, it refactors the connect code, removing most code that
handled synchronous connects. Since all sockets are now non-blocking,
the logic can be made simpler.
Introduced in commit 7d7df83198 curl would loop displaying "Whut?"
if it was trying to connect to an address and port that didn't have
anything listening on it.
This patch fixes a bug in Happy Eyeballs where curl would wait for a
connect response from socket1 before checking socket2.
Also, it updates error messages for failed connections, showing the ip
addresses that failed rather than just the host name repeatedly.
Bug: http://curl.haxx.se/mail/lib-2013-10/0236.html
Reported-by: Paul Marks
This patch invokes two socket connect()s nearly simultaneously, and
the socket that is first connected "wins" and is subsequently used for
the connection. The other is terminated.
There is a very slight IPv4 preference, in that if both sockets connect
simultaneously IPv4 is checked first and thus will win.
This is a regression since the switch to always-multi internally
c43127414d.
Test 1316 was modified since we now clearly call the Curl_client_write()
function when doing the LIST transfer part and then the
handler->protocol says FTP and ftpc.transfertype is 'A' which implies
text converting even though that the response is initially a HTTP
CONNECT response in this case.
As a remedy to the problem when a socket gets closed and a new one is
opened with the same file descriptor number and as a result
multi.c:singlesocket() doesn't detect the difference, the new function
Curl_multi_closed() gets told when a socket is closed so that it can be
removed from the socket hash. When the old one has been removed, a new
socket should be detected fine by the singlesocket() on next invoke.
Bug: http://curl.haxx.se/bug/view.cgi?id=1248
Reported-by: Erik Johansson
The code within #ifdef HAVE_SOCKADDR_IN6_SIN6_SCOPE_ID wrongly had two
closing braces when it should only have one, so builds without that
define would fail.
Bug: http://curl.haxx.se/mail/lib-2013-05/0000.html
I am using curl_easy_setopt(CURLOPT_INTERFACE, "if!something") to force
transfers to use a particular interface but the transfer fails with
CURLE_INTERFACE_FAILED, "Failed binding local connection end" if the
interface I specify has no IPv6 address. The cause is as follows:
The remote hostname resolves successfully and has an IPv6 address and an
IPv4 address.
cURL attempts to connect to the IPv6 address first.
bindlocal (in lib/connect.c) fails because Curl_if2ip cannot find an
IPv6 address on the interface.
This is a fatal error in singleipconnect()
This change will make cURL try the next IP address in the list.
Also included are two changes related to IPv6 address scope:
- Filter the choice of address in Curl_if2ip to only consider addresses
with the same scope ID as the connection address (mismatched scope for
local and remote address does not result in a working connection).
- bindlocal was ignoring the scope ID of addresses returned by
Curl_if2ip . Now it uses them.
Bug: http://curl.haxx.se/bug/view.cgi?id=1189
The Microsoft knowledge-base article
http://support.microsoft.com/kb/823764 describes how to use SNDBUF to
overcome a performance shortcoming in winsock, but it doesn't apply to
Windows Vista and later versions. If the described SNDBUF magic is
applied when running on those more recent Windows versions, it seems to
instead have the reversed effect in many cases and thus make libcurl
perform less good on those systems.
This fix thus adds a run-time version-check that does the SNDBUF magic
conditionally depending if it is deemed necessary or not.
Bug: http://curl.haxx.se/bug/view.cgi?id=1188
Reported by: Andrew Kurushin
Tested by: Christian Hägele
Remove timeout argument that's never used.
Make the actual connection get detected on a single spot to reduce code
duplication.
Store the IPv6 state already when the connection is attempted.
Remove internal separated behavior of the easy vs multi intercace.
curl_easy_perform() is now using the multi interface itself.
Several minor multi interface quirks and bugs have been fixed in the
process.
Much help with debugging this has been provided by: Yang Tse
This commit renames lib/setup.h to lib/curl_setup.h and
renames lib/setup_once.h to lib/curl_setup_once.h.
Removes the need and usage of a header inclusion guard foreign
to libcurl. [1]
Removes the need and presence of an alarming notice we carried
in old setup_once.h [2]
----------------------------------------
1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard
up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H,
this single inclusion guard is enough to ensure that inclusion of
lib/setup_once.h done from lib/setup.h is only done once.
Additionally lib/setup.h has always used __SETUP_ONCE_H macro to
protect inclusion of setup_once.h even after commit ec691ca3, this
was to avoid a circular header inclusion triggered when building a
c-ares enabled version with c-ares sources available which also has
a setup_once.h header. Commit ec691ca3 exposes the real nature of
__SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard
foreign to libcurl belonging to c-ares's setup_once.h
The renaming this commit does, fixes the circular header inclusion,
and as such removes the need and usage of a header inclusion guard
foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl.
2 - Due to the circular interdependency of old lib/setup_once.h and the
c-ares setup_once.h header, old file lib/setup_once.h has carried
back from 2006 up to now days an alarming and prominent notice about
the need of keeping libcurl's and c-ares's setup_once.h in sync.
Given that this commit fixes the circular interdependency, the need
and presence of mentioned notice is removed.
All mentioned interdependencies come back from now old days when
the c-ares project lived inside a curl subdirectory. This commit
removes last traces of such fact.
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:
f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files
This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:
c087374... curl_setup.h: remove redundant include guard
This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:
13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1
Start of related discussion thread:
http://curl.haxx.se/mail/lib-2013-01/0012.html
Asking for confirmation on pushing this revertion commit:
http://curl.haxx.se/mail/lib-2013-01/0048.html
Confirmation summary:
http://curl.haxx.se/mail/lib-2013-01/0079.html
NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.
lib/curl_imap.h
lib/curl_smtp.h
A bundle is a list of all persistent connections to the same host.
The connection cache consists of a hash of bundles, with the
hostname as the key.
The benefits may not be obvious, but they are two:
1) Faster search for connections to reuse, since the hash
lookup only finds connections to the host in question.
2) It lays out the groundworks for an upcoming patch,
which will introduce multiple HTTP pipelines.
This patch also removes the awkward list of "closure handles",
which were needed to send QUIT commands to the FTP server
when closing a connection.
Now we allocate a separate closure handle and use that
one to close all connections.
This has been tested in a live system for a few weeks, and of
course passes the test suite.
Fixed warning: dereferencing pointer does break strict-aliasing rules
by using a union instead of separate pointer variables.
Internal union sockaddr_u could probably be moved to generic header.
Thanks to Paul Howarth for the hint about using unions for this.
Important for winbuild: Separate declaration of sockaddr_u pointer.
The pointer variable *sock cannot be declared and initialized right
after the union declaration. Therefore it has to be a separate statement.
Commit 9109cdec11 brought this regression (shipped since 7.24.0).
The singleipconnect() function must not return an error if Curl_socket()
returns an error. It should then simply return OK and pass a SOCKET_BAD
back simply because that is how the user of this function expects it to
work and something else is not fine.
Reported by: Blaise Potard
Bug: http://curl.haxx.se/bug/view.cgi?id=3516508
Curl_socket returns CURLE_COULDNT_CONNECT when the opensocket callback
returns CURL_SOCKET_BAD. Previous return value CURLE_FAILED_INIT
conveys incorrect information to the user.
This adds three new options to control the behavior of TCP keepalives:
- CURLOPT_TCP_KEEPALIVE: enable/disable probes
- CURLOPT_TCP_KEEPIDLE: idle time before sending first probe
- CURLOPT_TCP_KEEPINTVL: delay between successive probes
While not all operating systems support the TCP_KEEPIDLE and
TCP_KEEPINTVL knobs, the library will still allow these options to be
set by clients, silently ignoring the values.
When connecting to a domain with multiple IP addresses, allow different,
decreasing connection timeout values. This should guarantee some
connections attempts with sufficiently long timeouts, while still
providing fallback.
First off the timeout for accepting a server connect back must of course
respect a global timeout. Then the timeleft function is only used by ftp
code so it was moved to ftp.c and made static.
1- Two new error codes are introduced.
CURLE_FTP_ACCEPT_FAILED to be set whenever ACCEPTing fails because of
FTP server connected.
CURLE_FTP_ACCEPT_TIMEOUT to be set whenever ACCEPTing timeouts.
Neither of these errors are considered fatal and control connection
remains OK because it could just be a firewall blocking server to
connect to the client.
2- One new setopt option was introduced.
CURLOPT_ACCEPTTIMEOUT_MS
It sets the maximum amount of time FTP client is going to wait for a
server to connect. Internal default accept timeout is 60 seconds.
Do not try to resolve interfaces names via DNS by recognizing interface
names in a few ways. If the interface option argument has a prefix of
"if!" then treat the argument as only an interface. Similarly, if the
interface argument is the name of an interface (even if it does not have
an IP address assigned), treat it as an interface name. Finally, if the
interface argument is prefixed by "host!" treat it as a hostname that
must be resolved by /etc/hosts or DNS.
These changes allow a client using the multi interfaces to avoid
blocking on name resolution if the interface loses its IP address or
disappears.
Keep track of which sockets that are the result of accept() calls and
refuse to call the closesocket callback for those sockets. Test case 596
now verifies that the open socket callback is called the same number of
times as the closed socket callback for active FTP connections.
Bug: http://curl.haxx.se/mail/lib-2011-12/0018.html
Reported by: Gokhan Sengun
