Make all code use connclose() and connkeep() when changing the "close
state" for a connection. These two macros take a string argument with an
explanation, and debug builds of curl will include that in the debug
output. Helps tracking connection re-use/close issues.
Depending on compiler line 3505 could generate the following warning or
error:
* warning: ISO C90 forbids mixed declarations and code
* A declaration cannot appear after an executable statement in a block
* error C2275: 'size_t' : illegal use of this type as an expression
set.infilesize in this case was modified in several places, which could
lead to repeated requests using the same handle to get unintendent/wrong
consequences based on what the previous request did!
This makes the findprotocol() function work as intended so that libcurl
can properly be restricted to not support HTTP while still supporting
HTTPS - since the HTTPS handler previously set both the HTTP and HTTPS
bits in the protocol field.
This fixes --proto and --proto-redir for most SSL protocols.
This is done by adding a few new convenience defines that groups HTTP
and HTTPS, FTP and FTPS etc that should then be used when the code wants
to check for both protocols at once. PROTO_FAMILY_[protocol] style.
Bug: https://github.com/bagder/curl/pull/97
Reported-by: drizzt
In addition to FTP, other connection based protocols such as IMAP, POP3,
SMTP, SCP, SFTP and LDAP require a new connection when different log-in
credentials are specified. Fixed the detection logic to include these
other protocols.
Bug: http://curl.haxx.se/docs/adv_20140326A.html
Port number zero is perfectly allowed to connect to. I moved to storing
the remote port number in an int so that -1 means undefined and 0-65535
can be used for legitimate port numbers.
When allowing NTLM, the re-use connection logic was too focused on
finding an existing NTLM connection to use and didn't properly allow
re-use of other ones. This made the logic not re-use perfectly re-usable
connections.
Added test case 1418 and 1419 to verify.
Regression brought in 8ae35102c (curl 7.35.0)
Reported-by: Jeff King
Bug: http://thread.gmane.org/gmane.comp.version-control.git/242213
Make sure that the special NTLM magic we do is for HTTP+NTLM only since
that's where the authenticated connection is a weird non-standard
paradigm.
Regression brought in 8ae35102c (curl 7.35.0)
Bug: http://curl.haxx.se/mail/lib-2014-02/0100.html
Reported-by: Dan Fandrich
when using --http2 one can now selectively disable NPN or ALPN with
--no-alpn and --no-npn. for now honored with NSS only.
TODO: honor this option with GnuTLS and OpenSSL
Removed some of the infof() calls that were added with the recent
pipeline improvements but they're not useful to the vast majority of
readers and the pipelining seems to fundamentaly work - the debugging
outputs can easily be added there if debugging these functions is needed
again.
When the requested authentication bitmask includes NTLM, we cannot
re-use a connection for another username/password as we then risk
re-using NTLM (connection-based auth).
This has the unfortunate downside that if you include NTLM as a possible
auth, you cannot re-use connections for other usernames/passwords even
if NTLM doesn't end up the auth type used.
Reported-by: Paras S
Patched-by: Paras S
Bug: http://curl.haxx.se/mail/lib-2014-01/0046.html
Following commit 0aafd77fa4, replaced the internal usage of
FORMAT_OFF_T and FORMAT_OFF_TU with the external versions that we
expect API programmers to use.
This negates the need for separate definitions which were subtly
different under different platforms/compilers.
To avoid the regression when users pass in passwords containing semi-
colons, we now drop the ability to set the login options with the same
options. Support for login options in CURLOPT_USERPWD was added in
7.31.0.
Test case 83 was modified to verify that colons and semi-colons can be
used as part of the password when using -u (CURLOPT_USERPWD).
Bug: http://curl.haxx.se/bug/view.cgi?id=1311
Reported-by: Petr Bahula
Assisted-by: Steve Holme
Signed-off-by: Daniel Stenberg <daniel@haxx.se>
Rather than set the authentication options as part of the login details
specified in the URL, or via the older CURLOPT_USERPWD option, added a
new libcurl option to allow the login options to be set separately.
This patch adds a 200ms delay between the first and second address
family socket connection attempts.
It also iterates over IP addresses in the order returned by the
system, meaning most dual-stack systems will try IPv6 first.
Additionally, it refactors the connect code, removing most code that
handled synchronous connects. Since all sockets are now non-blocking,
the logic can be made simpler.
This patch invokes two socket connect()s nearly simultaneously, and
the socket that is first connected "wins" and is subsequently used for
the connection. The other is terminated.
There is a very slight IPv4 preference, in that if both sockets connect
simultaneously IPv4 is checked first and thus will win.
This is a regression since the switch to always-multi internally
c43127414d.
Test 1316 was modified since we now clearly call the Curl_client_write()
function when doing the LIST transfer part and then the
handler->protocol says FTP and ftpc.transfertype is 'A' which implies
text converting even though that the response is initially a HTTP
CONNECT response in this case.
Otherwise, the FTP protocol would unnecessarily hang 60 seconds if
aborted in the CURLOPT_HEADERFUNCTION callback.
Reported by: Tomas Mlcoch
Bug: https://bugzilla.redhat.com/1005686
Added the ability to specify an XOAUTH2 bearer token [RFC6750] via the
option CURLOPT_XOAUTH2_BEARER for authentication using RFC6749 "OAuth
2.0 Authorization Framework".
We've announced this pending removal for a long time and we've
repeatedly asked if anyone would care or if anyone objects. Nobody has
objected. It has probably not even been working for a good while since
nobody has tested/used this code recently.
The stuff in krb4.h that was generic enough to be used by other sources
is now present in security.h
libcurl quietly truncates usernames, passwords, and options from
before an '@' sign in a URL to 255 (= MAX_CURL_PASSWORD_LENGTH - 1)
characters to fit in fixed-size buffers on the stack. Allocate a
buffer large enough to fit the parsed fields on the fly instead to
support longer passwords.
After this change, there are no more uses of MAX_CURL_OPTIONS_LENGTH
left, so stop defining that constant while at it. The hardcoded max
username and password length constants, on the other hand, are still
used in HTTP proxy credential handling (which this patch doesn't
touch).
Reported-by: Colby Ranger
Instead of nesting "if(success)" blocks and leaving the reader in
suspense about what happens in the !success case, deal with failure
cases early, usually with a simple goto to clean up and return from
the function.
No functional change intended. The main effect is to decrease the
indentation of this function slightly.
libcurl truncates usernames, passwords, and options set with
curl_easy_setopt to 255 (= MAX_CURL_PASSWORD_LENGTH - 1) characters.
This doesn't affect the return value from curl_easy_setopt(), so from
the caller's point of view, there is no sign anything strange has
happened, except that authentication fails.
For example:
# Prepare a long (300-char) password.
s=0123456789; s=$s$s$s$s$s$s$s$s$s$s; s=$s$s$s;
# Start a server.
nc -l -p 8888 | tee out & pid=$!
# Tell curl to pass the password to the server.
curl --user me:$s http://localhost:8888 & sleep 1; kill $pid
# Extract the password.
userpass=$(
awk '/Authorization: Basic/ {print $3}' <out |
tr -d '\r' |
base64 -d
)
password=${userpass#me:}
echo ${#password}
Expected result: 300
Actual result: 255
The fix is simple: allocate appropriately sized buffers on the heap
instead of trying to squeeze the provided values into fixed-size
on-stack buffers.
Bug: http://bugs.debian.org/719856
Reported-by: Colby Ranger
libcurl truncates usernames and passwords it reads from .netrc to
LOGINSIZE and PASSWORDSIZE (64) characters without any indication to
the user, to ensure the values returned from Curl_parsenetrc fit in a
caller-provided buffer.
Fix the interface by passing back dynamically allocated buffers
allocated to fit the user's input. The parser still relies on a
256-character buffer to read each line, though.
So now you can include an ~246-character password in your .netrc,
instead of the previous limit of 63 characters.
Reported-by: Colby Ranger
Instead of remembering before each "return" statement which temporary
allocations, if any, need to be freed, take care to set pointers to
NULL when no longer needed and use a goto to a common block to exit
the function and free all temporaries.
No functional change intended. Currently the only temporary buffer in
this function is "proxy" which is already correctly freed when
appropriate, but there will be more soon.
Moved Curl_easy_addmulti() from easy.c to multi.c, renamed it to
easy_addmulti and made it static.
Removed Curl_easy_initHandleData() and uses of it since it was emptied
in commit cdda92ab67b47d74a.
All protocol handler structs are now opaque (void *) in the
SessionHandle struct and moved in the request-specific sub-struct
'SingleRequest'. The intension is to keep the protocol specific
knowledge in their own dedicated source files [protocol].c etc.
There's some "leakage" where this policy is violated, to be addressed at
a later point in time.
1 - always allocate the struct in protocol->setup_connection. Some
protocol handlers had to get this function added.
2 - always free at the end of a request. This is also an attempt to keep
less memory in the handle after it is completed.
This is a regression as this logic used to work. It isn't clear when it
broke, but I'm assuming in 7.28.0 when we went all-multi internally.
This likely never worked with the multi interface. As the failed
connection is detected once the multi state has reached DO_MORE, the
Curl_do_more() function was now expanded somewhat so that the
ftp_do_more() function can request to go "back" to the previous state
when it makes another attempt - using PASV.
Added test case 1233 to verify this fix. It has the little issue that it
assumes no service is listening/accepting connections on port 1...
Reported-by: byte_bucket in the #curl IRC channel
CURLOPT_XFERINFOFUNCTION is now the preferred progress callback function
and CURLOPT_PROGRESSFUNCTION is considered deprecated.
This new callback uses pure 'curl_off_t' arguments to pass on full
resolution sizes. It otherwise retains the same characteristics: the
same call rate, the same meanings for the arguments and the return code
is used the same way.
The progressfunc.c example is updated to show how to use the new
callback for newer libcurls while supporting the older one if built with
an older libcurl or even built with a newer libcurl while running with
an older.
RFC3986 details how a path part passed in as part of a URI should be
"cleaned" from dot sequences before getting used. The described
algorithm is now implemented in lib/dotdot.c with the accompanied test
case in test 1395.
Bug: http://curl.haxx.se/bug/view.cgi?id=1200
Reported-by: Alex Vinnik
When performing COOKIELIST operations the cookie lock needs to be taken
for the cases where the cookies are shared among multiple handles!
Verified by Benjamin Gilbert's updated test 506
Bug: http://curl.haxx.se/bug/view.cgi?id=1215
Reported-by: Benjamin Gilbert
... in order to prevent an artificial timeout event based on stale
speed-check data from a previous network transfer. This commit fixes
a regression caused by 9dd85bced5.
Bug: https://bugzilla.redhat.com/906031
Fixed an issue in parse_proxy(), introduced in commit 11332577b3,
where an empty username or password (For example: http://:@example.com)
would cause a crash.
There is no need to perform separate clearing of data if a NULL option
pointer is passed in. Instead this operation can be performed by simply
not calling parse_login_details() and letting the rest of the code do
the work.
setstropt_userpwd() was calling setstropt() in commit fddb7b44a7 to
set each of the login details which would duplicate the strings and
subsequently cause a memory leak.
In addition to parsing the optional login options from the URL, added
support for parsing them from CURLOPT_USERPWD, to allow the following
supported command line:
--user username:password;options
Added bounds checking when searching for the separator characters within
the login string as this string may not be NULL terminated (For example
it is the login part of a URL). We do this in preference to allocating a
new string to copy the login details into which could then be passed to
parse_login_details() for performance reasons.
As well as parsing the username and password from the URL, added support
for parsing the optional options part from the login details, to allow
the following supported URL format:
schema://username:password;options@example.com/path?q=foobar
This will only be used by IMAP, POP3 and SMTP at present but any
protocol that may be given login options in the URL will be able to
add support for them.
Previously it only compared credentials if the requested needle
connection wasn't using a proxy. This caused NTLM authentication
failures when using proxies as the authentication code wasn't send on
the connection where the challenge arrived.
Added test 1215 to verify: NTLM server authentication through a proxy
(This is a modified copy of test 67)
At some point recently we lost the default value for the easy handle's
connection cache, and this change puts it back to 5 - which is the
former default value and it is documented in the curl_easy_setopt.3 man
page.
curl has been accepting URLs using slightly wrong syntax for a long
time, such as when completely missing as slash "http://example.org" or
missing a slash when a query part is given
"http://example.org?q=foobar".
curl would translate these into a legitimate HTTP request to servers,
although as was shown in bug #1206 it was not adjusted properly in the
cases where a HTTP proxy was used.
Test 1213 and 1214 were added to the test suite to verify this fix.
The test HTTP server was adjusted to allow us to specify test number in
the host name only without using any slashes in a given URL.
Bug: http://curl.haxx.se/bug/view.cgi?id=1206
Reported by: ScottJi
Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.
Two new options for limiting the number of connections:
CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.
CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.
Several new options for pipelining:
CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).
CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.
CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.
CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.
CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.
See the curl_multi_setopt() man page for details.
Always interprets the pointer passed with the CURLOPT_WRITEDATA or
CURLOPT_READDATA options of curl_easy_setopt() as a void pointer in
order to avoid problems in environments where FILE and void pointers
have non-trivial conversion.
This bug report properly identified that when doing SMTP and aborting
the transfer with a callback, it must be considered aborted prematurely
by the code to avoid QUIT etc to be attempted as that would cause a
hang.
The new test case 1507 verifies this behavior.
Reported by: Patricia Muscalu
Bug: http://curl.haxx.se/bug/view.cgi?id=1184
When a connection is no longer used, it is kept in the cache. If the
cache is full, the oldest idle connection is closed. If no connection is
idle, the current one is closed instead.
Remove internal separated behavior of the easy vs multi intercace.
curl_easy_perform() is now using the multi interface itself.
Several minor multi interface quirks and bugs have been fixed in the
process.
Much help with debugging this has been provided by: Yang Tse
