Eric Mertens posted bug report #3003005 pointing out that the
libcurl TFTP code was not sending the timeout option properly to
the server, and suggested a fix.
(http://curl.haxx.se/bug/view.cgi?id=3003005)
John-Mark Bell filed bug #3000052 that identified a problem (with
an associated patch) with the OpenSSL handshake state machine
when the multi interface is used:
Performing an https request using a curl multi handle and using
select or epoll to wait for events results in a hang. It appears
that the cause is the fix for bug #2958179, which makes
ossl_connect_common unconditionally return from the step 2 loop
when fetching from a multi handle.
When ossl_connect_step2 has completed, it updates
connssl->connecting_state to ssl_connect_3. ossl_connect_common
will then return to the caller, as a multi handle is in
use. Eventually, the client code will call curl_multi_fdset to
obtain an updated fdset to select or epoll on. For https
requests, curl_multi_fdset will cause https_getsock to be called.
https_getsock will only return a socket handle if the
connecting_state is ssl_connect_2_reading or
ssl_connect_2_writing. Therefore, the client will never obtain a
valid fdset, and thus not drive the multi handle, resulting in a
hang.
(http://curl.haxx.se/bug/view.cgi?id=3000052)
Sebastian V reported bug #3000056 identifying a problem with
redirect following. It showed that when curl followed redirects
it didn't properly ignore the response body of the 30X response
if that response was using compressed Content-Encoding!
(http://curl.haxx.se/bug/view.cgi?id=3000056)
"The BSD version of PolarSSL was made for migratory purposes only and is not
maintained. The GPL version of PolarSSL is actually the only actively
developed version, so I would be very reluctant to use the BSD version." /
Paul Bakker, PolarSSL hacker.
Signed-off-by: Hoi-Ho Chan <hoiho.chan@gmail.com>
FTP(S) use two connections that can be set to different recv and
send functions independently, so by introducing recv+send pairs
in the same manner we already have sockets/connections we can
work with FTPS fine.
This commit fixes the FTPS regression introduced in change d64bd82.
Dirk Manske reported a regression. When connecting with the multi
interface, there were situations where libcurl wouldn't store
connect time correctly as it used to (and is documented to) do.
Using his fine sample program we could repeat it, and I wrote up
test case 573 using that code. The problem does not easily show
itself using the local test suite though.
The fix, also as suggested by Dirk, is a bit on the ugly side as
it adds yet another call to Curl_verboseconnect() and setting the
TIMER_CONNECT time. That situation is subject for some closer
inspection in the future.
Howard Chu brought the bulk work of this patch that properly
moves out the sending and recving of data to the parts of the
code that are properly responsible for the various ways of doing
so.
Daniel Stenberg assisted with polishing a few bits and fixed some
minor flaws in the original patch.
Another upside of this patch is that we now abuse CURLcodes less
with the "magic" -1 return codes and instead use CURLE_AGAIN more
consistently.
This is Hoi-Ho Chan's patch with some minor fixes by me. There
are some potential issues in this, but none worse than we can
sort out on the list and over time.
The main change is to allow input from user-specified methods,
when they are specified with CURLOPT_READFUNCTION.
All calls to fflush(stdout) in telnet.c were removed, which makes
using 'curl telnet://foo.com' painful since prompts and other data
are not always returned to the user promptly. Use
'curl --no-buffer telnet://foo.com' instead. In general,
the user should have their CURLOPT_WRITEFUNCTION do a fflush
for interactive use.
Also fix assumption that reading from stdin never returns < 0.
Old code could crash in that case.
Call progress functions in telnet main loop.
Signed-off-by: Ben Greear <greearb@candelatech.com>
Make sure we don't call memcpy() if the argument is NULL even
though we also passed a zero length then, as the clang analyzer
whined and we want to limit warnings (even false positives) when
they're this easy to fix.
The change of (char) to (unsigned char) will fix long user names
and passwords on systems that have the char type signed by
default.
The recent overhaul of the SSL recv function made this treat a
zero returned from gnutls_record_recv() as an error, and this
caused our HTTPS test cases to fail. We leave it to upper layer
code to detect if an EOF is a problem or not.
This code would previously use dns_entry->addr->ai_canonname
instead of the given host name, which caused us grief and
problems since not all our resolver options do the reverse lookup
and I would also guess that it caused problems with KRB5/GSS with
virtual name-based hosts. Now the host name from the URL is used.
As reported in bug report #2987196, the code for ipv6 already did
the setting of this bit correctly so we copied that logic into
the Curl_ipv4_resolve_r() function as well. KRB code is the only
code we know that might need the cannonical name so only resolve
it for such requests!
Prefixing the FTP quote commands with an asterisk really only
worked for the postquote actions. This is now fixed and test case
227 has been extended to verify.
Matt Wixson found and fixed a bug in the SCP/SFTP area where the
code treated a 0 return code from libssh2 to be the same as
EAGAIN while in reality it isn't. The problem caused a hang in
SFTP transfers from a MessageWay server.
strlen() returns size_t, but ssh libraries are wanting 'unsigned int'. Add
explicit casts and use _ex versions of the ssh library calls.
Signed-off-by: Ben Greear <greearb@candelatech.com>
If you pass a URL to pop3 that does not contain a message ID as
part of the URL, it will currently ask for 'INBOX' which just
causes the pop3 server to return an error.
The change makes libcurl treat en empty message ID as a request
for LIST (list of pop3 message IDs). User's code could then
parse this and download individual messages as desired.
Ben Greear brought a patch that from now on allows all protocols
to specify name and user within the URL, in the same manner HTTP
and FTP have been allowed to in the past - although far from all
of the libcurl supported protocols actually have that feature in
their URL definition spec.
Bob Richmond: There's an annoying situation where libcurl will
read new HTTP response data from a socket, then check if it's a
timeout if one is set. If the last packet received constitutes
the end of the response body, libcurl still treats it as a
timeout condition and reports a message like:
"Operation timed out after 3000 milliseconds with 876 out of 876
bytes received"
It should only a timeout if the timer lapsed and we DIDN'T
receive the end of the response body yet.
This commit fixes the cmake build of curl, and cleans up the
cmake code a little. It removes some commented out code and
some trailing whitespace. To get curl to build the binary
tree include/curl directory needed to be added to the include
path. Also, SIZEOF_SHORT needed to be added. A check for the
lack of defines of SIZEOF_* for warnless.c was added.
Kenny To filed the bug report #2963679 with patch to fix a
problem he experienced with doing multi interface HTTP POST over
a proxy using PROXYTUNNEL. He found a case where it would connect
fine but bits.tcpconnect was not set correct so libcurl didn't
work properly.
(http://curl.haxx.se/bug/view.cgi?id=2963679)
Akos Pasztory filed debian bug report #572276http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=572276
mentioning a problem with a resource that returns chunked-encoded
_and_ with a Content-Length and libcurl failed to properly ignore
the latter information.
Hauke Duden provided an example program that made the multi
interface crash. His example simply used the multi interface and
did first one FTP transfer and after completion it used a second
easy handle and did another FTP transfer on the same FTP server.
This triggered a bug in the "delayed easy handle kill" system
that curl uses: when an FTP connection is left alive it must keep
an easy handle around internally - only for the purpose of having
an easy handle when it later disconnects it. The code assumed
that when the easy handle was removed and an internal reference
was made, that version could be killed later on when a new easy
handle came using the same connection. This was wrong as Hauke's
example showed that the removed handle wasn't killed for real
until later. This caused a double close attempt => segfault.
Looking at the code of Curl_resolv_timeout() in hostip.c, I think
that in case of a timeout, the signal handler for SIGALRM never
gets removed. I think that in my case it gets executed at some
point later on when execution has long left Curl_resolv_timeout()
or even the cURL library.
The code that is jumped to with siglongjmp() simply sets the
error message to "name lookup timed out" and then returns with
CURLRESOLV_ERROR. I guess that instead of simply returning
without cleaning up, the code should have a goto that jumps to
the spot right after the call to Curl_resolv().
Error codes were not properly returned to the main curl code (and on to apps
using libcurl).
tftp was crapping out when tsize == 0 on upload, but I see no reason to fail
to upload just because the remote file is zero-length. Ignore tsize option on
upload.
The problem mentioned on Dec 10 2009
(http://curl.haxx.se/bug/view.cgi?id=2905220) was only partially fixed.
Partially because an easy handle can be associated with many connections in
the cache (e.g. if there is a redirect during the lifetime of the easy
handle). The previous patch only cleaned up the first one. The new fix now
removes the easy handle from all connections, not just the first one.
makes sure that when using sub-second timeouts, there's no final bad 1000ms
wait. Previously, a sub-second timeout would often make the elapsed time end
up the time rounded up to the nearest second (e.g. 1s for 200ms timeout)
the global timeout if set. Also, as was reported in the bug report #2956437
by Ryan Chan, the time stamp to use as basis for the per command timeout was
not set properly in the DONE phase for FTP (and not for SMTP) so I fixed
that just now. This was a regression compared to 7.19.7 due to the
conversion of FTP code over to the generic pingpong concepts.
http://curl.haxx.se/bug/view.cgi?id=2956437
- SMTP falls back to RFC821 HELO when EHLO fails (and SSL is not required).
- Use of true local host name (i.e.: via gethostname()) when available, as default argument to SMTP HELO/EHLO.
- Test case 804 for HELO fallback.
properly in angle brackets. Recipients provided with CURLOPT_MAIL_RCPT now
get angle bracket wrapping automatically by libcurl unless the recipient
starts with an angle bracket as then the app is assumed to deal with that
properly on its own.
full DATA has been sent, and I modified the test SMTP server to also send
that response. As usual, the DONE operation that is made after a completed
transfer is still not doable in a non-blocking way so this waiting for 250
is unfortunately made blockingly.
in the same RCPT TO line, when they should be sent in separate single
commands. I updated test case 802 to verify this.
- I also fixed a bad use of my_setopt_str() of CURLOPT_MAIL_RCPT in the curl
tool which made it try to output it as string for the --libcurl feature
which could lead to crashes.
to automatically uncompress it with the CURLOPT_ENCODING option, libcurl
could wrongly provide the callback with more data than what the maximum
documented amount. An application could thus get tricked into badness if the
maximum limit was trusted to be enforced by libcurl itself (as it is
documented).
This is further detailed and explained in the libcurl security advisory
20100209 at
http://curl.haxx.se/docs/adv_20100209.html
simply check for CURLM_CALL_MULTI_PERFORM internally. This has the added
benefit that this goes in line with my long-term wishes to get rid of the
CURLM_CALL_MULTI_PERFORM all together from the public API.
from hostip.h to setup.h in order to allow proper inclusion in any file.
This represents no functional change at all in which resolver is used,
everything still works as usual, internally and externally there is no
difference in behavior.
HTTP Cookie: header _needs_ to be sorted on the path length in the cases
where two cookies using the same name are set more than once using
(overlapping) paths. Realizing this, identically named cookies must be
sorted correctly. But detecting only identically named cookies and take care
of them individually is harder than just to blindly and unconditionally sort
all cookies based on their path lengths. All major browsers also already do
this, so this makes our behavior one step closer to them in the cookie area.
Test case 8 was the only one that broke due to this change and I updated it
accordingly.
again when downloading files over FTP using ASCII and it turns out that the
final size of the file is not the same as the initial size the server
reported. This is very common since servers don't take the newline
conversions into account.
much as possible in one go, as long as it doesn't block and hasn't reached the
end of the state machine.
This avoids spurious -1 returns from curl_multi_fdset() simply because
previously it would return from this function without anything in EWOUDLBLOCK
and thus basically it wasn't actually waiting for anything!!
state, we return CURLM_CALL_MULTI_PERFORM unconditionally then so that we
can act faster like in the case the protocol-specific connect doesn't block
on anything and we can just persue on the next action immediately. It also
then avoids a case where curl_multi_fdset() would return -1.
ossl_connect_step3() increments an SSL session handle reference counter on
each call. When sessions are re-used this reference counter may be
incremented many times, but it will be decremented only once when done (by
Curl_ossl_session_free()); and the internal OpenSSL data will not be freed
if this reference count remains positive. When a session is re-used the
reference counter should be corrected by explicitly calling
SSL_SESSION_free() after each consecutive SSL_get1_session() to avoid
introducing a memory leak.
(http://curl.haxx.se/bug/view.cgi?id=2926284)
command is a special "hack" used by the drftpd server, but even though it is
a custom extension I've deemed it fine to add to libcurl since this server
seems to survive and people keep using it and want libcurl to support
it. The new libcurl option is named CURLOPT_FTP_USE_PRET, and it is also
usable from the curl tool with --ftp-pret. Using this option on a server
that doesn't support this command will make libcurl fail.
sequences in uploaded data. The test server doesn't "decode" escaped dot-lines
but instead test cases must be written to take them into account. Added test
case 803 to verify dot-escaping.
detects and uses proxies based on the environment variables. If the proxy
was given as an explicit option it worked, but due to the setup order
mistake proxies would not be used fine for a few protocols when picked up
from '[protocol]_proxy'. Obviously this broke after 7.19.4. I now also added
test case 1106 that verifies this functionality.
(http://curl.haxx.se/bug/view.cgi?id=2913886)
accessing alredy freed memory and thus crash when using HTTPS (with
OpenSSL), multi interface and the CURLOPT_DEBUGFUNCTION and a certain order
of cleaning things up. I fixed it.
(http://curl.haxx.se/bug/view.cgi?id=2891591)
curl_easy_setopt with CURLOPT_HTTPHEADER, the library should set
data->state.expect100header accordingly - the current code (in 7.19.7 at
least) doesn't handle this properly. Martin Storsjo provided the fix!
rework patch that now integrates TFTP properly into libcurl so that it can
be used non-blocking with the multi interface and more. BLKSIZE also works.
The --tftp-blksize option was added to allow setting the TFTP BLKSIZE from
the command line.
meter/callback during FTP command/response sequences. It turned out it was
really lame before and now the progress meter SHOULD get called at least
once per second.
CURLOPT_HTTPPROXYTUNNEL enabled over a proxy, a subsequent request using the
same proxy with the tunnel option disabled would still wrongly re-use that
previous connection and the outcome would only be badness.
end up with entries that wouldn't time-out:
1. Set up a first web server that redirects (307) to a http://server:port
that's down
2. Have curl connect to the first web server using curl multi
After the curl_easy_cleanup call, there will be curl dns entries hanging
around with in_use != 0.
(http://curl.haxx.se/bug/view.cgi?id=2891591)
the client certificate. It also disable the key name test as some engines
can select a private key/cert automatically (When there is only one key
and/or certificate on the hardware device used by the engine)
No need for a separate variable ndns.
The memory leak detection will detect code that fails to release a dns reference.
The DEBUGASSERT will detect code that releases too many references.
closed NSPR descriptor. The issue was hard to find, reported several times
before and always closed unresolved. More info at the RH bug:
https://bugzilla.redhat.com/534176
(http://curl.haxx.se/bug/view.cgi?id=2891595) which identified how an entry
in the DNS cache would linger too long if the request that added it was in
use that long. He also provided the patch that now makes libcurl capable of
still doing a request while the DNS hash entry may get timed out.
used during the FTP connection phase (after the actual TCP connect), while
it of course should be. I also made the speed check get called correctly so
that really slow servers will trigger that properly too.
wrong percentage for small files, most notable for <1000 bytes and could
easily end up showing more than 100% at the end. It also didn't show any
percentage, transfer size or estimated transfer times when transferring
less than 100 bytes.
auth is used, as it caused a crash. I failed to repeat the issue, but still
made a change that now forces the TCP connection used for a freed SCP
session to get closed and not be re-used.
POST using a read callback, with Digest authentication and
"Transfer-Encoding: chunked" enforced. I would then cause the first request
to be wrongly sent and then basically hang until the server closed the
connection. I fixed the problem and added test case 565 to verify it.
unparsable expiry dates and then treat them as session cookies - previously
libcurl would reject cookies with a date format it couldn't parse. Research
shows that the major browser treat such cookies as session cookies. I
modified test 8 and 31 to verify this.
fail to build when this happens, and show an appropriate error.
The brave of heart can circumvect this. Defining ALLOW_MSVC6_WITHOUT_PSDK
in lib/config-win32.h, although absolutely discouraged and unsupported,
this will allow the die hard MSVC hacker to build in such a discouraged
environment.
The actually supported 'fix' is to install 'February 2003 Platform SDK'
a.k.a. 'Windows Server 2003 PSDK' which can be freely downloaded from
http://www.microsoft.com/msdownload/platformsdk/sdkupdate/psdk-full.htm
(http://curl.haxx.se/bug/view.cgi?id=2873666) which identified a problem which
made libcurl loop infinitely when given incorrect credentials when using HTTP
GSS negotiate authentication.
(http://curl.haxx.se/bug/view.cgi?id=2870221) that libcurl returned an
incorrect return code from the internal trynextip() function which caused
him grief. This is a regression that was introduced in 7.19.1 and I find it
strange it hasn't hit us harder, but I won't persue into figuring out
exactly why.
SO_SNDBUF to CURL_WRITE_SIZE even if the SO_SNDBUF starts out larger. The
patch doesn't do a setsockopt if SO_SNDBUF is already greater than
CURL_WRITE_SIZE. This should help folks who have set up their computer with
large send buffers.
the define CURL_MAX_HTTP_HEADER which is even exposed in the public header
file to allow for users to fairly easy rebuild libcurl with a modified
limit. The rationale for a fixed limit is that libcurl is realloc()ing a
buffer to be able to put a full header into it, so that it can call the
header callback with the entire header, but that also risk getting it into
trouble if a server by mistake or willingly sends a header that is more or
less without an end. The limit is set to 100K.
saving received cookies with no given path, if the path in the request had a
query part. That is means a question mark (?) and characters on the right
side of that. I wrote test case 1105 and fixed this problem.
transfer.c for blocking. It is currently used only by SCP and SFTP protocols.
This enhancement resolves an issue with 100% CPU usage during SFTP upload,
reported by Vourhey.
(http://curl.haxx.se/bug/view.cgi?id=2861587) identifying that libcurl used
the OpenSSL function X509_load_crl_file() wrongly and failed if it would
load a CRL file with more than one certificate within. This is now fixed.
powered libcurl in 7.19.6. If there was a X509v3 Subject Alternative Name
field in the certficate it had to match and so even if non-DNS and non-IP
entry was present it caused the verification to fail.
POLLIN, and sets POLLERR without setting POLLIN and POLLOUT. In some
libcurl code execution paths this could trigger busy wait loops with
high CPU usage until a timeout condition aborted the loop.
This fix for Curl_poll adresses the above in a libcurl-wide mode.
Some systems poll function sets POLLHUP in revents without setting
POLLIN, and sets POLLERR without setting POLLIN and POLLOUT. In some
libcurl code execution paths this could trigger busy wait loops with
high CPU usage until a timeout condition aborted the loop.
The reverted patch addressed the above issue for a very specific case,
when awaiting c-ares to resolve. A libcurl-wide fix superceeds this one.
http://cool.haxx.se/cvs.cgi/curl/lib/select.c.diff?r1=1.52&r2=1.53
start second "Thu Jan 1 00:00:00 GMT 1970" as the date parser then returns 0
which internally then is treated as a session cookie. That particular date
is now made to get the value of 1.
libcurl to resolve 'localhost' whatever name you use in the URL *if* you set
the --interface option to (exactly) "LocalHost". This will enable us to
write tests for custom hosts names but still use a local host server.
