Running tests\libtest\libntlmconnect.exe reveals a 1 byte (!) leak in
./lib/curl_ntlm_msgs.c:
perl ..\memanalyze.pl c:memdebug.curl
Leak detected: memory still allocated: 1 bytes
At 9771e8, there's 1 bytes.
allocated by curl_ntlm_msgs.c:399
Snippet from curl_ntlm_msgs.c:
/* setup ntlm identity's domain and length */
dup_domain.tchar_ptr = malloc(sizeof(TCHAR) * (domlen + 1));
(my domlen == 0).
'dup_domain.tbyte_ptr' looks to be freed in Curl_ntlm_sspi_cleanup() via
'ntlm->identity.Domain'. But I see no freeing of 'dup_domain.tchar_ptr'.
This bug report properly identified that when doing SMTP and aborting
the transfer with a callback, it must be considered aborted prematurely
by the code to avoid QUIT etc to be attempted as that would cause a
hang.
The new test case 1507 verifies this behavior.
Reported by: Patricia Muscalu
Bug: http://curl.haxx.se/bug/view.cgi?id=1184
It turns out that Leopard (OS X 10.5) doesn't have constants for the ECDH
ciphers in its headers, so the cases for them have been taken out of the
build when building under Leopard. Also added a standard function for
getting a string description of a SecCertificateRef.
Changed the SMTP_AUTH_PASSWD state constant to SMTP_AUTH_LOGIN_PASSWD to
better describe the state as the second part of an AUTH LOGIN command,
as well as for consistency with the imap and pop3 modules.
Introduced detection of the SASL-IR capability, in order to add support
for sending the initial response with the AUTHENTICATE command, as per
RFC4959.
Updated the automatic response tag generation to follow the examples
given in RC3501, which list a 4 character string such as A001, A002,
etc.
As a unique identifier should be generated for each command the string
generation is based on the connection id and the incrementing command
id.
When negotiating SASL DIGEST-MD5 authentication, the function
Curl_sasl_create_digest_md5_message() uses the data provided from the
server without doing the proper length checks and that data is then
appended to a local fixed-size buffer on the stack.
This vulnerability can be exploited by someone who is in control of a
server that a libcurl based program is accessing with POP3, SMTP or
IMAP. For applications that accept user provided URLs, it is also
thinkable that a malicious user would feed an application with a URL to
a server hosting code targetting this flaw.
Bug: http://curl.haxx.se/docs/adv_20130206.html
config_h.com is a new file that generates a config.h file based on the
curl_config.h.in file and a quick scan of the configure script. This is
actually a generic procedure that is shared with other VMS packages.
The existing pre-built config-vms.h had over 100 entries that were not
correct and in some cases conflicted with the build options available in
the build_vms.com.
generate_config_vms_h_curl.com is a helper procedure to the
config_h.com. It covers the cases that the generic config_h.com is not
able to figure out, and accepts input from the build_vms.com procedure.
build_curlbuild_h.com is a new file to generate the curlbuild.h file
that Curl is now using when it is using a curl_config.h file.
post-config-vms.h is a new file that is needed to provide VMS specific
definitions, and most of them need to be set before the system header
files are included.
The VMS build procedure is fixed:
1. Fixed to link in the correct HP ssl library.
2. Fixed to detect if HP Kerberos is installed.
3. Fixed to detect if HP LDAP is installed.
4. Fixed to detect if gnv$libzshr is installed.
5. Simplified the input parameter parsing to not use a loop.
6. Warn that 64 bit pointer option support is not complete
in comments.
7. Default to IEEE floating if platform supports it so
resulting libcurl will be compatible with other
open source projects on VMS.
8. Default to LARGEFILE if platform supports it.
9. Default to enable SSL, LDAP, Kerberos, libz
if the libraries are present.
10. Build with exact case global symbols for libcurl.
11. Generate linker option file needed.
12. Compiler list option only commonly needed items.
13. fulllist option for those who really want it.
14. Create debug symbol file on Alpha, IA64.
By doing this unconditionally, we infer a simpler and more defined
behavior. This also has the upside that test 1021 no longer fails for me
even if I run with valgrind.
Also fixed some wrong comments.
According KB975858 this flag may cause problems on Windows 7 and
Windows Server 2008 R2 systems. Extended error information is not
currently used by libcurl and therefore not a requirement.
The flag may improve the SSL-connection shutdown in case of an
error. This means it might be a good improvement in the future.
Fixes bug/issue #1187 - thanks for the report
When a connection is no longer used, it is kept in the cache. If the
cache is full, the oldest idle connection is closed. If no connection is
idle, the current one is closed instead.
Tidied up code from commit 6b6bdc83bdUpdated where a few instances of
the pop3c struct variable used the longer conndata struct rather than
matching what other code in pop3_authenticate() used.
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
imap: Fixed no known authentication mechanism when fallback is required
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to Clear Text authentication.
Note: In order to fallback to Clear Text authentication properly this
fix adds support for the LOGINDISABLED server capability.
Related bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Fixed an issue where (lib)curl is compiled without support for a
supported challenge-response based SASL authentication mechanism, such
as CRAM-MD5 or NTLM, the server doesn't support the LOGIN or PLAIN
mechanisms and (lib)curl doesn't fallback to APOP or Clear Text
authentication.
Bug: http://curl.haxx.se/mail/lib-2013-02/0004.html
Reported by: Stanislav Ivochkin
Remove timeout argument that's never used.
Make the actual connection get detected on a single spot to reduce code
duplication.
Store the IPv6 state already when the connection is attempted.
There was a bug where, if SSLWrite() returned errSSLWouldBlock but did
succeed in transmitting at least something, then we'd incorrectly
resend the packet. Now we never take errSSLWouldBlock as a sign that
nothing was transferred to/from the server.
Bug: http://curl.haxx.se/mail/lib-2013-01/0295.html
Reported by: Bruno de Carvalho
Minor code tidy up to add comments similar to those used in the pop3
and imap end of resp functions, in order to assist anyone reading the
code and highlight the similarities between each of these protocols.
smtp_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to smtps and start the EHLO command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
pop3_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to pop3s and start the CAPA command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
imap_state_upgrade_tls() would attempt to incorrectly complete the
upgrade to imaps and start the CAPABILITY command if
Curl_ssl_connect_nonblocking() returned a failure code and if ssldone
was set to TRUE. This would only happen when a non-blocking API hadn't
been provided by the SSL implementation and curlssl_connect() was
called underneath.
- document the double-quote and backslash need be escaped if quoting.
- libcurl formdata escape double-quote in filename by backslash.
- curl formparse can parse filename both contains '"' and ',' or ';'.
- curl now can uploading file with ',' or ';' in filename.
Bug: http://curl.haxx.se/bug/view.cgi?id=1171
Fixed an issue where Curl_ssl_connect_nonblocking() wouldn't complete
correctly and the ssldone flag wouldn't be set to true for pop3s based
connections.
Bug introduced in commit: 4ffb8a6398.
Remove internal separated behavior of the easy vs multi intercace.
curl_easy_perform() is now using the multi interface itself.
Several minor multi interface quirks and bugs have been fixed in the
process.
Much help with debugging this has been provided by: Yang Tse
Fixes initial proxy response being processed by the tunneled protocol
handler instead of the HTTP wrapper handler. This issue would trigger
upon delayed CONNECT response from the proxy.
Additionally fixes a multi interface code-path in which connections
would not time out properly.
This does not fix known bug #39.
URL: http://curl.haxx.se/mail/lib-2013-01/0191.html
This commit fixes a regression introduced in 052a08ff.
NSS caches certs/keys returned by the SSL_GetClientAuthDataHook callback
and if we connect second time to the same server, the cached cert/key
pair is used. If we use multiple client certificates for different
paths on the same server, we need to clear the session cache to force
NSS to call the hook again. The commit 052a08ff prevented the session
cache from being cleared if a client certificate from file was used.
The condition is now fixed to cover both cases: consssl->client_nickname
is not NULL if a client certificate from the NSS database is used and
connssl->obj_clicert is not NULL if a client certificate from file is
used.
Review by: Kai Engert
This commit renames lib/setup.h to lib/curl_setup.h and
renames lib/setup_once.h to lib/curl_setup_once.h.
Removes the need and usage of a header inclusion guard foreign
to libcurl. [1]
Removes the need and presence of an alarming notice we carried
in old setup_once.h [2]
----------------------------------------
1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard
up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H,
this single inclusion guard is enough to ensure that inclusion of
lib/setup_once.h done from lib/setup.h is only done once.
Additionally lib/setup.h has always used __SETUP_ONCE_H macro to
protect inclusion of setup_once.h even after commit ec691ca3, this
was to avoid a circular header inclusion triggered when building a
c-ares enabled version with c-ares sources available which also has
a setup_once.h header. Commit ec691ca3 exposes the real nature of
__SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard
foreign to libcurl belonging to c-ares's setup_once.h
The renaming this commit does, fixes the circular header inclusion,
and as such removes the need and usage of a header inclusion guard
foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl.
2 - Due to the circular interdependency of old lib/setup_once.h and the
c-ares setup_once.h header, old file lib/setup_once.h has carried
back from 2006 up to now days an alarming and prominent notice about
the need of keeping libcurl's and c-ares's setup_once.h in sync.
Given that this commit fixes the circular interdependency, the need
and presence of mentioned notice is removed.
All mentioned interdependencies come back from now old days when
the c-ares project lived inside a curl subdirectory. This commit
removes last traces of such fact.
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:
f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files
This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:
c087374... curl_setup.h: remove redundant include guard
This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:
13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1
Start of related discussion thread:
http://curl.haxx.se/mail/lib-2013-01/0012.html
Asking for confirmation on pushing this revertion commit:
http://curl.haxx.se/mail/lib-2013-01/0048.html
Confirmation summary:
http://curl.haxx.se/mail/lib-2013-01/0079.html
NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.
lib/curl_imap.h
lib/curl_smtp.h
1. When the downloaded data file from Mozilla is current, but the output
bundle does not exist: continue processing to create the bundle. The
goal is to have the output file - not just download the latest input.
2. added -f option to force re-processing the file. Useful for
debugging/testing the process.
3. added support for output to '-' (stdout), allowing the output to be
piped.
4. All progress and error messages go to STDERR rather than STDOUT (3)
5. The script opened and closed the output file many times
unnecessarily. It now opens it once, does the output and closes it.
6. Backup of the input files happens after successful processing, not
before.
7. The output is written to a temporary file, and renamed to the
requested name after backup - this greatly reduces the window where the
file can be seen partially written.
8. all die calls have a \n at the end to suppress perl's traceback - the
traceback isn't useful to end users.
Patch: http://curl.haxx.se/mail/lib-2013-01/0045.html
lib/objnames.inc provides definition of curl_10char_object_name() shell
function. The intended purpose of this function is to transliterate a
(*.c) source file name that may be longer than 10 characters, or not,
into a string with at most 10 characters which may be used as an OS/400
object name.
Test case 1221 does unit testng of this function and also verifies
that it is possible to generate distinct short object names for all
curl and libcurl *.c source file names.
lib/objnames-test.sh is the shell script used for test case 1221.
tests/runtests.pl modified to accept shell script test cases.
More details inside lib/objnames.inc and lib/objnames-test.sh
* Changing the order of the state machine to represent the order in
which commands are sent to the server.
* Reworking the imap_endofresp() function as the FETCH response doesn't
include the command id and shouldn't be part of the length comparison
that takes into account the id string.
Fixed a problem with the state machine when attempting to log in with
invalid credentials. The server would report login failure but libcurl
would not read the response due to inappropriate IMAP_STOP states being
set after the login was sent.
Applied some of the comment and layout changes that had already been
applied to the pop3 and smtp code over the last 6 to 9 months.
This is in preparation of adding SASL based authentication.
... on Snow Leopard and Lion
Snow Leopard introduced the SSLSetSessionOption() function, but it
doesn't disable peer verification as expected on Snow Leopard or
Lion (it works as expected in Mountain Lion). So we now use sysctl()
to detect whether or not the user is using Snow Leopard or Lion,
and if that's the case, then we now use the deprecated
SSLSetEnableCertVerify() function instead to disable peer verification.
... it also clobbered the 'result' return value so that it wouldn't
return the error back to the parent function properly, which broke test
809 when run with 'multi-always'.
When prefixing a path with /~/ it is supposed to be used relative to the
user's home directory but it didn't work. Now we cut off the entire
three byte sequenct "/~/" which seems to be how OpenSSH does it.
Bug: http://curl.haxx.se/bug/view.cgi?id=1173
Reported by: Balaji Parasuram
Issue: When building a 32bit target with large file support HP-UX
<sys/socket.h> header file may simultaneously provide two different
sets of declarations for sendfile and sendpath functions, one with
static and another with external linkage. Given that we do not use
mentioned functions we really don't care which linkage is the
appropriate one, but on the other hand, the double declaration emmits
warnings when using the HP-UX compiler and errors when using modern
gcc versions resulting in fatal compilation errors.
Mentioned issue is now fixed as long as we don't use sendfile nor
sendpath functions.
A bundle is a list of all persistent connections to the same host.
The connection cache consists of a hash of bundles, with the
hostname as the key.
The benefits may not be obvious, but they are two:
1) Faster search for connections to reuse, since the hash
lookup only finds connections to the host in question.
2) It lays out the groundworks for an upcoming patch,
which will introduce multiple HTTP pipelines.
This patch also removes the awkward list of "closure handles",
which were needed to send QUIT commands to the FTP server
when closing a connection.
Now we allocate a separate closure handle and use that
one to close all connections.
This has been tested in a live system for a few weeks, and of
course passes the test suite.
BLANK_AT_MAKETIME may be used in our Makefile.am files to blank
LIBS variable used in generated makefile at makefile processing
time. Doing this functionally prevents LIBS from being used for
all link targets in given makefile.
This handling already works with the easy-interface code. When a request
is sent on a re-used connection that gets closed by the server at the
same time as the request is sent, the situation may occur so that we can
send the request and we discover the broken connection as a RECV_ERROR
in the PERFORM state and then the request needs to be retried on a fresh
connection. Test 64 broke with 'multi-always-internally'.
Although it is not explicitly stated in the documentation, NSS uses
*pRetCert and *pRetKey even if the client authentication hook returns
a failure. Namely, if we destroy *pRetCert without clearing *pRetCert
afterwards, NSS destroys the certificate once again, which causes a
double free.
Reported by: Bob Relyea
.. that are sent when auth-negotiating before a chunked
upload or when setting the 'Transfer-Encoding: chunked'
header and intentionally sending no content.
Adjust test565 and test1333 accordingly.
DNS cache entries populated with CURLOPT_RESOLVE were not properly freed
again when done using the multi interface.
Test case 1502 added to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=3575448
Reported by: Alex Gruz
If we use memory functions (malloc, free, strdup etc) in C sources in
libcurl and we fail to include curl_memory.h or memdebug.h we either
fail to properly support user-provided memory callbacks or the memory
leak system of the test suite fails.
After Ajit's report of a failure in the first category in http_proxy.c,
I spotted a few in the second category as well. These problems are now
tested for by test 1132 which runs a perl program that scans for and
attempts to check that we use the correct include files if a memory
related function is used in the source code.
Reported by: Ajit Dhumale
Bug: http://curl.haxx.se/mail/lib-2012-11/0125.html
When using only 1 second precision, curl doesn't create new cnonce
values quickly enough for all uses.
For example, issuing the following command multiple times to a recent
Tomcat causes authentication failures:
curl --digest -utest:test http://tomcat.test.com:8080/manager/list
This is because curl uses the same cnonce for several seconds, but
doesn't increment the nonce counter. Â Tomcat correctly interprets
this as a replay attack and rejects the request.
When microsecond-precision is available, this commit causes curl to
change cnonce values much more frequently.
With microsecond resolution, increasing the nounce length used in the
headers to 32 was made to further reduce the risk of duplication.
axTLS:
This will make the axTLS backend perform the RFC2818 checks, honoring
the VERIFYHOST setting similar to the OpenSSL backend.
Generic for OpenSSL and axTLS:
Move the hostcheck and cert_hostcheck functions from the lib/ssluse.c
files to make them genericly available for both the OpenSSL, axTLS and
other SSL backends. They are now in the new lib/hostcheck.c file.
CyaSSL:
CyaSSL now also has the RFC2818 checks enabled by default. There is a
limitation that the verifyhost can not be enabled exclusively on the
Subject CN field comparison. This SSL backend will thus behave like the
NSS and the GnuTLS (meaning: RFC2818 ok, or bust). In other words:
setting verifyhost to 0 or 1 will disable the Subject Alt Names checks
too.
Schannel:
Updated the schannel information messages: Split the IP address usage
message from the verifyhost setting and changed the message about
disabling SNI (Server Name Indication, used in HTTP virtual hosting)
into a message stating that the Subject Alternative Names checks are
being disabled when verifyhost is set to 0 or 1. As a side effect of
switching off the RFC2818 related servername checks with
SCH_CRED_NO_SERVERNAME_CHECK
(http://msdn.microsoft.com/en-us/library/aa923430.aspx) the SNI feature
is being disabled. This effect is not documented in MSDN, but Wireshark
output clearly shows the effect (details on the libcurl maillist).
PolarSSL:
Fix the prototype change in PolarSSL of ssl_set_session() and the move
of the peer_cert from the ssl_context to the ssl_session. Found this
change in the PolarSSL SVN between r1316 and r1317 where the
POLARSSL_VERSION_NUMBER was at 0x01010100. But to accommodate the Ubuntu
PolarSSL version 1.1.4 the check is to discriminate between lower then
PolarSSL version 1.2.0 and 1.2.0 and higher. Note: The PolarSSL SVN
trunk jumped from version 1.1.1 to 1.2.0.
Generic:
All the SSL backends are fixed and checked to work with the
ssl.verifyhost as a boolean, which is an internal API change.
The text "additional stuff not fine" text was added for debug purposes a
while ago, but it isn't really helping anyone and for some reason some
Linux distributions provide their libcurls built with debug info still
present and thus (far too many) users get to read this info.
The logic previously checked for a started NTLM negotiation only for
host and not also with proxy, leading to problems doing POSTs over a
proxy NTLM that are larger than 2000 bytes. Now it includes proxy in the
check.
Bug: http://curl.haxx.se/bug/view.cgi?id=3582321
Reported by: John Suprock
The existing logic only cut off the fragment from the separate 'path'
buffer which is used when sending HTTP to hosts. The buffer that held
the full URL used for proxies were not dealt with. It is now.
Test case 5 was updated to use a fragment on a URL over a proxy.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579813
As a handle can be re-used after having done HTTP auth in a previous
request, it must make sure to clear out the HTTP types that aren't
wanted in this new request.
This reverts commit ce8311c7e4.
The commit made test 2024 work but caused a regression with repeated
Digest authentication. We need to fix this differently.
After a research team wrote a document[1] that found several live source
codes out there in the wild that misused the CURLOPT_SSL_VERIFYHOST
option thinking it was a boolean, this change now bans 1 as a value and
will make libcurl return error for it.
1 was never a sensible value to use in production but was introduced
back in the days to help debugging. It was always documented clearly
this way.
1 was never supported by all SSL backends in libcurl, so this cleanup
makes the treatment of it unified.
The report's list of mistakes for this option were all PHP code and
while there's a binding layer between libcurl and PHP, the PHP team has
decided that they have an as thin layer as possible on top of libcurl so
they will not alter or specifically filter a 'TRUE' value for this
particular option. I sympathize with that position.
[1] = http://daniel.haxx.se/blog/2012/10/25/libcurl-claimed-to-be-dangerous/
Since automake 1.12.4, the warnings are issued on running automake:
warning: 'INCLUDES' is the old name for 'AM_CPPFLAGS' (or '*_CPPFLAGS')
Avoid INCLUDES and roll these flags into AM_CPPFLAGS.
Compile tested on:
Ubuntu 10.04 (automake 1:1.11.1-1)
Ubuntu 12.04 (automake 1:1.11.3-1ubuntu2)
Arch Linux (automake 1.12.4)
As pointed out in Bug report #3579064, curl_multi_perform() would
wrongly use a blocking mechanism internally for some commands which
could lead to for example a very long block if the LIST response never
showed.
The solution was to make sure to properly continue to use the multi
interface non-blocking state machine.
The new test 1501 verifies the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579064
Reported by: Guido Berhoerster
When given a string as 'srp' it didn't work, but required 'SRP'.
Starting now, the check disregards casing.
Bug: http://curl.haxx.se/bug/view.cgi?id=3578418
Reported by: Jeff Connelly
Previously the Metalink code used Apple's CommonCrypto library only if
curl was built using the --with-darwinssl option. Now we use CommonCrypto
on all Apple operating systems including Tiger or later, or iOS 5 or
later, so you don't need to build --with-darwinssl anymore. Also rolled
out this change to libcurl's md5 code.
The iOS build was broken by a reference to a function that only existed
under OS X; fixed. Also fixed a hard-to-reproduce problem where, if the
server disconnected before libcurl got the chance to hang up first and
SecureTransport was in use, then we'd raise an error instead of failing
gracefully.
This is a minor change in behavior after having been pointed out by Mark
Tully and discussed on the list. Initially this case would internally
call poll() with no sockets and a timeout which would equal a sleep for
that specified time.
Bug: http://curl.haxx.se/mail/lib-2012-10/0076.html
Reported by: Mark Tully
Since there are servers that seem to return very big encrypted
data packages, we need to be able to handle those without having
an internal size limit. To avoid the buffer growing to fast to
early the initial size was decreased and the minimum free space
in the buffer was decreased as well.
During the periods of rate limitation, the speedcheck function wasn't
called and thus the values weren't updated accordingly and it would then
easily trigger wrongly once data got transferred again.
Also, the progress callback's return code was not acknowledged in this
state so it could make an "abort" return code to get ignored and not
have the documented effect of aborting an ongoing transfer.
Bug: http://curl.haxx.se/mail/lib-2012-09/0081.html
Reported by: Jie He
The Curl_reconnect_request() function could end up returning a pointer
to a free()d struct when Curl_done() failed inside. Clearing the pointer
unconditionally after Curl_done() avoids this risk.
Reported by: Ho-chi Chen
Bug: http://curl.haxx.se/mail/lib-2012-09/0188.html
Selected socks proxy in Google's Chrome browser. Resulting in the
following environment variables:
NO_PROXY=localhost,127.0.0.0/8
ALL_PROXY=socks://localhost:1080/
all_proxy=socks://localhost:1080/
no_proxy=localhost,127.0.0.0/8
... and libcurl didn't treat 'socks://' as socks but instead picked HTTP
proxy.
Reported by: Scott Bailey
Bug: http://curl.haxx.se/bug/view.cgi?id=3566860
Each certificate section of the input certdata.txt file has a trust
section following it with details.
This script failed to detect the start of the trust for at least one
cert[*], which made the script continue pass that section into the next
one where it found an 'untrusted' marker and as a result that certficate
was not included in the output.
[*] = "Hellenic Academic and Research Institutions RootCA 2011"
Bug: http://curl.haxx.se/mail/lib-2012-09/0019.html
SMTP client will send SIZE parameter in MAIL FROM command only if server
supports it. Without this patch server might say "504 Command parameter
not implemented" and reject the message.
Bug: http://curl.haxx.se/bug/view.cgi?id=3564114
/*
* Name: curl_multi_wait()
*
* Desc: Poll on all fds within a CURLM set as well as any
* additional fds passed to the function.
*
* Returns: CURLMcode type, general multi error code.
*/
CURL_EXTERN CURLMcode curl_multi_wait(CURLM *multi_handle,
struct curl_waitfd extra_fds[],
unsigned int extra_nfds,
int timeout_ms);
In Mountain Lion, Apple added TLS 1.1 and 1.2, and deprecated a number
of SecureTransport functions, some of which we were using. We now check
to see if the replacement functions are present, and if so, we use them
instead. The old functions are still present for users of older
cats. Also fixed a build warning that started to appear under Mountain
Lion
Commit b91d29a28e170c16d65d956db79f2cd3a82372d2 introduces a bug and breaks Curl_closesocket function. sock_accepted flag for the second socket should be tagged as TRUE before the sockopt callback is called because in case the callback returns an error, Curl_closesocket function is going to call the - fclosesocket - callback for the accept()ed socket
For active FTP connections, applications may need setting the sockopt after accept() call returns successful. This fix gives a call to the callback registered with CURL_SOCKOPTFUNCTION option. Also a new sock type - CURLSOCKTYPE_ACCEPT - is added. This type is to be passed to application callbacks with - purpose - parameter. Applications may use this parameter to distinguish between socket types.
Commit e351972bc8 brought in the ssh agent support but some uses of
the libssh2 agent API was done unconditionally which wasn't good enough
since that API hasn't always been present.
By reading the ->head pointer and using that instead of the ->size
number to figure out if there's a list remaining we avoid the (false
positive) clang-analyzer warning that we might dereference of a null
pointer.
I suspect this is a regression introduced in commit 207cf150, included
since 7.24.0.
Avoid showing '(nil)' as hostname in verbose output by making sure the
hostname fixup function is called early enough to set the pointers that
are used for this. The name data is set again for each request even for
re-used connections to handle multiple hostnames over the same
connection (like with proxy) or that the casing etc of the host name is
changed between requests (which has proven to be important at least once
in the past).
Test1011 was modified to use a redirect with a re-used a connection
since it then showed the bug and now lo longer does. There's currently
no easy way to have the test suite detect 'nil' texts in verbose ouputs
so no tests will detect if this problem gets reintroduced.
Bug: http://curl.haxx.se/mail/lib-2012-07/0111.html
Reported by: Gisle Vanem
We found a problem with ftp transfer using libcurl (7.23 and 7.25)
inside an application which is receiving unix signals (SIGUSR1,
SIGUSR2...) almost continuously. (Linux 2.4, PowerPC, HAVE_POLL_FINE
defined).
Curl_socket_check() uses poll() to wait for the socket, and retries it
when a signal is received (EINTR). However, if a signal is received and
it also happens that the timeout has been reached, Curl_socket_check()
returns -1 instead of 0 (indicating an error instead of a timeout).
In our case, the result is an aborted connection even before the ftp
banner is received from the server, and a return value of
CURLE_OUT_OF_MEMORY from curl_easy_perform() (Curl_pp_multi_statemach(),
in pingpong.c, actually returns OOM if Curl_socket_check() fails :-)
Funny to debug on a system on which OOM is a possible cause).
Bug: http://curl.haxx.se/mail/lib-2012-07/0122.html
Due to WSAPoll bugs, libcurl does not work as intended. When the cURL
library is used to setup a connection to an incorrect port, normally the
result is CURLE_COULDNT_CONNECT, /* 7 */, but due to the bug in WSAPoll,
the result now is CURLE_OPERATION_TIMEDOUT, /* 28 - the timeout time was
reached */.
On August 1, Jan Koen Annot opened a case for this to Microsoft Premier
Online (https://premier.microsoft.com/). The support engineer handling
the case wrote that the case description is quite clear. He will try to
reproduce the issue and then proceed with troubleshooting it.
Reported by: Jan Koen Annot
Bug: http://curl.haxx.se/mail/lib-2012-07/0310.html
When figuring out if the data stream needs to be rewound when the
request is to be resent, we must not access the HTTP struct unless the
protocol used is indeed HTTP...
Bug: http://curl.haxx.se/bug/view.cgi?id=3544688
Previously the curl_multi interface would freeze if darwinssl was
enabled and at least one of the handles tried to connect to a Web site
using HTTPS. Removed the "wouldblock" state darwinssl was using because
I figured out a solution for our "would block but in which direction?"
dilemma.
In many states the easy_conn pointer is referenced and just assumed to
be working. This is an added extra check since analyzing indicates
there's a risk we can end up in these states with a NULL pointer there.
A HEAD response has no body length and gets the headers like the
corresponding GET would so it should not get closed after the response
based on the same rules. This mistake caused connections that did HEAD
to get closed too often without a valid reason.
Bug: http://curl.haxx.se/bug/view.cgi?id=3542731
Reported by: Eelco Dolstra
The function https_getsock was only implemented properly when USE_SSLEAY
or USE_GNUTLS is defined, but it is also necessary for USE_SCHANNEL.
The problem occurs when Curl_read_plain or Curl_write_plain returns
CURLE_AGAIN. In that case CURL_OK is returned to the multi-interface an
the used socket is set to state CURL_POLL_REMOVE and the easy-state is
set to CURLM_STATE_PROTOCONNECT. This is fine, because later the socket
should be set to CURL_POLL_IN or CURL_POLL_OUT via multi_getsock. That's
where https_getsock is called and doesn't return any sockets.
The code was printing a warning when SNI was set up successfully. Oops.
Printing the cipher number in verbose mode was something only TLS/SSL
programmers might understand, so I had it print the name of the cipher,
just like in the OpenSSL code. That'll be at least a little bit easier
to understand. The SecureTransport API doesn't have a method of getting
a string from a cipher like OpenSSL does, so I had to generate the
strings manually.
When doing CONNECT requests, libcurl must make sure the connection is
alive as much as possible. NTLM requires it and it is generally good for
other cases as well.
NTLM over CONNECT requests has been broken since this regression I
introduced in my CONNECT cleanup commits that started with 41b0237834,
included since 7.25.0.
Bug: http://curl.haxx.se/bug/view.cgi?id=3538625
Reported by: Marcel Raad
Allow NTLM authentication when building using SecureTransport (Darwin) for SSL.
This uses CommonCrypto, a cryptography library that ships with all versions of
iOS and Mac OS X. It's like OpenSSL's libcrypto, except that it's missing a few
less-common cyphers and doesn't have a big number data structure.
Before commit 2dded8fedb (dec 2010) there was logic that used
RAND_screen() at times and now I remove the leftover #ifdef check for
it.
The seeding code that uses Curl_FormBoundary() in ossl_seed() is dubious
to keep since it hardly increases randomness but I fear I'll break
something if I remove it now...
- Renamed st_ function prefix to darwinssl_
- Renamed Curl_st_ function prefix to Curl_darwinssl_
- Moved the duplicated ssl_connect_done out of the #ifdef in lib/urldata.h
- Fixed a teensy little bug that made non-blocking connection attempts block
- Made it so that it builds cleanly against the iOS 5.1 SDK
Removed two, not intended to exist, RESOURCE declarations.
Bug: http://curl.haxx.se/bug/view.cgi?id=3535977
And sorted configuration hunks to reflect same internal order
as the one shown in the usage message.
Increase decrypted and encrypted cache buffers using limitted
doubling strategy. More information on the mailinglist:
http://curl.haxx.se/mail/lib-2012-06/0255.html
It updates the two remaining reallocations that have already been there
and fixes the other one to use the same "do we need to increase the
buffer"-condition as the other two. CURL_SCHANNEL_BUFFER_STEP_SIZE was
renamed to CURL_SCHANNEL_BUFFER_FREE_SIZE since that is actually what it
is now. Since we don't know how much more data we are going to read
during the handshake, CURL_SCHANNEL_BUFFER_FREE_SIZE is used as the
minimum free space required in the buffer for the next operation.
CURL_SCHANNEL_BUFFER_STEP_SIZE was used for that before, too, but since
we don't have a step size now, the define was renamed.
Process extra data buffer before returning from schannel_connect_step2.
Without this change I've seen WinCE hang when schannel_connect_step2
returns and calls Curl_socket_ready.
If the encrypted handshake does not fit in the intial buffer (seen with
large certificate chain), increasing the encrypted data buffer is necessary.
Fixed warning in curl_schannel.c line 1215.
Implemented timeout loop in schannel_send while sending data. This
is as close as I think we can get to write buffering; I put a big
comment in to explain my thinking.
With some committer adjustments
Make the Schannel implementation use libcurl's default buffer size
for the initial received encrypted and decrypted data cache buffers.
The implementation still needs to handle more data since more data
might have already been received or decrypted during the handshake
or a read operation which needs to be cached for the next read.
curl_schannel.c - implemented graceful SSL shutdown. If we fail to
shutdown the connection gracefully, I've seen schannel try to use a
session ID for future connects and the server aborts the connection
during the handshake.
curl_schannel.c - auto certificate validation doesn't seem to work
right on CE. I added a method to perform the certificate validation
which uses CertGetCertificateChain and manually handles the result.
Coverity actually pointed out flawed logic in the previous call to
Curl_strntoupper() where the code used sizeof() of a pointer to pass in
a size argument. That code still worked since it only needed to
uppercase 4 letters. Still, the entire malloc/uppercase/free sequence
was pointless since the code has already matched the string once in the
condition that starts the block of code.
As spotted by Coverity, va_end() was not used previously. To make it
used I took away a bunch of return statements and made them into
assignments instead.
SSPI related code now compiles with ANSI and WCHAR versions of security
methods (WinCE requires WCHAR versions of methods).
Pulled UTF8 to WCHAR conversion methods out of idn_win32.c into their own file.
curl_sasl.c - include curl_memory.h to use correct memory functions.
getenv.c and telnet.c - WinCE compatibility fix
With some committer adjustments
Building with CyaSSL failed compilation. Reason being that OCSP_REQUEST and
OCSP_RESPONSE are enum values in CyaSSL and defines in <wincrypt.h> included
via <winldap.h> in ldap.c.
http://curl.haxx.se/mail/lib-2012-06/0196.html
Version number is removed in order to make this info consistent with
how we do it with other MS and Linux system libraries for which we don't
provide this info.
Identifier changed from 'WinSSPI' to 'schannel' given that this is the
actual provider of the SSL/TLS support. libcurl can still be built with
SSPI and without SCHANNEL support.
Removed obsolete minor status variable and parameter of status function
which was never used or set at all. Also Curl_sspi_strerror does support
only one status and there is no need for a second sub status.
Added Windows SSPI version information to the curl version string when
SCHANNEL SSL is not enabled, as the version of the library should also
be included when SSPI is used to generate security contexts.
Removed SSPI from the feature list as the features are GSS-Negotiate,
NTLM and SSL depending on the usage of the SSPI library.
Removed duplicate blank lines.
Removed spaces between the not and test in various if statements.
Removed explicit test of NULL in an if statement.
Placed function returns on same line as function declarations.
Replaced the use of curl_maprintf() with aprintf() as it is the
preprocessor job to do this substitution if ENABLE_CURLX_PRINTF
is set.
curl_sspi.c: Fixed mingw32-gcc compiler warnings
curl_sspi.c: Fixed length of error code hex output
The hex value was printed as signed 64-bit value on 64-bit systems:
SEC_E_WRONG_PRINCIPAL (0xFFFFFFFF80090322)
It is now correctly printed as the following:
SEC_E_WRONG_PRINCIPAL (0x80090322)
curl_sspi.c: Fallback to security function table version number
Instead of reporting an unknown version, the interface version is used.
curl_sspi.c: Removed SSPI/ version prefix from Curl_sspi_version
curl_schannel: Replaced static buffer sizes with defined names
curl_schannel.c: First brace when declaring functions on column 0
curl_schannel.c: Put the pointer sign directly at variable name
curl_schannel.c: Use structs directly instead of typedef'ed structs
curl_schannel.c: Removed space before opening brace
curl_schannel.c: Fixed lines being longer than 80 chars
Moved the error constant switch to curl_sspi.c and added two new helper
functions to curl_sspi.[ch] which either return the constant or a fully
translated message representing the SSPI security status.
Updated socks_sspi.c and curl_schannel.c to use the new functions.
Windows 2000 Professional: Schannel returns SEC_E_OK instead
of SEC_I_CONTEXT_EXPIRED. If the length of the output buffer
is zero and the first byte of the encrypted packet is 0x15,
the application can safely assume that the message was a
close_notify message and change the return value to
SEC_I_CONTEXT_EXPIRED.
Connection shutdown does not mean that there is no data to read
Correctly handle incomplete message and ask curl to re-read
Fixed buffer for decrypted being to small
Re-structured read condition to be more effective
Removed obsolete verbose messages
Changed memory reduction method to keep a minimum buffer of size 4096
Fixed warning: dereferencing pointer does break strict-aliasing rules
by using a union instead of separate pointer variables.
Internal union sockaddr_u could probably be moved to generic header.
Thanks to Paul Howarth for the hint about using unions for this.
Important for winbuild: Separate declaration of sockaddr_u pointer.
The pointer variable *sock cannot be declared and initialized right
after the union declaration. Therefore it has to be a separate statement.
Since Curl_pgrsDone() itself calls Curl_pgrsUpdate() which may return an
abort instruction or similar we need to return that info back and
subsequently properly handle return codes from Curl_pgrsDone() where
used.
(Spotted by a Coverity scan)
Previously it would use a 256 byte buffer and thus cut off very long
subject names. The limit is now upped to the receive buffer size, 16K.
Bug: http://curl.haxx.se/bug/view.cgi?id=3533045
Reported by: Anthony G. Basile
Re-factored the smtp_state_*_resp() functions to 1) Match the constants
that were refactored in commit 00fddba672, 2) To be more readable and
3) To match their counterparties in pop3.c.
Corrected lines longer than 78 characters.
Removed unnecessary braces in smtp_state_helo_resp().
Introduced some comments in data sending functions.
Tidied up comments to match changes made in pop3.c.
Corrected lines longer than 78 characters.
Changed POP3_AUTH_FINAL to POP3_AUTH to match SMTP code now that the
AUTH command is no longer sent on its own.
Introduced some comments in data sending functions.
Another attempt at trying to rational code and comment style.
Added a service type parameter to Curl_sasl_create_digest_md5_message()
to allow the function to be used by different services rather than being
hard coded to "smtp".
Not all SASL enabled POP3 servers support the AUTH command on its own
when trying to detect the supported mechanisms. As such changed the
mechanism detection to use the CAPA command instead.
Because pop3_endofresp() is called for each line of data yet is not
passed the line and line length, so we have to use the data pointed to
by pp->linestart_resp which contains the whole packet, the mechanisms
were being detected in one call yet the function would be called for
each line of data.
Using curl with verbose mode enabled would show that one line of data
would be received in response to the AUTH command, before the AUTH
<mechanism> command was sent to the server and then the next few lines
of the original AUTH command would be displayed before the response from
the AUTH <mechanism> command. This would then cause problems when
parsing the CRAM-MD5 challenge data as extra data was contained in the
buffer.
Changed the parsing so that each line is checked for the mechanisms
and the function returns FALSE until the whole of the AUTH response has
been processed.
Previously it wasn't possible to connect to POP3 and not specify the
user name as a CURLE_ACCESS_DENIED error would be returned. This error
occurred because USER would be sent to the server with a blank user name
if no mailbox user was specified as the server would reply with -ERR.
This wasn't a problem prior to the 7.26.0 release but with the
introduction of custom commands the user and/or application developer
might want to issue a CAPA command without having to log in as a
specific mailbox user.
Additionally this fix won't send the newly introduced AUTH command if no
user name is specified.
Rather than encoding the password message itself the
smtp_state_authpasswd_resp() function now delegates the work to the same
function that smtp_state_authlogin_resp() and smtp_authenticate() use
when constructing the encoded user name.
In preparation for moving to the SASL module re-factored the
smtp_auth_login_user() function to smtp_auth_login() so that it can be
used for both user names and passwords as sending both of these under
the login authentication mechanism is the same.
The POP3 protocol doesn't really have the concept of error codes and
uses +, +OK and -ERR in response to commands to indicate continue,
success and error.
The AUTH command is one of those commands that requires multiple pieces
of data to be sent to the server where the server will respond with + as
part of the handshaking. This meant changing the values before
continuing with the next stage of adding authentication support.
Changed the order of the state machine to match the order of actual
events.
Reworked some comments and function parameter positioning that I missed
the other day.
Added support for detecting the supported SASL authentication mechanisms
via the AUTH command. There are two ways of detecting them, either by
using the AUTH command, that will return -ERR if not supported or by
using the CAPA command which will return SASL and the list of mechanisms
if supported, not include SASL if SASL authentication is not supported
or -ERR if the CAPA command is not supported. As such it seems simpler
to use the AUTH command and fallback to normal clear text authentication
if the the command is not supported.
Additionally updated the test cases to return -ERR when the AUTH command
is encountered. Additional test cases will be added when support for the
individual authentication mechanisms is added.
Moved EOB definition into header file.
Switched the logic around in pop3_endofresp() to allow for the
introduction of auth-mechanism detection.
Repositioned second and third function variables where they will fit
within the 78 character line limit.
Tidied up some comments.
Move the SMTP_AUTH constants into a separate header file in
preparation for adding SASL based authentication to POP3 as the two
protocols will need to share them.
Due to the result code being reset to CURLE_OK when smtp_dophase_done()
was called, postdata would incorrectly be sent to the server when the
MAIL FROM or RCPT command was rejected.
As such, libcurl would return the wrong result code from performing the
operation and additionally set CURLINFO_RESPONSE_CODE to be that
returned by the postdata command.
Bug: http://curl.haxx.se/mail/lib-2012-05/0108.html
Reported by: Gokhan Sengun
In nettle/md5.h, md5_init and md5_update are defined as macros to
nettle_md5_init and nettle_md5_update respectively. This causes
error when using MD5_params.md5_init and md5_update. This patch
renames these members as md5_init_func and md5_update_func to
avoid name conflict. For completeness, MD5_params.md5_final was
also renamed as md5_final_func.
The changes in curl_ntlm_core.c is conversion error and fixed by
casting to proper type.
A dot character at the beginning of a line would not be escaped to a
double dot as required by RFC-2821, instead it would be deleted by the
mail server. Please see section 4.5.2 of the RFC for more information.
Note: This fix also simplifies the detection of repeated CRLF.CRLF
combinations, such as CRLF.CRLF.CRLF, a little rather than having to
advance the eob counter to 2.
Roman Mamedov spotted (in
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=670126) that curl would
not complain when given a URL with an IPv6 numerical address without
brackets. It would simply cut off the last ":[hex]" part and thus not
work correctly.
That's a URL using an illegal syntax and now libcurl will instead return
a clear error code and error message detailing the error.
The above mentioned bug report claims this to be a regression but
libcurl does not guarantee functionality when given URLs that aren't
following the URL spec (RFC3986 mostly). I consider the fact that it
used to handle this differently a mere coincidence.
When doing a chunked-encoded POST with -d (CURLOPT_POSTFIELDS) and the
size of the POST was zero length, it made libcurl first send a zero
chunk and then the terminating one. This could confuse a receiver and it
should rather just send the terminating chunk as it does with this fix.
Test case 1333 is added to verify.
Bug: http://curl.haxx.se/mail/archive-2012-04/0060.html
Reported by: Arnaud Compan
Commit 9109cdec11 brought this regression (shipped since 7.24.0).
The singleipconnect() function must not return an error if Curl_socket()
returns an error. It should then simply return OK and pass a SOCKET_BAD
back simply because that is how the user of this function expects it to
work and something else is not fine.
Reported by: Blaise Potard
Bug: http://curl.haxx.se/bug/view.cgi?id=3516508
Include stdbool.h only when it is available and configure is capable of
detecting a proper 'bool' data type when the header is included.
Compilation fix for old or unpatched versions of XL C compiler.
Report: http://curl.haxx.se/mail/archive-2012-04/0022.html
NSS_InitContext() was introduced in NSS 3.12.5 and helps to prevent
collisions on NSS initialization/shutdown with other libraries.
Bug: https://bugzilla.redhat.com/738456
configure script now provides conditional definitions for Makefile.am
that result in CURL_HIDDEN_SYMBOLS being defined by resulting makefiles
when appropriate.
Additionally, configure script option for symbol hiding control is now
named --enable-symbol-hiding --disable-symbol-hiding. While still valid,
old option name --enable-hidden-symbols --disable-hidden-symbols will
be deprecated in some future release.
BUILDING_LIBCURL and CURL_STATICLIB are no longer defined in curl_config.h,
configure will generate appropriate conditionals so that mentioned symbols
get defined and used in Makefiles at compilation time
Configuration files such as curl_config.h and all config-*.h no longer exist
nor are generated/copied into 'src' directory, now these only exist in 'lib'
directory from where curl tool sources uses them.
Additionally old src/setup.h has been refactored into src/tool_setup.h which
now pulls lib/setup.h
The possibility of a makefile needing an include path adjustment exists.
Curl_socket returns CURLE_COULDNT_CONNECT when the opensocket callback
returns CURL_SOCKET_BAD. Previous return value CURLE_FAILED_INIT
conveys incorrect information to the user.
Reworked the command sending from two specific LIST and RETR command
functions into a single command based function as well as the two
associated response handlers into a generic command handler.
If an empty string is passed to CURLOPT_SSH_PUBLIC_KEYFILE, libcurl will
pass no public key to libssh2 which then tries to compute it from the
private key. This is known to work when libssh2 1.4.0+ is linked against
OpenSSL.
This change replaces RFC 2818 based hostname check in OpenSSL build with
RFC 6125 [1] based one.
The hostname check in RFC 2818 is ambiguous and each project implements
it in the their own way and they are slightly different. I check curl,
gnutls, Firefox and Chrome and they are all different.
I don't think there is a bug in current implementation of hostname
check. But it is not as strict as the modern browsers do. Currently,
curl allows multiple wildcard character '*' and it matches '.'. (as
described in the comment in ssluse.c).
Firefox implementation is also based on RFC 2818 but it only allows at
most one wildcard character and it must be in the left-most label in the
pattern and the wildcard must not be followed by any character in the
label.[2] Chromium implementation is based on RFC 6125 as my patch does.
Firefox and Chromium both require wildcard in the left-most label in the
presented identifier.
This patch is more strict than the current implementation, so there may
be some cases where old curl works but new one does not. But at the same
time I think it is good practice to follow the modern browsers do and
follow the newer RFC.
[1] http://tools.ietf.org/html/rfc6125#section-6.4.3
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=159483
With FOLLOWLOCATION enabled. When a 3xx page is downloaded and the
download size was known (like with a Content-Length header), but the
subsequent URL (transfered after the 3xx page) was chunked encoded, then
the previous "known download size" would linger and cause the progress
meter to get incorrect information, ie the former value would remain
being sent in. This could easily result in downloads that were WAY
larger than "expected" and would cause >100% outputs with the curl
command line tool.
Test case 599 was created and it was used to repeat the bug and then
verify the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3510057
Reported by: Michael Wallner
It is now possible to calculate the md5 sum as the stream of buffers
becomes known where as previously it was only possible to calculate the
md5 sum of a pre-prepared buffer.
This feature allows the user to specify and use additional POP3
commands such as UIDL and DELE via libcurl's CURLOPT_CUSTOMREQUEST or
curl's -X command line option.
Simplified the code to remove the need for a separate "LIST <msg id>"
command handler and state machine and instead use the LIST command
handler for both operations.
Moved the server greeting response handling code from the statemach_act
functions to separate response functions. This makes the code simpler
to follow and provides consistency with the other responses that are
handled here.
The commit e650dbde86 that stripped off [brackets] from ipv6-only host
headers for the sake of cookie parsing wrongly incremented the host
pointer which would cause a bad free() call later on.
The refactoring of HTTP CONNECT handling in commit 41b0237834 that
made it protocol independent broke it for the multi interface. This fix
now introduce a better state handling and moved some logic to the
http_proxy.c source file.
Reported by: Yang Tse
Bug: http://curl.haxx.se/mail/lib-2012-03/0162.html
Changed the returned curl error codes for EHLO and HELO responses from
CURLE_LOGIN_DENIED to CURLE_REMOTE_ACCESS_DENIED as a negative response
from these commands represents no service as opposed to a login error.
An alternative would be:
1. specify HTTPS_CA_DIR and/or HTTPS_CA_FILE
2. ensure that Net::SSL is being used, and IO::Socket::SSL is NOT being
used
This question and answer explain:
http://stackoverflow.com/questions/74358/
Curl_protocol_connect() now does the tunneling through the HTTP proxy if
requested instead of letting each protocol specific connection function
do it.
Commit 466150bc64 fixed the Host: header with CONNECT, but I then
forgot the preceeding request-line. Now this too uses [brackets]
properly if a ipv6 numerical address was given.
Bug: http://curl.haxx.se/bug/view.cgi?id=3493129
Reported by: "Blacat"
Set the conn->data->info.httpcode variable in smtp_statemach_act() to
allow Curl_getinfo() to return the SMTP response code via the
CURLINFO_RESPONSE_CODE action.
Curl_pop3_write() would drop the final CRLF of a message as it was
considered part of the EOB as opposed to part of the message. Whilst
the EOB sequence needs to be searched for by the function only the
final 3 characters should be removed as per RFC-1939 section 3.
Reported by: Rich Gray
Bug: http://curl.haxx.se/mail/lib-2012-02/0051.html
Curl_smtp_escape_eob() would leave off final CRLFs from emails ending
in multiple blank lines additionally leaving the smtpc->eob variable
with the character count in, which would cause problems for additional
emails when sent through multiple calls to curl_easy_perform() after a
CURLOPT_CONNECT_ONLY.
Fixed the use of angled brackets "<>" in the optional AUTH parameter as
per RFC-2554 section 5. The address should not include them but an
empty address should be replaced by them.
Added a new CURLOPT_MAIL_AUTH option that allows the calling program to
set the optional AUTH parameter in the MAIL FROM command.
When this option is specified and an authentication mechanism is used
to communicate with the mail server then the AUTH parameter will be
included in the MAIL FROM command. This is particularly useful when the
calling program is acting as a relay in a trusted environment and
performing server to server communication, as it allows the relaying
server to specify the address of the mailbox that was used to
authenticate and send the original email.
Modify configure.ac to test for new CyaSSL Init function and remove
default install path to system. Change to CyaSSL OpenSSL header and
proper Init in code as well.
Note that this no longer detects or works with CyaSSL before v2
Fixed incorrect behavior in smtp_done() which would cause the end of
block data to be sent to the SMTP server if libcurl was operating in
connect only mode. This would cause the server to return an error as
data would not be expected which in turn caused libcurl to return
CURLE_RECV_ERROR.
... by making sure that the string is always freed after the invoke as
parse_proxy will always copy the data and this way there's a single
free() instead of multiple ones.
The proxy parser function strips off trailing slashes off the proxy name
which could lead to a mistaken zero length proxy name which would be
treated as no proxy at all by subsequent functions!
This is now detected and an error is returned. Verified by the new test
1329.
Reported by: Chandrakant Bagul
Bug: http://curl.haxx.se/mail/lib-2012-02/0000.html
Allow an appliction to set libcurl specific SSL options. The first and
only options supported right now is CURLSSLOPT_ALLOW_BEAST.
It will make libcurl to disable any work-arounds the underlying SSL
library may have to address a known security flaw in the SSL3 and TLS1.0
protocol versions.
This is a reaction to us unconditionally removing that behavior after
this security advisory:
http://curl.haxx.se/docs/adv_20120124B.html
... it did however cause a lot of programs to fail because of old
servers not liking this work-around. Now programs can opt to decrease
the security in order to interoperate with old servers better.
This adds three new options to control the behavior of TCP keepalives:
- CURLOPT_TCP_KEEPALIVE: enable/disable probes
- CURLOPT_TCP_KEEPIDLE: idle time before sending first probe
- CURLOPT_TCP_KEEPINTVL: delay between successive probes
While not all operating systems support the TCP_KEEPIDLE and
TCP_KEEPINTVL knobs, the library will still allow these options to be
set by clients, silently ignoring the values.
When CURLOPT_REFERER has been used, curl_easy_reset() did not properly
clear it.
Verified with the new test 598
Bug: http://curl.haxx.se/bug/view.cgi?id=3481551
Reported by: Michael Day
When the target host was given as a IPv6 numerical address, it was not
properly put within square brackets for the Host: header in the CONNECT
request. The "normal" request did fine.
Reported by: "zooloo"
Bug: http://curl.haxx.se/bug/view.cgi?id=3482093
When connecting to a domain with multiple IP addresses, allow different,
decreasing connection timeout values. This should guarantee some
connections attempts with sufficiently long timeouts, while still
providing fallback.
With advice from Nikos Mavrogiannopoulos, changed the priority string to
add "actual priorities" and favour ARCFOUR. This makes libcurl work
better when enforcing SSLv3 with GnuTLS. Both in the sense that the
libmicrohttpd test is now working again but also that it mitigates a
weakness in the older SSL/TLS protocols.
Bug: http://curl.haxx.se/mail/lib-2012-01/0225.html
Reported by: Christian Grothoff
Protocols (IMAP, POP3 and SMTP) that use the path part of a URL in a
decoded manner now use the new Curl_urldecode() function to reject URLs
with embedded control codes (anything that is or decodes to a byte value
less than 32).
URLs containing such codes could easily otherwise be used to do harm and
allow users to do unintended actions with otherwise innocent tools and
applications. Like for example using a URL like
pop3://pop3.example.com/1%0d%0aDELE%201 when the app wants a URL to get
a mail and instead this would delete one.
This flaw is considered a security vulnerability: CVE-2012-0036
Security advisory at: http://curl.haxx.se/docs/adv_20120124.html
Reported by: Dan Fandrich
OpenSSL added a work-around for a SSL 3.0/TLS 1.0 CBC vulnerability
(http://www.openssl.org/~bodo/tls-cbc.txt). In 0.9.6e they added a bit
to SSL_OP_ALL that _disables_ that work-around despite the fact that
SSL_OP_ALL is documented to do "rather harmless" workarounds.
The libcurl code uses the SSL_OP_ALL define and thus logically always
disables the OpenSSL fix.
In order to keep the secure work-around workding, the
SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS bit must not be set and this change
makes sure of this.
Reported by: product-security at Apple
Using a URL with embedded user name and password didn't work if the host
was given as a numerical IPv6 string, like ftp://user:password@[::1]/
Reported by: Brandon Wang
Bug: http://curl.haxx.se/mail/archive-2012-01/0047.html
SSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG option enabling allowed successfull
interoperability with web server Netscape Enterprise Server 2.0.1 released
back in 1996 more than 15 years ago.
Due to CVE-2010-4180, option SSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG has
become ineffective as of OpenSSL 0.9.8q and 1.0.0c. In order to mitigate
CVE-2010-4180 when using previous OpenSSL versions we no longer enable
this option regardless of OpenSSL version and SSL_OP_ALL definition.
Some functions using getaddrinfo and gethostbyname were still
mistakingly being used/linked even if c-ares was selected as resolver
backend.
Reported by: Arthur Murray
Bug: http://curl.haxx.se/mail/lib-2012-01/0160.html
Previously the code would create a dummy socket while resolving just to
have curl_multi_fdset() return something but the non-win32 version
doesn't do it this way and the creation and use of a socket that isn't
made with the common create-socket callback can be confusing to apps
using the multi_socket API etc.
This change removes the dummy socket and thus will cause
curl_multi_fdset() to return with maxfd == -1 more often.
Fixed a problem in POP3 and IMAP where a connection would fail when
CURLUSESSL_TRY was specified for a server that didn't support
SSL/TLS connections rather than continuing.
The STARTTLS response code in SMTP, POP3 and IMAP would return
CURLE_LOGIN_DENIED rather than CURLE_USE_SSL_FAILED when SSL/TLS
was not available on the server.
Reported by: Gokhan Sengun
Bug: http://curl.haxx.se/mail/lib-2012-01/0018.html
Unfortunately we have no test cases for this and I have no SSPI build or
server to verify this with. The change seems simple enough though.
Bug: http://curl.haxx.se/bug/view.cgi?id=3466497
Reported by: Patrice Guerin
When the buffer gets realloced to hold the file name in the
SSH_SFTP_READDIR_LINK state, the counter was not bumped accordingly.
Reported by: Armel Asselin
Patch by: Armel Asselin
Bug: http://curl.haxx.se/mail/lib-2011-12/0249.html
When a HTTP connection is re-used for a subsequent request without
proxy, it would always re-use the Host: header of the first request. As
host names are case insensitive it would make curl send another host
name case that what the particular request used.
Now it will instead always use the most recent host name to always use
the desired casing.
Added test case 1318 to verify.
Bug: http://curl.haxx.se/mail/lib-2011-12/0314.html
Reported by: Alex Vinnik
The load host names to DNS cache function was moved to hostip.c and it
now makes sure to not add host names that already are present in the
cache. It would previously lead to memory leaks when for example using
the --resolve and multiple URLs on the command line.
The commit 9dd85bc unintentionally changed the way we compute the time
spent waiting for 100-continue. In particular, when using a SSL client
certificate, the time spent by SSL handshake was included and could
cause the CURL_TIMEOUT_EXPECT_100 timeout to be mistakenly fired up.
Bug: https://bugzilla.redhat.com/767490
Reported by: Mamoru Tasaka
ftp_do_more() returns after accepting the server connect however it
needs to fall through and set "*complete" to TRUE before exit from the
function.
Bug: http://curl.haxx.se/mail/lib-2011-12/0250.html
Reported by: Gokhan Sengun
In the recent do_more fix the new logic was mistakenly checking the
pointer instead of what it points to.
Reported by: Gokhan Sengun
Bug: http://curl.haxx.se/mail/lib-2011-12/0250.html
When sending quote command to a SFTP server and 'mkdir' was used, it
would send fixed permissions and not use the CURLOPT_NEW_DIRECTORY_PERMS
as it should.
Reported by: Armel
Patch by: Armel
Bug: http://curl.haxx.se/mail/lib-2011-12/0249.html
CURLOPT_RESOLVE populates the DNS cache with entries that are marked as
eternally in use. Those entries need to be taken care of when the cache
is killed off.
Bug: http://curl.haxx.se/bug/view.cgi?id=3463121
Reported by: "tw84452852"
First off the timeout for accepting a server connect back must of course
respect a global timeout. Then the timeleft function is only used by ftp
code so it was moved to ftp.c and made static.
"wait_data_conn" was added to the connectionbits in commit c834213ad5 for
handling active FTP connections but as it is purely FTP specific and now
only ever accessed by ftp.c I moved it into the FTP connection struct.
Backpedaled out the funny double-change of state in the multi state
machine by adding a new argument to the do_more() function to signal
completion. This way it can remain in the DO_MORE state properly until
done. Long term, the entire DO_MORE logic should be moved into the FTP
code and be hidden from the multi code as the logic is only used for
FTP.
1- Two new error codes are introduced.
CURLE_FTP_ACCEPT_FAILED to be set whenever ACCEPTing fails because of
FTP server connected.
CURLE_FTP_ACCEPT_TIMEOUT to be set whenever ACCEPTing timeouts.
Neither of these errors are considered fatal and control connection
remains OK because it could just be a firewall blocking server to
connect to the client.
2- One new setopt option was introduced.
CURLOPT_ACCEPTTIMEOUT_MS
It sets the maximum amount of time FTP client is going to wait for a
server to connect. Internal default accept timeout is 60 seconds.
It makes it easier to introduce debug outputs in this function, and
everything in the function is using the value anyway so it might even be
more efficient.
Regression introduced in 7.23.0 with commit 9dd85bce. The function in
which the PRETRANSFER time stamp was recorded was moved in time causing
it be stored very quickly after the start timestamp. On most systems
shorter than 1 millisecond and thus it wouldn't even show with -w
"%{time_pretransfer}" using the command line tool.
Bug: http://curl.haxx.se/mail/archive-2011-12/0022.html
Reported by: Toni Moreno
Allow, at configure time, the production of versioned symbols. The
symbols will look like "CURL_<FLAVOUR>_<VERSION> <SYMBOL>", where
<FLAVOUR> represents the SSL flavour (e.g. OPENSSL, GNUTLS, NSS, ...),
<VERSION> is the major SONAME version and <SYMBOL> is the actual symbol
name. If no SSL library is enabled the symbols will be just
"CURL_<VERSION> <SYMBOL>".
This gets the appconnect time right for ssl backends, which don't
support non-blocking connects.
Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
Do not try to resolve interfaces names via DNS by recognizing interface
names in a few ways. If the interface option argument has a prefix of
"if!" then treat the argument as only an interface. Similarly, if the
interface argument is the name of an interface (even if it does not have
an IP address assigned), treat it as an interface name. Finally, if the
interface argument is prefixed by "host!" treat it as a hostname that
must be resolved by /etc/hosts or DNS.
These changes allow a client using the multi interfaces to avoid
blocking on name resolution if the interface loses its IP address or
disappears.
Fixed the connection reuse detection in ConnectionExists() when
comparing a new connection that is non-SSL based against that of a SSL
based connection that has become so by being upgraded via TLS.
This is a regression since who knows when. When spotting that a HTTP
proxy is used we must not uncondititionally enable the HTTP protocol
since if we do tunneling through the proxy we're still using the target
protocol.
Reported by: Naveen Chandran
If no SSLv2 was detected in OpenSSL by configure, then we enforce the
OPENSSL_NO_SSL2 define as it seems some people report it not being
defined properly in the OpenSSL headers.
When a 32 digit hex key is given as a hostkey md5 checksum, the code
would still run it against the knownhost check and not properly
acknowledge that the md5 should then be the sole guide for.
The verbose output now includes the evaluated MD5 hostkey checksum.
Some related source code comments were also updated.
Bug: http://curl.haxx.se/bug/view.cgi?id=3451592
Reported by: Reza Arbab
As there are different return codes for host vs proxy errors, this function
now properly returns the code properly depending on what was attempted to get
resolved.
Bug: http://curl.haxx.se/mail/archive-2011-12/0010.html
Reported by: Jason Liu
When making a distinction which return code to return, the code previously
only regarded HTTP proxies to be proxies and thus return host-related errors
for failures on other proxy types than HTTP. Now all proxy types will be
considered proxies...
Keep track of which sockets that are the result of accept() calls and
refuse to call the closesocket callback for those sockets. Test case 596
now verifies that the open socket callback is called the same number of
times as the closed socket callback for active FTP connections.
Bug: http://curl.haxx.se/mail/lib-2011-12/0018.html
Reported by: Gokhan Sengun
When the new socket is created for an active connection, it is now done
using the open socket callback.
Test case 596 was modified to run fine, although it hides the fact that
the close callback is still called too many times, as it also gets
called for closing sockets that were created with accept().
If the first name server is not available, the multi interface does
not invoke the socket_cb when the DNS request to the first name server
timesout. Ensure that the list of sockets are always updated after
calling Curl_resolver_is_resolved.
This bug can be reproduced if Curl is complied with --enable_ares and
your code uses the multi socket interfaces and the
CURLMOPT_SOCKETFUNCTION option. To test try:
iptables -I INPUT \
-s $(sed -n -e '/name/{s/.* //p;q}' /etc/resolv.conf)/32 \
-j REJECT
and then run a program which uses the multi-interface.
Changed the eob detection to work across the whole of the buffer so that
lines that begin with a dot (which the server will have escaped) are
passed to the client application correctly.
Curl_pop3_write() now has a state machine that scans for the end of a
POP3 body so that the CR LF '.' CR LF sequence can come in everything
from one up to five subsequent packets.
Test case 810 is modified to use SLOWDOWN which makes the server pause
between each single byte and thus makes the POP3 body get sent to curl
basically one byte at a time.
Added convenience macro to use to check if a handle is using a shared
SSL session, and fixed so that Curl_ssl_close_all() doesn't lock when
the session isn't shared.
Skip a floating point addition operation when integral part of time difference
is zero. This avoids potential floating point addition rounding problems while
preserving decimal part value.
Macros that look like function calls need to be made so that we can use
semicolons properly for indentation and for reducing the risk for
mistakes when using them.
1) enables the Window Size option
2) allows the server to enable the echo mode
3) allows an app using libcurl to disable the default binary mode
Signed-off-by: Laurent Rabret
By setting PROTOPT_NOURLQUERY in the protocol handler struct, the
protocol will get the "query part" of the URL cut off before the data is
handled by the protocol-specific code. This makes libcurl adhere to
RFC3986 section 2.2.
Test 1220 is added to verify a file:// URL with query-part.
Bugfix: https handshake fails using gnutls 3 on windows
http://sourceforge.net/tracker/index.php?func=detail&aid=3441084&group_id=976&atid=100976
New gnutls versions have an error handler that knows about Winsock
errors, which is why gnutls_transport_set_global_errno() was deprecated
and then removed.
This is a correction of commit f5bb370 (blame me) which meant to
reimplement gnutls_transport_set_global_errno(), which is not necessary.
Regression: commit b998d95b (shipped first in release 7.22.0) made the
condition always equal false that should reset the TIMER_CONNECT timer
and call the Curl_verboseconnect() function.
Reported by: "Captain Basil"
Bug: http://curl.haxx.se/mail/archive-2011-11/0035.html
When the user requests PORT with a specific port or port range, the code
could lock up in an endless loop. There's now an extra conditional that
makes sure to special treat the error and try the local address only
once so a second failure will abort the loop correctly.
Bug: http://curl.haxx.se/bug/view.cgi?id=3433968
Reported by: Gokhan Sengun
If a proxy offers several Authentication schemes where NTLM and
Negotiate are offered by the proxy and you tell libcurl not to use the
Negotiate scheme then the request never returns when the proxy answers
with its HTTP 407 reply.
It is reproducible by the following steps:
- Use a proxy that offers NTLM and Negotiate ( CURLOPT_PROXY and
CURLOPT_PROXYPORT )
- Tell libcurl NOT to use Negotiate CURL_EASY_SETOPT(CURLOPT_PROXYAUTH,
CURLAUTH_BASIC | CURLAUTH_DIGEST | CURLAUTH_NTLM )
- Start the request
The call to CURL_EASY_PERFORM never returns. If you switch on debug
logging you can see that libcurl issues a new request As soon as it
received the 407 reply. Instead it should return and set the response
code to 407.
Bug: http://curl.haxx.se/mail/lib-2011-10/0323.html
Move calling of ERR_remove_state(0) a.k.a ERR_remove_thread_state(NULL)
from Curl_ossl_close_all() to Curl_ossl_cleanup().
In this way ERR_remove_state(0) is now only called in libcurl by
curl_global_cleanup(). Previously it would get called by functions
curl_easy_cleanup(), curl_multi_cleanup and potentially each time a
connection was removed from a connection cache leading to premature
destruction of OpenSSL's thread local state hash.
Multi-threaded apps using OpenSSL enabled libcurl should still call
function ERR_remove_state(0) or ERR_remove_thread_state(NULL) at the
very end end of threads that do not call curl_global_cleanup().
Now called 'use_ssl' instead, which better matches the current CURLOPT
name and since the option is used for all pingpong protocols (at least)
it makes sense to not use 'ftp' in the name.
Use gnutls_priority_set_direct() instead of gnutls_protocol_set_priority().
Remove the gnutls_certificate_type_set_priority() use since x509 is the
default certificate type anyway.
Reported by: Vincent Torri
This extends the fix from commit d7934b8bd4
When the multi state is changed within the multi_runsingle from DOING to
DO_MORE, we didn't immediately start the FTP state machine again. That
then left the FTP state in FTP_STOP. When curl_multi_fdset() was
subsequently called, the ftp_domore_getsock() function would return the
wrong fd info.
Reported by: Gokhan Sengun
After a PORT has been issued, and the multi handle would switch to the
CURLM_STATE_DO_MORE state (which is unique for FTP), libcurl would
return the wrong fdset to wait for when curl_multi_fdset() is
called. The code would blindly assume that it was waiting for a connect
of the second connection, while that isn't true immediately after the
PORT command.
Also, the function multi.c:domore_getsock() was highly FTP-centric and
therefore ugly to keep in protocol-agnostic code. I solved this problem
by introducing a new function pointer in the Curl_handler struct called
domore_getsock() which is only called during the DOMORE state for
protocols that set that pointer.
The new ftp.c:ftp_domore_getsock() function now returns fdset info about
the control connection's command/response handling while such a state is
in use, and goes over to waiting for a writable second connection first
once the commands are done.
The original problem could be seen by running test 525 and checking the
time stamps in the FTP server log. I can verify that this fix at least
fixes this problem.
Bug: http://curl.haxx.se/mail/lib-2011-10/0250.html
Reported by: Gokhan Sengun
The fix is pretty much the one Nick Zitzmann provided, just edited to do
the right indent levels and with test case 1204 added to verify the fix.
Bug: http://curl.haxx.se/mail/lib-2011-10/0190.html
Reported by: Nick Zitzmann
The default lowat level for gnutls-2.12* is set to zero to avoid
unnecessary system calls and the gnutls_transport_set_lowat function has
been totally removed in >=gnutls-3 which causes build failures.
Therefore, the function shouldn't be used except for versions that
require it, <gnutls-2.12.0.
Previously the bit was set before the connection was found working so if
it would first fail to an ipv6 address and then connect fine to a IPv4
address the variable would still be TRUE.
Reported by: Thomas L. Shinnick
Bug: http://curl.haxx.se/bug/view.cgi?id=3421912
When doing a multipart formpost with a read callback, and that callback
returns CURL_READFUNC_ABORT, that return code must be properly
propagated back and handled accordingly. Previously it would be handled
as a zero byte read which would cause a hang!
Added test case 587 to verify. It uses the lib554.c source code with a
small ifdef.
Reported by: Anton Bychkov
Bug: http://curl.haxx.se/mail/lib-2011-10/0097.html
Save the errno value immediately after a connect() failure so that it
won't get reset to something else before we read it.
Bug: http://curl.haxx.se/mail/lib-2011-10/0066.html
Reported by: Frank Van Uffelen and Fabian Hiernaux
Set ACK timeout to 5 seconds.
If we are waiting for block X and receive block Y that is the expected one, we
should send ACK and increase X (which is already implemented). Otherwise drop
the packet and don't increase retry counter.
Prevent modification of easy handle being added with curl_multi_add_handle()
unless this function actually suceeds.
Run Curl_posttransfer() to allow restoring of SIGPIPE handler when
Curl_connect() fails early in multi_runsingle().
It makes much nicer and less convuluted code everywhere if this struct
member is always present even when libcurl is built without SSL support.
This reverts parts of commit 15e3e45170
Modified smtp_endofresp() to detect NTLM from the server specified list
of supported authentication mechanisms.
Modified smtp_authenticate() to start the sending of the NTLM data.
Added smtp_auth_ntlm_type1_message() which creates a NTLM type-1
message. This function is used by authenticate() to start the sending
of data and by smtp_state_auth_ntlm_resp() when the AUTH command
doesn't contain the type-1 message as part of the initial response.
This lack of initial response can happen if an OOM error occurs or the
type-1 message is longer than 504 characters. As the main AUTH command
is limited to 512 character the data has to be transmitted in two
parts; one containing the AUTH NTLM and the second containing the
type-1 message.
Added smtp_state_auth_ntlm_type2msg_resp() which handles the incoming
type-2 message and sends an outgoing type-3 message. This type-2
message is sent by the server in response to our type-1 message.
Modified smtp_state_auth_resp() to handle the response to: the AUTH
NTLM without the initial response and the type-2 response.
Modified smtp_disconnect() to cleanup the NTLM SSPI stack.
Added the output message length as a parameter to both
Curl_ntlm_create_type1_message() and Curl_ntlm_create_type3_message()
for use by future functions that require it.
Updated curl_ntlm.c to cater for the extra parameter on these two
functions.
Changed the name of variable l, in several functions, which represents
the length of strings being sent to the server, to len which is more
meaningful and consistent with other code in smtp.c and elsewhere.
Reworked smtp_authenticate() to be simpler and easier to follow.
Variables and now initialised in their definitions and if no username
and password are specified the function sets the state to SMTP_STOP and
returns immediately, rather than being part of a huge if statement.
Don't even declare the struct members for disabled features
Introducing the CURLSHE_NOT_BUILT_IN return code for the share interface
when trying to set a sharing option that has been disabled (or not
enabled) in the library.
When the progress function returns to cancel the request, we must mark
the connection to get closed and it must do to the DONE state.
do_init() must be called as early as possible so that state variables
for new connections are reset early. We could otherwise see that the old
values were still there when a connection was to be disconnected very
early and it would make it behave wrongly.
Bug: http://curl.haxx.se/mail/lib-2011-10/0006.html
Reported by: Vladimir Grishchenko
The size of the email can now be set via CURLOPT_INFILESIZE. This
allows the email to be rejected by the server, if supported, and the
maximum size has been configured on the server.
Removed the code that striped off the domain name when Curl_gethostname
returned the fully qualified domain name as the function has been
updated to return the un-qualified host name.
Replaced the use of HOSTNAME_MAX as the size of the buffer in the call
to Curl_gethostname with sizeof(host) as this is safer should the buffer
size ever be changed.
Allow (*curl_write_callback) write callbacks to return
CURL_WRITEFUNC_OUT_OF_MEMORY to properly indicate libcurl of OOM conditions
inside the callback itself.
If a socket is larger than FD_SETSIZE, avoid using FD_SET() on the
platforms where this is possible.
Bug: http://curl.haxx.se/bug/view.cgi?id=3413274
Reported by: Tim Starling
To avoid that the progress meter headers get output between each
transfer, make sure the bits gets kept when (re-)inited.
Reported by: Christopher Stone
I think curl should ignore this case and smtp.c should test for this.
Since RFC-2821 seems to allow a "null reverse-path". Ref. "MAIL
FROM:<>" in section 3.7, page 25.
Fixed Curl_gethostname() so that it always returns the un-qualified
machine name rather than being dependent on the socket provider.
Note: The return of getenv("CURL_GETHOSTNAME") is also parsed in case
the developer / test harness provided a fully qualified domain name as
it's value as well.
With this fix, it should work for PolarSSL-1.0.0 (and SVN-1091 trunk)
and retain compatibility with earlier versions. (Tested with 0.14.1)
PolarSSL still doesn't play nicely with curl's CA bundle (we discussed
this before) but I was at least able to retrieve the
https://www.gmail.com/ login page using a modified ca-certificates.crt
file with all 3 versions of PolarSSL.
Renamed the variable from 'proto' to 'level' simply because it is not
protocol you set but level and that is the name of the argument used in
man pages and the POSIX documentation of the setsockopt function.
This workarounds old libssh2 versions not properly initializing
some ssh session variables, which would trigger memory debuggers
warnings on memory being used without having been initialized.
The current version of speedcheck.c may disable timeout by setting zero
to Curl_expire. Which is fine using the curl_multi_perform, because it
recheck all timeout internals, but when using custom event poller (like
hiperfifo.c) it may keep stalle connection forever.
Calling sclose() both in the child and the parent fools the
socket leak detector into thinking it's been closed twice.
Calling close() in the child instead overcomes this problem. It's
not as portable as the sclose() macro, but this code is highly
POSIX-specific, anyway.
Just internal stuff...
Curl_safefree is now a macro defined in memdebug.h instead of a function
prototyped in url.h and implemented in url.c, so inclusion of url.h is no
longer required in order to simply use Curl_safefree.
Provide definition of macro WHILE_FALSE in setup_once.h in order to allow
other macros such as DEBUGF and DEBUGASSERT, and code using it, to compile
without 'conditional expression is constant' warnings.
The WHILE_FALSE stuff fixes 150+ MSVC compiler warnings.
Ensure existing logic in Curl_resolv_timeout() is not subverted upon getting a
negative timeout from resolve_server(). The timeout in resolve_server() could
be checked to avoid calling Curl_resolv_timeout() with an expired timeout, but
fixing this in this way allows existing logic in resolve_server() to be kept
unchanged.
Configure script option --enable-wb-ntlm-auth renamed to --enable-ntlm-wb
Configure script option --disable-wb-ntlm-auth renamed to --disable-ntlm-wb
Preprocessor symbol WINBIND_NTLM_AUTH_ENABLED renamed to NTLM_WB_ENABLED
Preprocessor symbol WINBIND_NTLM_AUTH_FILE renamed to NTLM_WB_FILE
Test harness env var CURL_NTLM_AUTH renamed to CURL_NTLM_WB_FILE
Static function wb_ntlm_close renamed to ntlm_wb_cleanup
Static function wb_ntlm_initiate renamed to ntlm_wb_init
Static function wb_ntlm_response renamed to ntlm_wb_response
Feature string literal NTLM_SSO renamed to NTLM_WB.
Preprocessor symbol USE_NTLM_SSO renamed to WINBIND_NTLM_AUTH_ENABLED.
curl's 'long' option 'ntlm-sso' renamed to 'ntlm-wb'.
Fix some comments to make clear that this is actually a NTLM delegation.
Fixed the order of the preferred SMTP authentication method to:
AUTH CRAM-MD5, AUTH LOGIN then AUTH PLAIN.
AUTH PLAIN should be the last as it slightly more insecure than AUTH LOGIN
as the username and password are sent together - there is no handshaking
between the client and server like there is with AUTH LOGIN.
Previous interfaces for these libcurl internal functions did not allow to tell
apart a legitimate zero size result from an error condition. These functions
now return a CURLcode indicating function success or otherwise specific error.
Output size is returned using a pointer argument.
All usage of these two functions, and others closely related, has been adapted
to the new interfaces. Relative error and OOM handling adapted or added where
missing. Unit test 1302 also adapted.
* Added function comments:
- Curl_ntlm_decode_type2_message
- Curl_ntlm_create_type1_message
- Curl_ntlm_create_type3_message
* Modification of ntlm processing state to NTLMSTATE_TYPE2 is now done
only when Curl_ntlm_decode_type2_message() has fully succeeded.
As a bonus, this lets our MemoryTracking subsystem track zlib operations.
And also fixes a shortcut some zlib 1.2.x versions took using malloc()
instead of calloc(), which would trigger memory debuggers warnings on
memory being used without having been initialized.
As I modified conn->bits.tcpconnect to become an array that holds one
bool for each potential connection all uses of that struct field must
index it correctly.
When using the multi interface, a SOCKS proxy, and a connection that
wouldn't immediately consider itself connected (which my Linux tests do
by default), libcurl would be tricked into doing _two_ connects to the
SOCKS proxy when it setup the data connection and then of course the
second attempt would fail miserably and cause error.
This problem is a regression that was introduced by commit
4a42e5cdaa that was introduced in the 7.21.7 release.
Bug: http://curl.haxx.se/mail/lib-2011-08/0199.html
Reported by: Fabian Keil
Until 2011-08-17 libcurl's Memory Tracking feature also performed
automatic malloc and free filling operations using 0xA5 and 0x13
values. Our own preinitialization of dynamically allocated memory
might be useful when not using third party memory debuggers, but
on the other hand this would fool memory debuggers into thinking
that all dynamically allocated memory is properly initialized.
As a default setting, libcurl's Memory Tracking feature no longer
performs preinitialization of dynamically allocated memory on its
own. If you know what you are doing, and really want to retain old
behavior, you can achieve this compiling with preprocessor symbols
CURL_MT_MALLOC_FILL and CURL_MT_FREE_FILL defined with appropriate
values.
"release-ssl-ssh2-zlib" and "debug-ssl-ssh2-zlib" are two new makefile
targets that build libcurl with MSVC and link with libssh2
Bug: http://curl.haxx.se/bug/view.cgi?id=3388920
Reported by: "kdekker"
Strict splitting of http_ntlm.[ch] may trigger 8 compiler warnings when
building with some compilers and strict compiler warnings enabled, depending
on other specific configuration options some could get triggered or not.
Seven are related with 'unused function parameters' and another one with
'var may be used before its value is set'.
For modularity purposes, huge chunks of NTLM existing code is transformed into
functions to allow future internal code reuse.
Resulting three new libcurl private functions:
- Curl_ntlm_create_type1_message()
- Curl_ntlm_create_type3_message()
- Curl_ntlm_decode_type2_message()
Changing static ntlm_sspi_cleanup() into non-static Curl_ntlm_sspi_cleanup()
This 'refactoring' has been prepared by previous commits to allow that this
specific one does not introduce any change to existing code. All existing
goodness and badness previous to this commit should remain the same once it is
applied, the only difference should be that existing code is moved into
functions.
Given the quite big portions of code being moved around, and the importance of
change traceability, this commit has been done in such a way that it is
possible to perform a three-way diff from initial http_ntlm.[ch] to resulting
http_ntlm.[ch] and curl_ntlm.[ch] to actually verify that no functional change
is introduced here.
Notice that Steve Holme has provided several patches, but these included this
refactoring along with 'extra' fixes. I really wanted this 'clean' refactoring
done first, in order to allow discussion or committing of 'extra' fixes on a
case by case basis, so, I had to bite the bullet ;-)
Comments, line adjustments, compiler warning fixes, whatever, may follow
afterwards.
Use preprocessor symbol NTLM_BUFSIZE to define private NTLM buffer fixed size.
Use a SessionHandle 'data' pointer variable to ease refactoring.
Update NTLM type-* message structure descriptions.
Fix some more spacing and typos (Steve Holme).
Moved NTLMSSP_SIGNATURE, HOSTNAME_MAX, SHORTPAIR and LONGQUARTET definitions in ready for move to curl_ntlm.c.
Used separate variables for Windows SSPI and native code to ease moving of code to curl_ntlm.c.
Fixed typographical erros where SPPI should be SSPI.
Fixed compilation warnings on 64-bit builds when calling Windows SSPI functions.
There are two keywords in cookie headers that don't follow the regular
name=value style: secure and httponly. Still we must support that they
are written like 'secure=' and then treat them as if they were written
'secure'. Test case 31 was much extended by Rob Ward to test this.
Bug: http://curl.haxx.se/bug/view.cgi?id=3349227
Reported by: "gnombat"
A regression where CURLFORM_BUFFER stopped to properly insert the file
name part in the formpart. Bug introduced in commit f851f76857.
Added CURLFORM_BUFFER use to test 554 to verify this.
Bug: http://curl.haxx.se/mail/lib-2011-07/0176.html
Reported by: Henry Ludemann
Allow configure builds to generate src/config-win32.h
Skip checksrc on src/config-win32.h contents already checked at lib/config-win32.h
Allow multiple -W in checksrc.pl
A proxy could be marked 'httpproxy' wrongly before if set with an
environment variable or with the CURLOPT_PROXY option with a socks*://
prefix or similar.
Added test 710 to verify
Bug: http://curl.haxx.se/mail/lib-2011-07/0194.html
Allow compilation of libcurl and curl using BSD-style lwIP on Win32.
In order to compile libcurl and curl with this support it is necessary
to edit files lib/config-win32.h and src/config-win32.h and uncomment
a line to make definition of preprocessor symbol USE_LWIPSOCK visible.
Afterwards you can compile as usual.
In order to use compiled library with BSD-style lwIP TCP/IP stack in
your program it is mandatory that you include lwIP header file <lwip/opt.h>
before including <curl/curl.h> or <curl/multi.h>
Compilation has been verified with lwIP 1.4.0 and contrib-1.4.0 from:
http://download.savannah.gnu.org/releases/lwip/lwip-1.4.0.ziphttp://download.savannah.gnu.org/releases/lwip/contrib-1.4.0.zip
Have fun!
Use preprocessor symbols WINBIND_NTLM_AUTH_ENABLED and WINBIND_NTLM_AUTH_FILE
for Samba's winbind daemon ntlm_auth helper code implementation and filename.
Retain preprocessor symbol USE_NTLM_SSO for NTLM single-sign-on feature
availability implementation independent.
For test harness, prefix NTLM_AUTH environment vars with CURL_
Refactor and rename configure option --with-ntlm-auth to --enable-wb-ntlm-auth[=FILE]
When an easy handle is used to download an URI which has no
Content-Length header (or equivalent) after downloading an URI which
does, the value from the previous transfer is reused and returned by
CURLINFO_CONTENT_LENGTH_DOWNLOAD. This is because the progress flags
(used to determine whether such a header was received) are not reset
between transfers.
Bug: http://curl.haxx.se/bug/view.cgi?id=3370895
When libcurl has said to the server that there's a POST or PUT coming
(with a content-length and all) it has to either deliver that amount of
data or it needs to close the connection before trying a second request.
Adds test case 1129, 1130 and 1131
The bug report is about when used with 100-continue, but the change is
more generic.
Bug: http://curl.haxx.se/mail/lib-2011-06/0191.html
Reported by: Steven Parkes
gssapi.h is used as a header name by Heimdal-style GSSAPI so it would
conflict with a private header using that name, and while renaming the
header I figured we should name the .c file accordingly as well.
Bug: http://curl.haxx.se/mail/lib-2011-07/0071.html
Reported by: Ben Greear
Modern gcc versions (4.6.X) get more picky by default and have started
to warn for unused parameters, but luckily gcc also allows us to mark
them as unused so that we can avoid the warnings.
adding unit test for Curl_llist_move, documenting unit-tested functions
in llist.c, changing unit-test to unittest, replacing assert calls with
abort_unless calls
The CURLFORM_STREAM is documented to only insert a file name (and thus
look like a file upload) in the part if CURLFORM_FILENAME is set, but in
reality it always inserted a filename="" and if CURLFORM_FILENAME wasn't
set, it would insert insert rubbish (or possibly crash).
This is now fixed to work as documented, and test 554 has been extended
to verify this.
Reported by: Sascha Swiercy
Bug: http://curl.haxx.se/mail/lib-2011-06/0070.html
Properly deal with the fact that the last fread() call most probably is
a short read, and when using callbacks in fact all calls can be short
reads. No longer consider a file read done until it returns a 0 from the
read function.
Reported by: Aaron Orenstein
Bug: http://curl.haxx.se/mail/lib-2011-06/0048.html
If a piece is set to use a callback to get the data, it should not be
treated as data. It unfortunately also requires that curl_easy_perform()
or similar has been used as otherwise the callback function hasn't been
figured out and curl_formget won't know how to get the content.
When closing a connection, the speedchecker's timestamp is now deleted
so that it cannot accidentally be used by a fresh connection on the same
handle when examining the transfer speed.
Bug: https://bugzilla.redhat.com/679709
When a time condition isn't met, so that no body is delivered to the
application even though a 2xx response is being read from the server, we
must close the connection to avoid a re-use of the connection to be
completely tricked.
Added test 1128 to verify.
When switching to HTTP because a HTTP proxy is being used, the existing
handler is now checked if it already is "compatible". This allows the https
handler remain while other non-http handlers will be redirected.
Bug: http://curl.haxx.se/mail/lib-2011-05/0214.html
Reported by: Jerome Robert
Fix compiler warning: `keycheck' might be used uninitialized in this function.
Fix compiler warning: `keybit' might be used uninitialized in this function.
Introduced the initial setup to allow closesocket callbacks by making
sure sclose() is only ever called from one place in the libcurl source
and still run all test cases fine.
The protocol handler's flags field now can set that the protocol
requires a password, so that the set_userpass function doesn't have to
have the specific knowledge of which protocols that do.
Made several functions static
Made one function defined to nothing when RTSP is disabled to avoid
the #ifdefs in code.
Removed explicit rtsp.h includes
Using 'socks5h' as proxy protocol will make it a
CURLPROXY_SOCKS5_HOSTNAME proxy which is SOCKS5 and asking the proxy to
resolve host names. I found no "standard" protocol name for this.
Introduce an INIT state for the SSH state machine and set libssh2
non-blocking in that so that it is set properly before
libssh2_session_startup() is called.
Bug: http://curl.haxx.se/mail/archive-2011-05/0001.html
Now use gai_strerror() to get proper error messages when getaddrinfo()
has failed. Detect the function in configure.
Code based on work and suggestions by Jeff Pohlmeyer and Guenter Knauf
Improved library search by check_function_exists_concat() macro:
it does not revert the list of libraries any more.
Improved OpenSSL library search: first find zlib, then search for
openssl libraries that may depend on zlib.
For Unix: openssl libraries can now be detected in nonstandard
locations. Supply CMAKE_LIBRARY_PATH to CMake on command line.
Added installation capability (very basic one yet).
When connecting to a socks or similar proxy we do the proxy handshake at
once when we know the TCP connect is completed and we only consider the
"connection" complete after the proxy handshake. This fixes test 564
which is now no longer considered disabled.
Reported by: Dmitri Shubin
Bug: http://curl.haxx.se/mail/lib-2011-04/0127.html
The make target checksrc now works in the root makefile and in both the
src and lib directories.
It is also run automatically on "all" if configure --enable-debug was
used.
It now scans multiple files and outputs an error+warning count summary
at the end in case at least one was detected.
-D can be used to specify in which dir the files are located
The script now scans for conditions that starts with a space for
if/while/for lines.
For now provide prototypes instead of including the
non-standard normalisation.h which is only available in the
"Internationalized Domain Names Mitigation APIs" download.
asyn-ares.c and asyn-thread.c are two separate backends that implement
the same (internal) async resolver API for libcurl to use. Backend is
specified at build time.
The internal resolver API is defined in asyn.h for asynch resolvers.
Fixed indents, coding conventions and white space edits.
Modified the c-ares completion callback function to again NOT read the
conn data when the ares handle is being taken down as then it may have
been freed already.
For now we directly import the Idn* symbols with the linker;
an upcoming release of OWC will have these added to the import
lib normaliz.lib, and prototypes are added to winnnls.h.
Make sure that files are closed before the post quote commands run as if
they operate on the just transferred file they could otherwise easily
fail.
Patch by: Rajesh Naganathan (edited)
libcurl failed to check the correct struct for HTTPS after CONNECT was
issued to the proxy, so it didn't do the TLS handshake and subsequently
failed the connection. A regression released in 7.21.5 (introduced
around commit 8831000bc0).
Bug: http://curl.haxx.se/mail/lib-2011-04/0134.html
Reported by: Josue Andrade Gomes
It is now possible to use any combination of features without
having to 1st add makefile targets to the main makefile. The
main makefile now passes the 'mingw32-feat1-feat2' as var CFG,
and the ./[lib|src]/Makefile.m32 parses the CFG var to determine
the features to be enabled.
changed windows.h include to system header;
changed obsolete 2nd check for str_w to str_utf8 in order to catch
malloc() failure and avoid a free(NULL);
changed calls to GetLastError() to void to kill unsused var compiler
warnings;
moved one call to GetLastError() into else case so that its only
called when WideCharToMultiByte() really fails.
Added CURLOPT_TRANSFER_ENCODING as the option to set to request Transfer
Encoding in HTTP requests (if built zlib enabled). I also renamed
CURLOPT_ENCODING to CURLOPT_ACCEPT_ENCODING (while keeping the old name
around) to reduce the confusion when we have to encoding options for
HTTP.
--tr-encoding is now the new command line option for curl to request
this, and thus I updated the test cases accordingly.
When TE: is inserted in the request, we must add a "Connection: TE" as
well to be HTTP 1.1 compliant. If a custom Connection: header is passed
in, we must use that and only append TE to it. Test case 1125 verifies
TE: + custom Connection:.
Since this struct member is used in the code to determine what and how
to decode automatically and since it is now also used for compressed
Transfer-Encodings, I renamed it to the more suitable 'auto_decoding'
Transfer-Encoding differs from Content-Encoding in a few subtle ways,
but primarily it concerns the transfer only and not the content so when
discovered to be compressed we know we have to uncompress it. There will
only arrive compressed transfers in a response after we have requested
them with the appropriate TE: header.
Test case 1122 and 1123 verify.
When checking if an existing RTSP connection is alive or not, the
checkconnection function might be called with a SessionHandle pointer
being NULL and then referenced causing a crash. This happened only using
the multi interface.
Reported by: Tinus van den Berg
Bug: http://curl.haxx.se/bug/view.cgi?id=3280739
In case a client certificate is used, invalidate SSL session cache
at the end of a session. This forces NSS to ask for a new client
certificate when connecting second time to the same host.
Bug: https://bugzilla.redhat.com/689031
* Rename the object object directory from 'objs' to 'BCC_obj'. I feel
it should be named properly. Ref. Makefile.Watcom where it's called
'WC_Win32.obj'.
* Turn off these warnings to keep the build totally silent (with CBuilder-6
that is).
-w-inl 8026 Functions X are not expanded inline.
-w-pia 8060 Possibly incorrect assignment
-w-pin 8061 Initialization is only partially bracketed
I'm sure the warnings could be fixed the "proper" way or with some added
"#pragma" statements. But that just clutters the sources IMHO.
* $(MKDIR) and $(RMDIR) have been replaced with the shell-commands 'md'
and 'rd'. When having MingW/Msys programs 'mkdir.exe' and 'rmdir.exe' in
$PATH, this confuses Borland's make and the result (the cleaning etc.) would
not be as expected.
* Added a ".path.int = $(OBJDIR)" to tell make where the $(PREPROCESSED)
files are. Why we need the preprocess step in the fist place is beyond me
(Yang?). But I'll leave that for now.
Stop the abuse of CURLE_FAILED_INIT as return code for things not being
init related by introducing two new return codes:
CURLE_NOT_BUILT_IN and CURLE_UNKNOWN_OPTION
CURLE_NOT_BUILT_IN replaces return code 4 that has been obsoleted for
several years. It is used for returning error when something is
attempted to be used but the feature/option was not enabled or
explictitly disabled at build-time. Getting this error mostly means that
libcurl needs to be rebuilt.
CURLE_FAILED_INIT is now saved and used strictly for init
failures. Getting this problem means something went seriously wrong,
like a resource shortage or similar.
CURLE_UNKNOWN_OPTION is the option formerly known as
CURLE_UNKNOWN_TELNET_OPTION (and the old name is still present,
separately defined to be removed in a very distant future). This error
code is meant to be used to return when an option is given to libcurl
that isn't known. This problem would mostly indicate a problem in the
program that uses libcurl.
In my attempts to reduce #ifdefs in code, the SOCKS functions are now
macros when libcurl is built without proxy support and therefore the FTP
code could avoid some #ifs.
The new http_proxy.* files now host HTTP proxy specific code (500+ lines
moved out from http.c), and as a consequence there is a macro introduced
for the Curl_proxyCONNECT() function so that code can use it without
actually supporting proxy (or HTTP) in builds.
1 - make sure to #define macros for cookie functions in the cookie
header when cookies are disabled to avoid having to use #ifdefs in code
using those functions.
2 - move cookie-specific code to cookie.c and use the functio
conditionally as mentioned in (1).
net result: 6 #if lines removed, and 9 lines of code less
Within multi_socket when conn is used as a shorthand, data could be
changed and multi_runsingle could modify the connectdata struct to deal
with. This bug has not been included in a public release.
Using 'conn' like that turned out to be ugly. This change is a partial
revert of commit f1c6cd42f4.
Reported by: Miroslav Spousta
Bug: http://curl.haxx.se/bug/view.cgi?id=3265485
When asked to bind the local end of a connection when doing a request,
the code will now disqualify other existing connections from re-use even
if they are connected to the correct remote host.
This will also affect which connections that can be used for pipelining,
so that only connections that aren't bound or bound to the same
device/port you're asking for will be considered.
The RTSP-specific function for checking for "dead" connection is better
located in rtsp.c. The code using this is now written without #ifdefs as
the function call is instead turned into a macro (in rtsp.h) when RTSP
is disabled.
Move ipv6-functional-probe into a single function that is used from all
places that need to know.
Make the probe function store the result in a static variable so that
subsequent invokes just returns the previous result and won't have to
probe again.
Curl_posttransfer is called too soon to add the final new line.
Moved the new line logic to pgrsDone as there is no more call to
update the progress status after this call.
Reported by: Dmitri Shubin <sbn_at_tbricks.com>
http://curl.haxx.se/mail/lib-2010-12/0162.html
When libcurl sends a HTTP request on a re-used connection and detects it
being closed (ie no data at all was read from it), it is important to
rewind if any data in the request was sent using the read callback or
was read from file, as otherwise the retried request will be broken.
Reported by: Chris Smowton
Bug: http://curl.haxx.se/bug/view.cgi?id=3195205
When NSS-powered libcurl connected to a SSL server with
CURLOPT_SSL_VERIFYPEER equal to zero, NSS remembered that the peer
certificate was accepted by libcurl and did not ask the second time when
connecting to the same server with CURLOPT_SSL_VERIFYPEER equal to one.
This patch turns off the SSL session cache for the particular SSL socket
if peer verification is disabled. In order to avoid any performance
impact, the peer verification is completely skipped in that case, which
makes it even faster than before.
Bug: https://bugzilla.redhat.com/678580
The PROT_* set of internal defines for the protocols is no longer
used. We now use the same bits internally as we have defined in the
public header using the CURLPROTO_ prefix. This is for simplicity and
because the PROT_* prefix was already used duplicated internally for a
set of KRB4 values.
The PROTOPT_* defines were moved up to just below the struct definition
within which they are used.
The protocol handler struct got a 'flags' field for special information
and characteristics of the given protocol.
This now enables us to move away central protocol information such as
CLOSEACTION and DUALCHANNEL from single defines in a central place, out
to each protocol's definition. It also made us stop abusing the protocol
field for other info than the protocol, and we could start cleaning up
other protocol-specific things by adding flags bits to set in the
handler struct.
The "protocol" field connectdata struct was removed as well and the code
now refers directly to the conn->handler->protocol field instead. To
make things work properly, the code now always store a conn->given
pointer that points out the original handler struct so that the code can
learn details from the original protocol even if conn->handler is
modified along the way - for example when switching to go over a HTTP
proxy.
The non-blocking connect improvement for IMAP showed that we didn't
properly define the Curl_ssl_connect_nonblocking function for non-SSL
builds.
Reported by: Tor Arntsen
Only download and convert the certdata to the ca-bundle.crt if Mozilla
changed the data
The Perl LWP module (which in a bit of a circular reference is used by
mk-ca-bundle.pl) is now indirectly using this script. I made this small
tweak to make it easier to automatically maintain the generated
ca-bundle.crt file in version control.
Some protocols have to call the underlying functions without regard to
what exact state the socket signals. For example even if the socket says
"readable", the send function might need to be called while uploading,
or vice versa. This is the case for libssh2 based protocols: SCP and
SFTP and we now introduce a define to set those protocols and we make
the multi interface code aware of this concept.
This is another fix to make test 582 run properly.
As a new state recently was added to the IMAP state machine it has to be
in the array of names as well as otherwise libcurl crashes when a debug
version runs...
For uploads we want to use the _sending_ function even when the socket
turns out readable as the underlying libssh2 sftp send function will
deal with both accordingly. This is what the cselect_bits magic is for.
Fixes test 582.
Make GSS authentication work when a curl handle is reused for multiple
authenticated requests, by always setting negdata->state in
output_auth_headers().
Signed-off-by: Marcus Sundberg <marcus.sundberg@aptilo.com>
When using the multi interface and a handle using SFTP was removed very
early on, we would get a segfault due to the code assumed data was there
that hadn't yet been setup.
Bug: http://curl.haxx.se/mail/lib-2011-03/0066.html
Reported by: Saqib Ali
Both SFTP and SCP are protocols that need to shut down stuff properly
when the connection is about to get torned down. The primary effect of
not doing this shows up as memory leaks (when using SCP or SFTP with the
multi interface).
This is one of the problems detected by test 582.
As we know how much to send, we can and should stop once we've sent that
much data as it avoids having to rely on other mechanisms to detect the
end.
This is one of the problems detected by test 582.
Reported by: Henry Ludemann <misc@hl.id.au>
When using the multi_socket API to do SFTP upload, it is important that
we set a quick expire when leaving the SSH_SFTP_UPLOAD_INIT state as
there's nothing happening on the socket so there's no read or write to
wait for, but the next libssh2 API function needs to be called to get
the ball rolling.
This is one of the problems detected by test 582.
Reported by: Henry Ludemann <misc@hl.id.au>
All C and H files now (should) feature the proper project curl source
code header, which includes basic info, a copyright statement and some
basic disclaimers.
CyaSSL (available from git@github.com:cyassl/cyassl.git) has been
added to the SSL abstraction layer.
To test:
1) git CyaSSL sources
2) autoreconf -i
3) ./configure --disable-static
4) make
5) sudo make install
6) autoreconf -i
7) git curl sources (and this patch)
8) ./configure --disable-shared --with-cyassl --without-ssl --enable-debug
9) make
10) normal testing
Please send questions or comments to todd@yassl.com .
libssh2_knownhost_readfile() returns a negative value on error or
otherwise number of parsed known hosts - this was previously not
documented correctly in the libssh2 man page for the function.
Bug: http://curl.haxx.se/mail/lib-2011-02/0327.html
Reported by: murat
Removed the "netrc_debug" keyword replaced with --netrc-file additions.
Removed the debug code from Curl_parsenetrc as it is superseeded by
--netrc-file.
After a request times out, the connection wasn't properly closed and
prevented to get re-used, so subsequent transfers could still mistakenly
get to use the previously aborted connection.
When failing to connect the protocol during the CURLM_STATE_PROTOCONNECT
state, Curl_done() has to be called with the premature flag set TRUE as
for the pingpong protocols this can be important.
When Curl_done() is called with premature == TRUE, it needs to call
Curl_disconnect() with its 'dead_connection' argument set to TRUE as
well so that any protocol handler's disconnect function won't attempt to
use the (control) connection for anything.
This problem caused the pingpong protocols to fail to disconnect when
STARTTLS failed.
Reported by: Alona Rossen
Bug: http://curl.haxx.se/mail/lib-2011-02/0195.html
Introducing a few CURL_SOCKOPT* defines for conveniance. The new
CURL_SOCKOPT_ALREADY_CONNECTED signals to libcurl that the socket is to
be treated as already connected and thus it will skip the connect()
call.
It turns out some systems rely on the gmtime or gmtime_r to be defined
already in the system headers and thus my "precaution" redefining of
them only caused trouble. They are now removed.
On second thought, I think CURLE_TLSAUTH_FAILED should be eliminated. It
was only being raised when an internal error occurred while allocating
or setting the GnuTLS SRP client credentials struct. For TLS
authentication failures, the general CURLE_SSL_CONNECT_ERROR seems
appropriate; its error string already includes "passwords" as a possible
cause. Having a separate TLS auth error code might also cause people to
think that a TLS auth failure means the wrong username or password was
entered, when it could also be a sign of a man-in-the-middle attack.
When the callback returns an error, this function must make sure to return
CURLE_ABORTED_BY_CALLBACK properly and not CURLE_OK as before to allow the
callback to properly abort the operation.
The main has not been updated from some time and is out of sync with
the code. The code is now tested by several test cases so no need for
a seperate code path.
Instead of polluting many places with #ifdefs, we create a single place
for this function, and also check return code properly so that a NULL
pointer returned won't cause problems.
The official Mozilla page at http://www.mozilla.org/projects/security/certs/
points out a new place as the "proper" place to get Mozilla's CA certs from
so this script is now updated to use that instead.
Reported by: Daniel Mentz
The official Mozilla page at
http://www.mozilla.org/projects/security/certs/ points out a new place
as the "proper" place to get Mozilla's CA certs from so this script is
now updated to use that instead.
Reported by: Daniel Mentz
The code in the toofast state needs to first recalculate the values
before it uses them again since it may have been a while since it last
did it when it reaches this point.
This will be used by file_do() and Curl_readwrite() as a unified method
of checking to see if a remote document meets the supplied
CURLOPT_TIMEVAL and CURLOPT_TIMECONDITION.
Signed-off-by: Dave Reisner <d@falconindy.com>
When this callback is called due to the destruction of the ares handle,
the connection pointer passed in as an argument may no longer pointing
to valid data and this function doesn't need to do anything with it
anyway so we make sure it doesn't.
Bug: http://curl.haxx.se/mail/lib-2011-01/0333.html
Reported by: Vsevolod Novikov
The HTTP parser allocated memory on each received Location: header
without properly freeing old data. Starting now, the code only considers
the first Location: header and will blissfully ignore subsequent ones.
Bug: http://curl.haxx.se/bug/view.cgi?id=3165129
Reported by: Martin Lemke
... and update the curl.1 and curl_easy_setopt.3 man pages such that
they do not suggest to use an OpenSSL utility if curl is not built
against OpenSSL.
Bug: https://bugzilla.redhat.com/669702
The idea that the protocol and socktype is part of name resolving in the
libc functions is nuts. We keep the name resolver functions assume
TCP/STREAM and we make sure that when we want to connect to a UDP
service we use the correct UDP/DGRAM set instead. This bug was because
the ->protocol field was not always set correctly.
This bug was only affecting ipv6-disabled non-cares non-threaded builds.
Bug: http://curl.haxx.se/bug/view.cgi?id=3154436
Reported by: "dperham"
When configure --enable-debug has been used, all files in lib/ are now
built twice and a separate static library crafted for unit-testing will
be linked. The unit tests in the tests/unit subdir will use that
library.
Since some systems don't have PATH_MAX and it isn't that clever to
assume a fixed maximum path length, the code now allocates buffer space
instead of using stack.
Reported by: Samuel Thibault
Bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=608521
Sending "pwd" as a QUOTE command only sent the reply to the
DEBUGFUNCTION. Now it also sends an FTP-like header to the header
callback to allow similar operations as with FTP, and apps can re-use
the same parser.
When built IPv6-enabled, we could do Curl_done() with one of the two
resolves having returned already, so when ares_cancel() is called the
resolve callback ends up doing funny things (sometimes resulting in a
segfault) since it would try to actually store the previous resolve even
though we're shutting down the resolve.
This bug was introduced in commit 8ab137b2bc so it hasn't been
included in any public release.
Bug: http://curl.haxx.se/bug/view.cgi?id=3145445
Reported by: Pedro Larroy
Providing multiple dots in a series in the domain field (domain=..com) could
trick the cookie engine to wrongly accept the cookie believing it to be
fine. Since the tailmatching would then match all .com sites, the cookie would
then be sent to all of them.
The code now requires at least one letter between each dot for them to be
counted. Edited test case 61 to verify this.
When using the multi interface and connecting to a host name that
resolves to multiple IP addresses, there was no logic that made it
continue to the next IP if connecting to the first address times
out. This is now corrected.
The info about pipe status and expire cleared are clearly debug-related
and not anything mere mortals will or should care about so they are now
ifdef'ed DEBUGBUILD
Similar to what is done already for RCPT TO, the code now checks for and
adds angle brackets (<>) around the email address that is provided for
CURLOPT_MAIL_RCPT unless the app has done so itself.
Make sure that Curl_cache_addr() errors are propagated to callers of
loadhostpairs().
(this loadhostpairs function caused a scan-build warning due to the
'dns' variable getting assigned but never used)
Doing curlx_strtoofft() on the size just to figure out the end of it
causes a compiler warning since the result wasn't used, but is also a
bit of a waste.
Since the original `conn' pointer was used after the `connectdata' it
points to has been closed/cleaned up by Curl_reconnect_request it caused
a crash. We must make sure to use the newly created connection instead!
URL: http://curl.haxx.se/mail/lib-2010-12/0202.html
Make the c-ares resolver code ask for both IPv4 and IPv6 addresses when
IPv6 is enabled.
This is a workaround for the missing ares_getaddrinfo() and is a lot
easier to implement.
Note that as long as c-ares returns IPv4 addresses when IPv6 addresses
were requested but missing, this will cause a host's IPv4 addresses to
occur twice in the DNS cache.
URL: http://curl.haxx.se/mail/lib-2010-12/0041.html
The SSL_SERVER_VERIFY_LATER bit in the ssl_ctx_new() call allows the
code to verify the peer certificate explicitly after the handshake and
then the "data->set.ssl.verifypeer" option works.
The public axTLS header (at least as of 1.2.7) redefines the memory
functions. We #undef those again immediately after the public header to
limit the damage. This should be fixed in axTLS.
Failed HTTPS tests: 301, 306, 311, 312, 313, 560
311, 312 need more detailed error reporting from axTLS.
313 relates to CRL, which hasn't been implemented yet.
Added axTLS to autotool files and glue code to misc other files.
axtls.h maps SSL API functions, but may change.
axtls.c is just a stub file and will definitely change.
The function that checks if pipelining is possible now requires the HTTP
bit to be set so that it doesn't mistakenly tries to do it for other
protocols.
Bug: http://curl.haxx.se/mail/lib-2010-12/0152.html
Reported by: Dmitri Shubin
The generic timeout code must not check easy handles that are already
completed. Going to completed (again) within there risked decreasing the
number of alive handles again and thus it could go negative.
This regression bug was added in 7.21.2 in commit ca10e28f06
ossl_connect_common() now checks whether or not 'struct
connectdata->state' is equal 'ssl_connection_complete' and if so, will
return CURLE_OK with 'done' set to 'TRUE'. This check prevents
ossl_connect_common() from creating a new ssl connection on an existing
ssl session which causes openssl to fail when it tries to parse an
encrypted TLS packet since the cipher data was effectively thrown away
when the new ssl connection was created.
Bug: http://curl.haxx.se/mail/lib-2010-11/0169.html
It helps to prevent a hangup with some FTP servers in case idle session
timeout has exceeded. But it may be useful also for other protocols
that send any quit message on disconnect. Currently used by FTP, POP3,
IMAP and SMTP.
When looping in this function and checking for the timeout being
expired, it was not updating the reference time when calculating the
timediff since previous round which made it think each subsequent loop
to have taken longer than it actually did.
I also modified the function to use the generic Curl_timeleft() function
instead of the custom logic.
Bug: http://curl.haxx.se/bug/view.cgi?id=3112579
Ensure that spurious results from system's getaddrinfo() ares not propagated
by Curl_getaddrinfo_ex() into the library.
Also ensure that the ai_addrlen member of Curl_getaddrinfo_ex()'s output linked
list of Curl_addrinfo structures has appropriate family-specific address size.
On Windows, translate WSAGetLastError() to errno values as GNU
TLS does it internally, too. This is necessary because send() and
recv() on Windows don't set errno when they fail but GNU TLS
expects a proper errno value.
Bug: http://curl.haxx.se/bug/view.cgi?id=3110991
When no timeout is set, we call the socket_ready function with a timeout
value of 0 during handshake, which makes it loop too much/fast in this
function. It also made this function return CURLE_OPERATION_TIMEDOUT
wrongly on a slow handshake.
However, the particular bug report that highlighted this problem is not
solved by this fix, as this fix only makes the more proper error get
reported instead.
Bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=594150
Reported by: Johannes Ernst
While changing Curl_sec_read_msg to accept an enum protection_level
instead of an int, I went ahead and fixed the usage of the associated
fields.
Some code was assuming that prot_clear == 0. Fixed those to use the
proper value. Added assertions prior to any code that would set the
protection level.
This is the advised way of checking for errors in the GSS-API RFC.
Also added some '\n' to the error message so that they are not mixed
with other outputs.
The IP version choice was previously only in the UserDefined struct
within the SessionHandle, but since we sometimes alter that option
during a request we need to have it on a per-connection basis.
I also moved more "init conn" code into the allocate_conn() function
which is designed for that purpose more or less.
CURLOPT_RESOLVE is a new option that sends along a curl_slist with
name:port:address sets that will populate the DNS cache with entries so
that request can be "fooled" to use another host than what otherwise
would've been used. Previously we've encouraged the use of Host: for
that when dealing with HTTP, but this new feature has the added bonus
that it allows the name from the URL to be used for TLS SNI and server
certificate name checks as well.
This is a first change. Surely more will follow to make it decent.
If the query result has a binary attribute, the binary attribute is
base64 encoded. But all following non binary attributes are also base64
encoded which is wrong.
This is a test (LDAP server is public).
curl
ldap://x500.bund.de:389/o=Bund,c=DE?userCertificate,certificateSerialNumber?sub
?cn=*Woehleke*
If you use a custom Host: name in a request to a SSL server, libcurl
will now use that given name when it verifies the server certificate to
be correct rather than using the host name used in the actual URL.
When given a custom host name in a Host: header, we can use it for
several different purposes other than just cookies, so we rename it and
use it for SSL SNI etc.
Some FTP servers (e.g. Pure-ftpd) end up hanging if we close the data
connection before transferring all the requested data. If we send ABOR
in that case, it prevents the server from hanging.
Bug: https://bugzilla.redhat.com/643656
Reported by: Pasi Karkkainen, Patrick Monnerat
These haven't worked in at least 8 years due to missing source
files, and most active RiscOS developers these days apparently
cross-compile anyway.
Signed-off-by: James Bursa <james@zamez.org>
In libssh2 1.2.8, libssh2_session_handshake() replaces
libssh2_session_startup() to fix the previous portability problem with
the socket type that was too small for win64 and thus easily could cause
crashes and more.
It is a bad idea to use the public prefix used by another library and
now we realize that libssh2 introduces a symbol in the upcoming version
1.2.8 that conflicts with our static function named libssh2_free.
When failing to build form post due to an error, the code now does a
proper failf(). Previously libcurl would report an error like "failed
creating formpost data" when a file wasn't possible to open which was
not easy for users to figure out.
I also lower cased a function name to be named more curl-style and
removed some unnecessary code.
The URL parser got a little stricter as it now considers a ? to be a
host name divider so that the slightly sloppier URLs work too. The
problem that made me do this change was the reported problem with an URL
like: www.example.com?email=name@example.com This form of URL is not
really a legal URL (due to the missing slash after the host name) but is
widely accepted by all major browsers and libcurl also already accepted
it, it was just the '@' letter that triggered the problem now.
The side-effect of this change is that now libcurl no longer accepts the
? letter as part of user-name or password when given in the URL, which
it used to accept (and is tested in test 191). That letter is however
mentioned in RFC3986 to be required to be percent encoded since it is
used as a divider.
Bug: http://curl.haxx.se/bug/view.cgi?id=3090268
In order to avoid for example the pingpong protocols to issue STARTTLS
(or equivalent) even though there's no SSL support built-in.
Reported by: Sune Ahlgren
Bug: http://curl.haxx.se/mail/archive-2010-10/0045.html
As the change in 5f0ae7a062 added a precaution against negative
file sizes that for some reason managed to get returned, this change now
introduces the same check at the second place in the code where the file
size from the libssh2 stat call is used.
This check might not be suitable for a 32 bit curl_off_t, but libssh2.h
assumes long long to work and to be 64 bit so I believe such a small
curl_off_t will be very unlikely to occur in the wild.
Renamed SDK_* to NDK_*; made NDK_* defines overwriteable from
environment; removed now obsolete YACC macro;
moved some curl_config.h defines to IPv6 section since they
are only needed when IPv6 is enabled - this makes libcurl compile
with older NDKs too which were not IPv6-aware.
We forgot to release the buffer passed to gss_init_sec_context.
The previous logic was difficult to read as we were reusing the same
variable (gssbuf) for both input buffer and output buffer. Splitted the
logic in 2 variables to better underline who needs to be released.
Also made the code break at 80 lines.
This fixes a memory leak related to the GSS-API code.
Added a krb5_init and krb5_end functions. Also removed a work-around
the lack of proper initialization of the GSS-API context.
It was pointed out that the special case libcurl did for 416 was
incorrect and wrong. 416 is not really different to other errors so the
response body must be handled like for other errors/http responses.
Reported by: Chris Smowton
Bug: http://curl.haxx.se/bug/view.cgi?id=3076808
It is still not clarified exactly why this happens, but libssh2
sometimes report a negative file size for the remote SFTP file and that
deeply confuses libcurl (or crashes it) so this precaution is added to
avoid badness.
Reported by: Ernest Beinrohr
Bug: http://curl.haxx.se/bug/view.cgi?id=3076430
Remove a leak seen on Kerberos/MIT (gss_OID is copied internally and
we were leaking it). Now we just pass NULL as advised in RFC2744.
|tmp| was never set back to buf->data.
Cleaned up Curl_sec_end to take into account failure in Curl_sec_login
(where conn->mech would be NULL but not conn->app_data or
conn->in_buffer->data).
Following a change in the way socket handler are registered, the custom
recv and send method were conditionaly registered.
We need to register them everytime to handle the ftp security
extensions.
Re-added the clear text handling in sec_recv.
Curl_sec_login was returning the opposite result that the code in ftp.c
was expecting. Simplified the return code (using a CURLcode) so to see
more clearly what is going on.
The functions Curl_disconnect() and Curl_done() are both used within the
scope of a single request so they cannot be allowed to use
Curl_expire(... 0) to kill all timeouts as there are some timeouts that
are set before a request that are supposed to remain until the request
is done.
The timeouts are now instead cleared at curl_easy_cleanup() and when the
multi state machine changes a handle to the complete state.
The date format in RFC822 allows that the seconds part of HH:MM:SS is
left out, but this function didn't allow it. This change also includes a
modified test case that makes sure that this now works.
Reported by: Matt Ford
Bug: http://curl.haxx.se/bug/view.cgi?id=3076529
tftpd-hpa has a bug where it will send an incorrect ack when the block
counter wraps and tftp options have been sent. Work around that by
accepting an ack for 65535 when we're expecting one for 0.
- |fd| is now a curl_socket_t and |len| a size_t to avoid conversions.
- Added 2 FIXMEs about the 2 unsigned -> signed conversions.
- Included 2 minor changes to Curl_sec_end.
- Renamed it to do_sec_send as it is the function doing the actual
transfer.
- Do not return any values as no one was checking it and it never
reported a failure (added a FIXME about checking for errors).
- Renamed the variables to make their use more specific.
- Removed some casts (int -> curl_socket_t, ...)
- Avoid doing the htnl <-> nthl twice by caching the 2 results.
- Renamed the variables name to better match their intend.
- Unified the |decoded_len| checks.
- Added some FIXMEs to flag some improvement that did not go in this
change.
- Removed sec_prot_internal as it is now inlined in the function (this removed
a redundant check).
- Changed the prototype to return an error code.
- Updated the method to use the new ftp_send_command function.
- Added a level_to_char helper method to avoid relying on the compiler's
bound checks. This default to the maximum security we have in case of a
wrong input.
Tighten the type of the |data| parameter to avoid a cast. Also made
it const as we should not modify it.
Added a DEBUGASSERT on the size to be written while changing it.
To do so, made block_read call Curl_read_plain instead of read.
While changing them renamed block_read to socket_read and sec_get_data
to read_data to better match their function.
Also fixed a potential memory leak in block_read.
Obviously, browsers ignore a colon without a following port number. Both
Firefox and Chrome just removes the colon for such URLs. This change
does not remove the colon for URLs sent over a HTTP proxy, so we should
consider doing that change as well.
Reported by: github user 'kreshano'
curl_easy_duphandle() was not properly duping the ares channel. The
ares_dup() function was introduced in c-ares 1.6.0 so by starting to use
this function we also raise the bar and require c-ares >= 1.6.0
(released Dec 9, 2008) for such builds.
Reported by: Ning Dong
Bug: http://curl.haxx.se/mail/lib-2010-08/0318.html
If built without HTTP or proxy support it would cause a compiler warning
due to the unused variable. I moved the declaration of it into the only
scope it is used.
bool_false is the internal name used in the setup_once.h definition
we fall back to for non-C99 non-stdbool systems, it's not the actual
name to use in assignments (we use bool_false, bool_true there to
avoid global namespace problems, see comment in setup_once.h).
The correct C99 value to use is 'false', but let's use FALSE as
used elsewhere when assigning to bits.close. FALSE is set equal
to 'false' in setup_once.h when possible.
This fixes a build problem on C99 targets.
As of curl-7.21.1 tunnelling ldap queries through HTTP Proxies is not
supported. Actually if --proxytunnel command-line option (or equivalent
CURLOPT_HTTPPROXYTUNNEL) is used for ldap queries like
ldap://ldap.my.server.com/... You are unable to successfully execute the
query. In facts ldap_*_bind is executed directly against the ldap server
and proxy is totally ignored. This is true for both openLDAP and
Microsoft LDAP API.
Step to reproduce the error:
Just launch "curl --proxytunnel --proxy 192.168.1.1:8080
ldap://ldap.my.server.com/dc=... "
This fix adds an invocation to Curl_proxyCONNECT against the provided
proxy address and on successful "CONNECT" it tunnels ldap query to the
final ldap server through the HTTP proxy. As far as I know Microsoft
LDAP APIs don't permit tunnelling in any way so the patch provided is
for OpenLDAP only. The patch has been developed against OpenLDAP 2.4.23
and has been tested with Microsoft ISA Server 2006 and works properly
with basic, digest and NTLM authentication.
Rodric provide an awesome recipe that proved libcurl didn't timeout at
the requested time - it instead often timed out at [connect time] +
[timeout time] instead of the documented and intended [timeout time]
only. This bug was due to the code using the wrong base offset when
comparing against "now". I could also take the oppurtinity to simplify
the code by properly using of the generic help function for this:
Curl_timeleft.
Reported by: Rodric Glaser
Bug: http://curl.haxx.se/bug/view.cgi?id=3061535
As this function uses return code 0 to mean that there is no timeout, it
needs to check that it doesn't return a time left value that is exactly
zero. It could lead to libcurl doing an extra 1000 ms select() call and
thus not timing out as accurately as it should.
I fell over this bug when working on the bug 3061535 but this fix does
not correct that problem alone, although this is a problem that needs to
be fixed.
Reported by: Rodric Glaser
Bug: http://curl.haxx.se/bug/view.cgi?id=3061535
The timeout is set for the connect phase already at the start of the
request so we should not add a new one, and we MUST not set expire to 0
as that will remove any other potentially existing timeouts.
The code reading chunked encoding attempts to rewind the code if it had
read more data than the chunky parser consumes. The rewinding can fail
and it will then cause an error. This change now makes the rewinding
only happen if pipelining is in use - as that's the only time it really
needs to be done.
Bug: http://curl.haxx.se/mail/lib-2010-08/0297.html
Reported by: Ron Parker
Curl_getconnectinfo() is changed to return a proper curl_socket_t for
the last socket so that it'll work more portably (and cause less
compiler warnings).
Add a timeout check for handles in the state machine so that they will
timeout in all states disregarding what actions that may or may not
happen.
Fixed a bug in socket_action introduced recently when looping over timed
out handles: it wouldn't assign the 'data' variable and thus it wouldn't
properly take care of handles.
In the update_timer function, the code now checks if the timeout has
been removed and then it tells the application. Previously it would
always let the remaining timeout(s) just linger to expire later on.
Each easy handle has a list of timeouts, so as soon as the main timeout
for a handle expires, we must make sure to get the next entry from the
list and re-add the handle to the splay tree.
This was attempted previously but was done poorly in my commit
232ad6549a.
When a new transfer is about to start we now set the proper timeouts to
expire for the multi interface if they are set for the handle. This is a
follow-up bugfix to make sure that easy handles timeout properly when
the times expire and the multi interface is used. This also improves
curl_multi_timeout().
The fix for the busyloop really only is a temporary work-around. It
causes a BLOCKING behavior which is a NO-NO. This function should rather
be split up in a do and a doing piece where the pieces that aren't
possible to send now will be sent in the doing function repeatedly until
the entire request is sent.
HTTP allows that a server sends trailing headers after all the chunks
have been sent WITHOUT signalling their presence in the first response
headers. The "Trailer:" header is only a SHOULD there and as we need to
handle the situation even without that header I made libcurl ignore
Trailer: completely.
Test case 1116 was added to verify this and to make sure we handle more
than one trailer header properly.
Reported by: Patrick McManus
Bug: http://curl.haxx.se/bug/view.cgi?id=3052450
The script works exactly same as the Perl one except for one thing:
when the text descriptions generated with openssl are included then
the md5 fingerprints are missing; seems openssl has either a bug or
a feature which prints the md5 fingerprint output to stdout instead
of writing them to specified file; this script could here do the same
as what the Perl scripr does (redirect stdout into file) but this
makes the script take up double the time because it needs to launch
cmd.exe 140 times (fo each openssl call). So I think for now we just
ommit the md5 fingerprints, and see if openssl will be fixed.
I fell over this bug report that mentioned that libcurl could wrongly
send more than one complete messages at the end of a transfer. Reading
the code confirmed this, so I've added a new multi state to make it not
happen. The mentioned bug report was made by Brad Jorsch but is (oddly
enough) filed in Debian's bug tracker for the "wmweather+" tool.
Bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=593390
There's an error in http_negotiation.c where a mistake is using only
userpwd even for proxy requests. Ludek provided a patch, but I decided
to write the fix slightly different using his patch as inspiration.
Reported by: Ludek Finstrle
Bug: http://curl.haxx.se/bug/view.cgi?id=3046066
When detecting that the send or recv speed, the multi interface changes
state to TOOFAST and previously there was no timeout set that would
force a recheck but it would rely on the application to somehow call
libcurl anyway. This now sets a timeout for a suitable future time to
check again if the average transfer speed is then below the threshold
again.
Curl_expire() is now expanded to hold a list of timeouts for each easy
handle. Only the closest in time will be the one used as the primary
timeout for the handle and will be used for the splay tree (which sorts
and lists all handles within the multi handle).
When the main timeout has triggered/expired, the next timeout in time
that is kept in the list will be moved to the main timeout position and
used as the key to splay with. This way, all timeouts that are set with
Curl_expire() internally will end up as a proper timeout. Previously any
Curl_expire() that set a _later_ timeout than what was already set was
just silently ignored and thus missed.
Setting Curl_expire() with timeout 0 (zero) will cancel all previously
added timeouts.
Corrects known bug #62.
Instead of looping over all attached easy handles, this now keeps a list
of messages in the multi handle. It allows curl_multi_info_read() to
perform O(1) no matter how many easy handles that are handled. This is
of importance since this function may be polled very frequently by apps
using the multi interface.
When the progress callback is called during the TCP connection, an error
return would accidentally not abort the operation as intended but would
instead be counted as a failure to connect to that particular IP and
libcurl would just continue to try the next. I made singleipconnect()
and trynextip() return CURLcode properly.
Added bonus: it corrected the error code for bad --interface usages,
like tested in test 1084 and test 1085.
Reported by: Adam Light
Bug: http://curl.haxx.se/mail/lib-2010-08/0105.html
Added the -br switch to dynamic builds which fixes the issue I saw
with curl's --version output. Added debug info and symfile for debug
builds to linker opts. Added DLL loader for wlink back, but this time
dependend on wlink version.
Patch posted to the list by malak.jiri AT gmail.com.
The var %MAKEFLAGS is only set in 3 cases: if set as environment
var or as macro definition from commandline, and either with the
-u or -ms switch. Since all these cases are unlikely for the average
user it should be safe to only test if %MAKEFLAGS is defined; this
has the benefit that now all other switches can be used again in
addition to the -u which was formerly not possible.
Curl_llist_init is never used outside of llist.c and thus it should be
static. I also removed the protos for Curl_llist_insert_prev and
Curl_llist_remove_next which are functions we removed from llist.c ages
ago.
Test 563 is enabled now and verifies that the combo FTP type=A URL,
CURLOPT_PORT set and proxy work fine. As a bonus I managed to remove the
somewhat odd FTP check in parse_remote_port() and instead converted it
to a better and more generic 'slash_removed' struct field. Checking the
->protocol field isn't right since when an FTP:// URL is sent over a
HTTP proxy, the protocol is HTTP but the URL was handled by the FTP code
and thus slash_removed is set TRUE for this case.
The struct used for storing the message for a completed transfer is now
no longer allocated separatly but is kept within the main struct kept
for each easy handle so that we avoid one malloc (and the subsequent
free).
When libcurl internally decided to wait for a 100-continue header, there
was no call to the timeout function so there was no timeout callback
called when the multi_socket API was used and thus applications became
either completely wrong or at least ineffecient depending on how they
handled the situation. We now set a timeout to get triggered.
Reported by: Ben Darnell
Bug: http://curl.haxx.se/bug/view.cgi?id=3039744
libssh2 1.2.6 and later handle >32bit file sizes properly even on 32bit
architectures and we make sure to use that ability.
Reported by: Mikael Johansson
Bug: http://curl.haxx.se/mail/lib-2010-08/0052.html
Simply because the TCP might be connected already we cannot skip the
proxy connect procedure. We need to be careful to not overload more
meaning to the bits.tcpconnect field like this.
With this fix, SOCKS proxies work again when the multi interface is
used. I believe this regression was added with commit 4b351d018e,
released as 7.20.1.
Left todo: add a test case that verifies this functionality that
prevents us from breaking it again in the future!
Reported by: Robin Cornelius
Bug: http://curl.haxx.se/bug/view.cgi?id=3033966
Commit 496002ea1c (released in 7.20.1) broke FTPS when using the
multi interface and OpenSSL was used. The condition for the non-blocking
connect was incorrect.
Reported by: Georg Lippitsch
Bug: http://curl.haxx.se/mail/lib-2010-07/0270.html
Previously the host name buffer was only used if gethostname() exists,
but since we converted that into a curl private function that function
always exists and will be used so the buffer needs to exist for all
cases/systems.
A shared library tests/libtest/.libs/lihostname.so is preloaded in NTLM
test-cases to override the system implementation of gethostname(). It
makes it possible to test the NTLM authentication for exact match, and
this way test the implementation of MD4 and DES.
If LD_PRELOAD doesn't work, a debug build willl also workk as debug
builds are now made to prefer a specific environment variable and will
then return that content as host name instead of the actual one.
Kamil wrote the bulk of this, Daniel Stenberg polished it.
lib/Makefile.Watcom works fine already, for src/Makefile.Watcom we
need first to tweak src/Makefile.inc a bit - therefore the handtweaked
list still exists for now.
- make both libcurl and curl makefiles use register calling convention
(previously libcurl had stack calling convention).
- added include paths to the Watcom headers so its no longer required
to set the environment vars for this.
- added -wcd=201 to supress compiler warning about unreachable code.
- use macros for all tools, and removed dependency on GNU tools like rm.
- make ipv6 and debug builds controlable via env vars and so make them
optional instead of default.
- commented WINLDAPAPI and WINBERAPI since they broke with OW 1.8, and
it seems they're not needed (anymore?).
- added rule for hugehelp.c.cvs so that it will be created when not
already exist - this is required for building from a release tarball
since there we have no hugehelp.c.cvs, thus compilation broke.
- removed C_ARG creation from lib/Makefile.Watcom and use CFLAGS
directly as done too in src/Makefile.Watcom - this has the benefit
that we will see all active cflags and defines during compile.
- added LINK-ARG to src/Makefile.Watcom in order to better control
linker input.
- a couple of other minor makefile tweaks here and there ...
- added largefile support for Watcom builds to config-win32.h. Not yet
tested if it really works, but should since Win32 supports it.
- added loaddll stuff to speed up builds if supported.
Win64's 32 bit long but 64 bit size_t caused a warning that we avoid
with a typecast. A small whitespace indent fix was also applied.
Reported by: Adam Light
This passes -Werror to gcc when building curl and libcurl,
allowing easy dection of compile warnings.
Signed-off-by: Ben Greear <greearb@candelatech.com>
... since FTP is using it as well, and potentially other protocols!
Also, an #endif CURL_DISABLE_HTTP was incorrectly marked, as it seems to
end the proxy block instead.
The FTP implementation was missing a timestamp reset point, making the
waiting for responses after sending a post-transfer "QUOTE" command not
working as supposedly. This bug was introduced in 7.20.0
curl_multi perform has two phases: run through every easy handle calling
multi_runsingle and remove expired timers (timer removal).
If a small timer (e.g. 1-10ms) is set during multi_runsingle, then it's
possible that the timer has passed by when the timer removal runs. The
timer which was just added is then removed. This will potentially cause
the timer list to be empty and cause the next call to curl_multi_timeout
to return -1. Ideally, curl_multi_timeout should return 0 in this case.
One way to fix this is to move the struct timeval now = Curl_tvnow(); to
the top of curl_multi_perform. The change does that.
As mentioned in bug report #2956968, the HTTP code wouldn't send the
first empty chunk during the auth negotiation phase of the HTTP request
sending, so the server would wait for data to come and libcurl would
wait for data to arrive... I've made the code not enable chunked
encoding until the auth negotiation is done and thus this scenario
doesn't occur anymore.
Reported by: Sidney San Martín
Bug: http://curl.haxx.se/bug/view.cgi?id=2956968
When curl_multi_remove_handle() is called and an easy handle is returned
to the connection cache held in the multi handle, then we cannot allow
CURLINFO_LASTSOCKET to extract it since that will more or less encourage
that the user uses the socket while it can get used by libcurl again.
Without this fix, we'd get a segfault in Curl_getconnectinfo() trying to
dereference the NULL pointer in 'data->state.connc'.
Bug: http://curl.haxx.se/bug/view.cgi?id=3023840
When configured with '--without-ssl --with-nss', NTLM authentication
now uses NSS crypto library for MD5 and DES. For MD4 we have a local
implementation in that case. More details are available at
https://bugzilla.redhat.com/603783
In order to get it working, curl_global_init() must be called with
CURL_GLOBAL_SSL or CURL_GLOBAL_ALL. That's necessary because NSS needs
to be initialized globally and we do so only when the NSS library is
actually required by protocol. The mentioned call of curl_global_init()
is responsible for creating of the initialization mutex.
There was also slightly changed the NSS initialization scenario, in
particular, loading of the NSS PEM module. It used to be loaded always
right after the NSS library was initialized. Now the library is
initialized as soon as any SSL or NTLM is required, while the PEM module
is prevented from being loaded until the SSL is actually required.
When a hostname resolves to multiple IP addresses and the first one
tried doesn't work, the socket for the second attempt may get dropped on
the floor, causing the request to eventually time out. The issue is that
when using kqueue (as on mac and bsd platforms) instead of select, the
kernel removes the first fd from kqueue when it is closed (in trynextip,
connect.c:503). Trynextip() then goes on to open a new socket, which
gets assigned the same number as the one it just closed. Later in
multi.c, socket_cb is not called because the fd is already in
multi->sockhash, so the new socket is never added to kqueue.
The correct fix is to ensure that socket_cb is called to remove the fd
when trynextip() closes the socket, and again to re-add it after
singleipsocket(). I'm not sure how to cleanly do that, but the attached
patch works around the problem in an admittedly kludgy way by delaying
the close to ensure that the newly-opened socket gets a different fd.
Daniel's added comment: I didn't spot a way to easily do a nicer fix so
I've proceeded with Ben's patch.
Bug: http://curl.haxx.se/bug/view.cgi?id=3017819
Patch by: Ben Darnell
For example the libssh2 based functions return other negative
values than -1 to signal errors and it is important that we catch
them properly. Right before this, various failures from libssh2
were treated as negative download amounts which caused havoc.
My additional call to Curl_pgrsUpdate() would sometimes get
called even though there's no connection (left) so a NULL pointer
would get passed, causing a segfault.
1) no need to call the progress function twice when in the
CURLM_STATE_TOOFAST state.
2) Make sure that the progress callback's return code is
acknowledged when used
As long as no error is reported, the progress function can get
called. This may be a little TOO often so we should keep an eye
on this and possibly make this conditional somehow.
