The integer literal 3123123123 doesn't fit into a 32-bit signed
integer, so GCC with 32-bit long warns in C90 mode:
this decimal constant is unsigned only in ISO C90 [enabled by default]
Fix this by using ULONG_MAX, which should fit in any curl_off_t and has
the correct suffix to not issue any warnings.
Also adds the missing CURLOPT_REQUEST_TARGET from commit
9b167fd090.
Closes https://github.com/curl/curl/pull/1611
... to enable sending "OPTIONS *" which wasn't possible previously.
This option currently only works for HTTP.
Added test cases 1298 + 1299 to verify
Fixes#1280Closes#1462
The list was freed incorrectly since the llist refactor of
cbae73e1dd. Added test 1550 to verify that it works and avoid future
regressions.
Reported-by: Pascal Terjan
Fixes#1584Closes#1585
... unless "--output -" is used. Binary detection is done by simply
checking for a binary zero in early data.
Added test 1425 1426 to verify.
Closes#1512
... the previous code would reset the header length wrongly (since
5113ad0424). This makes test 1060 reliable again.
Also: make sws send even smaller chunks of data to increase the
likeliness of this happening.
Both these tests run the same underlying test code: libntlmconnect.c -
this test code made some assumptions about socket ordering when it used
curl_easy_fdset() and when we changed timing or got accidental changes
in libcurl the tests would fail.
The tests verify that the different transfers keep using the same
connections, which I now instead made sure by adding the number of bytes
each transfer gets and then verifies that they always get the same
amount as when these tests worked.
Closes#1576
mk-lib1521.pl generates a test program (lib1521.c) that calls
curl_easy_setopt() for every known option with a few typical values to
make sure they work (ignoring the return codes).
Some small changes were necessary to avoid asserts and NULL accesses
when doing this.
The perl script needs to be manually rerun when we add new options.
Closes#1543
Use CURLMcode for variable 'res' and cast to int where necessary
instead of the other way around. Other tests do the same.
This fixes the following clang warning:
lib583.c:68:15: warning: cast from function call of type 'CURLMcode' to
non-matching type 'int' [-Wbad-function-cast]
Also removed a TODO suggesting caching the precheck results. Tests
showed this would save about 0.1 sec on the total test run time on a
relatively modern system, an unnoticeable gain at the cost of longer and
more complicated code. There would also be a danger that a cached test
result would be inappropriately returned, such as when other test
dependencies (like environment variables) are different or when the
precheck causes side effects (like filesystem changes).
@MarcelRaad noted that `test1399` causes infinite loop on MinGW.
Looking into this, seems like it is related to how Windows handles
CRLF. See https://github.com/curl/curl/commit/9e093f by @mback2k.
Removing `test1399` as it's identical to `test1326` then with such a
fix.
Test 1399 was broughy by commit 862b02f894Closes#1478
Include the test number in the names of files written out by tests to
reduce the chance of accidental duplication and to make it more clear
which test is associated with which file.
... using the docs/cmdline-opts/gen.pl script, so that we get all the
command line option documentation from the same source.
The generation of the list has to be done manually and pasted into the
source code.
Closes#1465
long is 32 bits while size_t is 64 bits on MinGW-w64, so
typecheck-gcc.h complains when using size_t for a long option.
Also, curl_socket_t is unsigned long long rather than int.
assign string literals to const char * instead of char * in order to
avoid a lot of these warnings:
cast from 'const char *' to 'char *' drops const qualifier
[-Wcast-qual]
Windows does not allow setting the locale with environment variables (as
the test attempted to do), so the test failed when run with a user
locale that has a comma as radixchar. Changed the test to call
setlocale() explicitly to ensure that a known working locale is set even
on Windows.
Previous TODO wanting to write in chunks. We should support writing more
at once since some TELNET servers may respond immediately upon first
byte written such as WHOIS servers.
Closes#1389
This fixes the following clang warning:
getpart.c:201:17: warning: cast from function call of type 'CURLcode'
to non-matching type 'int' [-Wbad-function-cast]
Test command 'time curl http://localhost/80GB -so /dev/null' on a Debian
Linux.
Before (middle performing run out 9):
real 0m28.078s
user 0m11.240s
sys 0m12.876s
After (middle performing run out 9)
real 0m26.356s (93.9%)
user 0m5.324s (47.4%)
sys 0m8.368s (65.0%)
Also, doing SFTP over a 200 millsecond latency link is now about 6 times
faster.
Closes#1446
The 'list element' struct now has to be within the data that is being
added to the list. Removes 16.6% (tiny) mallocs from a simple HTTP
transfer. (96 => 80)
Also removed return codes since the llist functions can't fail now.
Test 1300 updated accordingly.
Closes#1435
MinGW-w64 complains:
warning: conversion to 'long int' from 'time_t {aka long long int}' may
alter its value [-Wconversion]
Fix this by using the correct type.
This checks the new behavior of Curl_splaygetbest, so that the smallest
node not larger than the key is removed, and FIFO behavior is kept even
when there are multiple nodes with the same key.
Closes#1358
Multi handles repeatedly invert the queue of pending easy handles when
used with CURLMOPT_MAX_TOTAL_CONNECTIONS. This is caused by a multistep
process involving Curl_splaygetbest and violates the FIFO property of
the multi handle.
This patch fixes this issue by redefining the "best" node in the
context of timeouts as the "smallest not larger than now", and
implementing the necessary data structure modifications to do this
effectively, namely:
- splay nodes with the same key are now stored in a doubly-linked
circular list instead of a non-circular one to enable O(1)
insertion to the tail of the list
- Curl_splayinsert inserts nodes with the same key to the tail of
the same list
- in case of multiple nodes with the same key, the one on the head of
the list gets selected
system.h is aimed to replace curlbuild.h at a later point in time when
we feel confident system.h works sufficiently well.
curl/system.h is currently used in parallel with curl/curlbuild.h
curl/system.h determines a data sizes, data types and include file
status based on available preprocessor defines instead of getting
generated at build-time. This, in order to avoid relying on a build-time
generated file that makes it complicated to do 32 and 64 bit bields from
the same installed set of headers.
Test 1541 verifies that system.h comes to the same conclusion that
curlbuild.h offers.
Closes#1373
In ancient MinGW versions, in6addr_any was declared as extern, but not
defined. Because of that, 22a0c57746 added
definitions for in6addr_any when compiling with MinGW. The bug was fixed in
w32api version 3.6 from 2006, so this workaround is not needed anymore for
recent versions.
This fixes the following MinGW-w64 warnings because the MinGW-w64 version of
IN6ADDR_ANY_INIT has the two additional braces inside the macro:
util.c:59:14: warning: braces around scalar initializer
util.c:59:40: warning: excess elements in scalar initializer
Ref: e4803e0da2/tree/w32api/ChangeLog
Closes https://github.com/curl/curl/pull/1379
When receiving chunked encoded data with trailers, and the write
callback returns PAUSE, there might be both body and header to store to
resend on unpause. Previously libcurl returned error for that case.
Added test case 1540 to verify.
Reported-by: Stephen Toub
Fixes#1354Closes#1357
When using basic-auth, connections and proxy connections
can be re-used with different Authorization headers since
it does not authenticate the connection (like NTLM does).
For instance, the below command should re-use the proxy
connection, but it currently doesn't:
curl -v -U alice:a -x http://localhost:8181http://localhost/
--next -U bob:b -x http://localhost:8181http://localhost/
This is a regression since refactoring of ConnectionExists()
as part of: cb4e2be7c6
Fix the above by removing the username and password compare
when re-using proxy connection at proxy_info_matches().
However, this fix brings back another bug would make curl
to re-print the old proxy-authorization header of previous
proxy basic-auth connection because it wasn't cleared.
For instance, in the below command the second request should
fail if the proxy requires authentication, but would succeed
after the above fix (and before aforementioned commit):
curl -v -U alice:a -x http://localhost:8181http://localhost/
--next -x http://localhost:8181http://localhost/
Fix this by clearing conn->allocptr.proxyuserpwd after use
unconditionally, same as we do for conn->allocptr.userpwd.
Also fix test 540 to not expect digest auth header to be
resent when connection is reused.
Signed-off-by: Isaac Boukris <iboukris@gmail.com>
Closes https://github.com/curl/curl/pull/1350
Depend on the known behaviour of URLs for nonexistent files rather than
the undefined behaviour of URLs for directories (which fails on Windows).
The test isn't about file: URLs at all, so the URL used doesn't really
matter.
If a % ended the statement, the string's trailing NUL would be skipped
and memory past the end of the buffer would be accessed and potentially
displayed as part of the --write-out output. Added tests 1440 and 1441
to check for this kind of condition.
Reported-by: Brian Carpenter
- Add new option CURLOPT_SUPPRESS_CONNECT_HEADERS to allow suppressing
proxy CONNECT response headers from the user callback functions
CURLOPT_HEADERFUNCTION and CURLOPT_WRITEFUNCTION.
- Add new tool option --suppress-connect-headers to expose
CURLOPT_SUPPRESS_CONNECT_HEADERS and allow suppressing proxy CONNECT
response headers from --dump-header and --include.
Assisted-by: Jay Satiro
Assisted-by: CarloCannas@users.noreply.github.com
Closes https://github.com/curl/curl/pull/783
A client MUST ignore any Content-Length or Transfer-Encoding header
fields received in a successful response to CONNECT.
"Successful" described as: 2xx (Successful). RFC 7231 4.3.6
Prior to this change such a case would cause an error.
In some ways this bug appears to be a regression since c50b878. Prior to
that libcurl may have appeared to function correctly in such cases by
acting on those headers instead of causing an error. But that behavior
was also incorrect.
Bug: https://github.com/curl/curl/issues/1317
Reported-by: mkzero@users.noreply.github.com
Test 1903 is doing HTTP pipelining, and that is a timing and ordering
sensitive operation and this fails far too often on the Travis CI
leading to people more or less ignoring test failures there. Not good.
The end of pipelning is probably coming sooner rather than later
anyway...
... because it causes confusion with users. Example URLs:
"http://[127.0.0.1]:11211:80" which a lot of languages' URL parsers will
parse and claim uses port number 80, while libcurl would use port number
11211.
"http://user@example.com:80@localhost" which by the WHATWG URL spec will
be treated to contain user name 'user@example.com' but according to
RFC3986 is user name 'user' for the host 'example.com' and then port 80
is followed by "@localhost"
Both these formats are now rejected, and verified so in test 1260.
Reported-by: Orange Tsai
This is likely to be the case when building from a tar ball release
package which includes a prebuilt man page. In that case, test the
packaged man page instead. This only makes a difference when building
out-of-tree (in-tree, the location in both cases is identical).
The character set in POSIX is set by the locale defined by (in
decreasing order of precedence) the LC_ALL, LC_CTYPE and LANG
environment variables (CHARSET was used by libidn but not libidn2).
LC_ALL is cleared to ensure that LC_CTYPE takes effect, but LC_ALL is
not used to set the locale to ensure that other parts of the locale
aren't overridden. Since there doesn't seem to be a cross-platform way
of specifying a UTF-8 locale, and not all systems may support UTF-8, a
<precheck> is used to skip the test if UTF-8 can't be verified to be
available. Test 1035 was also converted to UTF-8 for consistency, as
the actual character set used there is irrelevant to the test.
This patch uses a different UTF-8 locale than the last attempt, namely
en_US.UTF-8. This one has been verified on 7 different Linux and BSD
distributions and is more complete and usable than the locale UTF-8 (on
at least some systems).
This reverts commit ecd1d020ab.
That commit caused test failures on my Debian Linux machine for all
changed test cases. We need to reconsider how that should get done.
Character set in POSIX is set by the locale defined (in decreasing order
of precedence) by the LC_ALL, LC_CTYPE and LANG environment variables (I
believe CHARSET is only historic). LC_ALL is cleared to ensure that
LC_CTYPE takes effect, but LC_ALL is not used to set the locale to
ensure that other parts of the locale aren't overriden, if set. Since
there doesn't seem to be a cross-platform way of specifying a UTF-8
locale, and not all systems may support UTF-8, a <precheck> is used
(where relevant) to skip the test if UTF-8 isn't in use. Test 1035 was
also converted to UTF-8 for consistency, as the actual character set
used there is irrelevant to the test.
- on the first invocation: keep security context returned by
InitializeSecurityContext()
- on subsequent invocations: use MakeSignature() instead of
InitializeSecurityContext() to generate HTTP digest response
Bug: https://github.com/curl/curl/issues/870
Reported-by: Andreas Roth
Closes https://github.com/curl/curl/pull/1251
Properly resolve, convert and log the proxy host names.
Support the "--connect-to" feature for SOCKS proxies and for passive FTP
data transfers.
Follow-up to cb4e2be
Reported-by: Jay Satiro
Fixes https://github.com/curl/curl/issues/1248
- While negotiating auth during PUT/POST if a user-specified
Content-Length header is set send 'Content-Length: 0'.
This is what we do already in HTTPREQ_POST_FORM and what we did in the
HTTPREQ_POST case (regression since afd288b).
Prior to this change no Content-Length header would be sent in such a
case.
Bug: https://curl.haxx.se/mail/lib-2017-02/0006.html
Reported-by: Dominik Hölzl
Closes https://github.com/curl/curl/pull/1242
This repairs cookies for localhost.
Non-PSL builds will now only accept "localhost" without dots, while PSL
builds okeys everything not listed as PSL.
Added test 1258 to verify.
This was a regression brought in a76825a5ef
Under condition using http_proxy env var, noproxy list was the
combination of --noproxy option and NO_PROXY env var previously. Since
this commit, --noproxy option overrides NO_PROXY environment variable
even if use http_proxy env var.
Closes#1140
The combination of --noproxy option and http_proxy env var works well
both for proxied hosts and non-proxied hosts.
However, when combining NO_PROXY env var with --proxy option,
non-proxied hosts are not reachable while proxied host is OK.
This patch allows us to access non-proxied hosts even if using NO_PROXY
env var with --proxy option.
Follow-up to 3463408.
Prior to 3463408 file:// hostnames were silently stripped.
Prior to this commit it did not work when a schemeless url was used with
file as the default protocol.
Ref: https://curl.haxx.se/mail/lib-2016-11/0081.html
Closes https://github.com/curl/curl/pull/1124
Also fix for drive letters:
- Support --proto-default file c:/foo/bar.txt
- Support file://c:/foo/bar.txt
- Fail when a file:// drive letter is detected and not MSDOS/Windows.
Bug: https://github.com/curl/curl/issues/1187
Reported-by: Anatol Belski
Assisted-by: Anatol Belski
Follow-up to 82245ea: Fix the example program sendrecv.c (handle
CURLE_AGAIN, handle incomplete send). Improve the documentation
for curl_easy_recv() and curl_easy_send().
Reviewed-by: Frank Meier
Assisted-by: Jay Satiro
See https://github.com/curl/curl/pull/1134
A server MUST NOT send any Transfer-Encoding or Content-Length header
fields in a 2xx (Successful) response to CONNECT. (RFC 7231 section
4.3.6)
Also fixes the three test cases that did this.
If a port number in a "connect-to" entry does not match, skip this
entry instead of connecting to port 0.
If a port number in a "connect-to" entry matches, use this entry
and look no further.
Reported-by: Jay Satiro
Assisted-by: Jay Satiro, Daniel Stenberg
Closes#1148
We're mostly saying just "curl" in lower case these days so here's a big
cleanup to adapt to this reality. A few instances are left as the
project could still formally be considered called cURL.
- Call Curl_initinfo on init and duphandle.
Prior to this change the statistical and informational variables were
simply zeroed by calloc on easy init and duphandle. While zero is the
correct default value for almost all info variables, there is one where
it isn't (filetime initializes to -1).
Bug: https://github.com/curl/curl/issues/1103
Reported-by: Neal Poole
... to make it less likely that we forget that the function actually
does case insentive compares. Also replaced several invokes of the
function with a plain strcmp when case sensitivity is not an issue (like
comparing with "-").
Cokie with the same domain but different tailmatching property are now
considered different and do not replace each other. If header contains
following lines then two cookies will be set: Set-Cookie: foo=bar;
domain=.foo.com; expires=Thu Mar 3 GMT 8:56:27 2033 Set-Cookie: foo=baz;
domain=foo.com; expires=Thu Mar 3 GMT 8:56:27 2033
This matches Chrome, Opera, Safari, and Firefox behavior. When sending
stored tokens to foo.com Chrome, Opera, Firefox store send them in the
stored order, while Safari pre-sort the cookies.
Closes#1050
Add the new option CURLOPT_KEEP_SENDING_ON_ERROR to control whether
sending the request body shall be completed when the server responds
early with an error status code.
This is suitable for manual NTLM authentication.
Reviewed-by: Jay Satiro
Closes https://github.com/curl/curl/pull/904
.. and add that --proto-redir and CURLOPT_REDIR_PROTOCOLS do not
override protocols denied by --proto and CURLOPT_PROTOCOLS.
- Add a test to enforce: --proto deny must override --proto-redir allow
Closes https://github.com/curl/curl/pull/1031
... like when a HTTP/0.9 response comes back without any headers at all
and just a body this now prevents that body from being sent to the
callback etc.
Adapted test 1144 to verify.
Fixes#973
Assisted-by: Ray Satiro
This only excludes building unit tests from default build ( 'all' Make
target or "Build Solution" in VisualStudio). The projects and Make
targets will still be generated and shown in supporting IDEs.
Fixes https://github.com/curl/curl/issues/981
Reported-by: Randy Armstrong
Closes https://github.com/curl/curl/pull/990
Detect support for compiler symbol visibility flags and apply those
according to CURL_HIDDEN_SYMBOLS option.
It should work true to the autotools build except it tries to unhide
symbols on Windows when requested and prints warning if it fails.
Ref: https://github.com/curl/curl/issues/981#issuecomment-242665951
Reported-by: Daniel Stenberg
Since we're using CURLE_FTP_WEIRD_SERVER_REPLY in imap, pop3 and smtp as
more of a generic "failed to parse" introduce an alias without FTP in
the name.
Closes https://github.com/curl/curl/pull/975
With HTTP/2 each transfer is made in an indivial logical stream over the
connection, making most previous errors that caused the connection to get
forced-closed now instead just kill the stream and not the connection.
Fixes#941
This fixes tests that were added after 113f04e664 as the tests would
fail otherwise.
We bring back "Proxy-Connection: Keep-Alive" now unconditionally to fix
regressions with old and stupid proxies, but we could possibly switch to
using it only for CONNECT or only for NTLM in a future if we want to
gradually reduce it.
Fixes#954
Reported-by: János Fekete
CMake build now using BUILD_TESTING=ON/OFF (default is OFF) to build
tests and enabling CTest integration. Options BUILD_CURL_TESTS and
BUILD_DASHBOARD_REPORTS was removed.
Closes#882
Reviewed-by: Brad King
The HTTP/2 tests brought with commit bf05606ef1 were using the internal
name 'http2' for the HTTP/2 server, while in fact that name was already
used for the second instance of the HTTP server. This made tests using
the second instance (like test 2050) fail after a HTTP/2 test had run.
The server is now known as HTTP/2 internally and within the <server>
section in test cases. 1700, 1701 and 1702 were updated accordingly.
It requires that 'nghttpx' is in the PATH, and it will run the tests
using nghttpx as a front-end proxy in front of the standard HTTP/1 test
server. This uses HTTP/2 over plain TCP.
If you like me have nghttpx installed in a custom path, you can run test 1700
like this:
$ PATH=$PATH:$HOME/build-nghttp2/bin/ ./runtests.pl 1700
Mostly in order to support broken web sites that redirect to broken URLs
that are accepted by browsers.
Browsers are typically even more leniant than this as the WHATWG URL
spec they should allow an _infinite_ amount. I tested 8000 slashes with
Firefox and it just worked.
Added test case 1141, 1142 and 1143 to verify the new parser.
Closes#791
Prior to this change a width arg could be erroneously output, and also
width and precision args could not be used together without crashing.
"%0*d%s", 2, 9, "foo"
Before: "092"
After: "09foo"
"%*.*s", 5, 2, "foo"
Before: crash
After: " fo"
Test 557 is updated to verify this and more
curl_printf.h defines printf to curl_mprintf, etc. This can cause
problems with external headers which may use
__attribute__((format(printf, ...))) markers etc.
To avoid that they cause problems with system includes, we include
curl_printf.h after any system headers. That makes the three last
headers to always be, and we keep them in this order:
curl_printf.h
curl_memory.h
memdebug.h
None of them include system headers, they all do funny #defines.
Reported-by: David Benjamin
Fixes#743
It does open up a miniscule risk that one of the other protocols that
libcurl could use would send back a Content-Disposition header and then
curl would act on it even if not HTTP.
A future mitigation for this risk would be to allow the callback to ask
libcurl which protocol is being used.
Verified with test 1312
Closes#760
This script now also scans src/tool_getparam.c, docs/curl.1 and
src/tool_help.c and will warn if any of them lists a command line option
not mentioned in one of the other places.
While being debated (in #716) and a violation of RFC 7230 section 5.4,
this test verifies that the existing functionality works as intended. It
strips the dot from the host name and uses the host without dot
throughout the internals.
... for checking ability to receive full HTTP response when POST request
is used with slow read callback function.
This test checks for bug #657 and verifies the work-around from
72d5e144fb.
Closes#720
warning: implicit declaration of function 'sprintf_was_used'
[-Wimplicit-function-declaration]
Follow up to the modications made to tests/libtest in commit 55452ebdff
as we prefer not to use sprintf() now.
The define is not in our name space and is therefore not protected by
our API promises.
It was only really used by libcurl internals but was mostly erased from
there already in 8aabbf5 (March 2015). This is supposedly the final
death blow to that define from everywhere.
As a side-effect, making sure _MPRINTF_REPLACE is gone and not used, I
made the lib tests in tests/libtest/ use curl_printf.h for its redefine
magic and then subsequently the use of sprintf() got banned in the tests
as well (as it is in libcurl internals) and I then replaced them all
with snprintf().
In the unlikely event that any users is actually using this define and
gets sad by this change, it is very easily copied to the user's own
code.
Fixed failed redirection of stderr with some options. At least on Msys2,
perl fails to redirect stderr if $value contains newline or other weird
characters.
It seems we may have some autobuild problems after this commit went
in. Trying to see if a revert helps to get them back.
This reverts commit 2716350d1f.
RFC 6265 section 4.1.1 spells out that the first name/value pair in the
header is the actual cookie name and content, while the following are
the parameters.
libcurl previously had a more liberal approach which causes significant
problems when introducing new cookie parameters, like the suggested new
cookie priority draft.
The previous logic read all n/v pairs from left-to-right and the first
name used that wassn't a known parameter name would be used as the
cookie name, thus accepting "Set-Cookie: Max-Age=2; person=daniel" to be
a cookie named 'person' while an RFC 6265 compliant parser should
consider that to be a cookie named 'Max-Age' with an (unknown) parameter
'person'.
Fixes#709
DSA is no longer supported by OpenSSH 7.0, which causes all SCP/SFTP
test cases to be skipped. Using RSA for host authentication works with
both old and new versions of OpenSSH.
Reported-by: Karlson2k
Closes#676
- Add tests.
- Add an example to CURLOPT_TFTP_NO_OPTIONS.3.
- Add --tftp-no-options to expose CURLOPT_TFTP_NO_OPTIONS.
Bug: https://github.com/curl/curl/issues/481
It turns out Firefox and Chrome both allow spaces in cookie names and
there are sites out there using that.
Turned out the code meant to strip off trailing space from cookie names
didn't work. Fixed now.
Test case 8 modified to verify both these changes.
Closes#639
- Add unit test 1604 to test the sanitize_file_name function.
- Use -DCURL_STATICLIB when building libcurltool for unit testing.
- Better detection of reserved DOS device names.
- New flags to modify sanitize behavior:
SANITIZE_ALLOW_COLONS: Allow colons
SANITIZE_ALLOW_PATH: Allow path separators and colons
SANITIZE_ALLOW_RESERVED: Allow reserved device names
SANITIZE_ALLOW_TRUNCATE: Allow truncating a long filename
- Restore sanitization of banned characters from user-specified outfile.
Prior to this commit sanitization of a user-specified outfile was
temporarily disabled in 2b6dadc because there was no way to allow path
separators and colons through while replacing other banned characters.
Now in such a case we call the sanitize function with
SANITIZE_ALLOW_PATH which allows path separators and colons to pass
through.
Closes https://github.com/curl/curl/issues/624
Reported-by: Octavio Schroeder
It isn't used by the code in current conditions but for safety it seems
sensible to at least not crash on such input.
Extended unit test 1395 to verify this too as well as a plain "/" input.
Before this patch, if a URL does not start with the protocol
name/scheme, effective URLs would be prefixed with upper-case protocol
names/schemes. This behavior might not be expected by library users or
end users.
For example, if `CURLOPT_DEFAULT_PROTOCOL` is set to "https". And the
URL is "hostname/path". The effective URL would be
"HTTPS://hostname/path" instead of "https://hostname/path".
After this patch, effective URLs would be prefixed with a lower-case
protocol name/scheme.
Closes#597
Signed-off-by: Mohammad AlSaleh <CE.Mohammad.AlSaleh@gmail.com>
The request needs to be read and send in binary mode in order to use
CRLF instead of LF. Adding --upload-file - causes curl to read stdin
in binary mode.
Make this the default for the curl tool (if built with HTTP/2 powers
enabled) unless a specific HTTP version is requested on the command
line.
This should allow more users to get HTTP/2 powers without having to
change anything.
Tests 842, 843, 844, 845, 887, 888, 889, 890, 946, 947, 948 and 949 fail
if a custom port number is specified via the -b option of runtests.pl.
Suggested by: Kamil Dudka
Bug: http://curl.haxx.se/mail/lib-2015-12/0003.html
As POP3 final and continuation responses both begin with a + character,
and both the finalcode and contcode variables in SASLprotoc are set as
such, we cannot tell the difference between them when we are expecting
an optional continuation from the server such as the following:
+ something else from the server
+OK final response
Disabled these tests until such a time we can tell the responses apart.
The hashes can vary between architectures (e.g. Sparc differs from x86_64).
This is not a fatal problem but just reduces the coverage of these white-box
tests, as the assumptions about into which hash bucket each key falls are no
longer valid.
- no point in repeating curl features that is already listed as features
from the curl -V output
- remove the port numbers/unix domain path from the output unless
verbose is used, as that is rarely interesting to users.
The tftpd test server now logs all received options and thus all TFTP
test cases need to match them exactly.
Extended test 283 to use and verify --tftp-blksize.
Apparently there are sites out there that do redirects to URLs they
provide in plain UTF-8 or similar. Browsers and wget %-encode such
headers when doing a subsequent request. Now libcurl does too.
Added test 1138 to verify.
Closes#473
- Add new option CURLOPT_DEFAULT_PROTOCOL to allow specifying a default
protocol for schemeless URLs.
- Add new tool option --proto-default to expose
CURLOPT_DEFAULT_PROTOCOL.
In the case of schemeless URLs libcurl will behave in this way:
When the option is used libcurl will use the supplied default.
When the option is not used, libcurl will follow its usual plan of
guessing from the hostname and falling back to 'http'.
New tool option --ssl-no-revoke.
New value CURLSSLOPT_NO_REVOKE for CURLOPT_SSL_OPTIONS.
Currently this option applies only to WinSSL where we have automatic
certificate revocation checking by default. According to the
ssl-compared chart there are other backends that have automatic checking
(NSS, wolfSSL and DarwinSSL) so we could possibly accommodate them at
some later point.
Bug: https://github.com/bagder/curl/issues/264
Reported-by: zenden2k <zenden2k@gmail.com>
This prevents valgrind from reporting possibly lost memory that NSPR
uses for file descriptor cache and other globally allocated internal
data structures.
Reported-by: Štefan Kremeň
When CURL_SOCKET_BAD is returned in the callback, it should be treated
as an error (CURLE_COULDNT_CONNECT) if no other socket is subsequently
created when trying to connect to a server.
Bug: http://curl.haxx.se/mail/lib-2015-06/0047.html
This function makes a platform-specific absolute path which uses
backslashes on Windows. This form works when passing it on the
command-line, as well as if the source is on another drive.
This avoids unnecessary dynamic allocs and as this also removed the last
users of *hash_alloc() and *hash_destroy(), those two functions are now
removed.
Make the HTTP headers separated by default for improved security and
reduced risk for information leakage.
Bug: http://curl.haxx.se/docs/adv_20150429.html
Reported-by: Yehezkel Horowitz, Oren Souroujon
This commit fixes a regression introduced in curl-7_41_0-186-g261a0fe.
It also introduces a regression test 1424 based on tests 78 and 1423.
Reported-by: Viktor Szakats
Bug: https://github.com/bagder/curl/issues/237
- cache entries must be also refreshed when they are in use
- have the cache count as inuse reference too, freeing timestamp == 0 special
value
- use timestamp == 0 for CURLOPT_RESOLVE entries which don't get refreshed
- remove CURLOPT_RESOLVE special inuse reference (timestamp == 0 will prevent refresh)
- fix Curl_hostcache_clean - CURLOPT_RESOLVE entries don't have a special
reference anymore, and it would also release non CURLOPT_RESOLVE references
- fix locking in Curl_hostcache_clean
- fix unit1305.c: hash now keeps a reference, need to set inuse = 1
Previously in Curl_http2_switched, we called nghttp2_session_mem_recv to
parse incoming data which were already received while curl was handling
upgrade. But we didn't call nghttp2_session_send, and it led to make
curl not send any response to the received frames. Most likely, we
received SETTINGS from server at this point, so we missed opportunity to
send SETTINGS + ACK. This commit adds missing nghttp2_session_send call
in Curl_http2_switched to fix this issue.
Bug: https://github.com/bagder/curl/issues/192
Reported-by: Stefan Eissing
"name =value" is fine and the space should just be skipped.
Updated test 31 to also test for this.
Bug: https://github.com/bagder/curl/issues/195
Reported-by: cromestant
Help-by: Frank Gevaerts
It seems that some systems (e.g. fairly consistently in some recent
Solaris autobuilds) would manage to get to the connect phase before the
progress callback was called, resulting in a CURLE_COULDNT_CONNECT
error. Reworked the test to point at a test server that never returns a
full result so the progress callback always gets a chance to be called
before the transfer can complete in some other way.
The certificates were missing the digitalSignature and keyAgreement
usage types, of which at least digitalSignature was checked by CyaSSL.
This caused the test server in test 310 (among others) to fail the
startup verification and therefore run (see
http://curl.haxx.se/mail/lib-2014-07/0303.html).
Since we just started make use of free(NULL) in order to simplify code,
this change takes it a step further and:
- converts lots of Curl_safefree() calls to good old free()
- makes Curl_safefree() not check the pointer before free()
The (new) rule of thumb is: if you really want a function call that
frees a pointer and then assigns it to NULL, then use Curl_safefree().
But we will prefer just using free() from now on.
The function "free" is documented in the way that no action shall occur for
a passed null pointer. It is therefore not needed that a function caller
repeats a corresponding check.
http://stackoverflow.com/questions/18775608/free-a-null-pointer-anyway-or-check-first
This issue was fixed by using the software Coccinelle 1.0.0-rc24.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
... by using the regular Curl_http_done() method which checks for
that. This makes test 1801 fail consistently with error 56 (which seems
fine) to that test is also updated here.
Reported-by: Ben Darnell
Bug: https://github.com/bagder/curl/issues/166
...after the method line:
"Since the Host field-value is critical information for handling a
request, a user agent SHOULD generate Host as the first header field
following the request-line." / RFC 7230 section 5.4
Additionally, this will also make libcurl ignore multiple specified
custom Host: headers and only use the first one. Test 1121 has been
updated accordingly
Bug: http://curl.haxx.se/bug/view.cgi?id=1491
Reported-by: Rainer Canavan
When checking for a connection to re-use, a proxy-using request must
check for and use a proxy connection and not one based on the host
name!
Added test 1421 to verify
Bug: http://curl.haxx.se/bug/view.cgi?id=1492
SSLeay was the name of the library that was subsequently turned into
OpenSSL many moons ago (1999). curl does not work with the old SSLeay
library since years. This is now reflected by only using USE_OPENSSL in
code that depends on OpenSSL.
sockfilt.c:288: warning: conversion to 'DWORD' from 'size_t' may alter
its value
sockfilt.c:291: warning: conversion to 'DWORD' from 'size_t' may alter
its value
sockfilt.c:323: warning: conversion to 'DWORD' from 'size_t' may alter
its value
sockfilt.c:326: warning: conversion to 'DWORD' from 'size_t' may alter
its value
* Missing initialisation of upload status caused a seg fault
* Missing data termination caused corrupt data to be uploaded
* Data verification should be performed in <upload> element
* Added missing recipient list cleanup
For consistency, as we seem to have a bit of a mixed bag, changed all
instances of ipv4 and ipv6 in comments and documentations to use the
correct case.
Merge multiple internal arrays into one, even if some variables
will not not be used. They are all created with the number of
file descriptors as their size.
Also fix possible thread handle leak in CloseHandle-loop.
Improves performance of test cases 574 and 575 by 50%.
A value of zero causes the thread to relinquish the remainder
of its time slice to any other thread of equal priority that is
ready to run. If there are no other threads of equal priority
ready to run, the function returns immediately, and the thread
continues execution.
http://msdn.microsoft.com/library/windows/desktop/ms686307.aspx
This fixes the test 506 torture test. The internal cookie API really
ought to be improved to separate cookie parsing errors (which may be
ignored) with OOM errors (which should be fatal).
The ability to do HTTP requests over a UNIX domain socket has been
requested before, in Apr 2008 [0][1] and Sep 2010 [2]. While a
discussion happened, no patch seems to get through. I decided to give it
a go since I need to test a nginx HTTP server which listens on a UNIX
domain socket.
One patch [3] seems to make it possible to use the
CURLOPT_OPENSOCKETFUNCTION function to gain a UNIX domain socket.
Another person wrote a Go program which can do HTTP over a UNIX socket
for Docker[4] which uses a special URL scheme (though the name contains
cURL, it has no relation to the cURL library).
This patch considers support for UNIX domain sockets at the same level
as HTTP proxies / IPv6, it acts as an intermediate socket provider and
not as a separate protocol. Since this feature affects network
operations, a new feature flag was added ("unix-sockets") with a
corresponding CURL_VERSION_UNIX_SOCKETS macro.
A new CURLOPT_UNIX_SOCKET_PATH option is added and documented. This
option enables UNIX domain sockets support for all requests on the
handle (replacing IP sockets and skipping proxies).
A new configure option (--enable-unix-sockets) and CMake option
(ENABLE_UNIX_SOCKETS) can disable this optional feature. Note that I
deliberately did not mark this feature as advanced, this is a
feature/component that should easily be available.
[0]: http://curl.haxx.se/mail/lib-2008-04/0279.html
[1]: http://daniel.haxx.se/blog/2008/04/14/http-over-unix-domain-sockets/
[2]: http://sourceforge.net/p/curl/feature-requests/53/
[3]: http://curl.haxx.se/mail/lib-2008-04/0361.html
[4]: https://github.com/Soulou/curl-unix-socket
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
test1435: a simple test that checks whether a HTTP request can be
performed over the UNIX socket. The hostname/port are interpreted
by sws and should be ignored by cURL.
test1436: test for the ability to do two requests to the same host,
interleaved with one to a different hostname.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
The variable `$ipvnum` can now contain "unix" besides the integers 4
and 6 since the variable. Functions which receive this parameter
have their `$port` parameter renamed to `$port_or_path` to support a
path to the UNIX domain socket (as a "port" is only meaningful for TCP).
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
If sws is killed it might leave a stale socket file on the filesystem
which would cause an EADDRINUSE error. After this patch, it is checked
whether the socket is really stale and if so, the socket file gets
removed and another bind is executed.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
This extends sws with a --unix-socket option which causes the port to
be ignored (as the server now listens on the path specified by
--unix-socket). This feature will be available in the following patch
that enables checking for UNIX domain socket support.
Proxy support (CONNECT) is not considered nor tested. It does not make
sense anyway, first connecting through a TCP proxy, then let that TCP
proxy connect to a UNIX socket.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Instead of depending the socket domain type on use_ipv6, specify the
domain type (AF_INET / AF_INET6) as variable. An enum is used here with
switch to avoid compiler warnings in connect_to, complaining that rc
is possibly undefined (which is not possible as socket_domain is
always set).
Besides abstracting the socket type, make the debugging messages be
independent on IP (introduce location_str which points to "port XXXXX").
Rename "ipv_inuse" to "socket_type" and tighten the scope (main).
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Commit curl-7_23_1-143-g8218064 changed the parameter of
responsive_http_server to accept types other than IPv6 (converting
from a boolean to a string), but only considered the lower-case "ipv6"
and not the "IPv6" variant. This caused all servers to start in IPv4
mode instead.
This patch converts the remaining cases to "ipv6". While not strictly
necessary for the run*server variants, these got also converted for
consistency and to prevent future errors.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
This is the only user of the backtick operator in the command. As the
commands will soon not be executed by a shell anymore (but by perl),
replace the command with its output.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Makes test1119 pass when building with cmake.
configurehelp.pm is generated by configure (autotools). As cmake does
not provide a separate variable for the C preprocessor, default to cpp.
Before commit ef24ecde68 ("symbol-scan:
use configure script knowledge about how to run the C preprocessor"),
this tool would also use 'cpp'.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Fix detection of the AsynchDNS feature which not just depends on
pthreads support, but also on whether USE_POSIX_THREADS is set or not.
Caught by test 1014.
This patch adds a new ENABLE_THREADED_RESOLVER option (corresponding to
--enable-threaded-resolver of autotools) which also needs a check for
HAVE_PTHREAD_H.
For symmetry with autotools, CURL_USE_ARES is renamed to ENABLE_ARES
(--enable-ares). Checks that test for the availability actually use
USE_ARES instead as that is the result of whether a-res is available or
not (in practice this does not matter as CARES is marked as required
package, but nevertheless it is better to write the intent).
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
I noticed that a patched cmake build would pass tests with a fake local
hostname, but the autotools build skips them:
got unexpected host name back, LD_PRELOAD failed
It turns out that -fvisibility=hidden hides the symbol, and since the
tests are not part of libcurl, it fails too. Just remove the LIBCURL
guard.
Broken since cURL 7.30 (commit 83a42ee20e,
"curl.h: stricter CURL_EXTERN linkage decorations logic").
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
Added !SSPI to the features list of the HTTP digest tests, as SSPI
based builds now use the Windows SSPI messaging API rather than the
internal functions, and we can't control the random numbers that get
used as part of the digest.
Basically since servers often then don't respond well to this and
instead send the full contents and then libcurl would instead error out
with the assumption that the server doesn't support resume. As the data
is then already transfered, this is now considered fine.
Test case 1434 added to verify this. Test case 1042 slightly modified.
Reported-by: hugo
Bug: http://curl.haxx.se/bug/view.cgi?id=1443
HTTP 1.1 is clearly specified to only allow three digit response codes,
and libcurl used sscanf("%3d") for that purpose. This made libcurl
support smaller numbers but not larger. It does now, but we will not
make any specific promises nor document this further since it is going
outside of what HTTP is.
Bug: http://curl.haxx.se/bug/view.cgi?id=1441
Reported-by: Balaji
CURLOPT_COPYPOSTFIELDS with a given CURLOPT_POSTFIELDSIZE does not
require a trailing zero of the data and by making sure this test doesn't
use one we know it works (combined with valgrind).
This change allows runtests.pl to be run from the CMake builddir:
export srcdir=/tmp/curl/tests;
perl -I$srcdir $srcdir/runtests.pl -l
In order to make this possible, all test cases have been moved from
Makefile.am to Makefile.inc.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
The 2to3 tool converted socketserver (which I manually fixed up with an
import fallback) and the print(e) line. The xrange option was converted
to range, but it seems better to use the '*' operator here for
simplicity.
Signed-off-by: Peter Wu <peter@lekensteyn.nl>
There is no need for such function. Include_directories propagate by
themselves and having a function with one simple link statement makes
little sense.
Option --pinnedpubkey takes a path to a public key in DER format and
only connect if it matches (currently only implemented with OpenSSL).
Provides CURLOPT_PINNEDPUBLICKEY for curl_easy_setopt().
Extract a public RSA key from a website like so:
openssl s_client -connect google.com:443 2>&1 < /dev/null | \
sed -n '/-----BEGIN/,/-----END/p' | openssl x509 -noout -pubkey \
| openssl rsa -pubin -outform DER > google.com.der
By not detecting and rejecting domain names for partial literal IP
addresses properly when parsing received HTTP cookies, libcurl can be
fooled to both send cookies to wrong sites and to allow arbitrary sites
to set cookies for others.
CVE-2014-3613
Bug: http://curl.haxx.se/docs/adv_20140910A.html
Historically the default "unknown" value for progress.size_dl and
progress.size_ul has been zero, since these values are initialized
implicitly by the calloc that allocates the curl handle that these
variables are a part of. Users of curl that install progress
callbacks may expect these values to always be >= 0.
Currently it is possible for progress.size_dl and progress.size_ul
to by set to a value of -1, if Curl_pgrsSetDownloadSize() or
Curl_pgrsSetUploadSize() are passed a "size" of -1 (which a few
places currently do, and a following patch will add more). So
lets update Curl_pgrsSetDownloadSize() and Curl_pgrsSetUploadSize()
so they make sure that these variables always contain a value that
is >= 0.
Updates test579 and test599.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
... to handle "*/[total]". Also, removed the strange hack that made
CURLOPT_FAILONERROR on a 416 response after a *RESUME_FROM return
CURLE_OK.
Reported-by: Dimitrios Siganos
Bug: http://curl.haxx.se/mail/lib-2014-06/0221.html
If a non-standard $TESTDIR is used the file may not be necessary.
Previously a "missing" file resulted in the warning:
readline() on closed filehandle D at ./runtests.pl line 4940.
This seems to have become necessary for SRP support to work starting
with GnuTLS ver. 2.99.0. Since support for SRP was added to GnuTLS
before the function that takes this priority string, there should be no
issue with backward compatibility.
Curl_rand() will return a dummy and repatable random value for this
case. Makes it possible to write test cases that verify output.
Also, fake timestamp with CURL_FORCETIME set.
Only when built debug enabled of course.
Curl_ssl_random() was not used anymore so it has been
removed. Curl_rand() is enough.
create_digest_md5_message: generate base64 instead of hex string
curl_sasl: also fix memory leaks in some OOM situations
Added required "debug" feature, missed in commit 1c9aaa0bac, as NTLMv2
calls Curl_rand() which can only be fixed to a specific entropy in
debug builds.
gcc spit out warning: variable 'x' might be clobbered by 'longjmp' or
'vfork' messages for a few variables. These automatic variables were
expected to be changed between a setjmp/longjmp and hold their values,
so are now marked volatile.
Follow-up to commit 121bcfee5d. curl-config --features now lists
GSS-API but it is not a listed feature in curl -V. This should probably
be synchronized.
Verifies that the change in 68f0166a92 works as intended and that
different HTTP auth credentials to the same host still re-uses the
connection properly.
In commit 0b3750b5c2 (released in 7.36.0) we fixed a timeout issue
but instead broke the timings.
To fix this, I introduce a new timestamp to use for the timeouts and
restored the previous timestamp and timestamp position so that the old
timer functionality is restored.
In addition to that, that change also broke connection timeouts for when
more than one connect was used (as it would then count the total time
from the first connect and not for the most recent one). Now
Curl_timeleft() has been modified so that it checks against different
start times depending on which timeout it checks.
Test 1303 is updated accordingly.
Bug: http://curl.haxx.se/mail/lib-2014-05/0147.html
Reported-by: Ryan Braud
If the precision is indeed shorter than the string, don't strlen() to
find the end because that's not how the precision operator works.
I also added a unit test for curl_msnprintf to make sure this works and
that the fix doesn't a few other basic use cases. I found a POSIX
compliance problem that I marked TODO in the unit test, and I figure we
need to add more tests in the future.
Reported-by: Török Edwin
Updated the docs to clarify and the code accordingly, with test 1528 to
verify:
When CURLHEADER_SEPARATE is set and libcurl is asked to send a request
to a proxy but it isn't CONNECT, then _both_ header lists
(CURLOPT_HTTPHEADER and CURLOPT_PROXYHEADER) will be used since the
single request is then made for both the proxy and the server.
Since all present tests now have <keywords> listed, this script will now
refuse to run a given test case if no such section is provided.
Hopefully this will help us make sure new test cases get keywords added
at start.
This makes it possible to fetch from an IPv6 literal without specifying
the -g option. Globbing remains available elsehwere in the URL.
For example:
curl http://[::1]/file[1-3].txt
This creates no ambiguity, because there is no overlap between the
syntax of valid globs and valid IPv6 literals. Globs contain hyphens
and at most 1 colon, while IPv6 literals have no hyphens, and at least 2
colons.
The peek_ipv6() parser simply whitelists a set of characters and counts
colons, because the real validation happens later on. The character set
includes A-Z, in case someone decides to implement support for scopes
like [fe80::1%25eth0] in the future.
Signed-off-by: Paul Marks <pmarks@google.com>
When the protocol part fails, the data usually does too but the protocol
part is often more fundamental and often provide the clues you need to
fix the test case.
As the email protocols implement SASL authentication rather than IMAP,
POP3 and SMTP specific authentication, updated the authentication
keywords to reflect this.
The improved connection reuse logic would otherwise create a new
connection for each one, which isn't supported by the test
server, nor expected by the test.
To better allow arguments like "1 to 9999" without flooding the terminal
with error messages, the given test cases range is now checked and only
test numbers with existing files are actually run.
The previous test certificate contained a MD5 hash which is not
supported using TLSv1.2 with Schannel on Windows 7 or newer.
See the update to this blog post on IEInternals / MSDN:
http://blogs.msdn.com/b/ieinternals/archive/2011/03/25/
misbehaving-https-servers-impair-tls-1.1-and-tls-1.2.aspx
"Update: If the server negotiates a TLS1.2 connection with a
Windows 7 or 8 schannel.dll-using client application, and it
provides a certificate chain which uses the (weak) MD5 hash
algorithm, the client will abort the connection (TCP/IP FIN)
upon receipt of the certificate."
When allowing NTLM, the re-use connection logic was too focused on
finding an existing NTLM connection to use and didn't properly allow
re-use of other ones. This made the logic not re-use perfectly re-usable
connections.
Added test case 1418 and 1419 to verify.
Regression brought in 8ae35102c (curl 7.35.0)
Reported-by: Jeff King
Bug: http://thread.gmane.org/gmane.comp.version-control.git/242213
This one is needed with the gcc options -fstack-protector-all -O2
That brings the number of suppressions for test 165 to four, and I
suspect I could find another two missing without trying very hard. I'm
beginning to think suppressions isn't the best way to handle these
kinds of cases.
Do not try to convert line-endings to CRLF on Windows by setting stdout
to binary mode, just like the curl tool does if --ascii is not specified.
This should prevent corrupted stdout line-ending output like CRCRLF.
In order to make the previously naive text-aware tests work with
binary mode on Windows, text-mode is disabled for them if it is not
actually part of the test case and line-endings are corrected.
According to RFC 2616 and RFC 2326 individual protocol elements, like
headers and except the actual content, are terminated by using CRLF.
Therefore the test data files for these protocols need to contain
mixed line-endings if the actual protocol elements use CRLF while
the file uses LF.
gcc 4.7.2 with -O2 will optimize Curl_connect by inlining some
functions two levels deep, which makes the valgrind suppression
fail to match. The underlying reason for these idna suppressions is
a gcc strlen optimization when compiling libidn; compiling it with
-fno-builtin-strlen makes this suppression unnecessary.
It seems the fips config option causes an error if FIPS mode was
not enabled at stunnel compile-time. FIPS support was disabled
by default in stunnel 5.00, so this is probably really only needed
on versions between 4.32 and 5.00.
This was already mostly being done, except that analysis after the
test still assumed that the valgrind log files would be available. An
alternative way to handle the valgrind + gdb combination could be to
enable one of the valgrind debugger hooks.
lib1515.c:38:26 warning: unused parameter 'curl'
lib1515.c:38:81 warning: unused parameter 'ptr'
lib1515.c:38:5 warning: no previous prototype for 'debug_callback'
lib1515.c:46:5 warning: no previous prototype for 'do_one_request'
lib1515.c:120:3 warning: ISO C90 forbids mixed declarations and code
As well as some code policing such as white space and braces.
Not comma, which is an inconsistency and a mistake probably inherited
from the examples section of RFC1867.
This bug has been present since the day curl started to support
multipart formposts, back in the 90s.
Reported-by: Rob Davies
Bug: http://curl.haxx.se/bug/view.cgi?id=1333
Fix for bug #1303 (030a2b8cb) was not complete.
libcurl still pruned DNS entries added manually
after detecting a dead connection. This test
checks such behavior.
Test-case 1515 reproduces bug #1303, where libcurl
would incorrectly prune DNS entries added via
CURLOPT_RESOLVE after the DNS_CACHE_TIMEOUT had
expired.
The test contains a cookie jar file where one of the cookies has an
expiry date of 1391252187 -- Sat, 1 Feb 2014 10:56:27 GMT which has
now expired. Updated to Wed, 14 Oct 2037 16:36:33 GMT as per test
179.
Reported-by: Adam Sampson
Bug: http://curl.haxx.se/bug/view.cgi?id=1330
Since the timer resolution is lower, there are actually cases that
the compared values are equal. Therefore we check for previous
timestamps being greater than the current one instead.
According to section 2.2 of RFC959 the End-of-Line is defined as:
The end-of-line sequence defines the separation of printing
lines. The sequence is Carriage Return, followed by Line Feed.
Verified by sniffing traffic between a Windows FTP client (FileZilla)
and Unix-hosted FTP server (ProFTPD).
It makes more sense to convert the expected output to [CR][LF] on
Windows than to force the actual, probably correct, output to [LF].
This way it is actually possible to see if curl outputs the correct
line-ending excepted by a text-aware test case.
Since the previous complex select function with initial support for
non-socket file descriptors, did not actually work correctly for
Console handles, this change simplifies the whole procedure by using
an internal waiting thread for the stdin console handle.
The previous implementation made it continuously trigger for the stdin
handle if it was being redirected to a parent process instead of
an actual Console input window.
This approach supports actual Console input handles as well as
anonymous Pipe handles which are used during input redirection.
It depends on the fact that ReadFile supports trying to read zero bytes
which makes it wait for the handle to become ready for reading.
Removed Unix-specific functionality in order to support Windows:
- select.epoll replaced with select.select
- SocketServer.ForkingMixIn replaced with SocketServer.ForkingMixIn
- socket.MSG_DONTWAIT replaced with socket.setblocking(False)
Even though epoll has a better performance and improved socket handling
than select, this change should not affect the actual test case.
Also, make the ftp server return a canned response that doesn't
cause XML verification problems. Although the test file format
isn't technically XML, it's still handy to be able to use XML
tools to verify and manipulate them.
Since /dev/stdout is not always emulated on Windows,
just skip the output option on Windows.
MinGW/msys support /dev/stdout only from a new login shell.
tstunnel on Windows does not support the pid option and is unable
to write to an output log that is already being used as a redirection
target for stdout. Therefore it does now output all log data to stdout
by default and secureserver.pl creates a fake pidfile on Windows.
The built-in memory debug system doesn't work with multi-threaded use so
instead of causing annoying false positives, disable the memory tracking
if the threaded resolver is used.
The Windows console version of stunnel is called "tstunnel", while
running "stunnel" on Windows spawns a new console window which
cannot be handled by the testsuite.
Previously LIST always returned a fixed hardcoded list that the ftp
server code knew about, mostly since the server didn't get any test case
number in the LIST scenario. Starting now, doing a CWD to a directory
named test-[number] will make the test server remember that number and
consider it a test case so that a subsequent LIST command will send the
<data> section of that test case back.
It allows LIST tests to be made more similar to how all other tests
work.
Test 100 was updated to provide its own directory listing.
Verify the change brought in commit 8e11731653061. It makes sure that
returning a failure from the progress callback even very early results
in the correct return code.
memdebug.h already contains all required definitions and including
curl_memory.h causes errors like the following:
tests/unit/unit1394.c:119: undefined reference to `Curl_cfree'
tests/unit/unit1394.c:120: undefined reference to `Curl_cfree'
Following commit 0aafd77fa4, replaced the internal usage of
FORMAT_OFF_T and FORMAT_OFF_TU with the external versions that we
expect API programmers to use.
This negates the need for separate definitions which were subtly
different under different platforms/compilers.
Following the addition of informational commands to the SMTP protocol,
the test server is no longer required to return the verified server
information in responses that curl only outputs in verbose mode.
Instead, a similar detection mechanism to that used by FTP, IMAP and
POP3 can now be used.
This commit replaces that of 9f260b5d66 because according to RFC-2449,
section 6, there is no APOP capability "...even though APOP is an
optional command in [POP3]. Clients discover server support of APOP by
the presence in the greeting banner of an initial challenge enclosed in
angle brackets."
SASL downgrade tests: 833, 835, 879, 881, 935 and 937 would fail as
they contained a minus sign in their authentication mechanism and this
would be missed by the custom reply parser.
Added support for downgrading the SASL authentication mechanism when the
decoding of CRAM-MD5, DIGEST-MD5 and NTLM messages fails. This enhances
the previously added support for graceful cancellation by allowing the
client to retry a lesser SASL mechanism such as LOGIN or PLAIN, or even
APOP / clear text (in the case of POP3 and IMAP) when supported by the
server.
To avoid the regression when users pass in passwords containing semi-
colons, we now drop the ability to set the login options with the same
options. Support for login options in CURLOPT_USERPWD was added in
7.31.0.
Test case 83 was modified to verify that colons and semi-colons can be
used as part of the password when using -u (CURLOPT_USERPWD).
Bug: http://curl.haxx.se/bug/view.cgi?id=1311
Reported-by: Petr Bahula
Assisted-by: Steve Holme
Signed-off-by: Daniel Stenberg <daniel@haxx.se>
A failure during authentication, which is performed as part of the
CONNECT phrase (for IMAP, POP3 and SMTP) is considered by the multi-
interface as being closed prematurely (aka a dead connection). As such
these protocols cannot issue the relevant QUIT or LOGOUT command.
Temporarily fixed the test cases until we can fix this properly.
The error code should not be sent as data as it isn't passed onto the
client as body data, so cannot be compared in the test suite against
expected data.
Although this option should have already been set, the SMTP module can
now download information from and send instructional commands to, an
SMTP server, requiring the option to be set in order to perform a mail
transfer.
As the IMAP regex could fail and $1 would not contain the command id
updated the unrecognised command response to be more generic and
realistic (like those used in the command handlers).
Additionally updated the POP3, SMTP and FTP responses.
A base64 string should be a multiple of 4 characters in length, not
contain any more than 2 padding characters and only contain padding
characters at the end of string. For example: Y3VybA==
Strings such as the following are considered invalid:
Y= - Invalid length
Y== - Invalid length
Y=== - More than two padding characters
Y=x= - Padding character contained within string
This is a regression since the switch to always-multi internally
c43127414d.
Test 1316 was modified since we now clearly call the Curl_client_write()
function when doing the LIST transfer part and then the
handler->protocol says FTP and ftpc.transfertype is 'A' which implies
text converting even though that the response is initially a HTTP
CONNECT response in this case.
As the URI, which is contained within the DIGEST-MD5 response, is
constructed from the service and realm, the encoded message differs
from that generated under POP3.
...as it is no longer required following capability and authentication
changes and is now causing problems following commit 49341628b5 as
the test number is obtained from the client address in the EHLO.
...to the client address as this frees the RCPT strings to contain
just an email address and by passing the test number into curl as the
client address remains consistent with POP3 and IMAP tests as they are
specified in the URL.
As someone reported on the mailing list a while back, the hard-coded
arbitrary timeout of 7s in test 1112 is not sufficient in some build
environments. At Arista Networks we build and test curl as part of our
automated build system, and we've run into this timeout 170 times so
far. Our build servers are typically quite busy building and testing a
lot of code in parallel, so despite being beefy machines with 32 cores
and 128GB of RAM we still hit this 7s timeout regularly.
URL: http://curl.haxx.se/mail/lib-2010-02/0200.html
As curl will send a HELO command after an negative EHLO response, added
the same detection from commit b07709f741 to the HELO handler to
ensure the test server is identified correctly and an upload isn't
performed.
Rather than detecting the TO address as missing in the DATA handler,
moved the detection to the RCPT command handler where an error response
can be generated.
Moved the special SMTP server detection code from the DATA command
handler, which happens further down the operation chain after EHLO,
MAIL and RCPT commands, to the EHLO command as it is the first command
to be generated by a SMTP operation as well as containing the special
"verifiedserver" string from the URL.
This not only makes it easier and quicker to detect but also means that
cURL doesn't need to specify "verifiedserver" as --mail-from and
--mail-rcpt arguments.
More importantly, this also makes the upcoming verification changes to
the RCPT handler easier to implement.
The libcurl date parser returns INT_MAX for all dates > 2037 so this
test is now made to use 2037 instead of 2038 to work the same for both
32bit and 64bit time_t systems.
Following changes to ftpserver.pl fixed the mail from address to be a
correctly formatted address otherwise the server response will be 501
Invalid address.
MAIl_smtp() will now check for a correctly formatted FROM address as
well as the optional SIZE parameter comparing it against the server
capability when specified.
Implement: Expired Cookies These following situation, curl removes
cookie(s) from struct CookieInfo if the cookie expired.
- Curl_cookie_add()
- Curl_cookie_getlist()
- cookie_output()
Renamed SUPPORTAUTH to AUTH and added support for specifying a list of
supported SASL mechanisms to return to the client.
Additionally added the directive to the FILEFORMAT document.
Renamed SUPPORTCAPA to CAPA and added support for specifying a list of
supported capabilities to return to the client.
Additionally added the directive to the FILEFORMAT document.
The message numbers given in the LIST response are an index into the
list, which are only valid for the current session, rather than being a
unique message identifier. An index would only be missing from the LIST
response if a DELE command had been issued within the same session and
had not been committed by the end of session QUIT command. Once
committed the POP3 server will regenerate the message numbers in the
next session to be contiguous again. As such our LIST response should
list message numbers contiguously until we support a DELE command in the
same session.
Should a POP3 user require the unique message ID for any or all
messages then they should use the extended UIDL command. This command
will be supported by the test ftpserver in an upcoming commit.
Corrected the call to logmsg() in the IMAP SEARCH handler from commit
4ae7b7ea69 as it should have been outputting the what argument and
not the test number.
The specified curl binary will then be used to verify the running
server(s) instead of the development version. This is very useful in
some cases when the development version fails to verify correctly as
then the test case may not run at all.
The actual test will still be run with the "normal" curl executable
(unless the test case specifies something differently).
... this also makes sure that the progess callback gets called more
often during TFTP transfers.
Added test 1238 to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=1269
Reported-by: Jo3
This function is meant to work *exactly* as curl_easy_perform() but will
use the event-based libcurl API internally instead of
curl_multi_perform(). To avoid relying on an actual event-based library
and to not use non-portable functions (like epoll or similar), there's a
rather inefficient emulation layer implemented on top of Curl_poll()
instead.
There's currently some convenience logging done in curl_easy_perform_ev
which helps when tracking down problems. They may be suitable to remove
or change once things seem to be fine enough.
curl has a new --test-event option when built with debug enabled that
then uses curl_easy_perform_ev() instead of curl_easy_perform(). If
built without debug, using --test-event will only output a warning
message.
NOTE: curl_easy_perform_ev() is not part if the public API on purpose.
It is only present in debug builds of libcurl and MUST NOT be considered
stable even then. Use it for libcurl-testing purposes only.
runtests.pl now features an -e command line option that makes it use
--test-event for all curl command line tests. The man page is updated.
libcurl truncates usernames and passwords it reads from .netrc to
LOGINSIZE and PASSWORDSIZE (64) characters without any indication to
the user, to ensure the values returned from Curl_parsenetrc fit in a
caller-provided buffer.
Fix the interface by passing back dynamically allocated buffers
allocated to fit the user's input. The parser still relies on a
256-character buffer to read each line, though.
So now you can include an ~246-character password in your .netrc,
instead of the previous limit of 63 characters.
Reported-by: Colby Ranger
The new multiply() function detects range value overflows. 32bit
machines will overflow on a 32bit boundary while 64bit hosts support
ranges up to the full 64 bit range.
Added test 1236 to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=1267
Reported-by: Will Dietz
A rather big overhaul and cleanup.
1 - curl wouldn't properly detect and reject globbing that ended with an
open brace if there were brackets or braces before it. Like "{}{" or
"[0-1]{"
2 - curl wouldn't properly reject empty lists so that "{}{}" would
result in curl getting (nil) strings in the output.
3 - By using strtoul() instead of sscanf() the code will now detected
over and underflows. It now also better parses the step argument to only
accept positive numbers and only step counters that is smaller than the
delta between the maximum and minimum numbers.
4 - By switching to unsigned longs instead of signed ints for the
counters, the max values for []-ranges are now very large (on 64bit
machines).
5 - Bumped the maximum number of globs in a single URL to 100 (from 10)
6 - Simplified the code somewhat and now it stores fixed strings as
single- entry lists. That's also one of the reasons why I did (5) as now
all strings between "globs" will take a slot in the array.
Added test 1234 and 1235 to verify. Updated test 87.
This commit fixes three separate bug reports.
Bug: http://curl.haxx.se/bug/view.cgi?id=1264
Bug: http://curl.haxx.se/bug/view.cgi?id=1265
Bug: http://curl.haxx.se/bug/view.cgi?id=1266
Reported-by: Will Dietz
CURLOPT_DNS_USE_GLOBAL_CACHE broke in commit c43127414d (been
broken since the libcurl 7.29.0 release). While this option has been
documented as deprecated for almost a decade and nobody even reported
this bug, it should remain functional.
Added test case 1512 to verify
This is a regression as this logic used to work. It isn't clear when it
broke, but I'm assuming in 7.28.0 when we went all-multi internally.
This likely never worked with the multi interface. As the failed
connection is detected once the multi state has reached DO_MORE, the
Curl_do_more() function was now expanded somewhat so that the
ftp_do_more() function can request to go "back" to the previous state
when it makes another attempt - using PASV.
Added test case 1233 to verify this fix. It has the little issue that it
assumes no service is listening/accepting connections on port 1...
Reported-by: byte_bucket in the #curl IRC channel
The internal function that's used to detect known file extensions for
the default Content-Type got the the wrong pointer passed in when
CURLFORM_BUFFER + CURLFORM_BUFFERPTR were used. This had the effect that
strlen() would be used which could lead to an out-of-bounds read (and
thus segfault). In most cases it would only lead to it not finding or
using the correct default content-type.
It also showed that test 554 and test 587 were testing for the
previous/wrong behavior and now they're updated as well.
Bug: http://curl.haxx.se/bug/view.cgi?id=1262
Reported-by: Konstantin Isakov
This reverts commit 7ed25cc, reinstating commit 8ec2cb5.
As of 18-jul-2013 we still do have code in libcurl that makes use of these
memory functions. Commit 8ec2cb5 comment still applies and is yet valid.
These memory functions are solely used in Windows builds, so all related
code is protected with '#ifdef WIN32' preprocessor conditional compilation
directives.
Specifically, wcsdup() _wcsdup() are used when building a Windows target with
UNICODE and USE_WINDOWS_SSPI preprocessor symbols defined. This is the case
when building a Windows UNICODE target with Windows native SSL/TLS support
enabled.
Realizing that wcsdup() _wcsdup() are used is a bit tricky given that usage
of these is hidden behind _tcsdup() which is MS way of dealing with code
that must tolerate UNICODE and non-UNICODE compilation. Additionally, MS
header files and those compatible from other compilers use this preprocessor
conditional compilation directive in order to select at compilation time
whether 'wide' or 'ansi' MS API functions are used.
Without this code, Windows build targets with Windows native SSL/TLS support
enabled and MemoryTracking support enabled misbehave in tracking memory usage,
regardless of being a UNICODE enabled build or not.
Fixed issue with static build for MSVC2010.
After some investigation I've discovered known issue
http://public.kitware.com/Bug/view.php?id=11240 When .rc file is linked
to static lib it fails with following linker error
LINK : warning LNK4068: /MACHINE not specified; defaulting to X86
file.obj : fatal error LNK1112: module machine type 'x64' conflicts with
target machine type 'X86'
Fix add target property /MACHINE: for MSVC generation.
Also removed old workarounds - it caused errors during msvc build.
Bug: http://curl.haxx.se/mail/lib-2013-07/0046.html
When doing multi-part formposts, libcurl used a pseudo-random value that
was seeded with time(). This turns out to be bad for users who formpost
data that is provided with users who then can guess how the boundary
string will look like and then they can forge a different formpost part
and trick the receiver.
My advice to such implementors is (still even after this change) to not
rely on the boundary strings being cryptographically strong. Fix your
code and logic to not depend on them that much!
I moved the Curl_rand() function into the sslgen.c source file now to be
able to take advantage of the SSL library's random function if it
provides one. If not, try to use the RANDOM_FILE for seeding and as a
last resort keep the old logic, just modified to also add microseconds
which makes it harder to properly guess the exact seed.
The formboundary() function in formdata.c is now using 64 bit entropy
for the boundary and therefore the string of dashes was reduced by 4
letters and there are 16 hex digits following it. The total length is
thus still the same.
Bug: http://curl.haxx.se/bug/view.cgi?id=1251
Reported-by: "Floris"
RFC3986 details how a path part passed in as part of a URI should be
"cleaned" from dot sequences before getting used. The described
algorithm is now implemented in lib/dotdot.c with the accompanied test
case in test 1395.
Bug: http://curl.haxx.se/bug/view.cgi?id=1200
Reported-by: Alex Vinnik
The initial fix to only compare full path names were done in commit
04f52e9b4d but found out to be incomplete. This takes should make the
change more complete and there's now two additional tests to verify
(test 31 and 62).
After curl_multi_wait() returns, this test checked that we got exactly
one file descriptor told to read from, but we cannot be sure that is
true. curl_multi_wait() will sometimes return earlier without any file
descriptor to handle, just just because it is a suitable time to call
*perform().
This problem showed up with commit 29bf0598.
Bug: http://curl.haxx.se/mail/lib-2013-06/0029.html
Reported-by: Fabian Keil
If an ipv6-address is provided to CONNECT, the last hexadecimal group in
the address will be used as the test number! For example the address
"[1234::ff]" would be treated as test case 255.
When sending the HTTP Authorization: header for digest, the user name
needs to be escaped if it contains a double-quote or backslash.
Test 1229 was added to verify
Reported and fixed by: Nach M. S
Bug: http://curl.haxx.se/bug/view.cgi?id=1230
I found a bug which cURL sends cookies to the path not to aim at.
For example:
- cURL sends a request to http://example.fake/hoge/
- server returns cookie which with path=/hoge;
the point is there is NOT the '/' end of path string.
- cURL sends a request to http://example.fake/hogege/ with the cookie.
The reason for this old "feature" is because that behavior is what is
described in the original netscape cookie spec:
http://curl.haxx.se/rfc/cookie_spec.html
The current cookie spec (RFC6265) clarifies the situation:
http://tools.ietf.org/html/rfc6265#section-5.2.4
This reverts commit 8ec2cb5544.
We don't have any code anywhere in libcurl (or the curl tool) that use
wcsdup so there's no such memory use to track. It seems to cause mild
problems with the Borland compiler though that we may avoid by reverting
this change again.
Bug: http://curl.haxx.se/mail/lib-2013-05/0070.html
If the mail sent during the transfer contains a terminating <CRLF> then
we should not send the first <CRLF> of the EOB as specified in RFC-5321.
Additionally don't send the <CRLF> if there is "no mail data" as the
DATA command already includes it.
