These verfy that the 'memory tracking' subsystem is actually doing its
job when using curl tool (#96), a test in libtest (#558) and also a unit
test (#1330), in order to prevent regressions in this functionallity.
Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.
Two new options for limiting the number of connections:
CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.
CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.
Several new options for pipelining:
CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).
CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.
CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.
CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.
CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.
See the curl_multi_setopt() man page for details.
This bug report properly identified that when doing SMTP and aborting
the transfer with a callback, it must be considered aborted prematurely
by the code to avoid QUIT etc to be attempted as that would cause a
hang.
The new test case 1507 verifies this behavior.
Reported by: Patricia Muscalu
Bug: http://curl.haxx.se/bug/view.cgi?id=1184
When a connection is no longer used, it is kept in the cache. If the
cache is full, the oldest idle connection is closed. If no connection is
idle, the current one is closed instead.
Remove internal separated behavior of the easy vs multi intercace.
curl_easy_perform() is now using the multi interface itself.
Several minor multi interface quirks and bugs have been fixed in the
process.
Much help with debugging this has been provided by: Yang Tse
Automake documents that doing this will make it choose a different name
for intermediate object files even when sharing source files across
targets of same Makefile.am.
Up to automake 1.13.1 target's intermediate object files were placed
in the build subdirectory of the target. We depended on this, probably
undocumented behavior, to achieve same behavior as if a per-target flag
had been specified when building targets that actually belong to
different Makefile.am files.
It seems automake 1.13.2 is going to break behavior mentioned above.
So, lets use a documented behavior in order to achieve same purpose,
across automake versions, no matter where automake wishes to place
intermediate object files.
Our build targets that already were using a per-target '_CFLAGS' or
'_CPPFLAGS' need no 'fixing', these were already 'fixed'. The only
Makefile.am or Makefile.in files in libcurl's source tree touched by
this 'fix' are tests/libtest/Makefile.inc and tests/unit/Makefile.inc.
This commit renames lib/setup.h to lib/curl_setup.h and
renames lib/setup_once.h to lib/curl_setup_once.h.
Removes the need and usage of a header inclusion guard foreign
to libcurl. [1]
Removes the need and presence of an alarming notice we carried
in old setup_once.h [2]
----------------------------------------
1 - lib/setup_once.h used __SETUP_ONCE_H macro as header inclusion guard
up to commit ec691ca3 which changed this to HEADER_CURL_SETUP_ONCE_H,
this single inclusion guard is enough to ensure that inclusion of
lib/setup_once.h done from lib/setup.h is only done once.
Additionally lib/setup.h has always used __SETUP_ONCE_H macro to
protect inclusion of setup_once.h even after commit ec691ca3, this
was to avoid a circular header inclusion triggered when building a
c-ares enabled version with c-ares sources available which also has
a setup_once.h header. Commit ec691ca3 exposes the real nature of
__SETUP_ONCE_H usage in lib/setup.h, it is a header inclusion guard
foreign to libcurl belonging to c-ares's setup_once.h
The renaming this commit does, fixes the circular header inclusion,
and as such removes the need and usage of a header inclusion guard
foreign to libcurl. Macro __SETUP_ONCE_H no longer used in libcurl.
2 - Due to the circular interdependency of old lib/setup_once.h and the
c-ares setup_once.h header, old file lib/setup_once.h has carried
back from 2006 up to now days an alarming and prominent notice about
the need of keeping libcurl's and c-ares's setup_once.h in sync.
Given that this commit fixes the circular interdependency, the need
and presence of mentioned notice is removed.
All mentioned interdependencies come back from now old days when
the c-ares project lived inside a curl subdirectory. This commit
removes last traces of such fact.
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:
f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files
This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:
c087374... curl_setup.h: remove redundant include guard
This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:
13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1
Start of related discussion thread:
http://curl.haxx.se/mail/lib-2013-01/0012.html
Asking for confirmation on pushing this revertion commit:
http://curl.haxx.se/mail/lib-2013-01/0048.html
Confirmation summary:
http://curl.haxx.se/mail/lib-2013-01/0079.html
NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.
lib/curl_imap.h
lib/curl_smtp.h
BLANK_AT_MAKETIME may be used in our Makefile.am files to blank
LIBS variable used in generated makefile at makefile processing
time. Doing this functionally prevents LIBS from being used for
all link targets in given makefile.
DNS cache entries populated with CURLOPT_RESOLVE were not properly freed
again when done using the multi interface.
Test case 1502 added to verify.
Bug: http://curl.haxx.se/bug/view.cgi?id=3575448
Reported by: Alex Gruz
Since automake 1.12.4, the warnings are issued on running automake:
warning: 'INCLUDES' is the old name for 'AM_CPPFLAGS' (or '*_CPPFLAGS')
Avoid INCLUDES and roll these flags into AM_CPPFLAGS.
Compile tested on:
Ubuntu 10.04 (automake 1:1.11.1-1)
Ubuntu 12.04 (automake 1:1.11.3-1ubuntu2)
Arch Linux (automake 1.12.4)
As pointed out in Bug report #3579064, curl_multi_perform() would
wrongly use a blocking mechanism internally for some commands which
could lead to for example a very long block if the LIST response never
showed.
The solution was to make sure to properly continue to use the multi
interface non-blocking state machine.
The new test 1501 verifies the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3579064
Reported by: Guido Berhoerster
Fixed tests/libtest/libntlmconnect.c:52: warning: call to
'_curl_easy_getinfo_err_long' declared with attribute warning:
curl_easy_getinfo expects a pointer to long for this info
The test would hang and get aborted with a "ABORTING TEST, since it
seems that it would have run forever." until I prevented that from
happening.
I also fixed the data file which got broken CRLF line endings when I
sucked down the path from Joe's repo == my fault.
Removed #37 from KNOWN_BUGS as this fix and test case verifies exactly
this.
Explicit conversion to 'long' of curl_easy_setopt() third argument for options
CURLOPT_HTTPAUTH and CURLOPT_PROXYAUTH given that this is how its bitmasks are
docummented to be used.
configure script now provides conditional definitions for Makefile.am
that result in CURL_HIDDEN_SYMBOLS being defined by resulting makefiles
when appropriate.
Additionally, configure script option for symbol hiding control is now
named --enable-symbol-hiding --disable-symbol-hiding. While still valid,
old option name --enable-hidden-symbols --disable-hidden-symbols will
be deprecated in some future release.
BUILDING_LIBCURL and CURL_STATICLIB are no longer defined in curl_config.h,
configure will generate appropriate conditionals so that mentioned symbols
get defined and used in Makefiles at compilation time
With FOLLOWLOCATION enabled. When a 3xx page is downloaded and the
download size was known (like with a Content-Length header), but the
subsequent URL (transfered after the 3xx page) was chunked encoded, then
the previous "known download size" would linger and cause the progress
meter to get incorrect information, ie the former value would remain
being sent in. This could easily result in downloads that were WAY
larger than "expected" and would cause >100% outputs with the curl
command line tool.
Test case 599 was created and it was used to repeat the bug and then
verify the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3510057
Reported by: Michael Wallner
When CURLOPT_REFERER has been used, curl_easy_reset() did not properly
clear it.
Verified with the new test 598
Bug: http://curl.haxx.se/bug/view.cgi?id=3481551
Reported by: Michael Day
Allows tests from the libtest subdir to generate log traces
similar to those of curl with --tracetime and --trace-ascii
options but with output going to stderr.
1- Two new error codes are introduced.
CURLE_FTP_ACCEPT_FAILED to be set whenever ACCEPTing fails because of
FTP server connected.
CURLE_FTP_ACCEPT_TIMEOUT to be set whenever ACCEPTing timeouts.
Neither of these errors are considered fatal and control connection
remains OK because it could just be a firewall blocking server to
connect to the client.
2- One new setopt option was introduced.
CURLOPT_ACCEPTTIMEOUT_MS
It sets the maximum amount of time FTP client is going to wait for a
server to connect. Internal default accept timeout is 60 seconds.
As commit ce896875f8 fixed a timer that accidentally had been moved in
code and then returned a bad timer, the lib500.c code (used in test 500
and some others) now verifies 5 timers against each other to verify that
they have the correct relative values. We cannot compare against
absolute values as the timings will vary a lot.
Skip a floating point addition operation when integral part of time difference
is zero. This avoids potential floating point addition rounding problems while
preserving decimal part value.
591 -> FTP multi PORT and 425 on upload
592 -> FTP multi PORT and 421 on upload
593 -> FTP multi PORT upload, no data conn and no transient neg. reply
594 -> FTP multi PORT upload, no data conn and no positive prelim. reply
1206 -> FTP PORT and 425 on download
1207 -> FTP PORT and 421 on download
1208 -> FTP PORT download, no data conn and no transient negative reply
1209 -> FTP PORT download, no data conn and no positive preliminary reply
This test is created to verify Rene Bernhardt's patch which makes sure
libcurl properly _not_ deals with Negotiate if not asked to even if the
proxy says it can serve it.
When doing a multipart formpost with a read callback, and that callback
returns CURL_READFUNC_ABORT, that return code must be properly
propagated back and handled accordingly. Previously it would be handled
as a zero byte read which would cause a hang!
Added test case 587 to verify. It uses the lib554.c source code with a
small ifdef.
Reported by: Anton Bychkov
Bug: http://curl.haxx.se/mail/lib-2011-10/0097.html
Added missing memoryTracking to test cases 560 and 583. If this triggers
leak detection on these, it only means that previously it was going unnoticed.
A regression where CURLFORM_BUFFER stopped to properly insert the file
name part in the formpart. Bug introduced in commit f851f76857.
Added CURLFORM_BUFFER use to test 554 to verify this.
Bug: http://curl.haxx.se/mail/lib-2011-07/0176.html
Reported by: Henry Ludemann
CURLM_CALL_MULTI_PERFORM stopped being a valid return code from
curl_multi_perform back in 7.20.0. All the libcurl tests are ajusted to
this and no longer check for this return code. Makes them simpler.
The CURLFORM_STREAM is documented to only insert a file name (and thus
look like a file upload) in the part if CURLFORM_FILENAME is set, but in
reality it always inserted a filename="" and if CURLFORM_FILENAME wasn't
set, it would insert insert rubbish (or possibly crash).
This is now fixed to work as documented, and test 554 has been extended
to verify this.
Reported by: Sascha Swiercy
Bug: http://curl.haxx.se/mail/lib-2011-06/0070.html
curl-config --version didn't output the correct version string (bug
introduced in commit 0355e33b5f), and unfortunately the test
case 1022 that was supposed to check for this was broken.
This change fixes the test to detect this problem and it fixes the
output.
Bug: http://curl.haxx.se/bug/view.cgi?id=3288727
This test case is meant to verify that the logic in commit
60172a0446 actually works. This test failed for me before that
change and it works after it.
All C and H files now (should) feature the proper project curl source
code header, which includes basic info, a copyright statement and some
basic disclaimers.
Add test 582 for uploading a file using sftp and the multi interface.
(Patch and test slightly tweaked by Daniel Stenberg)
Initially marked as disabled until it is fixed in the source.
The HTTP parser allocated memory on each received Location: header
without properly freeing old data. Starting now, the code only considers
the first Location: header and will blissfully ignore subsequent ones.
Bug: http://curl.haxx.se/bug/view.cgi?id=3165129
Reported by: Martin Lemke
Test 580 is removed again for two reasons:
1) Some compilers aren't satisfied by just a data variable called 'test'
when first.o wants a function called 'test'. The Solaris compiler says
"ld: warning: symbol `test' has differing types:" while the AIX compiler
downright rejects it.
2) Test case 1119 that was added after this test is way more complete
and cover everything test 580 does and more without introducing the same
problems.
The new perl script mk580.pl generates a C table in a fresh source file
named lib580.c and if that compiles fine we know that the file
docs/libcurl/symbols-in-versions at least doesn't include any symbols
that are misspelled.
An additional feature would be to somehow scan curl/curl.h and compare
with symbols-in-versions to see if there are symbols missing.
The date format in RFC822 allows that the seconds part of HH:MM:SS is
left out, but this function didn't allow it. This change also includes a
modified test case that makes sure that this now works.
Reported by: Matt Ford
Bug: http://curl.haxx.se/bug/view.cgi?id=3076529
When curl calls a function from that library then it needs to
explicitly link to the library instead of piggybacking on
libcurl's own dependency. Without this, GNU ld with the
--no-add-needed flag fails when linking (which Fedora now does
by default).
Reported by: Quanah Gibson-Mount
Bug: http://curl.haxx.se/mail/lib-2010-09/0085.html
The 66 bytes checked are those 38 bytes with the chunked encoding
headers added: 8+8+10+35+5 = 66
The three-letter words become 8 bytes on the wire because they are sent
like: "3\r\none\r\n"
... and there's the trailing 5 bytes write after the four lines since
the final chunk is sent (which is "0\r\n\r\n").
A shared library tests/libtest/.libs/lihostname.so is preloaded in NTLM
test-cases to override the system implementation of gethostname(). It
makes it possible to test the NTLM authentication for exact match, and
this way test the implementation of MD4 and DES.
If LD_PRELOAD doesn't work, a debug build willl also workk as debug
builds are now made to prefer a specific environment variable and will
then return that content as host name instead of the actual one.
Kamil wrote the bulk of this, Daniel Stenberg polished it.
curl_easy_getinfo() called with a pointer to long instead of double
would sigbus on RISC processors (e.g. MIPS) due to wrong alignment
of pointer address.
Dirk Manske reported a regression. When connecting with the multi
interface, there were situations where libcurl wouldn't store
connect time correctly as it used to (and is documented to) do.
Using his fine sample program we could repeat it, and I wrote up
test case 573 using that code. The problem does not easily show
itself using the local test suite though.
The fix, also as suggested by Dirk, is a bit on the ugly side as
it adds yet another call to Curl_verboseconnect() and setting the
TIMER_CONNECT time. That situation is subject for some closer
inspection in the future.
POST using a read callback, with Digest authentication and
"Transfer-Encoding: chunked" enforced. I would then cause the first request
to be wrongly sent and then basically hang until the server closed the
connection. I fixed the problem and added test case 565 to verify it.
mail posted to the http-state mailing list, from Adam Barth, and is said to be
the set of date formats the Chrome browser code is tested against:
http://www.ietf.org/mail-archive/web/http-state/current/msg00129.html
libcurl parses most of them identically, but not all of them.
If the CURLOPT_PORT option is used on an FTP URL like
"ftp://example.com/file;type=A" the ";type=A" is stripped off.
I added test case 562 to verify, only to find out that I couldn't repeat
this bug so I hereby consider it not a bug anymore!
This test was added after the HTTPS-using-multi-interface with OpenSSL
regression of 7.19.1 to hopefully prevent this embarassing mistake from
appearing again... Unfortunately the bug wasn't triggered by this test, which
presumably is because the connect to a local server is too fast/different
compared to the real/distant servers we saw the bug happen with.
fix for it. It occured when you did a FTP transfer using
CURLFTPMETHOD_SINGLECWD and then did another one on the same easy handle but
switched to CURLFTPMETHOD_NOCWD. Due to the "dir depth" variable not being
cleared properly. Scott's test case is now known as test 539 and it
verifies the fix.
curl_easy_getinfo. It returns a pointer to a string with the most recently
used IP address. Modified test case 500 to also verify this feature. The
implementing of this feature was sponsored by Lenny Rachitsky at NeuStar.
when function clock_gettime() is available and the monotonic timer is
also available. Otherwise, in some cases, librt or libposix4 could be used
for linking even when finally not using the clock_gettime() function due
to lack of the monotonic clock.
application to provide data for a multipart with the read callback. Note
that the size needs to be provided with CURLFORM_CONTENTSLENGTH when the
stream option is used. This feature is verified by the new test case
554. This feature was sponsored by Xponaut.
such as the CURLOPT_SSL_CTX_FUNCTION one treat that as if it was a Location:
following. The patch that introduced this feature was done for 7.11.0, but
this code and functionality has been broken since about 7.15.4 (March 2006)
with the introduction of non-blocking OpenSSL "connects".
It was a hack to begin with and since it doesn't work and hasn't worked
correctly for a long time and nobody has even noticed, I consider it a very
suitable subject for plain removal. And so it was done.
(http://curl.haxx.se/bug/view.cgi?id=1850730) I wrote up test case 552. The
test is doing a 70K POST with a read callback and an ioctl callback over a
proxy requiring Digest auth. The test case code is more or less identical to
the test recipe code provided by Spacen Jasset (who submitted the bug report).
callback) over a proxy when NTLM is used as auth with the proxy. The bug
also concerned Digest and was limited to using callback only. Spacen worked
with us to provide a useful patch. I added the test case 547 and 548 to
verify two variations of POST over proxy with NTLM.
is supposed to repeat the bug report "NTLM proxy authentication with
CURLOPT_READDATA seems broken." posted on the curl-library mailing list on dec
3 2007.
function do wrong on all input bytes that are >= 0x80 (decimal 128) due to a
signed / unsigned mistake in the code. I fixed it and added test case 543 to
verify.
CURLOPT_NOBODY enabled but not CURLOPT_HEADER, libcurl wouldn't do TYPE
before it does SIZE which makes it less useful. I walked over the code and
made it do this properly, and added test case 542 to verify it.
- Bug report #1792649 (http://curl.haxx.se/bug/view.cgi?id=1792649) pointed
out a problem with doing an empty upload over FTP on a re-used connection.
I added test case 541 to reproduce it and to verify the fix.
- I noticed while writing test 541 that the FTP code wrongly did a CWD on the
second transfer as it didn't store and remember the "" path from the
previous transfer so it would instead CWD to the entry path as stored. This
worked, but did a superfluous command. Thus, test case 541 now also verifies
this fix.
a new directory listing format that newer libssh2's can provide. This
is probably NOT sufficient to handle all directory listing formats that
server's can provide and should be revisited.
open file limit. In this case the test is skipped with a
message showing this limitation when the number of open files
needed for the test is greater than 256.
Check for lowercase 'bool' type at configuration stage. If not available
provide a suitable replacement with a type definition of 'unsigned char'
in setup_once.h
Move definitions of TRUE and FALSE to setup_once.h
part of the official libcurl API http://curl.haxx.se/lxr/source/lib/README.curlx
The documented way of using them would be to use timeval.c as a source code file.
The above described method works very well when statically linking libcurl and
apps, test programs, but has several drawbacks when you build a true shared
libcurl (i.e. Name space clash at linkage stage as functions are defined more
than once. Windows makefiles are not capable of handling this system of
source-level sharing)
So...
Now testutil.h and testutil.c define and implement tutil_tvnow and tutil_tvdiff
which replace curlx_tvnow and curlx_tvdiff for the libtest programs. Doing this
we avoid the above described problems, and the code in the testsuite does not
impose the need to keep those functions public in libcurl even when not part of
the API.
part of the official libcurl API http://curl.haxx.se/lxr/source/lib/README.curlx
The documented way of using them would be to use timeval.c as a source code file.
The above described method works very well when statically linking libcurl and
apps, test programs, but has several drawbacks when you build a true shared
libcurl (i.e. Name space clash at linkage stage as functions are defined more
than once. Windows makefiles are not capable of handling this system of
source-level sharing)
So...
Now testutil.h and testutil.c define and implement tutil_tvnow and tutil_tvdiff
which replace curlx_tvnow and curlx_tvdiff for the libtest programs. Doing this
we avoid the above described problems, and the code in the testsuite does not
impose the need to keep those functions public in libcurl even when not part of
the API.
open file descriptors is greater than FD_SETSIZE minus SAFETY_MARGIN,
also skip the test if any of the open file descriptors has a number
greater than FD_SETSIZE minus SAFETY_MARGIN.
