Maximum amount of data a header callback is supposed to get in
a single call from libcurl is limited by the lowest value of
CURL_MAX_WRITE_SIZE and CURL_MAX_HTTP_HEADER.
Allow (*curl_write_callback) write callbacks to return
CURL_WRITEFUNC_OUT_OF_MEMORY to properly indicate libcurl of OOM conditions
inside the callback itself.
When we use binary posts and regular ones intermixed on a single command
line, we cannot do strlen() etc on the data to figure out the length
(when inserting '&' and more). We must therefore keep track of the post
data length. Then we also end up setting the libcurl option with the
known size, so that we don't risk that libcurl will do strlen() on the
data.
This has the minor side-effect that --libcurl source codes now always
will use CURLOPT_POSTFIELDSIZE but I don't consider that terribly
damaging.
Bug: http://curl.haxx.se/bug/view.cgi?id=3413181
Reported by: Taneli Vhkangas
Reorder handling of fields to match same order as the one given by current
definition order of 'Configurable' struct fields. Fields currently not handled
marked for further inspection.
Re-enabling MemoryTracking capability on 'src' subdirectory files and torture
tests have uncovered many issues which were going unnoticed in curl tool. So
here we go fixing some of them. Others still remain and should be addressed
ASAP, given that curl tool is used in our test harness also for test server
start verification purposes. There are even non-socket file descriptor leaks.
Feature string literal NTLM_SSO renamed to NTLM_WB.
Preprocessor symbol USE_NTLM_SSO renamed to WINBIND_NTLM_AUTH_ENABLED.
curl's 'long' option 'ntlm-sso' renamed to 'ntlm-wb'.
Fix some comments to make clear that this is actually a NTLM delegation.
As a bonus, this lets our MemoryTracking subsystem track zlib operations.
And also fixes a shortcut some zlib 1.2.x versions took using malloc()
instead of calloc(), which would trigger memory debuggers warnings on
memory being used without having been initialized.
Follow-up to commit 5eb2396cd as that wasn't complete.
At times HEADERFUNCTION+HEADERDATA was set only to have only HEADERDATA
set in the subsequent loop which could cause a NULL to get sent as
userdata to 'header_callback' which wasn't made to handle that.
Now HEADERFUNCTION is explicitly set to NULL if it isn't set to the
callback.
Using this option with an argument being set to one of
none/policy/always instructs libcurl how to deal with GSS
credentials. Or rather how it tells the server that delegation is fine
or not.
Allow configure builds to generate src/config-win32.h
Skip checksrc on src/config-win32.h contents already checked at lib/config-win32.h
Allow multiple -W in checksrc.pl
Allow compilation of libcurl and curl using BSD-style lwIP on Win32.
In order to compile libcurl and curl with this support it is necessary
to edit files lib/config-win32.h and src/config-win32.h and uncomment
a line to make definition of preprocessor symbol USE_LWIPSOCK visible.
Afterwards you can compile as usual.
In order to use compiled library with BSD-style lwIP TCP/IP stack in
your program it is mandatory that you include lwIP header file <lwip/opt.h>
before including <curl/curl.h> or <curl/multi.h>
Compilation has been verified with lwIP 1.4.0 and contrib-1.4.0 from:
http://download.savannah.gnu.org/releases/lwip/lwip-1.4.0.ziphttp://download.savannah.gnu.org/releases/lwip/contrib-1.4.0.zip
Have fun!
Content-disposition headers can provide file names with semicolons which
previously would be cut off at that point.
Added test case 1311 and 1312 to verify -J.
Bug: http://curl.haxx.se/bug/view.cgi?id=3375603
Reported by: Peter Hjalmarsson
First, the -J/--remote-header-name was wrongly sorted in the --help
output as pointed out in bug report #3349271.
Then, I changed the format of the texts to follow the man page better in
that it now uses "-A, --long" intead of "-A/--long". I also made all
additional arguments get written as in "-A, --long FILENAME" instead of
the previous "<filename>" style.
Reported by: Herve Amblard
Bug: http://curl.haxx.se/bug/view.cgi?id=3349271
Improved library search by check_function_exists_concat() macro:
it does not revert the list of libraries any more.
Improved OpenSSL library search: first find zlib, then search for
openssl libraries that may depend on zlib.
For Unix: openssl libraries can now be detected in nonstandard
locations. Supply CMAKE_LIBRARY_PATH to CMake on command line.
Added installation capability (very basic one yet).
The make target checksrc now works in the root makefile and in both the
src and lib directories.
It is also run automatically on "all" if configure --enable-debug was
used.
For now we directly import the Idn* symbols with the linker;
an upcoming release of OWC will have these added to the import
lib normaliz.lib, and prototypes are added to winnnls.h.
It is now possible to use any combination of features without
having to 1st add makefile targets to the main makefile. The
main makefile now passes the 'mingw32-feat1-feat2' as var CFG,
and the ./[lib|src]/Makefile.m32 parses the CFG var to determine
the features to be enabled.
Added CURLOPT_TRANSFER_ENCODING as the option to set to request Transfer
Encoding in HTTP requests (if built zlib enabled). I also renamed
CURLOPT_ENCODING to CURLOPT_ACCEPT_ENCODING (while keeping the old name
around) to reduce the confusion when we have to encoding options for
HTTP.
--tr-encoding is now the new command line option for curl to request
this, and thus I updated the test cases accordingly.
* Rename the object object directory from 'objs' to 'BCC_obj' to be in
sync with my previous patch for lib/Makefile.b32.
* Turn off these warnings to keep the build totally silent (with CBuilder-6
that is).
-w-inl 8026 Functions X are not expanded inline.
-w-pia 8060 Possibly incorrect assignment
-w-pin 8061 Initialization is only partially bracketed
(same added in src/Makefile.b32)
* $(MKDIR) and $(RMDIR) have been replaced with the shell-commands 'md'
and 'rd'. When having MingW/Msys programs 'mkdir.exe' and 'rmdir.exe' in
$PATH, this confuses Borland's make and the result (the cleaning etc.) would
not be as expected.
* Removed the preprocessing step; no need for PP_CMD and the .int files.
curl.exe builds fine w/o and the makefile gets simpler.
* Added a target for creating a compressed hugehelp.c if WITH_ZLIB is defined.
It assumes groff, gzip and perl is available if such an "advanced" users
requests it. Okay? BTW. My groff and Perl needs unix-slashes ('/').
Other perls should handle both forms ('/' and '\').
All C and H files now (should) feature the proper project curl source
code header, which includes basic info, a copyright statement and some
basic disclaimers.
This enables people to specify a path to the netrc file to use.
The new option override --netrc if both are present. However it
does follow --netrc-optional if specified.
Since the feature requires support for TCP_KEEPIDLE and TCP_KEEPINTVL to
function as documented, it now warns if that support is missing when the
option is used.
... to not make the connection between the tool and the libcurl used
tighter than necessary, the tlsauth options are now always present but
if the used libcurl doesn't have TLSAUTH support it will return failure.
Also, replaced strncmp() with strequal to get case insensitive matching.
Instead of reopening the downloaded file, fsetxattr uses the (already
open) file descriptor to attach extended attributes. This makes the
procedure more robust against errors caused by moved or deleted files.
CURLOPT_RESOLVE is a new option that sends along a curl_slist with
name:port:address sets that will populate the DNS cache with entries so
that request can be "fooled" to use another host than what otherwise
would've been used. Previously we've encouraged the use of Host: for
that when dealing with HTTP, but this new feature has the added bonus
that it allows the name from the URL to be used for TLS SNI and server
certificate name checks as well.
This is a first change. Surely more will follow to make it decent.
setxattr is a glibc call to set extended attributes, so configure now
checks for it and the code is adapted to only build when the
functionality is present.
It is often convinient to track back the source of a once downloaded
file; this patch makes curl store the source URL and other metadata
alongside the retrieved file by using the extended attributes (if
supported by the file system and enabled by --xattr).
These haven't worked in at least 8 years due to missing source
files, and most active RiscOS developers these days apparently
cross-compile anyway.
Signed-off-by: James Bursa <james@zamez.org>
Some options, such as the automatic decompression and some SSL related
ones now will bail out if the underlying libcurl doesn't have support
for the particular feature needed.
If the filename contains a backslash, only use filename portion. The
idea is that even systems that don't handle backslashes as path
separators probably want that path removed for convenience.
This flaw is considered a security problem, see the curl security
vulnerability http://curl.haxx.se/docs/adv_20101013.html
Having an open brace without a closing brace caused a segfault.
Having a closing brace too many caused a silent error to occur, which
caused curl to bail out and return an error code but no error message
was shown. It does now!
All error message outputs no longer wrongly get _two_ newlines written
after the error message.
Reported by: Vlad Ureche
Bug: http://curl.haxx.se/bug/view.cgi?id=3083942
Renamed SDK_* to NDK_*; made NDK_* defines overwriteable from
environment; removed now obsolete YACC macro;
moved some curl_config.h defines to IPv6 section since they
are only needed when IPv6 is enabled - this makes libcurl compile
with older NDKs too which were not IPv6-aware.
It was introduced in commit eeb2cb05 along with the -F type=
change. Also fixed a typo in the name of the magic filename=
parameter. Tweaked tests 39 and 173 to better test this path.
The -F option allows some custom parameters within the given string, and
those strings are separated with semicolons. You can for example specify
"name=daniel;type=text/plain" to set content-type for the
field. However, the use of semicolons like that made it not work fine if
you specified one within the content-type, like for:
"name=daniel;type=text/plain;charset=UTF-8"
... as the second one would be seen as a separator and "charset" is no
parameter curl knows anything about so it was just silently discarded.
The new logic now checks if the semicolon and following keyword looks
like a parameter it knows about and if it isn't it is assumed to be
meant to be used within the content-type string itself.
I modified test case 186 to verify that this works as intended.
Reported by: Larry Stone
Bug: http://curl.haxx.se/bug/view.cgi?id=3048988
Added the -br switch to dynamic builds which fixes the issue I saw
with curl's --version output. Added debug info and symfile for debug
builds to linker opts. Added DLL loader for wlink back, but this time
dependend on wlink version.
Patch posted to the list by malak.jiri AT gmail.com.
The var %MAKEFLAGS is only set in 3 cases: if set as environment
var or as macro definition from commandline, and either with the
-u or -ms switch. Since all these cases are unlikely for the average
user it should be safe to only test if %MAKEFLAGS is defined; this
has the benefit that now all other switches can be used again in
addition to the -u which was formerly not possible.
The --retry logic does retry HTTP when some specific response codes are
returned, but because the -f option sets the CURLOPT_FAILONERROR to
libcurl, the return codes are different for such situations and then the
curl tool failed to consider it for retrying.
Reported by: Mike Power
Bug: http://curl.haxx.se/bug/view.cgi?id=3037362
lib/Makefile.Watcom works fine already, for src/Makefile.Watcom we
need first to tweak src/Makefile.inc a bit - therefore the handtweaked
list still exists for now.
- make both libcurl and curl makefiles use register calling convention
(previously libcurl had stack calling convention).
- added include paths to the Watcom headers so its no longer required
to set the environment vars for this.
- added -wcd=201 to supress compiler warning about unreachable code.
- use macros for all tools, and removed dependency on GNU tools like rm.
- make ipv6 and debug builds controlable via env vars and so make them
optional instead of default.
- commented WINLDAPAPI and WINBERAPI since they broke with OW 1.8, and
it seems they're not needed (anymore?).
- added rule for hugehelp.c.cvs so that it will be created when not
already exist - this is required for building from a release tarball
since there we have no hugehelp.c.cvs, thus compilation broke.
- removed C_ARG creation from lib/Makefile.Watcom and use CFLAGS
directly as done too in src/Makefile.Watcom - this has the benefit
that we will see all active cflags and defines during compile.
- added LINK-ARG to src/Makefile.Watcom in order to better control
linker input.
- a couple of other minor makefile tweaks here and there ...
- added largefile support for Watcom builds to config-win32.h. Not yet
tested if it really works, but should since Win32 supports it.
- added loaddll stuff to speed up builds if supported.
This passes -Werror to gcc when building curl and libcurl,
allowing easy dection of compile warnings.
Signed-off-by: Ben Greear <greearb@candelatech.com>
The --remote-header-name option for the command-line tool assumes that
everything beyond the filename= field is part of the filename, but that
might not always be the case, for example:
Content-Disposition: attachment; filename=file.txt; modification-date=...
This fix chops the filename off at the next semicolon, if there is one.
When getting multiple URLs, curl didn't properly reset the byte counter
after a successful transfer so if the subsequent transfer failed it
would wrongly use the previous byte counter and behave badly (segfault)
because of that. The code assumes that the byte counter and the 'stream'
pointer is well in synch.
Reported by: Jon Sargeant
Bug: http://curl.haxx.se/bug/view.cgi?id=3028241
Since uploading from stdin is very likely to not work with anyauth and
its multi-phase probing for what authentication to actually use, alert
the user about it. Multi-phase negotiate almost certainly will involve
sending data and thus libcurl will need to rewind the stream to send
again, and it cannot do that with stdin.
I think the [REMARK] and commented function calls cluttered the code a
bit too much and made the generated code ugly to read. Now we instead
track the remarks one specially and just lists them at the end of the
generated code more as additional information.
And additionally, don't show function or object pointers actual value
since they make no sense to anyone. Show 'functionpointer' and
'objectpointer' instead.
In the generated code --libcurl makes, all calls to curl_easy_setopt()
that use *_LARGE options now have the value typecasted to curl_off_t, so
that it works correctly for 32bit systems with 64bit curl_off_t type.
curl didn't properly handle escaping characters in a URL with the use of
backslash. It did an attempt, but that failed as reported in bug
3022551. The described example was using the URL
"http://example.com?{AB,C\,D}".
I've now removed the special-handling of letters following the backslash
and I also removed the bad extra check that triggered this particular
bug.
Bug: http://curl.haxx.se/bug/view.cgi?id=3022551
Reported by: Jon Sargeant
The main change is to allow input from user-specified methods,
when they are specified with CURLOPT_READFUNCTION.
All calls to fflush(stdout) in telnet.c were removed, which makes
using 'curl telnet://foo.com' painful since prompts and other data
are not always returned to the user promptly. Use
'curl --no-buffer telnet://foo.com' instead. In general,
the user should have their CURLOPT_WRITEFUNCTION do a fflush
for interactive use.
Also fix assumption that reading from stdin never returns < 0.
Old code could crash in that case.
Call progress functions in telnet main loop.
Signed-off-by: Ben Greear <greearb@candelatech.com>
The feature that uses the file name given in a
Content-disposition: header didn't properly skip trailing
carriage returns and linefeed characters from the end of the file
name when it was given without quotes.
(http://curl.haxx.se/bug/view.cgi?id=2958074) that curl on Windows with
option --trace-time did not use local time when timestamping trace lines.
This could also happen on other systems depending on time souurce.
versions --ftp-ssl and --ftp-ssl-reqd as these options are now used to
control SSL/TLS for IMAP, POP3 and SMTP as well in addition to FTP. The old
option names are still working but the new ones are the prefered ones
(listed and documented).
command is a special "hack" used by the drftpd server, but even though it is
a custom extension I've deemed it fine to add to libcurl since this server
seems to survive and people keep using it and want libcurl to support
it. The new libcurl option is named CURLOPT_FTP_USE_PRET, and it is also
usable from the curl tool with --ftp-pret. Using this option on a server
that doesn't support this command will make libcurl fail.
on FTP errors in the transient 5xx range. Transient FTP errors are in the
4xx range. The code itself only tried on 5xx errors that occured _at login_.
Now the retry code retries on all FTP transfer failures that ended with a
4xx response.
(http://curl.haxx.se/bug/view.cgi?id=2911279)
rework patch that now integrates TFTP properly into libcurl so that it can
be used non-blocking with the multi interface and more. BLKSIZE also works.
The --tftp-blksize option was added to allow setting the TFTP BLKSIZE from
the command line.
though it failed to write a very small download to disk (done in a single
fwrite call). It turned out to be because fwrite() returned success, but
there was insufficient error-checking for the fclose() call which tricked
curl to believe things were fine.
by user 'koresh' introduced the --crlfile option to curl, which makes curl
tell libcurl about a file with CRL (certificate revocation list) data to
read.
read stdin in a non-blocking fashion. This also brings back -T- (minus) to
the previous blocking behavior since it could break stuff for people at
times.
This allows curl(1) to be used as a client-side tunnel for arbitrary stream
protocols by abusing chunked transfer encoding in both the HTTP request and
HTTP response. This requires server support for sending a response while a
request is still being read, of course.
If attempting to read from stdin returns EAGAIN, then we pause our sender.
This leaves curl to attempt to read from the socket while reading from stdin
(and thus sending) is paused.
With the curl memory tracking feature decoupled from the debug build feature,
CURLDEBUG and DEBUGBUILD preprocessor symbol definitions are used as follows:
CURLDEBUG used for curl debug memory tracking specific code (--enable-curldebug)
DEBUGBUILD used for debug enabled specific code (--enable-debug)
Chen pointed out how curl couldn't upload with resume when reading from a
pipe.
This ended up with the introduction of a new return code for the
CURLOPT_SEEKFUNCTION callback that basically says that the seek failed but
that libcurl may try to resolve the situation anyway. In our case this means
libcurl will attempt to instead read that much data from the stream instead
of seeking and that way curl can now upload with resume when data is read
from a stream!
OBJECTPOINT options. Now we've introduced a new function - my_setopt_str -
within the app for setting plain string options to avoid the risk of this
mistake happening.
curl didn't use sprintf() in a way that is documented to work in POSIX but
since we use our own printf() code (from libcurl) that shouldn't be a
problem. Nonetheless I modified the code to not rely on such particular
features and to not cause further raised eyebrowse with no good reason.
plain FTP connections, and it will then allow MKD to fail once and retry the
CWD afterwards. This is especially useful if you're doing many simultanoes
connections against the same server and they all have this option enabled,
as then CWD may first fail but then another connection does MKD before this
connection and thus MKD fails but trying CWD works! The numbers can
(should?) now be set with the convenience enums now called
CURLFTP_CREATE_DIR and CURLFTP_CREATE_DIR_RETRY.
Tests has proven that if you're making an application that uploads a set of
files to an ftp server, you will get a noticable gain in speed if you're
using multiple connections and this option will be then be very useful.
version 1.1 instead of 1.0 like before. This change also introduces the new
proxy type for libcurl called 'CURLPROXY_HTTP_1_0' that then allows apps to
switch (back) to CONNECT 1.0 requests. The curl tool also got a --proxy1.0
option that works exactly like --proxy but sets CURLPROXY_HTTP_1_0.
I updated all test cases cases that use CONNECT and I tried to do some using
--proxy1.0 and some updated to do CONNECT 1.1 to get both versions run.
Curl_sspi_global_init() and Curl_sspi_global_cleanup() which previously were
named Curl_ntlm_global_init() and Curl_ntlm_global_cleanup() in http_ntlm.c
Also adjusted socks_sspi.c to remove the link-time dependency on the Windows
SSPI library using it now in the same way as it was done in http_ntlm.c.
CURLOPT_SOCKS5_GSSAPI_SERVICE and CURLOPT_SOCKS5_GSSAPI_NEC to allow libcurl
to do GSS-style authentication with SOCKS5 proxies. The curl tool got the
options called --socks5-gssapi-service and --socks5-gssapi-nec to enable
these.
They basically offer the same thing the NO_PROXY environment variable only
offered previously: list a set of host names that shall not use the proxy
even if one is specified.
decided it was a good idea to properly document my thoughts in a comment near
the code that was identified as a possible flaw. A false positive as far as I
can see.
(http://curl.haxx.se/bug/view.cgi?id=2154627) which pointed out that libcurl
uses strcasecmp() in multiple places where it causes failures when the
Turkish locale is used. This is because 'i' and 'I' isn't the same letter so
strcasecmp() on those letters are different in Turkish than in English (or
just about all other languages). I thus introduced a totally new internal
function in libcurl (called Curl_ascii_equal) for doing case insentive
comparisons for english-(ascii?) style strings that thus will make "file"
and "FILE" match even if the Turkish locale is selected.
CURLOPT_POST301 (but adds a define for backwards compatibility for you who
don't define CURL_NO_OLDIES). This option allows you to now also change the
libcurl behavior for a HTTP response 302 after a POST to not use GET in the
subsequent request (when CURLOPT_FOLLOWLOCATION is enabled). I edited the
patch somewhat before commit. The curl tool got a matching --post302
option. Test case 1076 was added to verify this.
which caused an error when the second header was dumped due to stdout
being closed. Added test case 1066 to verify. Also fixed a potential
problem where a closed file descriptor might be used for an upload
when more than one URL is given.
when including the OpenSSL header files. This is the recommended setting, this
prevents the undesired inclusion of header files with the same name as those
of OpenSSL but which do not belong to the OpenSSL package. The visible change
from previously released libcurl versions is that now OpenSSl enabled NetWare
builds also define USE_OPENSSL in config files, and that OpenSSL header files
must be located in a subdirectory named 'openssl'.
