Metalink file contains several hash types of checksums, such as
md5, sha-1, sha-256, etc. To deal with these checksums, I created
abstraction layer based on lib/curl_md5.h and
lib/md5.c. Basically, they are almost the same but I changed the
code so that it is not hash type dependent. Currently,
GNUTLS(nettle or gcrypt) and OpenSSL functions are supported.
Checksum checking is done by reopening download file. If there
is an I/O error, the current implementation just prints error
message and does not try next resource.
In this patch, the supported hash types are: md5, sha-1 and sha-256.
Filenames contained in Metalink file can include directory information.
Filenames are unique in Metalink file, taking into account the directory
information. So we need to create the directory hierarchy.
Curl has --create-dirs option, but we create directory hierarchy for
Metalink downloads regardless of the option value.
This patch also put metalink int variable outside of HAVE_LIBMETALINK
guard. This reduces the number of #ifdefs.
This change adds experimental Metalink support to curl.
To enable Metalink support, run configure with --with-libmetalink.
To feed Metalink file to curl, use --metalink option like this:
$ curl -O --metalink foo.metalink
We use libmetalink to parse Metalink files.
To achieve this, first new structure HeaderData is defined to hold
necessary data to perform header-related work. Then tool_header_cb now
receives HeaderData pointer as userdata. All header-related work
(currently, dumping header and Content-Disposition inspection) are done
in this callback function. HeaderData.outs->config is used to determine
whether each work is done.
Unit tests were also updated because after this change, curl code always
sets CURLOPT_HEADERFUNCTION and CURLOPT_HEADERDATA.
Tested with -O -J -D, -O -J -i and -O -J -D -i and all worked fine.
Explicit conversion to 'long' of curl_easy_setopt() third argument for options
CURLOPT_HTTPAUTH and CURLOPT_PROXYAUTH given that this is how its bitmasks are
docummented to be used.
By comparing if a different "progress point" is reached or not since the
previous update, the progress function callback for this now avoids many
superfluous screen updates. This has the nice side-effect that it fixes
a problem that causes a second progress meter line.
The second line output happened because when we use the -# progress
meter, we force a newline output after the transfer in the main loop in
curl, but when libcurl calls the progress callback from
curl_easy_cleanup() it would then output the progress display
again. Possibly the naive newline output is wrong but this optimization
was suitable anyway...
Reported by: Daniel Theron
Bug: http://curl.haxx.se/bug/view.cgi?id=3517418
BUILDING_LIBCURL and CURL_STATICLIB are no longer defined in curl_config.h,
configure will generate appropriate conditionals so that mentioned symbols
get defined and used in Makefiles at compilation time
Configuration files such as curl_config.h and all config-*.h no longer exist
nor are generated/copied into 'src' directory, now these only exist in 'lib'
directory from where curl tool sources uses them.
Additionally old src/setup.h has been refactored into src/tool_setup.h which
now pulls lib/setup.h
The possibility of a makefile needing an include path adjustment exists.
By modifying the parameter list for ourWriteOut() and passing the
OutStruct that collects data in tool_operate, we get access to the
remote name that we're writing to. Shell scripters should find this
useful when used in conjuntion with the --remote-header-name option.
This patch improves the output of curl's --libcurl option by
generating code which builds curl_httppost and curl_slist lists, and
uses symbolic names for enum and flag values. Variants of the
my_setopt macro in tool_setopt.h are added in order to pass extra type
information to the code-generation step in tool_setopt.c.
If curl is configured with --disable-libcurl-option then the macros
call curl_easy_setopt directly.
Fix the str2num() function to not check if the input string starts with a
digit, since strtol() supports numbers prepended with '-' (and '+') too.
This makes the --max-redirs option work as documented.
This new option tells curl to not work around a security flaw in the
SSL3 and TLS1.0 protocols. It uses the new libcurl option
CURLOPT_SSL_OPTIONS with the CURLSSLOPT_ALLOW_BEAST bit set.
Use the new library CURLOPT_TCP_KEEPALIVE rather than disabling this via
the sockopt callback. If --keepalive-time is used, apply the value to
CURLOPT_TCP_KEEPIDLE and CURLOPT_TCP_KEEPINTVL.
We want to continue to the next URL to try even on failures returned
from libcurl. This makes -f with ranges still get subsequent URLs even
if occasional ones return error. This was a regression as it used to
work and broke in the 7.23.0 release.
Added test case 1328 to verify the fix.
Bug: http://curl.haxx.se/bug/view.cgi?id=3481223
Reported by: Juan Barreto
Test case 1315 was added to verify this functionality. When passing in
multiple files to a single -F, the parser would get all confused if one
of the specified files had a custom type= assigned.
Reported by: Colin Hogben
Skip a floating point addition operation when integral part of time difference
is zero. This avoids potential floating point addition rounding problems while
preserving decimal part value.
A regression between 7.22.0 and 7.23.0 -- downloading a file with the
flags -O and -J results in the content being written to stdout if and
only if there was no Content-Disposition header in the http response. If
there is a C-D header with a filename attribute, the output is correctly
written.
Reported by: Dave Reisner
Bug: http://curl.haxx.se/mail/archive-2011-11/0030.html
The progress bar output function would blindly use the terminal width
without bounds checking. When using a very wide terminal that caused a
buffer overflow and segfault.
We now limit the max bar with to 255 columns, and I simplified the code
to avoid an extra snprintf and buffer.
Bug: http://curl.haxx.se/bug/view.cgi?id=3435710
Reported by: Alexey Zakhlestin
As commit 5850cc4808 clarifies, libcurl can deliver header lines that
are longer than CURL_MAX_WRITE_SIZE, only body data is limited to that
size. The curl tool has check (when built debug-enabled) that made the
wrong checks and this new test 1205 verifies that larger headers work.
Previously we required that -S/--show-error was used _after_
-s/--silent. This was slightly confusing since we strive to make
arguments as position independent as possible.
Now, you can use them in any order and the result should still be the
same.
Bug: http://curl.haxx.se/bug/view.cgi?id=3424286
Reported by: Andreas Olsson
Maximum amount of data a header callback is supposed to get in
a single call from libcurl is limited by the lowest value of
CURL_MAX_WRITE_SIZE and CURL_MAX_HTTP_HEADER.
Allow (*curl_write_callback) write callbacks to return
CURL_WRITEFUNC_OUT_OF_MEMORY to properly indicate libcurl of OOM conditions
inside the callback itself.
When we use binary posts and regular ones intermixed on a single command
line, we cannot do strlen() etc on the data to figure out the length
(when inserting '&' and more). We must therefore keep track of the post
data length. Then we also end up setting the libcurl option with the
known size, so that we don't risk that libcurl will do strlen() on the
data.
This has the minor side-effect that --libcurl source codes now always
will use CURLOPT_POSTFIELDSIZE but I don't consider that terribly
damaging.
Bug: http://curl.haxx.se/bug/view.cgi?id=3413181
Reported by: Taneli Vähäkangas
