Two fixes:
1. Force output file format to be stream-lf so that partial downloads
can be continued.
This should have minor impact as if the file does not exist, it was
created with stream-lf format. The only time this was an issue is if
there was already an existing file with a different format.
2. Fix file uploads are now fixed.
a. VMS binary files such as ZIP archives are now uploaded
correctly.
b. VMS text files are read once to get the correct size
and then converted to line-feed terminated records as
they are read into curl.
The default VMS text formats do not contain either line-feed or
carriage-return terminated records. Those delimiters are added by the
operating system file read calls if the application requests them.
Bug: http://curl.haxx.se/bug/view.cgi?id=496
We no longer pass our 'bool' data type variables nor constants as
an argument to my_setopt(), instead we use proper 1L or 0L values.
This also fixes macro used to pass string argument for CURLOPT_SSLCERT,
CURLOPT_SSLKEY and CURLOPT_EGDSOCKET using my_setopt_str() instead of
my_setopt().
This also casts enum or int argument data types to long when passed to
my_setopt_enum().
Implement wrappers around strtod to convert the user argument to a
double with sane error checking. Use this to allow --max-time and
--connect-timeout to accept decimal values instead of strictly integers.
The manpage is updated to make mention of this feature and,
additionally, forewarn that the actual timeout of the operation can
vary in its precision (particularly as the value increases in its
decimal precision).
The list of unsafe functions currently consists of sprintf, vsprintf,
strcat, strncat and gets.
Subsequently, some existing code needed updating to avoid warnings on
this.
The this_url pointer wasn't being initialized, so if strdup() would return
null when copying the filename in a metalink file, then hilarity would
ensue during the cleanup phase. This change was brought to you by clang,
which noticed this and raised a warning.
config_h.com is a new file that generates a config.h file based on the
curl_config.h.in file and a quick scan of the configure script. This is
actually a generic procedure that is shared with other VMS packages.
The existing pre-built config-vms.h had over 100 entries that were not
correct and in some cases conflicted with the build options available in
the build_vms.com.
generate_config_vms_h_curl.com is a helper procedure to the
config_h.com. It covers the cases that the generic config_h.com is not
able to figure out, and accepts input from the build_vms.com procedure.
build_curlbuild_h.com is a new file to generate the curlbuild.h file
that Curl is now using when it is using a curl_config.h file.
post-config-vms.h is a new file that is needed to provide VMS specific
definitions, and most of them need to be set before the system header
files are included.
The VMS build procedure is fixed:
1. Fixed to link in the correct HP ssl library.
2. Fixed to detect if HP Kerberos is installed.
3. Fixed to detect if HP LDAP is installed.
4. Fixed to detect if gnv$libzshr is installed.
5. Simplified the input parameter parsing to not use a loop.
6. Warn that 64 bit pointer option support is not complete
in comments.
7. Default to IEEE floating if platform supports it so
resulting libcurl will be compatible with other
open source projects on VMS.
8. Default to LARGEFILE if platform supports it.
9. Default to enable SSL, LDAP, Kerberos, libz
if the libraries are present.
10. Build with exact case global symbols for libcurl.
11. Generate linker option file needed.
12. Compiler list option only commonly needed items.
13. fulllist option for those who really want it.
14. Create debug symbol file on Alpha, IA64.
This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:
f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files
This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:
c087374... curl_setup.h: remove redundant include guard
This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:
13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1
Start of related discussion thread:
http://curl.haxx.se/mail/lib-2013-01/0012.html
Asking for confirmation on pushing this revertion commit:
http://curl.haxx.se/mail/lib-2013-01/0048.html
Confirmation summary:
http://curl.haxx.se/mail/lib-2013-01/0079.html
NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.
lib/curl_imap.h
lib/curl_smtp.h
Print "parsing (...) OK" only when no warnings are generated. If
no file is found in Metalink, treat it FAILED.
If no digest is provided, print WARNING in parse_metalink().
Also print validating FAILED after download.
These changes make tests 2012 to 2016 pass.
Including headers in response body will break Metalink XML parser.
If it is included in the file described in Metalink XML, hash check
will fail. Therefore, --include should be ignored if --metalink is
used.
The noprogress and isatty in Configurable are global, in a sense
that they persist in one curl invocation. Currently once one
download writes its response data to tty, they are set to FALSE
and they are not restored on successive downloads. This change
first backups the current noprogress and isatty, and restores
them when download does not write its data to tty.
In this change, --metalink option no longer takes argument. If
it is specified, given URIs are processed as Metalink XML file.
If given URIs are remote (e.g., http URI), curl downloads it
first. Regardless URI is local file (e.g., file URI scheme) or
remote, Metalink XML file is not written to local file system and
the received data is fed into Metalink XML parser directly. This
means with --metalink option, filename related options like -O
and -o are ignored.
Usage examples:
$ curl --metalink http://example.org/foo.metalink
This will download foo.metalink and parse it and then download
the URI described there.
$ curl --metalink file://foo.metalink
This will parse local file foo.metalink and then download the URI
described there.
Additionally, make hash checking ability mandatory in order to allow metalink
support in curl.
A command line option could be introduced to skip hash checking at runtime,
but the ability to check hashes should always be built-in when providing
metalink support.
Metalink file contains several hash types of checksums, such as
md5, sha-1, sha-256, etc. To deal with these checksums, I created
abstraction layer based on lib/curl_md5.h and
lib/md5.c. Basically, they are almost the same but I changed the
code so that it is not hash type dependent. Currently,
GNUTLS(nettle or gcrypt) and OpenSSL functions are supported.
Checksum checking is done by reopening download file. If there
is an I/O error, the current implementation just prints error
message and does not try next resource.
In this patch, the supported hash types are: md5, sha-1 and sha-256.
Filenames contained in Metalink file can include directory information.
Filenames are unique in Metalink file, taking into account the directory
information. So we need to create the directory hierarchy.
Curl has --create-dirs option, but we create directory hierarchy for
Metalink downloads regardless of the option value.
This patch also put metalink int variable outside of HAVE_LIBMETALINK
guard. This reduces the number of #ifdefs.
This change adds experimental Metalink support to curl.
To enable Metalink support, run configure with --with-libmetalink.
To feed Metalink file to curl, use --metalink option like this:
$ curl -O --metalink foo.metalink
We use libmetalink to parse Metalink files.
To achieve this, first new structure HeaderData is defined to hold
necessary data to perform header-related work. Then tool_header_cb now
receives HeaderData pointer as userdata. All header-related work
(currently, dumping header and Content-Disposition inspection) are done
in this callback function. HeaderData.outs->config is used to determine
whether each work is done.
Unit tests were also updated because after this change, curl code always
sets CURLOPT_HEADERFUNCTION and CURLOPT_HEADERDATA.
Tested with -O -J -D, -O -J -i and -O -J -D -i and all worked fine.
Explicit conversion to 'long' of curl_easy_setopt() third argument for options
CURLOPT_HTTPAUTH and CURLOPT_PROXYAUTH given that this is how its bitmasks are
docummented to be used.
Configuration files such as curl_config.h and all config-*.h no longer exist
nor are generated/copied into 'src' directory, now these only exist in 'lib'
directory from where curl tool sources uses them.
Additionally old src/setup.h has been refactored into src/tool_setup.h which
now pulls lib/setup.h
The possibility of a makefile needing an include path adjustment exists.
By modifying the parameter list for ourWriteOut() and passing the
OutStruct that collects data in tool_operate, we get access to the
remote name that we're writing to. Shell scripters should find this
useful when used in conjuntion with the --remote-header-name option.
Yang Tse