moparisthebest/wget
1
0
Fork 0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Code Issues Releases Wiki Activity
3,420 Commits 1 Branch 0 Tags 9.6 MiB
fc8a545bfd
Commit Graph

2148 Commits

Author SHA1 Message Date
Giuseppe Scrivano
9e12b8ca39 fix compiler warnings 
* src/utils.h: Include <stdlib.h>
* src/recur.c: Include "exits.h"
 2015-07-20 15:37:52 +02:00
Hubert Tarasiuk
6064f21c66 Geolocation support for Metalink resources. 
* doc/wget.text: Add information about --preferred-location.
* src/init.c: Add --preferred-location option.
* src/main.c (option_data): Handle --preferred-location argument.
(main): Sort resources based on location if requested.
* src/metalink.c (metalink_res_cmp): Compare based on location if
priority and preference are equal.
* src/options.h (options): Add preferred_location option.
 2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
97389a7497 Support at most one file signature. Adapt comments to libmetalink 0.13. 
* src/metalink.c (retrieve_from_metalink): Add comment about new
libmetalink version. Do not iterate over signatures - support just one.
 2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
225a87d4a2 Move some Metalink-related code from http.c to metalink.c. 
* src/http.c: Move find_key_value, has_key, find_key_values.
* src/metalink.c: To here.
* src/metalink.h: Make them non-static and add prototypes here.
 2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
92a889b278 Unit test for find_key_values. 
* src/http.c: Add test_find_key_values.
* src/test.c (main): Run new test.
* src/test.h: Add test_find_key_values.
 2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
1113e78534 Unit test for has_key. 
* src/http.c: Add test_has_key.
* src/test.c (main): Run new test.
* src/test.h: Add test_has_key.
 2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
70cbd59ed6 Unit test for find_key_value. 
* src/http.c: Add test_find_key_value.
* src/test.c (main): Run new test.
* src/test.h: Add test_find_key_value.
 2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
37b58e3976 Metalink support. 
* bootstrap.conf: Add crypto/sha256
* configure.ac: Look for libmetalink and GPGME
* doc/wget.texi: Add --input-metalink and --metalink-over-http
options description.
* po/POTFILES.in: Add metalink.c
* src/Makefile.am: Add new translation unit (metalink.c)
* src/http.c (http_stat): Add metalink field.
(free_stat): Free metalink field.
(find_key_value): Find value of given key in header string.
(has_key): Check if token exists in header string.
(find_key_values): Find all key=value pairs in header string.
(metalink_from_http): Obtain Metalink metadata from HTTP response.
(gethttp): Call metalink_from_http if requested.
(http_loop): Request Metalink metadata from HTTP response if should be.
Fall back to regular download if no Metalink metadata found.
* src/init.c: Add --input-metalink and --metalink-over-http options
* src/main.c (option_data): Handle --input-metalink and
--metalink-over-http cmd arguments.
(print_help): Print --input-metalink option description.
(main): Retrieve files from Metalink file
* src/metalink.c (retrieve_from_metalink): Download files described by
metalink.
(metalink_res_cmp): Comparator for resources priority-sorting.
* src/metalink.h: Create header for metalink.c
(RES_TYPE_SUPPORTED): Define supported resources media.
(DEFAULT_PRI): Default mirror priority for Metalink over HTTP.
(VALID_PRI_RANGE): Valid priority range.
* src/options.h (options): Add input_metalink option and metalink_over_http
options.
* src/utils.c (hex_to_string): Convert binary data to ASCII-hex.
* src/utils.h (hex_to_string): Add prototype.
* src/wget.h: Add metalink-related error enums
Add METALINK_METADATA flag for document type.
 2015-07-20 15:30:39 +02:00
Romain Bentz
80303366ae Add NULL value check to fix #45289 
* src/recur.c (retrieve_tree): Check return value of url_parse()
 2015-07-15 18:10:08 +02:00
Tim Rühsen
25c9b462bf Change function params to const in src/iri.[ch] 
* iri.h, iri.c: Added const attribute for params of parse_charsset(),
	check_encoding_name(), idn_encode(), idn_decode(),
	remote_to_utf8(), set_uri_encoding(), set_content_encoding().
 2015-07-01 17:15:10 +02:00
Tim Rühsen
77f5a27e65 Work around a libidn <= 1.30 vulnerability 
* src/iri.c: Add _utf8_is_valid() to check UTF-8 sequences before
  passing them to idna_to_ascii_8z().
 2015-07-01 17:15:05 +02:00
Ángel González
ae58d8a78b Fix wgetrc filename creation for Windows 
* init.c/wgetrc_file_name: Remove obsolete code in WINDOWS code path

Reported-by: Gisle Vanem <gvanem@yahoo.no>
 2015-06-27 21:32:48 +02:00
Tim Rühsen
c6ac51d5bc Move test_* function protoypes from test.c to test.h 
* src/test.c: Remove test_* function prototypes, make tests_run static
* src/test.h: Add test_* function protoypes
 2015-06-13 22:34:36 +02:00
Hubert Tarasiuk
8a8d138dcc Support If-Modified-Since header in timestamping mode. 
* src/wget.h: Add IF_MODIFIED_SINCE enum for dt. Add TIMECONV_ERR
enum to uerr_t.
* src/http.c (time_to_rfc1123): Convert time_t do http time.
* src/http.c (initialize_request): Include If-Modified-Since header
if appropriate.
* src/http.c (set_file_timestamp): Separate this code from check_file_output.
* src/http.c (check_file_output): Use set_file_timestamp.
* src/http.c (gethttp): Handle properly 304 return code and 200 if server
ignores If-Modified-Since headers.
* src/http.c (http_loop): Load filename to hstat if condget was requested,
use IF_MODIFIED_SINCE if requested and current timestamp can be obtained.
 2015-05-22 11:08:30 +02:00
Hubert Tarasiuk
0e8d2d4251 Add --if-modified-since option 
* src/init.c: Add to commands array.
* src/main.c: Add to cmdline_option. Add to help message.
* src/options.h: Add to options struct.
 2015-05-22 11:08:30 +02:00
Ander Juaristi
b0820d553b Fixed incorrect handling of reserved chars. 
* src/iri.c (do_conversion): Call url_unescape_except_reserved,
instead of url_unescape.

* src/url.c (url_unescape_1): New static function.
(url_unescape): Calls url_unescape_1 with mask zero. Preserves
same behavior as before. Only code changes.
(url_unescape_except_reserved): New function.

* src/url.h: Added prototype for url_unescape_except_reserved().

When the locale is US-ASCII, URIs that contain special characters
in them are converted to IRIs according to RFC 3987, section 3.2
"Converting URIs to IRIs".
 2015-05-12 21:24:06 +02:00
Darshit Shah
b6b1388fb7 Fix documentation for update_speed_ring() 
* progress.c (update_speed_ring): The comment for the function
    incorrectly stated that the function uses thirty samples from the
    past instead of twenty.

    Reported-By: Yi Li <lovelylich@gmail.com>
 2015-05-07 11:29:07 +05:30
Darshit Shah
9b1dd6dab8 Remove shadowed variable in http.c 
* http.c (gethttp): Rename err to conn_err to prevent shadowed
    variable
 2015-05-04 21:45:26 +05:30
Tim Ruehsen
6b8dfe1d6e Fix format specifier warning 
* src/utils.c (aprintf): Use %d for int argument
 2015-05-03 21:18:47 +02:00
Nikolay Merinov
0e6d6ca963 Fix timestamping and continue behaviour with ftp protocol. 
* src/ftp.c (ftp_loop_internal): Add option `force_full_retrieve' that force to
retrieve full file.
(ftp_retrieve_list): Pass `true' as `force_full_retrieve' option to
`ftp_loop_internal' if we want to download file with newer timestamp than local
copy.
 2015-05-01 00:28:08 +02:00
Rohit Mathulla
3765a1b266 openssl: Read cert from private key file when needed 
* src/openssl.c (ssl_init): Assign opt.cert_{file, type}
  from opt.private_key(_type)
 2015-04-27 19:52:18 +02:00
Rohit Mathulla
8654f7e2e7 Fix double free bug in SSL code 
* src/openssl.c, src/gnutls.c (ssl_init): Copy options using xstrdup
 2015-04-27 19:48:51 +02:00
Hubert Tarasiuk
566696cb82 Single exit point and common cleanup code in gethttp 
* src/http.c (gethttp): Common cleanup for type, message,
  req, resp, head.  Single exit point.
 2015-04-20 10:54:34 +02:00
Tim Rühsen
c579c7bf1e Check memory allocations in WARC code 
* src/warc.c: Remove some memory allocations,
              use xmalloc instead of malloc

Reported-by: Bill Parker <wp02855@gmail.com>
 2015-04-17 22:42:59 +02:00
Tim Rühsen
4dde3e200f Add more const usage to function params 
* warc.c, warc.h: Add const specifier to several function args
 2015-04-17 22:42:59 +02:00
Ángel González
bef5945202 Remove memory leak in idn_encode. 
* src/iri.c (idn_encode): Free buffer from remote_to_utf8
when needed; give meaningful names to variables;
remove excessive comment.
 2015-04-11 21:55:17 +02:00
Hubert Tarasiuk
ac40b84ee1 Fix error in free_vec. 
* src/utils.c (free_vec): Increment pointer instead of its value.

Reported-by: Gisle Vanem <gvanem@yahoo.no>
 2015-04-10 18:06:14 +02:00
Ángel González
45463eaad7 Fix const usage in iri.c 
* src/iri.c (remote_to_utf8): Do not qualify with const the output pointer.
(do_conversion): Use the provided input parameter as const.
(idn_encode): casts to remote_to_utf8 parameters are no longer needed.
* src/iri.h: Adjusted remote_to_utf8 prototype.
* src/url.c: It is no longer necessary to cast new_url to const char.
 2015-04-10 10:23:01 +02:00
Miquel Llobet
d03b40e31e Fixed #44628 honoring RFC 6266 content-disposition 
src/http.c (parse_content_disposition): stores filename* and filename
separately and choses filename* if available.
(test_parse_content_disposition): added new tests.
 2015-04-06 10:30:30 +02:00
Steven M. Schweda
5efb24e4a2 Add option to restrict filenames used VMS. 
* src/options.h (enum restrict_files_os): Define "restrict_vms".
* src/init.c (defaults) [__VMS]: Set "opt.restrict_files_os" to
"restrict_vms".
(cmd_spec_restrict_file_names): honor "vms".
* src/url.c (filechr_not_unix): Define "filechr_not_vms".
(filechr_table): Update for VMS.
(append_uri_pathel): Honor opt.restrict_files_os.
(FN_QUERY_SEP): Update for VMS.
(FN_QUERY_SEP_STR): Update for VMS.
 2015-04-02 15:36:42 +02:00
Hubert Tarasiuk
eae8b1d565 Change semantics of resp_free and request_free in http.c 
* src/http.c (resp_free): Change the semantics of this function.
(request_free): Change the semantics of this function.
(initialize_request): Adjust request_free call.
(establish_connection): Adjust request_free, resp_free calls.
(gethttp): Adjust request_free, resp_free calls.
 2015-04-01 17:01:07 +02:00
Hubert Tarasiuk
045463b814 Do not free request in establish_connection; do it in gethttp 
* src/http.c (establish_connection): Do not free request here (it is
* never allocated here).
* src/http.c (gethttp): Free request before returning if error in
* establish_connection encountered.
 2015-04-01 17:01:07 +02:00
Hubert Tarasiuk
621c313b94 Transform read_header label and goto into a loop 
* src/http.c (gethttp): Replace label and goto statement with a do
loop.
 2015-04-01 17:00:40 +02:00
Hubert Tarasiuk
52a7d0ad85 Factor out set_content_type function from gethttp 
* src/http.c (gethttp): Move some code in...
(set_content_type): ... a new function.
 2015-03-31 17:00:07 +02:00
Giuseppe Scrivano
59e9ef00e6 Factor out some gethttp code 
* src/http.c (gethttp): Move some code in...
(open_output_stream): ... a new function.
 2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
14bbc18512 Factor out some auth gethttp code 
* src/http.c (gethttp): Move some code in...
(check_auth): ... a new function.
 2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
8aa63e482e Factor out some gethttp code 
* src/http.c (gethttp): Move some code in...
(check_file_output): ... a new function.
 2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
0bc2757713 Factor out some connection initialization code for gethttp 
* src/http.c (gethttp): Move some initialization code in...
(establish_connection): ... a new function.
 2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
f8abb9dd00 Factor out some proxy initialization code for gethttp 
* src/http.c (gethttp): Move some initialization code in...
(initialize_proxy_configuration): ... a new function.
 2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
29850e77d0 Factor out some initialization code for gethttp 
* src/http.c (gethttp): Move some initialization code in...
(initialize_request): ... a new function.
 2015-03-18 12:09:54 +01:00
Tim Ruehsen
799c545722 src/ftp.c: make sure warc_tmp becomes closed before return 
Reported-by: Coverity bug #1188044
 2015-03-18 10:46:11 +01:00
Tim Ruehsen
014b1d6041 src/http.c: fix error return of digest_authentication_encode() 
Reported-by: Coverity bug #1188036
 2015-03-18 10:46:05 +01:00
Darshit Shah
cc9f76c5a4 retr.c: Fix memory leak in retrieve_from_file() 
Reported by: Coverity Bug 1188045
 2015-03-14 16:48:30 +05:30
Darshit Shah
53b22974cb html-url.c: Fix potential memory leaks 
Reported by: Coverity Bug 1188050
 2015-03-14 16:48:30 +05:30
Darshit Shah
7d5a7ef9ca main.c: Fix two potential memory leaks 
Reported by: Coverity bug 1188048
 2015-03-14 16:48:30 +05:30
Darshit Shah
735cc220e3 retr.c: Fix two memory leaks when proxy URL is bad 
Reported by: Coverity bug 1188047
 2015-03-14 16:48:29 +05:30
Giuseppe Scrivano
16f1fb1d1f maint: update copyright year ranges to include 2015 2015-03-09 16:32:01 +01:00
Yousong Zhou
91e9a20752 Fix --content-on-error option handling. 
* src/http.c: Log --content-on-error downloads.
* src/retr.c (retrieve_url): Register the download of an error page
when --content-on-error is specified.
 2015-03-09 11:45:01 +01:00
Anderson Goulart
882ed28d59 src/main.c (--no-verbose): don't show progress bar 
Fixes #44431
 2015-03-09 00:44:23 +05:30
Darshit Shah
e316d253fa main.c: Use assertion to test buffer size 2015-03-07 00:38:04 +05:30