1
0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Commit Graph

3494 Commits

Author SHA1 Message Date
Hubert Tarasiuk
8a8d138dcc Support If-Modified-Since header in timestamping mode.
* src/wget.h: Add IF_MODIFIED_SINCE enum for dt. Add TIMECONV_ERR
enum to uerr_t.
* src/http.c (time_to_rfc1123): Convert time_t do http time.
* src/http.c (initialize_request): Include If-Modified-Since header
if appropriate.
* src/http.c (set_file_timestamp): Separate this code from check_file_output.
* src/http.c (check_file_output): Use set_file_timestamp.
* src/http.c (gethttp): Handle properly 304 return code and 200 if server
ignores If-Modified-Since headers.
* src/http.c (http_loop): Load filename to hstat if condget was requested,
use IF_MODIFIED_SINCE if requested and current timestamp can be obtained.
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk
0e8d2d4251 Add --if-modified-since option
* src/init.c: Add to commands array.
* src/main.c: Add to cmdline_option. Add to help message.
* src/options.h: Add to options struct.
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk
be4f91737a Add test for condget requests.
* testenv/Test-condget.py: the test
* testenv/Makefile.am: add to tests list
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk
901bc98edf Support conditional GET in testenv server.
* src/exc/server_error.py: Add exception for GET to HEAD fallback.
* src/server/http/http_server.py: Do not send body if 304 return
code requested for a file.
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk
e397a48f6a Implement timestamp support for local files in testenv
* testenv/README: Change timestamp format definition
* testenv/conf/local_files.py: Set proper timestamps
2015-05-22 11:08:30 +02:00
Pär Karlsson
83537f2415 Fix undeclared loop variable in Perl test suite
Reported-by: Hubert Tarasiuk <hubert.tarasiuk@gmail.com>
2015-05-20 10:04:07 +02:00
Ander Juaristi
8682c2612f Make sure Wget does not unescape reserved chars.
* testenv/Test-reserved-chars.py: New file.

* testenv/Makefile.am: Added new test Test-reserved-chars.py.

When following redirections, Wget should not unescape the reserved
characters that might appear in target URLs.
2015-05-12 21:24:11 +02:00
Ander Juaristi
b0820d553b Fixed incorrect handling of reserved chars.
* src/iri.c (do_conversion): Call url_unescape_except_reserved,
instead of url_unescape.

* src/url.c (url_unescape_1): New static function.
(url_unescape): Calls url_unescape_1 with mask zero. Preserves
same behavior as before. Only code changes.
(url_unescape_except_reserved): New function.

* src/url.h: Added prototype for url_unescape_except_reserved().

When the locale is US-ASCII, URIs that contain special characters
in them are converted to IRIs according to RFC 3987, section 3.2
"Converting URIs to IRIs".
2015-05-12 21:24:06 +02:00
Darshit Shah
b6b1388fb7 Fix documentation for update_speed_ring()
* progress.c (update_speed_ring): The comment for the function
    incorrectly stated that the function uses thirty samples from the
    past instead of twenty.

    Reported-By: Yi Li <lovelylich@gmail.com>
2015-05-07 11:29:07 +05:30
Darshit Shah
9b1dd6dab8 Remove shadowed variable in http.c
* http.c (gethttp): Rename err to conn_err to prevent shadowed
    variable
2015-05-04 21:45:26 +05:30
Steven Schubiger
1cc835dc5b paramcheck: use explicit quoting for here-docs
* util/paramcheck.pl: Adjust here-docs
2015-05-04 10:13:24 +05:30
Tim Ruehsen
6b8dfe1d6e Fix format specifier warning
* src/utils.c (aprintf): Use %d for int argument
2015-05-03 21:18:47 +02:00
Nikolay Merinov
0e6d6ca963 Fix timestamping and continue behaviour with ftp protocol.
* src/ftp.c (ftp_loop_internal): Add option `force_full_retrieve' that force to
retrieve full file.
(ftp_retrieve_list): Pass `true' as `force_full_retrieve' option to
`ftp_loop_internal' if we want to download file with newer timestamp than local
copy.
2015-05-01 00:28:08 +02:00
Rohit Mathulla
3765a1b266 openssl: Read cert from private key file when needed
* src/openssl.c (ssl_init): Assign opt.cert_{file, type}
  from opt.private_key(_type)
2015-04-27 19:52:18 +02:00
Rohit Mathulla
8654f7e2e7 Fix double free bug in SSL code
* src/openssl.c, src/gnutls.c (ssl_init): Copy options using xstrdup
2015-04-27 19:48:51 +02:00
Hubert Tarasiuk
566696cb82 Single exit point and common cleanup code in gethttp
* src/http.c (gethttp): Common cleanup for type, message,
  req, resp, head.  Single exit point.
2015-04-20 10:54:34 +02:00
Tim Rühsen
bebe462a67 Silence warning in perl test suite
* tests/WgetTests.pm: Use string comparisons for $valgrind variable
2015-04-17 22:42:59 +02:00
Tim Rühsen
c579c7bf1e Check memory allocations in WARC code
* src/warc.c: Remove some memory allocations,
              use xmalloc instead of malloc

Reported-by: Bill Parker <wp02855@gmail.com>
2015-04-17 22:42:59 +02:00
Tim Rühsen
4dde3e200f Add more const usage to function params
* warc.c, warc.h: Add const specifier to several function args
2015-04-17 22:42:59 +02:00
Darshit Shah
8e0dd0d870 PEP8'ify the Python Test Suite
* testenv/conf/{__init__,authentication,files_crawled,
      hook_sample,reject_header,server_files}.py: Aesthetic changes to
      meet Python PEP8 guidelines
    * testenv/exc/{server_error,test_failed}.py: Same
    * testenv/misc/{colour_terminal,wget_file}.py: Same
    * testenv/server/http/http_server.py: Same
    * testenv/test/base_test.py: Same
2015-04-14 10:59:17 +05:30
Ander Juaristi
c6af2fddee Improved test suite documentation
* README.checkout: Added reference to Python. New section
  "Testing and development". Updated information about test suites.
2015-04-13 19:36:39 +02:00
Ander Juaristi
e7e3227b34 Added support for GDB in Python tests
* base_test.py (gen_cmd_line): Check GDB_TESTS environment variable.
2015-04-13 19:32:57 +02:00
Tim Ruehsen
5c5d45ae49 Added GDB support to Perl tests
* WgetTests.pm (run): Check GDB_TESTS environment variable.
2015-04-13 19:29:43 +02:00
Hubert Tarasiuk
f4072e5d0b Add Valgrind suppression for libidn.so at idna_to_ascii_4z
* tests/WgetTests.pm (run): Include suppression file when running Valgrind.
* tests/valgrind-suppressions: Add suppression for idn_to_ascii_4z.
* tests/Makefile.am: Add valgrind-suppressions to EXTRA_DIST.
2015-04-11 23:16:32 +02:00
Ángel González
bef5945202 Remove memory leak in idn_encode.
* src/iri.c (idn_encode): Free buffer from remote_to_utf8
when needed; give meaningful names to variables;
remove excessive comment.
2015-04-11 21:55:17 +02:00
Hubert Tarasiuk
ac40b84ee1 Fix error in free_vec.
* src/utils.c (free_vec): Increment pointer instead of its value.

Reported-by: Gisle Vanem <gvanem@yahoo.no>
2015-04-10 18:06:14 +02:00
Ángel González
45463eaad7 Fix const usage in iri.c
* src/iri.c (remote_to_utf8): Do not qualify with const the output pointer.
(do_conversion): Use the provided input parameter as const.
(idn_encode): casts to remote_to_utf8 parameters are no longer needed.
* src/iri.h: Adjusted remote_to_utf8 prototype.
* src/url.c: It is no longer necessary to cast new_url to const char.
2015-04-10 10:23:01 +02:00
Miquel Llobet
d03b40e31e Fixed #44628 honoring RFC 6266 content-disposition
src/http.c (parse_content_disposition): stores filename* and filename
separately and choses filename* if available.
(test_parse_content_disposition): added new tests.
2015-04-06 10:30:30 +02:00
Giuseppe Scrivano
268e6aef72 NEWS: cite last change 2015-04-02 15:39:02 +02:00
Steven M. Schweda
5efb24e4a2 Add option to restrict filenames used VMS.
* src/options.h (enum restrict_files_os): Define "restrict_vms".
* src/init.c (defaults) [__VMS]: Set "opt.restrict_files_os" to
"restrict_vms".
(cmd_spec_restrict_file_names): honor "vms".
* src/url.c (filechr_not_unix): Define "filechr_not_vms".
(filechr_table): Update for VMS.
(append_uri_pathel): Honor opt.restrict_files_os.
(FN_QUERY_SEP): Update for VMS.
(FN_QUERY_SEP_STR): Update for VMS.
2015-04-02 15:36:42 +02:00
Hubert Tarasiuk
eae8b1d565 Change semantics of resp_free and request_free in http.c
* src/http.c (resp_free): Change the semantics of this function.
(request_free): Change the semantics of this function.
(initialize_request): Adjust request_free call.
(establish_connection): Adjust request_free, resp_free calls.
(gethttp): Adjust request_free, resp_free calls.
2015-04-01 17:01:07 +02:00
Hubert Tarasiuk
045463b814 Do not free request in establish_connection; do it in gethttp
* src/http.c (establish_connection): Do not free request here (it is
* never allocated here).
* src/http.c (gethttp): Free request before returning if error in
* establish_connection encountered.
2015-04-01 17:01:07 +02:00
Hubert Tarasiuk
621c313b94 Transform read_header label and goto into a loop
* src/http.c (gethttp): Replace label and goto statement with a do
loop.
2015-04-01 17:00:40 +02:00
Hubert Tarasiuk
52a7d0ad85 Factor out set_content_type function from gethttp
* src/http.c (gethttp): Move some code in...
(set_content_type): ... a new function.
2015-03-31 17:00:07 +02:00
Giuseppe Scrivano
59e9ef00e6 Factor out some gethttp code
* src/http.c (gethttp): Move some code in...
(open_output_stream): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
14bbc18512 Factor out some auth gethttp code
* src/http.c (gethttp): Move some code in...
(check_auth): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
8aa63e482e Factor out some gethttp code
* src/http.c (gethttp): Move some code in...
(check_file_output): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
0bc2757713 Factor out some connection initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(establish_connection): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
f8abb9dd00 Factor out some proxy initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(initialize_proxy_configuration): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
29850e77d0 Factor out some initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(initialize_request): ... a new function.
2015-03-18 12:09:54 +01:00
Tim Ruehsen
799c545722 src/ftp.c: make sure warc_tmp becomes closed before return
Reported-by: Coverity bug #1188044
2015-03-18 10:46:11 +01:00
Tim Ruehsen
014b1d6041 src/http.c: fix error return of digest_authentication_encode()
Reported-by: Coverity bug #1188036
2015-03-18 10:46:05 +01:00
Rohan Prinja
8d4bb928b9 Convert wget.texi to UTF-8
* doc/wget.texi: convert to UTF-8 by adding @documentencoding
* doc/Makefile.am: Pass argument --utf8 to pod2man.
2015-03-17 09:52:23 +01:00
Ander Juaristi Alamos
d0f406a13f * doc/wget.texi: The default is GnuTLS, not OpenSSL. 2015-03-16 22:46:08 +01:00
Darshit Shah
87b52e510c testenv/http_server: Allow overriding Headers
We would like to override existing headers in the response from the
server to allow the test cases to define their own headers. This patch
allows this behaviour through the use of the add_header() method while
also allowing sending multiple headers of the same name.
2015-03-16 23:24:14 +05:30
Darshit Shah
cc9f76c5a4 retr.c: Fix memory leak in retrieve_from_file()
Reported by: Coverity Bug 1188045
2015-03-14 16:48:30 +05:30
Darshit Shah
53b22974cb html-url.c: Fix potential memory leaks
Reported by: Coverity Bug 1188050
2015-03-14 16:48:30 +05:30
Darshit Shah
7d5a7ef9ca main.c: Fix two potential memory leaks
Reported by: Coverity bug 1188048
2015-03-14 16:48:30 +05:30
Darshit Shah
735cc220e3 retr.c: Fix two memory leaks when proxy URL is bad
Reported by: Coverity bug 1188047
2015-03-14 16:48:29 +05:30
Darshit Shah
524f26a200 Docs: --post-file is binary data
Wget considers the file mentioned in the --post-file argument as a
binary file and does not strip any control characters. The lack of this
information in the documentation can cause a lot of headaches debugging
for a simple issue
2015-03-13 23:51:59 +05:30