Commit Graph

3399 Commits

Author SHA1 Message Date
Tim Rühsen 5f0818d9f1 Fix usage of CFLAGS in contrib/check-hard
* contrib/check-hard: Set CFLAGS per command line instead of using export.

'make distcheck' changes CFLAGS. So using ./configure -C together with
exported CFLAGS fails. Setting CFLAGS per command line works smoothly.
2015-06-14 20:30:07 +02:00
Tim Rühsen c6ac51d5bc Move test_* function protoypes from test.c to test.h
* src/test.c: Remove test_* function prototypes, make tests_run static
* src/test.h: Add test_* function protoypes
2015-06-13 22:34:36 +02:00
Giuseppe Scrivano fd3a3245eb NEWS: cite --if-modified-since 2015-05-23 14:54:17 +02:00
Giuseppe Scrivano 48acb6693d gnulib: update gnulib 2015-05-23 14:51:52 +02:00
Hubert Tarasiuk 885eaaa214 Include --if-modified-since option in user manual.
* doc/wget.texi: Add --if-modified-since section.
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk 8a8d138dcc Support If-Modified-Since header in timestamping mode.
* src/wget.h: Add IF_MODIFIED_SINCE enum for dt. Add TIMECONV_ERR
enum to uerr_t.
* src/http.c (time_to_rfc1123): Convert time_t do http time.
* src/http.c (initialize_request): Include If-Modified-Since header
if appropriate.
* src/http.c (set_file_timestamp): Separate this code from check_file_output.
* src/http.c (check_file_output): Use set_file_timestamp.
* src/http.c (gethttp): Handle properly 304 return code and 200 if server
ignores If-Modified-Since headers.
* src/http.c (http_loop): Load filename to hstat if condget was requested,
use IF_MODIFIED_SINCE if requested and current timestamp can be obtained.
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk 0e8d2d4251 Add --if-modified-since option
* src/init.c: Add to commands array.
* src/main.c: Add to cmdline_option. Add to help message.
* src/options.h: Add to options struct.
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk be4f91737a Add test for condget requests.
* testenv/Test-condget.py: the test
* testenv/Makefile.am: add to tests list
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk 901bc98edf Support conditional GET in testenv server.
* src/exc/server_error.py: Add exception for GET to HEAD fallback.
* src/server/http/http_server.py: Do not send body if 304 return
code requested for a file.
2015-05-22 11:08:30 +02:00
Hubert Tarasiuk e397a48f6a Implement timestamp support for local files in testenv
* testenv/README: Change timestamp format definition
* testenv/conf/local_files.py: Set proper timestamps
2015-05-22 11:08:30 +02:00
Pär Karlsson 83537f2415 Fix undeclared loop variable in Perl test suite
Reported-by: Hubert Tarasiuk <hubert.tarasiuk@gmail.com>
2015-05-20 10:04:07 +02:00
Ander Juaristi 8682c2612f Make sure Wget does not unescape reserved chars.
* testenv/Test-reserved-chars.py: New file.

* testenv/Makefile.am: Added new test Test-reserved-chars.py.

When following redirections, Wget should not unescape the reserved
characters that might appear in target URLs.
2015-05-12 21:24:11 +02:00
Ander Juaristi b0820d553b Fixed incorrect handling of reserved chars.
* src/iri.c (do_conversion): Call url_unescape_except_reserved,
instead of url_unescape.

* src/url.c (url_unescape_1): New static function.
(url_unescape): Calls url_unescape_1 with mask zero. Preserves
same behavior as before. Only code changes.
(url_unescape_except_reserved): New function.

* src/url.h: Added prototype for url_unescape_except_reserved().

When the locale is US-ASCII, URIs that contain special characters
in them are converted to IRIs according to RFC 3987, section 3.2
"Converting URIs to IRIs".
2015-05-12 21:24:06 +02:00
Darshit Shah b6b1388fb7 Fix documentation for update_speed_ring()
* progress.c (update_speed_ring): The comment for the function
    incorrectly stated that the function uses thirty samples from the
    past instead of twenty.

    Reported-By: Yi Li <lovelylich@gmail.com>
2015-05-07 11:29:07 +05:30
Darshit Shah 9b1dd6dab8 Remove shadowed variable in http.c
* http.c (gethttp): Rename err to conn_err to prevent shadowed
    variable
2015-05-04 21:45:26 +05:30
Steven Schubiger 1cc835dc5b paramcheck: use explicit quoting for here-docs
* util/paramcheck.pl: Adjust here-docs
2015-05-04 10:13:24 +05:30
Tim Ruehsen 6b8dfe1d6e Fix format specifier warning
* src/utils.c (aprintf): Use %d for int argument
2015-05-03 21:18:47 +02:00
Nikolay Merinov 0e6d6ca963 Fix timestamping and continue behaviour with ftp protocol.
* src/ftp.c (ftp_loop_internal): Add option `force_full_retrieve' that force to
retrieve full file.
(ftp_retrieve_list): Pass `true' as `force_full_retrieve' option to
`ftp_loop_internal' if we want to download file with newer timestamp than local
copy.
2015-05-01 00:28:08 +02:00
Rohit Mathulla 3765a1b266 openssl: Read cert from private key file when needed
* src/openssl.c (ssl_init): Assign opt.cert_{file, type}
  from opt.private_key(_type)
2015-04-27 19:52:18 +02:00
Rohit Mathulla 8654f7e2e7 Fix double free bug in SSL code
* src/openssl.c, src/gnutls.c (ssl_init): Copy options using xstrdup
2015-04-27 19:48:51 +02:00
Hubert Tarasiuk 566696cb82 Single exit point and common cleanup code in gethttp
* src/http.c (gethttp): Common cleanup for type, message,
  req, resp, head.  Single exit point.
2015-04-20 10:54:34 +02:00
Tim Rühsen bebe462a67 Silence warning in perl test suite
* tests/WgetTests.pm: Use string comparisons for $valgrind variable
2015-04-17 22:42:59 +02:00
Tim Rühsen c579c7bf1e Check memory allocations in WARC code
* src/warc.c: Remove some memory allocations,
              use xmalloc instead of malloc

Reported-by: Bill Parker <wp02855@gmail.com>
2015-04-17 22:42:59 +02:00
Tim Rühsen 4dde3e200f Add more const usage to function params
* warc.c, warc.h: Add const specifier to several function args
2015-04-17 22:42:59 +02:00
Darshit Shah 8e0dd0d870 PEP8'ify the Python Test Suite
* testenv/conf/{__init__,authentication,files_crawled,
      hook_sample,reject_header,server_files}.py: Aesthetic changes to
      meet Python PEP8 guidelines
    * testenv/exc/{server_error,test_failed}.py: Same
    * testenv/misc/{colour_terminal,wget_file}.py: Same
    * testenv/server/http/http_server.py: Same
    * testenv/test/base_test.py: Same
2015-04-14 10:59:17 +05:30
Ander Juaristi c6af2fddee Improved test suite documentation
* README.checkout: Added reference to Python. New section
  "Testing and development". Updated information about test suites.
2015-04-13 19:36:39 +02:00
Ander Juaristi e7e3227b34 Added support for GDB in Python tests
* base_test.py (gen_cmd_line): Check GDB_TESTS environment variable.
2015-04-13 19:32:57 +02:00
Tim Ruehsen 5c5d45ae49 Added GDB support to Perl tests
* WgetTests.pm (run): Check GDB_TESTS environment variable.
2015-04-13 19:29:43 +02:00
Hubert Tarasiuk f4072e5d0b Add Valgrind suppression for libidn.so at idna_to_ascii_4z
* tests/WgetTests.pm (run): Include suppression file when running Valgrind.
* tests/valgrind-suppressions: Add suppression for idn_to_ascii_4z.
* tests/Makefile.am: Add valgrind-suppressions to EXTRA_DIST.
2015-04-11 23:16:32 +02:00
Ángel González bef5945202 Remove memory leak in idn_encode.
* src/iri.c (idn_encode): Free buffer from remote_to_utf8
when needed; give meaningful names to variables;
remove excessive comment.
2015-04-11 21:55:17 +02:00
Hubert Tarasiuk ac40b84ee1 Fix error in free_vec.
* src/utils.c (free_vec): Increment pointer instead of its value.

Reported-by: Gisle Vanem <gvanem@yahoo.no>
2015-04-10 18:06:14 +02:00
Ángel González 45463eaad7 Fix const usage in iri.c
* src/iri.c (remote_to_utf8): Do not qualify with const the output pointer.
(do_conversion): Use the provided input parameter as const.
(idn_encode): casts to remote_to_utf8 parameters are no longer needed.
* src/iri.h: Adjusted remote_to_utf8 prototype.
* src/url.c: It is no longer necessary to cast new_url to const char.
2015-04-10 10:23:01 +02:00
Miquel Llobet d03b40e31e Fixed #44628 honoring RFC 6266 content-disposition
src/http.c (parse_content_disposition): stores filename* and filename
separately and choses filename* if available.
(test_parse_content_disposition): added new tests.
2015-04-06 10:30:30 +02:00
Giuseppe Scrivano 268e6aef72 NEWS: cite last change 2015-04-02 15:39:02 +02:00
Steven M. Schweda 5efb24e4a2 Add option to restrict filenames used VMS.
* src/options.h (enum restrict_files_os): Define "restrict_vms".
* src/init.c (defaults) [__VMS]: Set "opt.restrict_files_os" to
"restrict_vms".
(cmd_spec_restrict_file_names): honor "vms".
* src/url.c (filechr_not_unix): Define "filechr_not_vms".
(filechr_table): Update for VMS.
(append_uri_pathel): Honor opt.restrict_files_os.
(FN_QUERY_SEP): Update for VMS.
(FN_QUERY_SEP_STR): Update for VMS.
2015-04-02 15:36:42 +02:00
Hubert Tarasiuk eae8b1d565 Change semantics of resp_free and request_free in http.c
* src/http.c (resp_free): Change the semantics of this function.
(request_free): Change the semantics of this function.
(initialize_request): Adjust request_free call.
(establish_connection): Adjust request_free, resp_free calls.
(gethttp): Adjust request_free, resp_free calls.
2015-04-01 17:01:07 +02:00
Hubert Tarasiuk 045463b814 Do not free request in establish_connection; do it in gethttp
* src/http.c (establish_connection): Do not free request here (it is
* never allocated here).
* src/http.c (gethttp): Free request before returning if error in
* establish_connection encountered.
2015-04-01 17:01:07 +02:00
Hubert Tarasiuk 621c313b94 Transform read_header label and goto into a loop
* src/http.c (gethttp): Replace label and goto statement with a do
loop.
2015-04-01 17:00:40 +02:00
Hubert Tarasiuk 52a7d0ad85 Factor out set_content_type function from gethttp
* src/http.c (gethttp): Move some code in...
(set_content_type): ... a new function.
2015-03-31 17:00:07 +02:00
Giuseppe Scrivano 59e9ef00e6 Factor out some gethttp code
* src/http.c (gethttp): Move some code in...
(open_output_stream): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano 14bbc18512 Factor out some auth gethttp code
* src/http.c (gethttp): Move some code in...
(check_auth): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano 8aa63e482e Factor out some gethttp code
* src/http.c (gethttp): Move some code in...
(check_file_output): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano 0bc2757713 Factor out some connection initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(establish_connection): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano f8abb9dd00 Factor out some proxy initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(initialize_proxy_configuration): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano 29850e77d0 Factor out some initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(initialize_request): ... a new function.
2015-03-18 12:09:54 +01:00
Tim Ruehsen 799c545722 src/ftp.c: make sure warc_tmp becomes closed before return
Reported-by: Coverity bug #1188044
2015-03-18 10:46:11 +01:00
Tim Ruehsen 014b1d6041 src/http.c: fix error return of digest_authentication_encode()
Reported-by: Coverity bug #1188036
2015-03-18 10:46:05 +01:00
Rohan Prinja 8d4bb928b9 Convert wget.texi to UTF-8
* doc/wget.texi: convert to UTF-8 by adding @documentencoding
* doc/Makefile.am: Pass argument --utf8 to pod2man.
2015-03-17 09:52:23 +01:00
Ander Juaristi Alamos d0f406a13f * doc/wget.texi: The default is GnuTLS, not OpenSSL. 2015-03-16 22:46:08 +01:00
Darshit Shah 87b52e510c testenv/http_server: Allow overriding Headers
We would like to override existing headers in the response from the
server to allow the test cases to define their own headers. This patch
allows this behaviour through the use of the add_header() method while
also allowing sending multiple headers of the same name.
2015-03-16 23:24:14 +05:30