* Makefile.am: Added new source files hsts.c and hsts.h.
* http.c (parse_strict_transport_security): new function for STS header
parsing.
(gethttp): update the HSTS store.
* http.h: new include "hsts.h".
* init.c: new options --hsts and --hsts-file.
* main.c (get_hsts_database, load_hsts, save_hsts): new functions.
New options --no-hsts and --hsts-file added to help.
(main): load and save HSTS store.
* options.h: new variables for supporting --hsts and --hsts-file.
* retr.c (retrieve_url): rewrite the URI according to the HSTS policy before
entering http_loop.
* test.c, test.h: new unit tests for HSTS.
* utils.c, utils.h (countchars): new function.
* wget.h: new preprocessor check.
* hsts.c, hsts.h: new files with the HSTS engine implementation.
Added support for HTTP Strict Transport Security (HSTS), as defined by RFC
6797.
* doc/wget.text: Add information about --preferred-location.
* src/init.c: Add --preferred-location option.
* src/main.c (option_data): Handle --preferred-location argument.
(main): Sort resources based on location if requested.
* src/metalink.c (metalink_res_cmp): Compare based on location if
priority and preference are equal.
* src/options.h (options): Add preferred_location option.
* src/http.c: Move find_key_value, has_key, find_key_values.
* src/metalink.c: To here.
* src/metalink.h: Make them non-static and add prototypes here.
* testenv/README: Describe how to use repeated header name.
* testenv/server/http/http_server.py (finish_headers): Send all
values from list if the header value is a Python list.
* configure.ac: Export WITH_SSL for use in Makefile.am
* testenv/Makefile.am: Add HTTPS tests to XFAIL_TESTS when !WITH_SSL
Reported-by: Ander Juaristi <ajuaristi@gmx.es>
* testenv/test/base_test.py: Use Valgrind SSL suppressions file for
tests
* testenv/valgrind-suppression-ssl, tests/valgrind-suppression-ssl:
Add new suppression files to suppress OpenSSL errors in valgrind
* tests/test-proxied-https-auth.px: Use the valgrind SSL
suppressions file for the test
* tests/test-proxied-https-auth-keepalive.px: Same
* contrib/check-hard: Set CFLAGS per command line instead of using export.
'make distcheck' changes CFLAGS. So using ./configure -C together with
exported CFLAGS fails. Setting CFLAGS per command line works smoothly.
* src/wget.h: Add IF_MODIFIED_SINCE enum for dt. Add TIMECONV_ERR
enum to uerr_t.
* src/http.c (time_to_rfc1123): Convert time_t do http time.
* src/http.c (initialize_request): Include If-Modified-Since header
if appropriate.
* src/http.c (set_file_timestamp): Separate this code from check_file_output.
* src/http.c (check_file_output): Use set_file_timestamp.
* src/http.c (gethttp): Handle properly 304 return code and 200 if server
ignores If-Modified-Since headers.
* src/http.c (http_loop): Load filename to hstat if condget was requested,
use IF_MODIFIED_SINCE if requested and current timestamp can be obtained.
* src/exc/server_error.py: Add exception for GET to HEAD fallback.
* src/server/http/http_server.py: Do not send body if 304 return
code requested for a file.
* testenv/Test-reserved-chars.py: New file.
* testenv/Makefile.am: Added new test Test-reserved-chars.py.
When following redirections, Wget should not unescape the reserved
characters that might appear in target URLs.
* src/iri.c (do_conversion): Call url_unescape_except_reserved,
instead of url_unescape.
* src/url.c (url_unescape_1): New static function.
(url_unescape): Calls url_unescape_1 with mask zero. Preserves
same behavior as before. Only code changes.
(url_unescape_except_reserved): New function.
* src/url.h: Added prototype for url_unescape_except_reserved().
When the locale is US-ASCII, URIs that contain special characters
in them are converted to IRIs according to RFC 3987, section 3.2
"Converting URIs to IRIs".
* progress.c (update_speed_ring): The comment for the function
incorrectly stated that the function uses thirty samples from the
past instead of twenty.
Reported-By: Yi Li <lovelylich@gmail.com>
* src/ftp.c (ftp_loop_internal): Add option `force_full_retrieve' that force to
retrieve full file.
(ftp_retrieve_list): Pass `true' as `force_full_retrieve' option to
`ftp_loop_internal' if we want to download file with newer timestamp than local
copy.
* testenv/conf/{__init__,authentication,files_crawled,
hook_sample,reject_header,server_files}.py: Aesthetic changes to
meet Python PEP8 guidelines
* testenv/exc/{server_error,test_failed}.py: Same
* testenv/misc/{colour_terminal,wget_file}.py: Same
* testenv/server/http/http_server.py: Same
* testenv/test/base_test.py: Same
