1
0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Commit Graph

468 Commits

Author SHA1 Message Date
Tim Rühsen
4e37fb6191 Fix regression in HTTP authentication
* src/http.c (initialize_request): Fix wrong params to search_netrc()

Regression introduced in commit 29850e77
Reported-by: Axel Reinhold <axel@freakout.de>
2015-11-24 10:39:39 +01:00
Tim Rühsen
218d81f6e5 Fix SIGSEGV in -N / --content-disposition combination
* src/http.c (http_loop): Fix SIGSEGV

Reported-by: "Schleusener, Jens" <Jens.Schleusener@t-online.de>
2015-11-23 15:10:00 +01:00
Tim Rühsen
6cdfc9c143 Do not download/save file on error when --spider enabled
* src/http.c (gethttp,http_loop):
  Do not download/save file on error when --spider is enabled and not
  working recursive.

Reported-by: Сковорода Никита Андреевич chalkerx@gmail.com
Fixes #45821
2015-11-03 14:29:36 +01:00
Darshit Shah
c387db6451 Do not test for impossible qop value
* http.c (digest_authentication_encode): Wget already errors out if
    qop != "auth". Then it makes no sense to test for qop == "auth-int"
    later on. Currently, Wget does not support the "auth-int" qop value
    and till nobidy requests, it may remain so.
2015-09-22 16:36:40 +05:30
Darshit Shah
12dfc03116 Fix #46024. Support RFC 2069 Digest Authentication
* http.c (digest_authentication_encode): Some servers are still
    using the obsolete RFC 2069 Digest Authentication. Allow Digest
    authentication without the qop parameter for this.

    Reported-by: Andreas Longwitz  <longwitz@incore.de>
2015-09-22 15:41:22 +05:30
Ander Juaristi
f8901af4e0 Added support for FTPS
* doc/wget.texi: updated documentation to reflect the new FTPS functionality.
 * src/ftp-basic.c (ftp_greeting): new function to read the server's greeting.
   (ftp_login): greeting code was previously here. Moved to ftp_greeting to
   support FTPS implicit mode.
   (ftp_auth): wrapper around the AUTH TLS command.
   (ftp_ccc): wrapper around the CCC command.
   (ftp_pbsz): wrapper around the PBSZ command.
   (ftp_prot): wraooer around the PROT command.
 * src/ftp.c (get_ftp_greeting): new static function.
   (init_control_ssl_connection): new static function to start SSL/TLS on the
   control channel.
   (getftp): added hooks to support FTPS commands (RFCs 2228 and 4217).
   (ftp_loop_internal): test for new FTPS error codes.
 * src/ftp.h: new enum 'prot_level' with available FTPS protection levels +
   prototypes of previous functions. New flag for enum 'wget_ftp_fstatus' to track
   whether the data channel has some security mechanism enabled or not.
 * src/gnutls.c (struct wgnutls_transport_context): new field 'session_data'.
   (wgnutls_close): free GnuTLS session data before exiting.
   (ssl_connect_wget): save/resume SSL/TLS session.
 * src/http.c (establish_connection): refactor ssl_connect_wget call.
   (metalink_from_http): take into account SCHEME_FTPS as well.
 * src/init.c, src/main.c, src/options.h: new command line/wgetrc options.
   (main): in recursive downloads, check for SCHEME_FTPS as well.
 * src/openssl.c (struct openssl_transport_context): new field 'sess'.
   (ssl_connect_wget): save/resume SSL/TLS session.
 * src/retr.c (retrieve_url): check new scheme SCHEME_FTPS.
 * src/ssl.h (ssl_connect_wget): refactor. New parameter of type 'int *'.
 * src/url.c. src/url.h: new scheme SCHEME_FTPS.
 * src/wget.h: new FTPS error codes.
 * src/metalink.h: support FTPS scheme.
2015-09-14 10:16:44 +02:00
Darshit Shah
187edb604a Fix coding style violation in last commit
* http.c (test_parse_range_header): Declare loop variable
    explicitly. Not in gnu99 standard.
2015-08-31 21:04:54 +05:30
Darshit Shah
b06fca60ac Add unit test for parse_content_range() method
* http.c (test_parse_range_header): New function to test the
    function for parsing the HTTP/1.1 Content-Range header.
    * test.[ch]: Same
    * http.c (parse_content_range): Fix parsing code. Fail on scenarios
    mentioned in rfc 7233.
2015-08-30 21:34:32 +05:30
Tim Rühsen
88a1a79bc1 Fix leaks found by Coverity
* src/http.c (parse_strict_transport_security): Free c_max_age
             (open_output_stream): Fix indentation
* src/iri.c (locale_to_utf8): Free new
2015-08-30 14:10:25 +02:00
Tim Rühsen
398699c438 Fix two leaks foudn by Coverity
* src/http.c (gethttp): Do not leak 'message'.
* src/main.c (format_and_print_line): Do not leak 'line_dup'.
2015-08-29 22:35:29 +02:00
Darshit Shah
6b5acff566 Fix memory leaks in unit-test
* hsts.c (get_hsts_store_filename): Free the homedir value
    (close_hsts_test_store): Actually free the store struct too
    (test_hsts_new_entry): Pass store to close_hsts_test_store()
    (test_hsts_url_rewrite_superdomain): Same
    (test_hsts_url_rewrite_congruent): Same
    (test_hsts_read_database): Same and homedir and store filename
    * http.c (test_parse_content_disposition): Free the returned
    filename
    * url.c (test_append_uri_pathel): Free allocated string
2015-08-29 22:52:49 +05:30
Darshit Shah
5c4489db9b Fix mixed-indentation in http.c
* http.c: Fix mix indentation. Visual change only.
2015-08-29 09:45:13 +05:30
Ander Juaristi
d080a70a3a Fix resource leak.
* src/http.c (parse_strict_transport_security): Freed memory to avoid resource leak.
   Comply with GNU coding style.
2015-08-26 17:50:26 +05:30
Darshit Shah
4e56a91001 Fix function name collision with OpenSSL library
* src/utils.[ch], src/http.c, src/metalink.c: Rename function
    hex_to_string() to wg_hex_to_string sine it collides with a
    similarly named function in OpenSSL Library.
2015-07-24 23:52:43 +05:30
Ander Juaristi
b60131a399 Added support for HSTS.
* Makefile.am: Added new source files hsts.c and hsts.h.
 * http.c (parse_strict_transport_security): new function for STS header
   parsing.
   (gethttp): update the HSTS store.
 * http.h: new include "hsts.h".
 * init.c: new options --hsts and --hsts-file.
 * main.c (get_hsts_database, load_hsts, save_hsts): new functions.
   New options --no-hsts and --hsts-file added to help.
   (main): load and save HSTS store.
 * options.h: new variables for supporting --hsts and --hsts-file.
 * retr.c (retrieve_url): rewrite the URI according to the HSTS policy before
   entering http_loop.
 * test.c, test.h: new unit tests for HSTS.
 * utils.c, utils.h (countchars): new function.
 * wget.h: new preprocessor check.
 * hsts.c, hsts.h: new files with the HSTS engine implementation.

Added support for HTTP Strict Transport Security (HSTS), as defined by RFC
6797.
2015-07-20 15:55:57 +02:00
Hubert Tarasiuk
225a87d4a2 Move some Metalink-related code from http.c to metalink.c.
* src/http.c: Move find_key_value, has_key, find_key_values.
* src/metalink.c: To here.
* src/metalink.h: Make them non-static and add prototypes here.
2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
92a889b278 Unit test for find_key_values.
* src/http.c: Add test_find_key_values.
* src/test.c (main): Run new test.
* src/test.h: Add test_find_key_values.
2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
1113e78534 Unit test for has_key.
* src/http.c: Add test_has_key.
* src/test.c (main): Run new test.
* src/test.h: Add test_has_key.
2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
70cbd59ed6 Unit test for find_key_value.
* src/http.c: Add test_find_key_value.
* src/test.c (main): Run new test.
* src/test.h: Add test_find_key_value.
2015-07-20 15:31:06 +02:00
Hubert Tarasiuk
37b58e3976 Metalink support.
* bootstrap.conf: Add crypto/sha256
* configure.ac: Look for libmetalink and GPGME
* doc/wget.texi: Add --input-metalink and --metalink-over-http
options description.
* po/POTFILES.in: Add metalink.c
* src/Makefile.am: Add new translation unit (metalink.c)
* src/http.c (http_stat): Add metalink field.
(free_stat): Free metalink field.
(find_key_value): Find value of given key in header string.
(has_key): Check if token exists in header string.
(find_key_values): Find all key=value pairs in header string.
(metalink_from_http): Obtain Metalink metadata from HTTP response.
(gethttp): Call metalink_from_http if requested.
(http_loop): Request Metalink metadata from HTTP response if should be.
Fall back to regular download if no Metalink metadata found.
* src/init.c: Add --input-metalink and --metalink-over-http options
* src/main.c (option_data): Handle --input-metalink and
--metalink-over-http cmd arguments.
(print_help): Print --input-metalink option description.
(main): Retrieve files from Metalink file
* src/metalink.c (retrieve_from_metalink): Download files described by
metalink.
(metalink_res_cmp): Comparator for resources priority-sorting.
* src/metalink.h: Create header for metalink.c
(RES_TYPE_SUPPORTED): Define supported resources media.
(DEFAULT_PRI): Default mirror priority for Metalink over HTTP.
(VALID_PRI_RANGE): Valid priority range.
* src/options.h (options): Add input_metalink option and metalink_over_http
options.
* src/utils.c (hex_to_string): Convert binary data to ASCII-hex.
* src/utils.h (hex_to_string): Add prototype.
* src/wget.h: Add metalink-related error enums
Add METALINK_METADATA flag for document type.
2015-07-20 15:30:39 +02:00
Hubert Tarasiuk
8a8d138dcc Support If-Modified-Since header in timestamping mode.
* src/wget.h: Add IF_MODIFIED_SINCE enum for dt. Add TIMECONV_ERR
enum to uerr_t.
* src/http.c (time_to_rfc1123): Convert time_t do http time.
* src/http.c (initialize_request): Include If-Modified-Since header
if appropriate.
* src/http.c (set_file_timestamp): Separate this code from check_file_output.
* src/http.c (check_file_output): Use set_file_timestamp.
* src/http.c (gethttp): Handle properly 304 return code and 200 if server
ignores If-Modified-Since headers.
* src/http.c (http_loop): Load filename to hstat if condget was requested,
use IF_MODIFIED_SINCE if requested and current timestamp can be obtained.
2015-05-22 11:08:30 +02:00
Darshit Shah
9b1dd6dab8 Remove shadowed variable in http.c
* http.c (gethttp): Rename err to conn_err to prevent shadowed
    variable
2015-05-04 21:45:26 +05:30
Hubert Tarasiuk
566696cb82 Single exit point and common cleanup code in gethttp
* src/http.c (gethttp): Common cleanup for type, message,
  req, resp, head.  Single exit point.
2015-04-20 10:54:34 +02:00
Miquel Llobet
d03b40e31e Fixed #44628 honoring RFC 6266 content-disposition
src/http.c (parse_content_disposition): stores filename* and filename
separately and choses filename* if available.
(test_parse_content_disposition): added new tests.
2015-04-06 10:30:30 +02:00
Hubert Tarasiuk
eae8b1d565 Change semantics of resp_free and request_free in http.c
* src/http.c (resp_free): Change the semantics of this function.
(request_free): Change the semantics of this function.
(initialize_request): Adjust request_free call.
(establish_connection): Adjust request_free, resp_free calls.
(gethttp): Adjust request_free, resp_free calls.
2015-04-01 17:01:07 +02:00
Hubert Tarasiuk
045463b814 Do not free request in establish_connection; do it in gethttp
* src/http.c (establish_connection): Do not free request here (it is
* never allocated here).
* src/http.c (gethttp): Free request before returning if error in
* establish_connection encountered.
2015-04-01 17:01:07 +02:00
Hubert Tarasiuk
621c313b94 Transform read_header label and goto into a loop
* src/http.c (gethttp): Replace label and goto statement with a do
loop.
2015-04-01 17:00:40 +02:00
Hubert Tarasiuk
52a7d0ad85 Factor out set_content_type function from gethttp
* src/http.c (gethttp): Move some code in...
(set_content_type): ... a new function.
2015-03-31 17:00:07 +02:00
Giuseppe Scrivano
59e9ef00e6 Factor out some gethttp code
* src/http.c (gethttp): Move some code in...
(open_output_stream): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
14bbc18512 Factor out some auth gethttp code
* src/http.c (gethttp): Move some code in...
(check_auth): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
8aa63e482e Factor out some gethttp code
* src/http.c (gethttp): Move some code in...
(check_file_output): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
0bc2757713 Factor out some connection initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(establish_connection): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
f8abb9dd00 Factor out some proxy initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(initialize_proxy_configuration): ... a new function.
2015-03-18 12:09:55 +01:00
Giuseppe Scrivano
29850e77d0 Factor out some initialization code for gethttp
* src/http.c (gethttp): Move some initialization code in...
(initialize_request): ... a new function.
2015-03-18 12:09:54 +01:00
Tim Ruehsen
014b1d6041 src/http.c: fix error return of digest_authentication_encode()
Reported-by: Coverity bug #1188036
2015-03-18 10:46:05 +01:00
Giuseppe Scrivano
16f1fb1d1f maint: update copyright year ranges to include 2015 2015-03-09 16:32:01 +01:00
Yousong Zhou
91e9a20752 Fix --content-on-error option handling.
* src/http.c: Log --content-on-error downloads.
* src/retr.c (retrieve_url): Register the download of an error page
when --content-on-error is specified.
2015-03-09 11:45:01 +01:00
Tim Ruehsen
103cbf1751 src/http.c: Revert commit d81a8d5f56
The removal of the 'redundant' condition was a failure.
Fixes: #43876
Reported-by: Sean Jensen-Grey <seanj@xyke.com>
2014-12-27 23:27:20 +01:00
Darshit Shah
4b845615fa Define MIN and MAx macros in a single location
MIN and MAx are macros that a developer will universally expect
throughout the source. Yet, they were being defined in multiple places
across the source. Instead, define them in a single location in the
common wget.h header file and use them consistently everywhere.
2014-12-04 18:36:54 +05:30
Tim Ruehsen
4850e9c873 Replaced xfree_null() by xfree() and nullify argument after freeing. 2014-12-01 16:15:37 +01:00
Tim Ruehsen
5edc97f3f8 Select most secure challenge from WWW-Authenticate
This patch also adds support for multiple challenges per
WWW-Authenticate header line.
The test Test-auth-both.py now succeeds and thus is taken away
from XFAIL_TESTS (expected to fail tests).
2014-11-26 16:41:56 +01:00
Tim Rühsen
0c18773308 Add size of buffer to warc_timestamp() 2014-11-24 10:41:08 +01:00
Darshit Shah
e94a542d0f Declare extern numurls in common header 2014-11-22 17:26:06 +05:30
Darshit Shah
c6b750061a Add extern declaration for version.c strings 2014-11-22 17:26:06 +05:30
Tim Rühsen
3c51ad7f02 Removed form feeds from sources and NEWS 2014-11-20 16:35:34 +01:00
Gisle Vanem
6a9b2d36e1 Fix C89 issue in http.c found by MSVC 16 2014-11-20 16:20:20 +01:00
Tim Rühsen
7b43510fe3 Fixes possible issues with Wget running in a turkish locale 2014-11-20 10:56:21 +01:00
Tim Rühsen
1356e90a14 Trivial fixes for C89 compliancy 2014-11-20 09:56:57 +01:00
Darshit Shah
c6ee033425 Make 504 Gateway Timeout non fatal 2014-11-19 18:03:17 +05:30
Tim Ruehsen
50ec4d9c3d Fix warnings from clang-analyzer 3.6 2014-11-18 20:44:56 +01:00