1
0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Commit Graph

114 Commits

Author SHA1 Message Date
Hubert Tarasiuk
37b58e3976 Metalink support.
* bootstrap.conf: Add crypto/sha256
* configure.ac: Look for libmetalink and GPGME
* doc/wget.texi: Add --input-metalink and --metalink-over-http
options description.
* po/POTFILES.in: Add metalink.c
* src/Makefile.am: Add new translation unit (metalink.c)
* src/http.c (http_stat): Add metalink field.
(free_stat): Free metalink field.
(find_key_value): Find value of given key in header string.
(has_key): Check if token exists in header string.
(find_key_values): Find all key=value pairs in header string.
(metalink_from_http): Obtain Metalink metadata from HTTP response.
(gethttp): Call metalink_from_http if requested.
(http_loop): Request Metalink metadata from HTTP response if should be.
Fall back to regular download if no Metalink metadata found.
* src/init.c: Add --input-metalink and --metalink-over-http options
* src/main.c (option_data): Handle --input-metalink and
--metalink-over-http cmd arguments.
(print_help): Print --input-metalink option description.
(main): Retrieve files from Metalink file
* src/metalink.c (retrieve_from_metalink): Download files described by
metalink.
(metalink_res_cmp): Comparator for resources priority-sorting.
* src/metalink.h: Create header for metalink.c
(RES_TYPE_SUPPORTED): Define supported resources media.
(DEFAULT_PRI): Default mirror priority for Metalink over HTTP.
(VALID_PRI_RANGE): Valid priority range.
* src/options.h (options): Add input_metalink option and metalink_over_http
options.
* src/utils.c (hex_to_string): Convert binary data to ASCII-hex.
* src/utils.h (hex_to_string): Add prototype.
* src/wget.h: Add metalink-related error enums
Add METALINK_METADATA flag for document type.
2015-07-20 15:30:39 +02:00
Tim Rühsen
bd0ffcf8bc Let HTTPS tests XFAIL when no TLS support configured
* configure.ac: Export WITH_SSL for use in Makefile.am
* testenv/Makefile.am: Add HTTPS tests to XFAIL_TESTS when !WITH_SSL

Reported-by: Ander Juaristi <ajuaristi@gmx.es>
2015-07-14 07:54:03 +02:00
Tim Ruehsen
6f62bc5cd8 configure.ac: Skip pkg-config for opensl and gnutls when prefix is given
Make --with-libssl-prefix and --with-libgnutls-prefix do the right thing,
no matter if pkg-config is installed or not.

Reported-by: Charles Diza <chdiza@gmail.com>
2014-12-25 15:21:44 +01:00
Tim Rühsen
05d73a926a configure.ac: Fix libuuid and uuid_create detection
--with-libuuid now explictly asks for libuuid.
--without-libuuid ignores libuuid and tries to use libc builtin functions.
Else try builtin functions first, libuuid second and fallback to Wget's own
function.
2014-12-18 10:42:26 +01:00
Tim Rühsen
0d9615292e configure.ac: Check for RAND_egd() in pkg-config success code 2014-12-17 12:39:00 +01:00
Jérémie Courrèges-Anglas
b5778699f0 openssl: Detect the availability of RAND_egd (tiny change)
Alternatives like LibreSSL don't provide RAND_egd() anymore.
Fixes compilation on OpenBSD.
2014-12-17 11:47:15 +01:00
Jérémie Courrèges-Anglas
b8c567a3ef Fix use of uuid libc functions (tiny change)
* Always attempt to detect uuid.h and uuid_create().
* Split libuuid and uuid.h implementations of warc_uuid_str(), since
  those APIs vary significantly.
* Correctly use the uuid.h functions
2014-12-17 09:24:31 +01:00
Tim Rühsen
71425f1bb3 Check for pcre.h in configure.ac 2014-12-10 10:22:39 +01:00
Tim Rühsen
a80986a8c9 Fix message in configure.ac for when GnuTLS has not been found 2014-12-05 12:18:30 +01:00
Darshit Shah
841efb0108 Assertions are now disabled by default 2014-11-21 17:45:19 +05:30
Tim Ruehsen
b19d8032c2 Fix libpsl<0.6.0 detection in configure.ac 2014-11-20 21:08:21 +01:00
Darshit Shah
acc76b1e02 Use pkg-config for libpsl checking 2014-11-20 16:57:45 +05:30
Darshit Shah
e4be5e3b9a Autotoolize configure.ac 2014-11-20 16:49:28 +05:30
Darshit Shah
bfbc2866c5 Add the -Wextra flag by default
After the Spring Cleaning on warnings in early 2014, GCC and Clang emit
only about two warnings when compiling Wget with -Wall -Wextra.

Enable both these flags by default so that new warnings in these areas
are more prominent.
2014-11-20 16:45:34 +05:30
Tim Rühsen
7a7a241e5b Use random() and srandom() if available.
Reported-by: Coverity scanner
2014-11-19 16:06:19 +01:00
Darshit Shah
ff6fc1f8dd Add a configure option to disable assertions
Adds a --disable-assert configure option. We **highly recommend** the
use of this option by distro maintainers so as to ensure a smooth
experience with Wget. Sometimes an assertion may fail even though Wget
continues to work perfectly. In such scenarios, we do not wish for
client installations of Wget to crash. Hence, for a stable experience,
please use the --disable-assert configure flag.
2014-11-15 13:39:25 +05:30
Tim Ruehsen
831b9d1880 Skip python tests if python3 is not available
Reported-by: kdevel@vogtner.de
2014-11-06 11:07:33 +01:00
Darshit Shah
fdd8b8c997 Fix libpsl configure code 2014-11-01 23:49:55 +05:30
Mike Frysinger
f48be4e0c6 use pkg-config to check gnutls deps if available
Newer versions of these packages ship with pkg-config files, so if we can
detect it via those, do so.  If that fails, fall back to the old methods.
2014-11-01 17:09:24 +01:00
Mike Frysinger
5ae5d2eaeb use pkg-config to check openssl deps if available
Newer versions of these packages ship with pkg-config files, so if we can
detect it via those, do so.  If that fails, fall back to the old methods.
2014-11-01 17:09:10 +01:00
Mike Frysinger
47d0d4c331 use pkg-config to check pcre deps if available
Newer versions of these packages ship with pkg-config files, so if we can
detect it via those, do so.  If that fails, fall back to the old methods.

Also add a configure flag to explicitly control its usage.
2014-11-01 17:08:52 +01:00
Mike Frysinger
d272339fda use pkg-config to check libuuid deps if available
Newer versions of these packages ship with pkg-config files, so if we can
detect it via those, do so.  If that fails, fall back to the old methods.

Also add a configure flag to explicitly control its usage.
2014-11-01 17:07:08 +01:00
Mike Frysinger
297d040ce1 use pkg-config to check zlib deps if available
Newer versions of these packages ship with pkg-config files, so if we can
detect it via those, do so.  If that fails, fall back to the old methods.
2014-11-01 17:06:33 +01:00
Tim Ruehsen
4152e98bb0 content for commit 605d9053e5 2014-10-28 20:21:33 +01:00
Tim Rühsen
9c34d4ed09 add ./configure valgrind support to test suites 2014-10-28 12:32:12 +01:00
Tim Rühsen
b64fa2eb84 Switched to parallel test harness 2014-10-01 14:41:32 +05:30
Darshit Shah
7cbe8141d1 Introducing Python based Test Environment
Squashed Commit, of the following commits:

7743384 Update documentation to reflect changes in code
b703633 Add feature that allows to ensure that Wget correctly crawls the website in recursive mode
0758f47 Add new test for recursive spider mode
43bb61b Smartly guess content type header
d4d0e63 Support substring replace in File Contents too
f578500 Compatibility fix with multiple servers
8b1a9b6 Extend Functionality to support spawning multiple servers
e84192a Use the provided calls to shutdown server instead of rewriting it
99659f3 Improve Documentation
cb94e52 Slight code cleanup. Remove unused code
886ac1a Shift to new Threading Model from Multiprocessing model
e74c2ec Add new test for POST Requests
48644f1 Print diff when file contents don't match
b6f9efe Add tests for Cookie support
4c9e6b4 Document pending work
e13bc90 Add new test to ensure Content Disposition and Auth work together
60d1f4d Add new Test for Continue command
738b299 Add test, Test-Head
9b9d16b Edit non-unique TEST_NAME variable
ae958db Minor optimizations to the way Server Rules are executed
50b4f0c The rules need not be a defaultdict.
dccc154 Introducing Python based Test Environment
2014-07-24 16:51:58 +05:30
Darshit Shah
cae32d41c1 Fix broken check for libpsl 2014-07-22 22:11:32 +05:30
Darshit Shah
a44841cbe2 Fix potential memory leak and libpsl configure 2014-07-21 13:25:54 +05:30
Giuseppe Scrivano
601401da71 warc: Fix UUID generation on FreeBSD 2014-06-24 15:05:19 +02:00
Giuseppe Scrivano
ffccc7d3be Do not depend on always defined macros 2014-06-12 18:49:15 +02:00
Giuseppe Scrivano
a22cd7394b Remove trailing whitespaces 2014-06-12 18:49:14 +02:00
Darshit Shah
854ebbf4dd Support libpsl for cookie domain checking 2014-06-05 16:20:34 +05:30
Giuseppe Scrivano
59080465d1 configure.ac: update copyright years 2014-02-06 18:34:25 +01:00
Tim Ruehsen
7e85e9509e fix bug #39844 2013-09-16 10:25:11 +02:00
Tim Ruehsen
ffb94036f2 openssl: fix build. 2013-07-23 15:45:30 +02:00
Tim Ruehsen
c19d76c024 ntlm: support libnettle. 2013-07-22 14:17:03 +02:00
Giuseppe Scrivano
22bd8011e8 Fallback to PATH_MAX when pathconf is not available 2012-10-07 13:09:56 +02:00
Giuseppe Scrivano
f4122c5094 Use git-version-gen instead of bzr-version-gen. 2012-05-13 16:46:15 +02:00
Gijs van Tulder
f5a1097871 Add support for -accept-regex and --reject-regex. 2012-05-09 21:18:23 +02:00
Ray Satiro
44ea82bc67 Fix build under mingw when OpenSSL is used. 2012-03-25 13:47:53 +02:00
Gijs van Tulder
0a8a898fbe Fix a linker error if zlib is not found. 2012-01-11 15:27:06 +01:00
Gijs van Tulder
e3820953b2 Add support for WARC files. 2011-11-04 22:25:00 +01:00
Alan Hourihane
85797041a6 configure: check for libz when GNU TLS is used. 2011-09-04 14:19:12 +02:00
Giuseppe Scrivano
baed30f057 Under mingw don't check for different OpenSSL when one is found. 2011-08-26 12:35:12 +02:00
Giuseppe Scrivano
5e3c9b55f2 Prefers utime over futimens when available. 2011-08-25 11:41:51 +02:00
Giuseppe Scrivano
6330e4f8b8 configure: do not hardcode libraries 2011-08-11 12:20:25 +02:00
Giuseppe Scrivano
f3c2686ef7 allow --with-libgnutls-prefix and --with-libssl-prefix. 2011-08-11 12:07:32 +02:00
Giuseppe Scrivano
13028b1bae configure: print usage string for --with-ssl. 2011-08-10 22:23:53 +02:00
Giuseppe Scrivano
228299b4f7 Fix a linker error on systems with an older gnutls version. 2011-08-10 21:35:59 +02:00