mirror of
https://github.com/moparisthebest/wget
synced 2024-07-03 16:38:41 -04:00
![]() Squashed Commit, of the following commits: 7743384 Update documentation to reflect changes in code b703633 Add feature that allows to ensure that Wget correctly crawls the website in recursive mode 0758f47 Add new test for recursive spider mode 43bb61b Smartly guess content type header d4d0e63 Support substring replace in File Contents too f578500 Compatibility fix with multiple servers 8b1a9b6 Extend Functionality to support spawning multiple servers e84192a Use the provided calls to shutdown server instead of rewriting it 99659f3 Improve Documentation cb94e52 Slight code cleanup. Remove unused code 886ac1a Shift to new Threading Model from Multiprocessing model e74c2ec Add new test for POST Requests 48644f1 Print diff when file contents don't match b6f9efe Add tests for Cookie support 4c9e6b4 Document pending work e13bc90 Add new test to ensure Content Disposition and Auth work together 60d1f4d Add new Test for Continue command 738b299 Add test, Test-Head 9b9d16b Edit non-unique TEST_NAME variable ae958db Minor optimizations to the way Server Rules are executed 50b4f0c The rules need not be a defaultdict. dccc154 Introducing Python based Test Environment |
||
---|---|---|
build-aux | ||
doc | ||
gnulib@98ca2c063c | ||
m4 | ||
msdos | ||
po | ||
src | ||
testenv | ||
tests | ||
util | ||
vms | ||
.gitmodules | ||
.x-sc_po_check | ||
.x-sc_trailing_blank | ||
ABOUT-NLS | ||
AUTHORS | ||
bootstrap | ||
bootstrap.conf | ||
cfg.mk | ||
ChangeLog | ||
ChangeLog.README | ||
configure.ac | ||
COPYING | ||
MAILING-LIST | ||
Makefile.am | ||
NEWS | ||
README | ||
README.checkout |
-*- text -*- GNU Wget ======== Current Web home: http://www.gnu.org/software/wget/ GNU Wget is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies. It can follow links in HTML pages and create local versions of remote web sites, fully recreating the directory structure of the original site. This is sometimes referred to as "recursive downloading." While doing that, Wget respects the Robot Exclusion Standard (/robots.txt). Wget can be instructed to convert the links in downloaded HTML files to the local files for offline viewing. Recursive downloading also works with FTP, where Wget can retrieves a hierarchy of directories and files. With both HTTP and FTP, Wget can check whether a remote file has changed on the server since the previous run, and only download the newer files. Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off. If you are behind a firewall that requires the use of a socks style gateway, you can get the socks library and compile wget with support for socks. Most of the features are configurable, either through command-line options, or via initialization file .wgetrc. Wget allows you to install a global startup file (/usr/local/etc/wgetrc by default) for site settings. Wget works under almost all Unix variants in use today and, unlike many of its historical predecessors, is written entirely in C, thus requiring no additional software, such as Perl. The external software it does work with, such as OpenSSL, is optional. As Wget uses the GNU Autoconf, it is easily built on and ported to new Unix-like systems. The installation procedure is described in the INSTALL file. As with other GNU software, the latest version of Wget can be found at the master GNU archive site ftp.gnu.org, and its mirrors. Wget resides at <ftp://ftp.gnu.org/pub/gnu/wget/>. Please report bugs in Wget to <bug-wget@gnu.org>. See the file `MAILING-LIST' for information about Wget mailing lists. Wget's home page is at <http://www.gnu.org/software/wget/>. If you would like to contribute code for Wget, please read http://wget.addictivecode.org/PatchGuidelines. Wget was originally written and mainained by Hrvoje Niksic. Please see the file AUTHORS for a list of major contributors, and the ChangeLogs for a detailed listing of all contributions. Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004 2005, 2006, 2007, 2008, 2009 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. Additional permission under GNU GPL version 3 section 7 If you modify this program, or any covered work, by linking or combining it with the OpenSSL project's OpenSSL library (or a modified version of that library), containing parts covered by the terms of the OpenSSL or SSLeay licenses, the Free Software Foundation grants you additional permission to convey the resulting work. Corresponding Source for a non-source form of such a combination shall include the source code for the parts of OpenSSL used as well as that of the covered work.