mirror of
https://github.com/moparisthebest/wget
synced 2024-07-03 16:38:41 -04:00
b0b1c815c1
- use mmap() to read whole files in core instead of allocating memory and read'ing it. - use a new, more general, HTML parser (html-parse.c) and interface to it from Wget (html-url.c). - respect <meta name=robots content=nofollow> (easy with the new HTML parser). - use hash tables instead of linked lists in places where the lists were used to facilitate mappings. - rewrite the code in host.c to be more readable and faster (hash tables instead of home-grown lists.) - make convert_links properly convert partial URLs to complete ones for those URLs that have *not* been downloaded. - use HTTP persistent connections where available. very simple-minded, caches the last connection to the server. Published in <sxshf533d5r.fsf@florida.arsdigita.de>. |
||
---|---|---|
doc | ||
po | ||
src | ||
util | ||
windows | ||
aclocal.m4 | ||
AUTHORS | ||
ChangeLog | ||
config.guess | ||
config.sub | ||
configure | ||
configure.bat | ||
configure.in | ||
COPYING | ||
INSTALL | ||
install-sh | ||
MACHINES | ||
MAILING-LIST | ||
Makefile.in | ||
mkinstalldirs | ||
NEWS | ||
PATCHES | ||
README | ||
stamp-h.in | ||
TODO |
-*- text -*- GNU Wget ======== Home: http://sunsite.dk/wget/ GNU Wget is a free network utility to retrieve files from the World Wide Web using HTTP and FTP, the two most widely used Internet protocols. It works non-interactively, thus enabling work in the background, after having logged off. The recursive retrieval of HTML pages, as well as FTP sites is supported -- you can use Wget to make mirrors of archives and home pages, or traverse the web like a WWW robot (Wget understands /robots.txt). Wget works exceedingly well on slow or unstable connections, keeping getting the document until it is fully retrieved. Re-getting files from where it left off works on servers (both HTTP and FTP) that support it. Matching of wildcards and recursive mirroring of directories are available when retrieving via FTP. Both HTTP and FTP retrievals can be time-stamped, thus Wget can see if the remote file has changed since last retrieval and automatically retrieve the new version if it has. Wget supports proxy servers, which can lighten the network load, speed up retrieval and provide access behind firewalls. If you are behind a firewall that requires the use of a socks style gateway, you can get the socks library and compile wget with support for socks. Most of the features are configurable, either through command-line options, or via initialization file .wgetrc. Wget allows you to install a global startup file (/usr/local/etc/wgetrc by default) for site settings. Wget works under almost all modern Unix variants and, unlike many other similar utilities, is written entirely in C, thus requiring no additional software (like perl). As Wget uses the GNU Autoconf, it is easily built on and ported to other Unix's. Installation procedure is described in the INSTALL file. Like all GNU utilities, the latest version of Wget can be found at the master GNU archive site ftp.gnu.org, and its mirrors. For example, Wget 1.5.3 is at: <URL:ftp://ftp.gnu.org/pub/gnu/wget/wget-1.5.3.tar.gz>. Please report bugs in Wget to <bug-wget@gnu.org>. Wget has a mailing list at <wget@sunsite.auc.dk>. To subscribe, mail to <wget-subscribe@sunsite.auc.dk>. Wget's home page is at <URL:http://sunsite.dk/wget/>. Wget is free in all senses -- it is freely redistributable, and no payment is required. If you still wish to donate money to the developers or fund the development for a specific purpose, please send mail to the development mailing list at <wget@sunsite.auc.dk>. MAINTAINERS: Dan Harkless <dan-wget@dilvish.speed.net> Hrvoje Niksic <hniksic@arsdigita.com> Wget was originally written by Hrvoje Niksic. Copyright (C) 1995, 1996, 1997, 1998, 2000 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.