moparisthebest/wget
1
0
Fork 0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Code Issues Releases Wiki Activity
1,896 Commits 1 Branch 0 Tags 9.6 MiB
6493450947
Commit Graph

237 Commits

Author SHA1 Message Date
hniksic
261183c609 [svn] retr.c (fd_read_body): Pass total size to progress_create, not 
the remaining amount.
 2003-12-12 14:55:20 -08:00
hniksic
a8155e7bcc [svn] Correctly calculate bandwidth as total data read divided with download 
time.
 2003-12-05 18:32:57 -08:00
hniksic
bf8350af62 [svn] Support the 303 See Other response. 2003-12-05 06:37:47 -08:00
hniksic
78706dc5ea [svn] * retr.c (fd_read_body): Report the amount of data *written* as 
amount_read.  This is not entirely logical, but that's what the
callers expect, and it's not easy to change.

* ftp.c (ftp_loop_internal): Ditto.

* http.c (http_loop): Be smarter about assigning restval; if we're
in the nth pass of a download, simply use the information we have
about how much data has been retrieved as restval.

* ftp.c (getftp): Ditto for FTP "REST" command.

* http.c (gethttp): When the server doesn't respect range, skip
the first RESTVAL bytes of the read body.  Never truncate the
output file.

* retr.c (fd_read_body): Support skipping initial STARTPOS octets.
 2003-11-30 15:39:04 -08:00
hniksic
381457408a [svn] Read the data in skip_short_body directly. 2003-11-30 06:33:00 -08:00
hniksic
2f357b5eb8 [svn] Better documentation for fd_read_body args. 2003-11-29 20:41:08 -08:00
hniksic
3390f7eb45 [svn] Try to reuse connections that return error codes. 2003-11-29 17:34:33 -08:00
hniksic
63d492c0ba [svn] Initialize SSL only the first time when an SSL URL is encountered. 2003-11-29 14:24:55 -08:00
hniksic
ddd2644bc8 [svn] Docfix. 2003-11-29 14:19:46 -08:00
hniksic
0ff9eb4727 [svn] Restructure generation of HTTP requests. Allow headers specified with 
--header to override generated headers.
 2003-11-29 10:40:01 -08:00
hniksic
83b0e38940 [svn] Don't include the Proxy-Authorization header in the request tunneled 
through proxy.
 2003-11-28 03:28:05 -08:00
hniksic
76b59b1ad7 [svn] Use the CONNECT method to establish passthrough over SSL traffic 
over proxies.
 2003-11-27 18:24:53 -08:00
hniksic
dfe1f43dc0 [svn] Use fd_read_hunk for reading HTTP response head. Support HTTP/0.9 
responses.
 2003-11-27 15:29:36 -08:00
hniksic
7736d5dc98 [svn] Remove headers.c and headers.h. 2003-11-21 00:48:45 -08:00
hniksic
d9fea91a0a [svn] Remove the "rbuf" buffering layer. Provide peeking primitives instead. 2003-11-20 17:48:11 -08:00
hniksic
0716c335a0 [svn] Renamed xread/xwrite/xclose to fd_read/fd_write/fd_close. The "x" prefix is 
not appropriate because the semantics have diverged from read/write/close
too significantly.
 2003-11-20 07:19:59 -08:00
hniksic
0fb0ee87c7 [svn] Renamed address_list_find to address_list_contains. 2003-11-18 14:18:44 -08:00
hniksic
36387204b0 [svn] Fix call to socket_ip_address. 2003-11-14 18:03:23 -08:00
hniksic
9d907933ad [svn] Don't attempt to talk to two different SSL sites over the same secure 
connection.
 2003-11-13 14:48:03 -08:00
hniksic
509154dc81 [svn] Use the new function retryable_socket_connect_error instead of the 
CONNECT_ERROR macro.
 2003-11-13 14:27:38 -08:00
hniksic
c4ba08110d [svn] Mark clearly where the response begins and ends in the debug output. 2003-11-08 17:05:35 -08:00
hniksic
d7e592d797 [svn] Replace conaddr with socket_ip_address. 2003-11-07 18:57:51 -08:00
hniksic
1a0adb90b4 [svn] Don't force silent lookup. 2003-11-07 16:11:29 -08:00
hniksic
bc764dbb7c [svn] Improve SSL code. Only initialize PRNG when needed. 2003-11-06 05:06:59 -08:00
hniksic
c8e92cdbd6 [svn] Get rid of ssl_iread/ssl_iwrite. Have the SSL code register its 
routines for reading and writing data.
 2003-11-05 17:12:03 -08:00
hniksic
38b37ed104 [svn] New option --keep-session-cookies. 2003-11-04 16:11:33 -08:00
hniksic
e2e9b753e4 [svn] Retired the `boolean' type. Renamed FREE_MAYBE to xfree_null and moved the 
definition from wget.h to xmalloc.h.  Moved the DEFAULT_LOGFILE
define to log.h.  Moved the INFINITE_RECURSION define to recur.h.
 2003-11-02 11:56:37 -08:00
hniksic
370ff7a576 [svn] Moved malloc-related code to xmalloc.c. Defined new macros xnew, xnew0, 
xnew_array, and xnew0_array.  Created xmalloc.h and log.h to unclutter
wget.h.
 2003-10-31 06:31:56 -08:00
hniksic
add61a2d9c [svn] Networking improvements: get rid of the MSOCK global variable, 
move the sockaddr handling to connect.c, make sure Wget refreshes
the DNS lookup after it becomes stale.
 2003-10-30 16:18:08 -08:00
hniksic
6f1250b1c6 [svn] Updated IPv6 code. 2003-10-29 15:13:25 -08:00
hniksic
90cb3309da [svn] Imported Mauro's IPv6 changes. 2003-10-29 10:23:56 -08:00
hniksic
31d919b23d [svn] Accept HTTP status 307 as redirect. 2003-10-14 16:32:15 -07:00
hniksic
711bf72609 [svn] Remove VERY_LONG_TYPE; use LARGE_INT instead. Remove special code 
for handling VERY_LONG_TYPE overflows.
Make opt.quota a LARGE_INT.
 2003-10-11 06:57:11 -07:00
hniksic
38ea4d5dad [svn] Renamed DEBUG to ENABLE_DEBUG. 2003-10-07 17:05:51 -07:00
hniksic
097923f7b1 [svn] Move fnmatch() to cmpt.c and don't use it under GNU libc. 2003-10-07 16:53:31 -07:00
hniksic
254291cc03 [svn] Fix crash when post-file is missing. 2003-10-04 15:26:58 -07:00
hniksic
95c647eb44 [svn] Split off non-URL related stuff from url.c to convert.c. 2003-09-21 15:47:14 -07:00
hniksic
c06d32a309 [svn] Treat xhtml files just like regular html. 
By Matthew J. Mellon.
 2003-09-21 05:02:57 -07:00
hniksic
7b5fb50cb1 [svn] Renamed wget.h XDIGIT-related macros to (hopefully) clearer names. 2003-09-19 07:08:37 -07:00
hniksic
0e77942233 [svn] Don't pad shrunk POST'ed files. 
Message-ID: <m3u17dn1qk.fsf@hniksic.iskon.hr>
 2003-09-15 15:09:17 -07:00
hniksic
9228f0bf53 [svn] Update progress code to use higher timer resolution. 
Message-ID: <m37k49oivp.fsf@hniksic.iskon.hr>
 2003-09-15 14:14:15 -07:00
hniksic
0a3697ad65 [svn] New mechanism for quoting file names. 
Published in <m3smmzt4px.fsf@hniksic.iskon.hr>.
 2003-09-14 15:04:13 -07:00
hniksic
30ac043b0a [svn] New option --retry-connrefused from Ahmod Dancy. 2003-09-04 14:34:58 -07:00
hniksic
b2be7522c7 [svn] Update the license to include the OpenSSL exception. 2002-05-17 19:16:36 -07:00
abbotti
043d305fb1 [svn] Don't ftruncate standard output. 
Submitted by Bill Richardson  <bill@riverstonenet.com>.
Final version of patch published in
<9vv1eus668tn7v3vn95e7l2lcjbbph8bi2@farscape.privy.mev.co.uk>.
 2002-05-14 05:23:51 -07:00
hniksic
29a66192cf [svn] Cookie interface and implementation changes 
from <sxsofge9yao.fsf@florida.arsdigita.de>
and <sxslmbim7dg.fsf@florida.arsdigita.de>.
 2002-04-20 13:46:38 -07:00
hniksic
426bf23ae1 [svn] If content-type is not given, assume text/html. 2002-04-15 18:13:51 -07:00
hniksic
9f2cc07924 [svn] Minor cleanup of iread/iwrite. 
Published in <sxsk7raj1t2.fsf@florida.arsdigita.de>.
 2002-04-13 22:19:27 -07:00
hniksic
5cb8a6f44d [svn] Add the POST method. 
Published in <sxssn5yj4eb.fsf@florida.arsdigita.de>.
 2002-04-13 21:22:47 -07:00
abbotti
4aaf8fdcea [svn] Compensate for MS Windows two-second granularity of file modification 
timestamps.
Submitted by Ian Abbott in <3CB7355C.21011.2134FDA@localhost>.
 2002-04-12 12:21:02 -07:00
abbotti
cfd7b9a951 [svn] Use new function to test filename for common html suffixes. 
Submitted by Ian Abbott in <3CB72D29.4898.1F34872@localhost> with minor
changes to formatting and comments.
 2002-04-12 11:53:39 -07:00
hniksic
1375a89141 [svn] Allow Wget to be compiled with a K&R compiler. 
Published in <sxszo0af9k6.fsf@florida.arsdigita.de>.
 2002-04-11 10:49:32 -07:00
hniksic
a9ff5853a9 [svn] when -c used, mark already fully retrieved file as successfully retrieved. 
Submitted by Ian Abbott in <3C6019F4.30803.FF43E9@localhost>.
 2002-02-18 21:18:43 -08:00
hniksic
f2bb99d102 [svn] Wrap numeric IPv6 addresses in square brackets when sending out HTTP 
`Host' header.
Published in <sxs7kq4vgzf.fsf@florida.arsdigita.de>.
 2002-01-26 15:19:10 -08:00
hniksic
cafe798fec [svn] Invalidate socket if get_contents encountered an error. 
Published in <sxsita54sqy.fsf@florida.arsdigita.de>.
 2002-01-13 17:39:16 -08:00
hniksic
5b575eef31 [svn] Check for conn->ssl before using ssl_iwrite. 2001-12-13 08:46:56 -08:00
hniksic
dd84231c6a [svn] Minor fixes prompted by `lint'. 
Published in <sxsadwt2nkg.fsf@florida.arsdigita.de>.
 2001-12-08 17:24:41 -08:00
hniksic
6d93a95e37 [svn] Declare h_errno in host.c, but not elsewhere. 2001-12-03 09:22:06 -08:00
hniksic
569fd61c95 [svn] Use the full path when building the authorization line. 
Published in <sxsitbqu9iw.fsf@florida.arsdigita.de>.
 2001-12-01 09:39:07 -08:00
hniksic
2c37aef2d8 [svn] Don't forget to shutdown the SSL structure if the socket times out. 
Submitted by T. Bharath in <3B3DEBC7.2537186A@responsenetworks.com>.
 2001-11-30 00:57:03 -08:00
hniksic
b65661a879 [svn] Remove char/unsigned char warnings emitted by Sun cc. 2001-11-29 10:48:43 -08:00
hniksic
a3d3d452ab [svn] Make the MD5 references even more generic. 
Use OpenSSL's MD5 where available.
Published in <sxsg06x4f41.fsf@florida.arsdigita.de>.
 2001-11-29 10:17:54 -08:00
hniksic
6218fbed2b [svn] Accept the broken Content-Range format emitted by Sun's server. 
Published in <sxsg06x7kld.fsf@florida.arsdigita.de>.
 2001-11-29 06:03:33 -08:00
hniksic
278fb2412d [svn] Declare ERR in gethttp. 
Submitted by Ian Abbott in <3C022295.15135.464B54@localhost>.
 2001-11-26 04:51:25 -08:00
hniksic
84396de673 [svn] Allocate a timer directly in get_contents. 
Published in <sxsherhbsvl.fsf@florida.arsdigita.de>.
 2001-11-26 04:49:10 -08:00
hniksic
024cb5ed3a [svn] A lot of host name changes. 
Published in <sxs3d32856s.fsf@florida.arsdigita.de>.
 2001-11-25 21:36:33 -08:00
hniksic
3afb9c659a [svn] Recursion and progress bar tweaks. 
Published in <sxsd727cvc0.fsf@florida.arsdigita.de>.
 2001-11-25 13:03:30 -08:00
hniksic
95631700f7 [svn] Remove host canonicalization code. 
Published in <sxsg072ai5v.fsf@florida.arsdigita.de>.
 2001-11-25 09:13:20 -08:00
hniksic
05463c7121 [svn] Commit progress bar tweaks. 
Published in <sxsd727cvc0.fsf@florida.arsdigita.de>.
 2001-11-24 20:46:26 -08:00
hniksic
222e9465b7 [svn] Implemented breadth-first retrieval. 
Published in <sxsherjczw2.fsf@florida.arsdigita.de>.
 2001-11-24 19:10:34 -08:00
hniksic
d5be8ecca4 [svn] Rewrite parsing and handling of URLs. 
Published in <sxs4rnnlklo.fsf@florida.arsdigita.de>.
 2001-11-21 16:24:28 -08:00
hniksic
f178e6c613 [svn] Clean up handling of schemes. 
Published in <sxswv0n7h7s.fsf@florida.arsdigita.de>.
 2001-11-18 16:12:05 -08:00
hniksic
d408dec0bd [svn] Use Solaris's libmd5 when available. 
Published in <sxsn11kbssm.fsf@florida.arsdigita.de>.
 2001-11-17 20:36:21 -08:00
hniksic
6ef5baa07b [svn] Prettier printing of HTTP headers. 
Published in <sxs4rnsdcw1.fsf@florida.arsdigita.de>.
 2001-11-17 18:36:30 -08:00
hniksic
ac74e16638 [svn] Applied Christian Fraenkel's patch from <3BAF5B30.189077D6@gmx.net>. 
Print SSL errors when connecting fails.
 2001-11-16 09:06:25 -08:00
hniksic
c77d880776 [svn] Search netrc with the proper host name, not the proxy one. 2001-06-14 13:55:02 -07:00
hniksic
0b056d1720 [svn] Update copyright notices. 2001-05-27 12:35:15 -07:00
hniksic
36eba40905 [svn] Make sure cookie code works with the real URL data, not the proxy one. 
Published in <sxsg0e8z3bp.fsf@florida.arsdigita.de>.
 2001-05-14 02:32:11 -07:00
hniksic
9588e1af80 [svn] Before concluding that the file is already fully retrieved, make sure 
that the file existed and `Range' was actually requested.

Published in <sxseltwf4ks.fsf@florida.arsdigita.de>.
 2001-05-11 05:37:37 -07:00
hniksic
105627124e [svn] Make sure hstat.no_truncate gets properly initialized. 
Published in <sxspudk6pdl.fsf@florida.arsdigita.de>.
 2001-05-08 04:47:05 -07:00
hniksic
50980de841 [svn] Avoid useless file_exists_p. Published in 
<sxsr8y17zav.fsf@florida.arsdigita.de>.
 2001-05-07 12:08:29 -07:00
hniksic
b44f1701b0 [svn] Use alloca to allocate filename_plus_orig_suffix. 
Published in <sxsu239htnl.fsf@florida.arsdigita.de>.
 2001-04-28 09:36:46 -07:00
hniksic
d70101ea4e [svn] Commit various HTTP fixes. 
Published in <sxsy9slhu7g.fsf@florida.arsdigita.de>.

* http.c (gethttp): Return RETRUNNEEDED when the retrieval is
unneeded because the file is already there and fully downloaded,
and -c is specified.
(http_loop): Handle RETRUNNEEDED.

* wget.h (uerr_t): New value RETRUNNEEDED.

* http.c (http_loop): Set no_truncate for files that both exist
and are non-empty.
(gethttp): Consider the download finished when restval >= contlen,
not only when restval==contlen.
(gethttp): Handle redirection before giving up due to -c.
(gethttp): Clarify error message which explains that -c will not
truncate the file.
(gethttp): When returning CONTNOTSUPPORTED, don't forget to free
the stuff that needs freeing and release the socket.

* main.c (print_help): Wget booleans accept "off", not "no".
 2001-04-28 09:22:20 -07:00
hniksic
c6e565fd33 [svn] Send the no-cache directive when required regardless of whether we're 
going through a proxy.
Published in <sxsitjqncbx.fsf@florida.arsdigita.de>.
 2001-04-26 22:26:44 -07:00
hniksic
26ddb28a7c [svn] Improve documentation of mktime_from_utc. 
Rearrange http_atotm.
Published in <sxsoftjlyku.fsf@florida.arsdigita.de>.
 2001-04-26 22:11:38 -07:00
hniksic
f0eb1fb758 [svn] Fix loading of cookies. 
Published in <sxslmopyao6.fsf@florida.arsdigita.de>.
 2001-04-24 19:29:54 -07:00
hniksic
9ae0328c3d [svn] Applied Roger Beeman's mktime_from_utc fix published in 
<Pine.HPX.4.02.10104181128180.6232-100000@mail1.cisco.com>.
Also, minor doc fixes.
 2001-04-24 17:50:22 -07:00
hniksic
45eea7387f [svn] New timer functions. Published in <sxs8zkp28vk.fsf@florida.arsdigita.de>. 2001-04-24 16:09:45 -07:00
hniksic
61bb00adc0 [svn] Various url.c-related changes. 
Published in <sxsvgo8nmub.fsf@florida.arsdigita.de>.

* retr.c (retrieve_url): Call uri_merge, not url_concat.
* html-url.c (collect_tags_mapper): Call uri_merge, not
url_concat.
* url.c (mkstruct): Use encode_string instead of xstrdup followed
by URL_CLEANSE.
(path_simplify_with_kludge): Deleted.
(contains_unsafe): Deleted.
(construct): Renamed to uri_merge_1.
(url_concat): Renamed to uri_merge.
* url.c (str_url): Use encode_string instead of the unnecessary
CLEANDUP.
(encode_string_maybe): New function, returns input string if no
encoding is needed.
(encode_string): Call encode_string_maybe to do the dirty work,
xstrdup if no work needed.
* wget.h (XDIGIT_TO_xchar): Define here.
* url.c (decode_string): Use new name.
(encode_string): Ditto.
* http.c (XDIGIT_TO_xchar): Rename HEXD2asc to XDIGIT_TO_xchar.
(dump_hash): Use new name.
* wget.h: Rename ASC2HEXD and HEXD2ASC to XCHAR_TO_XDIGIT and
XDIGIT_TO_XCHAR respectively.
 2001-04-13 21:11:35 -07:00
hniksic
5fa53b5a1d [svn] Implement better sorting and matching of cookies. 
Fix previously broken command-line options.
Published in <sxspuek9255.fsf@florida.arsdigita.de>.
 2001-04-10 09:04:18 -07:00
hniksic
579d051d57 [svn] Rewrite SSL #ifdef for better indentation. 2001-04-09 07:58:55 -07:00
hniksic
2cfb2d2ef6 [svn] Added support for cookies. 2001-04-08 15:25:24 -07:00
hniksic
31ea5aae27 [svn] Prepend literal newline with `\n\' inside quoted string. 2001-04-05 20:06:00 -07:00
hniksic
1d51748034 [svn] Fix support for rfc850 time recognition on systems where strptime 
format specs %a and %A are not the same.
Published in <49022.983849922@ideon.st.ryukoku.ac.jp>.
 2001-04-03 11:54:32 -07:00
hniksic
f1d534cd8c [svn] Make -c' and -O' work together. 
Published in <sxsitkmgokh.fsf@florida.arsdigita.de>.
 2001-04-03 05:24:49 -07:00
hniksic
66a0c39e22 [svn] Make sure opt.dfp is rewound only on the first retrieval. 
(See the code comment for a more detailed explanation.)
Published in <sxsy9tkqizz.fsf@florida.arsdigita.de>.
 2001-04-01 16:50:04 -07:00
hniksic
a52aaeb451 [svn] Make sure -c doesn't truncate the file. 
Published in <sxsofuhi4kc.fsf@florida.arsdigita.de>.
 2001-04-01 13:04:21 -07:00
hniksic
8f048b7be8 [svn] Committed my patch that makes HTTP code close its socket in premature 
returns (before the body was drained).  Published in
<sxsd7ayn0kt.fsf@florida.arsdigita.de>.
 2001-03-30 18:20:16 -08:00
hniksic
1a6058b1ec [svn] Applied Philipp Thomas's safe-ctype patch. Published in 
<20010330025159.U21662@jeffreys.suse.de>.
 2001-03-30 14:36:59 -08:00
janp
5014d32c3a [svn] Skip `:port' in the host header if it is the DEFAULT_HTTPS_PORT when 
using SSL. Patch submitted by Hack Kampbjorn <hack@hackdata.com>.
 2001-03-08 15:11:03 -08:00
hniksic
366b832e3f [svn] Applied Jonas Jensen's download rate patch with my additions. 
Published in <015b01c09084$ce2d9330$0100000a@bob> and
<sxspugq3xai.fsf@florida.arsdigita.de>.
 2001-02-10 16:00:30 -08:00
hniksic
07f60f19a4 [svn] Applied Christian Fraenkel's patch "custom certificate patch for wget-1.7+dev; 
2nd try", published in <200102082132220160.00474E73@pop.gmx.net>.
It adds two options for custom certificates.
 2001-02-10 14:33:31 -08:00
dan
e979053779 [svn] 2000-12-31 Dan Harkless <wget@harkless.org> 
* ftp.c, http.c:  Applied Hack Kampbj?rn <hack@hackdata.com>'s
        patch to deal with h_errno not being defined in netdb.h under Cygwin.

2000-12-30  Dan Harkless  <wget@harkless.org>

        * version.c: Released Wget version 1.6.  Note that on this branch we
        never actually had the version set to 1.6, but we still need the
        ChangeLog comment for posterity.
 2000-12-31 04:04:14 -08:00
hniksic
23cbd15095 [svn] Igor Khristophorov's fix for check_end(). From 
<sxs4s027snq.fsf@florida.arsdigita.de>.
 2000-12-17 11:31:30 -08:00
hniksic
5099ec0306 [svn] Apply lint-expired fixes from <sxsn1du7ufa.fsf@florida.arsdigita.de>. 2000-12-17 10:52:52 -08:00
hniksic
2e8fc46b7b [svn] Include <netdb.h> where h_errno is used. Likewise for <errno.h> and errno. 
From <sxsvgsi7wcw.fsf@florida.arsdigita.de>.
 2000-12-17 10:12:02 -08:00
hniksic
d5cd691b12 [svn] Fix -O timestamp bug. 
From <sxs7l57u6o6.fsf@florida.arsdigita.de>.
 2000-12-10 16:47:44 -08:00
hniksic
62b876401e [svn] Set the timestamp even when -O is used. 2000-12-10 16:31:14 -08:00
hniksic
afe131624b [svn] Add , before `int ssl'. 2000-12-06 09:22:27 -08:00
hniksic
cab759659e [svn] Move inclusion of gen_ssl.h below Wget's headers. 2000-12-06 09:18:14 -08:00
hniksic
b0e9dfd4e2 [svn] Fix opt.wait. 2000-12-05 16:24:40 -08:00
hniksic
e754c99b46 [svn] Cosmetic cleanup of SSL stuff. 2000-12-05 15:35:56 -08:00
hniksic
7828e81c79 [svn] Committed C. Frankel's SSL patch. 2000-12-05 15:09:41 -08:00
hniksic
2ffb47eabf [svn] Committed <sxsbsv854j9.fsf@florida.arsdigita.de>. 2000-11-22 08:58:28 -08:00
hniksic
6e598c81e3 [svn] Committed a bunch of different tweaks of mine. 
Published in <sxsr9463wrx.fsf@florida.arsdigita.de>.
 2000-11-20 18:06:36 -08:00
hniksic
fb42069e51 [svn] Constify check_end(). 2000-11-20 02:25:28 -08:00
hniksic
7302e69dc2 [svn] Improved comments for the persistent connections feature. 2000-11-20 02:20:39 -08:00
hniksic
6f7fd37186 [svn] Auth tweaks. 
Published in <sxs3dgnz955.fsf@florida.arsdigita.de>.
 2000-11-19 18:10:54 -08:00
hniksic
f2f77d87fd [svn] New option --no-http-keep-alive. 
Published in <sxsd7fr1pdf.fsf@florida.arsdigita.de>.
 2000-11-19 16:04:06 -08:00
hniksic
b27144fcce [svn] My patch "persistent connection tweaks". 
Published in <sxshf531qhj.fsf@florida.arsdigita.de>.

(Applied with the addition of correct calculation for the
length of the request.)
 2000-11-19 15:42:13 -08:00
hniksic
b0b1c815c1 [svn] A bunch of new features: 
- use mmap() to read whole files in core instead of allocating memory
  and read'ing it.

- use a new, more general, HTML parser (html-parse.c) and interface to
  it from Wget (html-url.c).

- respect <meta name=robots content=nofollow> (easy with the new HTML
  parser).

- use hash tables instead of linked lists in places where the lists
  were used to facilitate mappings.

- rewrite the code in host.c to be more readable and faster (hash
  tables instead of home-grown lists.)

- make convert_links properly convert partial URLs to complete ones
  for those URLs that have *not* been downloaded.

- use HTTP persistent connections where available.  very
  simple-minded, caches the last connection to the server.

Published in <sxshf533d5r.fsf@florida.arsdigita.de>.
 2000-11-19 12:50:10 -08:00
hniksic
0e2b74ce3b [svn] Commit "minor fixes". 2000-11-06 13:24:57 -08:00
hniksic
eef4a668b7 [svn] Update copyright blurbs with the year 2000. 2000-11-01 17:50:03 -08:00
hniksic
c6575c6382 [svn] Another contributed fix... 2000-11-01 17:06:57 -08:00
hniksic
b7a8c6d3f5 [svn] Gracefully handle opt.downloaded overflowing. 
Published in <sxsd7gfnv17.fsf@florida.arsdigita.de>.
 2000-11-01 15:17:31 -08:00
hniksic
0dd418242a [svn] Committed my patches from <sxsbsw16sbu.fsf@florida.arsdigita.de> 
and <sxsvgu824xk.fsf@florida.arsdigita.de>.
 2000-10-31 11:25:32 -08:00
dan
b3e2c0ff97 [svn] Implemented and documented new -E / --html-extension / html_extension option. 2000-10-19 22:55:46 -07:00
hniksic
ee6065f581 [svn] Committed my patch from <dpd7mj3sap.fsf@mraz.iskon.hr>. 2000-05-19 00:37:22 -07:00
hniksic
6b4a85888e [svn] Commit several fixes. 2000-04-12 06:23:35 -07:00
hniksic
bc7060a81d [svn] More old fixes. 2000-03-31 06:14:58 -08:00
dan
3a8c75cac4 [svn] Dan Berger's query string patch is totally bogus. If you have two different 
URLs, gen_page.cgi?page1 and get_page.cgi?page2, they'll both be saved as
get_page.cgi and the second will overwrite the first.  Also, parameters to
implicit CGIs, like "http://www.host.com/db/?2000-03-02" cause the URLs to be
printed with trailing garbage characters, and could seg fault.  I'm not sure
what Dan had in mind with this patch (no explanatory comments), but I'm removing
it for now.  If he can rewrite it so it doesn't break stuff, okay.
 2000-03-02 14:48:07 -08:00
dan
d2e1d7fe9d [svn] Hrvoje didn't regenerate the .info files after changing wget.texi. 
Got rid of newly-introduced nested-if warnings in ftp.c and http.c.  Fixed
apparently completely untested code in main.c that was trying to provide --wait
/ --waitretry backwards compatibility, but had multiple fundamental bugs.
 2000-03-02 13:17:47 -08:00
hniksic
5d8cfbd904 [svn] Applied contributed patches (see ChangeLog for details.) 2000-03-02 06:45:37 -08:00
hniksic
2b2fd2924a [svn] Added user-contributed patches. 2000-03-02 06:16:12 -08:00
hniksic
cf35e2009d [svn] Applied contributed patch. 2000-03-02 05:34:05 -08:00
dan
4331c39c9a [svn] Implemented the item I formerly had in the TODO: When -K and -N are used 
together, we compare local file X.orig (if extant) against server file X.
Previously -k and -N were worthless in combination because the local converted
files always differed from the server versions.
 2000-03-01 22:33:48 -08:00
kwget
31d6616c48 [svn] Initial revision 1999-12-01 23:42:23 -08:00