moparisthebest/wget
1
0
Fork 0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Code Issues Releases Wiki Activity
1,896 Commits 1 Branch 0 Tags 9.6 MiB
6493450947
Commit Graph

237 Commits

Author SHA1 Message Date
abbotti
cfd7b9a951 [svn] Use new function to test filename for common html suffixes. 
Submitted by Ian Abbott in <3CB72D29.4898.1F34872@localhost> with minor
changes to formatting and comments.
 2002-04-12 11:53:39 -07:00
hniksic
1375a89141 [svn] Allow Wget to be compiled with a K&R compiler. 
Published in <sxszo0af9k6.fsf@florida.arsdigita.de>.
 2002-04-11 10:49:32 -07:00
hniksic
a9ff5853a9 [svn] when -c used, mark already fully retrieved file as successfully retrieved. 
Submitted by Ian Abbott in <3C6019F4.30803.FF43E9@localhost>.
 2002-02-18 21:18:43 -08:00
hniksic
f2bb99d102 [svn] Wrap numeric IPv6 addresses in square brackets when sending out HTTP 
`Host' header.
Published in <sxs7kq4vgzf.fsf@florida.arsdigita.de>.
 2002-01-26 15:19:10 -08:00
hniksic
cafe798fec [svn] Invalidate socket if get_contents encountered an error. 
Published in <sxsita54sqy.fsf@florida.arsdigita.de>.
 2002-01-13 17:39:16 -08:00
hniksic
5b575eef31 [svn] Check for conn->ssl before using ssl_iwrite. 2001-12-13 08:46:56 -08:00
hniksic
dd84231c6a [svn] Minor fixes prompted by `lint'. 
Published in <sxsadwt2nkg.fsf@florida.arsdigita.de>.
 2001-12-08 17:24:41 -08:00
hniksic
6d93a95e37 [svn] Declare h_errno in host.c, but not elsewhere. 2001-12-03 09:22:06 -08:00
hniksic
569fd61c95 [svn] Use the full path when building the authorization line. 
Published in <sxsitbqu9iw.fsf@florida.arsdigita.de>.
 2001-12-01 09:39:07 -08:00
hniksic
2c37aef2d8 [svn] Don't forget to shutdown the SSL structure if the socket times out. 
Submitted by T. Bharath in <3B3DEBC7.2537186A@responsenetworks.com>.
 2001-11-30 00:57:03 -08:00
hniksic
b65661a879 [svn] Remove char/unsigned char warnings emitted by Sun cc. 2001-11-29 10:48:43 -08:00
hniksic
a3d3d452ab [svn] Make the MD5 references even more generic. 
Use OpenSSL's MD5 where available.
Published in <sxsg06x4f41.fsf@florida.arsdigita.de>.
 2001-11-29 10:17:54 -08:00
hniksic
6218fbed2b [svn] Accept the broken Content-Range format emitted by Sun's server. 
Published in <sxsg06x7kld.fsf@florida.arsdigita.de>.
 2001-11-29 06:03:33 -08:00
hniksic
278fb2412d [svn] Declare ERR in gethttp. 
Submitted by Ian Abbott in <3C022295.15135.464B54@localhost>.
 2001-11-26 04:51:25 -08:00
hniksic
84396de673 [svn] Allocate a timer directly in get_contents. 
Published in <sxsherhbsvl.fsf@florida.arsdigita.de>.
 2001-11-26 04:49:10 -08:00
hniksic
024cb5ed3a [svn] A lot of host name changes. 
Published in <sxs3d32856s.fsf@florida.arsdigita.de>.
 2001-11-25 21:36:33 -08:00
hniksic
3afb9c659a [svn] Recursion and progress bar tweaks. 
Published in <sxsd727cvc0.fsf@florida.arsdigita.de>.
 2001-11-25 13:03:30 -08:00
hniksic
95631700f7 [svn] Remove host canonicalization code. 
Published in <sxsg072ai5v.fsf@florida.arsdigita.de>.
 2001-11-25 09:13:20 -08:00
hniksic
05463c7121 [svn] Commit progress bar tweaks. 
Published in <sxsd727cvc0.fsf@florida.arsdigita.de>.
 2001-11-24 20:46:26 -08:00
hniksic
222e9465b7 [svn] Implemented breadth-first retrieval. 
Published in <sxsherjczw2.fsf@florida.arsdigita.de>.
 2001-11-24 19:10:34 -08:00
hniksic
d5be8ecca4 [svn] Rewrite parsing and handling of URLs. 
Published in <sxs4rnnlklo.fsf@florida.arsdigita.de>.
 2001-11-21 16:24:28 -08:00
hniksic
f178e6c613 [svn] Clean up handling of schemes. 
Published in <sxswv0n7h7s.fsf@florida.arsdigita.de>.
 2001-11-18 16:12:05 -08:00
hniksic
d408dec0bd [svn] Use Solaris's libmd5 when available. 
Published in <sxsn11kbssm.fsf@florida.arsdigita.de>.
 2001-11-17 20:36:21 -08:00
hniksic
6ef5baa07b [svn] Prettier printing of HTTP headers. 
Published in <sxs4rnsdcw1.fsf@florida.arsdigita.de>.
 2001-11-17 18:36:30 -08:00
hniksic
ac74e16638 [svn] Applied Christian Fraenkel's patch from <3BAF5B30.189077D6@gmx.net>. 
Print SSL errors when connecting fails.
 2001-11-16 09:06:25 -08:00
hniksic
c77d880776 [svn] Search netrc with the proper host name, not the proxy one. 2001-06-14 13:55:02 -07:00
hniksic
0b056d1720 [svn] Update copyright notices. 2001-05-27 12:35:15 -07:00
hniksic
36eba40905 [svn] Make sure cookie code works with the real URL data, not the proxy one. 
Published in <sxsg0e8z3bp.fsf@florida.arsdigita.de>.
 2001-05-14 02:32:11 -07:00
hniksic
9588e1af80 [svn] Before concluding that the file is already fully retrieved, make sure 
that the file existed and `Range' was actually requested.

Published in <sxseltwf4ks.fsf@florida.arsdigita.de>.
 2001-05-11 05:37:37 -07:00
hniksic
105627124e [svn] Make sure hstat.no_truncate gets properly initialized. 
Published in <sxspudk6pdl.fsf@florida.arsdigita.de>.
 2001-05-08 04:47:05 -07:00
hniksic
50980de841 [svn] Avoid useless file_exists_p. Published in 
<sxsr8y17zav.fsf@florida.arsdigita.de>.
 2001-05-07 12:08:29 -07:00
hniksic
b44f1701b0 [svn] Use alloca to allocate filename_plus_orig_suffix. 
Published in <sxsu239htnl.fsf@florida.arsdigita.de>.
 2001-04-28 09:36:46 -07:00
hniksic
d70101ea4e [svn] Commit various HTTP fixes. 
Published in <sxsy9slhu7g.fsf@florida.arsdigita.de>.

* http.c (gethttp): Return RETRUNNEEDED when the retrieval is
unneeded because the file is already there and fully downloaded,
and -c is specified.
(http_loop): Handle RETRUNNEEDED.

* wget.h (uerr_t): New value RETRUNNEEDED.

* http.c (http_loop): Set no_truncate for files that both exist
and are non-empty.
(gethttp): Consider the download finished when restval >= contlen,
not only when restval==contlen.
(gethttp): Handle redirection before giving up due to -c.
(gethttp): Clarify error message which explains that -c will not
truncate the file.
(gethttp): When returning CONTNOTSUPPORTED, don't forget to free
the stuff that needs freeing and release the socket.

* main.c (print_help): Wget booleans accept "off", not "no".
 2001-04-28 09:22:20 -07:00
hniksic
c6e565fd33 [svn] Send the no-cache directive when required regardless of whether we're 
going through a proxy.
Published in <sxsitjqncbx.fsf@florida.arsdigita.de>.
 2001-04-26 22:26:44 -07:00
hniksic
26ddb28a7c [svn] Improve documentation of mktime_from_utc. 
Rearrange http_atotm.
Published in <sxsoftjlyku.fsf@florida.arsdigita.de>.
 2001-04-26 22:11:38 -07:00
hniksic
f0eb1fb758 [svn] Fix loading of cookies. 
Published in <sxslmopyao6.fsf@florida.arsdigita.de>.
 2001-04-24 19:29:54 -07:00
hniksic
9ae0328c3d [svn] Applied Roger Beeman's mktime_from_utc fix published in 
<Pine.HPX.4.02.10104181128180.6232-100000@mail1.cisco.com>.
Also, minor doc fixes.
 2001-04-24 17:50:22 -07:00
hniksic
45eea7387f [svn] New timer functions. Published in <sxs8zkp28vk.fsf@florida.arsdigita.de>. 2001-04-24 16:09:45 -07:00
hniksic
61bb00adc0 [svn] Various url.c-related changes. 
Published in <sxsvgo8nmub.fsf@florida.arsdigita.de>.

* retr.c (retrieve_url): Call uri_merge, not url_concat.
* html-url.c (collect_tags_mapper): Call uri_merge, not
url_concat.
* url.c (mkstruct): Use encode_string instead of xstrdup followed
by URL_CLEANSE.
(path_simplify_with_kludge): Deleted.
(contains_unsafe): Deleted.
(construct): Renamed to uri_merge_1.
(url_concat): Renamed to uri_merge.
* url.c (str_url): Use encode_string instead of the unnecessary
CLEANDUP.
(encode_string_maybe): New function, returns input string if no
encoding is needed.
(encode_string): Call encode_string_maybe to do the dirty work,
xstrdup if no work needed.
* wget.h (XDIGIT_TO_xchar): Define here.
* url.c (decode_string): Use new name.
(encode_string): Ditto.
* http.c (XDIGIT_TO_xchar): Rename HEXD2asc to XDIGIT_TO_xchar.
(dump_hash): Use new name.
* wget.h: Rename ASC2HEXD and HEXD2ASC to XCHAR_TO_XDIGIT and
XDIGIT_TO_XCHAR respectively.
 2001-04-13 21:11:35 -07:00
hniksic
5fa53b5a1d [svn] Implement better sorting and matching of cookies. 
Fix previously broken command-line options.
Published in <sxspuek9255.fsf@florida.arsdigita.de>.
 2001-04-10 09:04:18 -07:00
hniksic
579d051d57 [svn] Rewrite SSL #ifdef for better indentation. 2001-04-09 07:58:55 -07:00
hniksic
2cfb2d2ef6 [svn] Added support for cookies. 2001-04-08 15:25:24 -07:00
hniksic
31ea5aae27 [svn] Prepend literal newline with `\n\' inside quoted string. 2001-04-05 20:06:00 -07:00
hniksic
1d51748034 [svn] Fix support for rfc850 time recognition on systems where strptime 
format specs %a and %A are not the same.
Published in <49022.983849922@ideon.st.ryukoku.ac.jp>.
 2001-04-03 11:54:32 -07:00
hniksic
f1d534cd8c [svn] Make -c' and -O' work together. 
Published in <sxsitkmgokh.fsf@florida.arsdigita.de>.
 2001-04-03 05:24:49 -07:00
hniksic
66a0c39e22 [svn] Make sure opt.dfp is rewound only on the first retrieval. 
(See the code comment for a more detailed explanation.)
Published in <sxsy9tkqizz.fsf@florida.arsdigita.de>.
 2001-04-01 16:50:04 -07:00
hniksic
a52aaeb451 [svn] Make sure -c doesn't truncate the file. 
Published in <sxsofuhi4kc.fsf@florida.arsdigita.de>.
 2001-04-01 13:04:21 -07:00
hniksic
8f048b7be8 [svn] Committed my patch that makes HTTP code close its socket in premature 
returns (before the body was drained).  Published in
<sxsd7ayn0kt.fsf@florida.arsdigita.de>.
 2001-03-30 18:20:16 -08:00
hniksic
1a6058b1ec [svn] Applied Philipp Thomas's safe-ctype patch. Published in 
<20010330025159.U21662@jeffreys.suse.de>.
 2001-03-30 14:36:59 -08:00
janp
5014d32c3a [svn] Skip `:port' in the host header if it is the DEFAULT_HTTPS_PORT when 
using SSL. Patch submitted by Hack Kampbjorn <hack@hackdata.com>.
 2001-03-08 15:11:03 -08:00
hniksic
366b832e3f [svn] Applied Jonas Jensen's download rate patch with my additions. 
Published in <015b01c09084$ce2d9330$0100000a@bob> and
<sxspugq3xai.fsf@florida.arsdigita.de>.
 2001-02-10 16:00:30 -08:00
hniksic
07f60f19a4 [svn] Applied Christian Fraenkel's patch "custom certificate patch for wget-1.7+dev; 
2nd try", published in <200102082132220160.00474E73@pop.gmx.net>.
It adds two options for custom certificates.
 2001-02-10 14:33:31 -08:00
dan
e979053779 [svn] 2000-12-31 Dan Harkless <wget@harkless.org> 
* ftp.c, http.c:  Applied Hack Kampbj?rn <hack@hackdata.com>'s
        patch to deal with h_errno not being defined in netdb.h under Cygwin.

2000-12-30  Dan Harkless  <wget@harkless.org>

        * version.c: Released Wget version 1.6.  Note that on this branch we
        never actually had the version set to 1.6, but we still need the
        ChangeLog comment for posterity.
 2000-12-31 04:04:14 -08:00
hniksic
23cbd15095 [svn] Igor Khristophorov's fix for check_end(). From 
<sxs4s027snq.fsf@florida.arsdigita.de>.
 2000-12-17 11:31:30 -08:00
hniksic
5099ec0306 [svn] Apply lint-expired fixes from <sxsn1du7ufa.fsf@florida.arsdigita.de>. 2000-12-17 10:52:52 -08:00
hniksic
2e8fc46b7b [svn] Include <netdb.h> where h_errno is used. Likewise for <errno.h> and errno. 
From <sxsvgsi7wcw.fsf@florida.arsdigita.de>.
 2000-12-17 10:12:02 -08:00
hniksic
d5cd691b12 [svn] Fix -O timestamp bug. 
From <sxs7l57u6o6.fsf@florida.arsdigita.de>.
 2000-12-10 16:47:44 -08:00
hniksic
62b876401e [svn] Set the timestamp even when -O is used. 2000-12-10 16:31:14 -08:00
hniksic
afe131624b [svn] Add , before `int ssl'. 2000-12-06 09:22:27 -08:00
hniksic
cab759659e [svn] Move inclusion of gen_ssl.h below Wget's headers. 2000-12-06 09:18:14 -08:00
hniksic
b0e9dfd4e2 [svn] Fix opt.wait. 2000-12-05 16:24:40 -08:00
hniksic
e754c99b46 [svn] Cosmetic cleanup of SSL stuff. 2000-12-05 15:35:56 -08:00
hniksic
7828e81c79 [svn] Committed C. Frankel's SSL patch. 2000-12-05 15:09:41 -08:00
hniksic
2ffb47eabf [svn] Committed <sxsbsv854j9.fsf@florida.arsdigita.de>. 2000-11-22 08:58:28 -08:00
hniksic
6e598c81e3 [svn] Committed a bunch of different tweaks of mine. 
Published in <sxsr9463wrx.fsf@florida.arsdigita.de>.
 2000-11-20 18:06:36 -08:00
hniksic
fb42069e51 [svn] Constify check_end(). 2000-11-20 02:25:28 -08:00
hniksic
7302e69dc2 [svn] Improved comments for the persistent connections feature. 2000-11-20 02:20:39 -08:00
hniksic
6f7fd37186 [svn] Auth tweaks. 
Published in <sxs3dgnz955.fsf@florida.arsdigita.de>.
 2000-11-19 18:10:54 -08:00
hniksic
f2f77d87fd [svn] New option --no-http-keep-alive. 
Published in <sxsd7fr1pdf.fsf@florida.arsdigita.de>.
 2000-11-19 16:04:06 -08:00
hniksic
b27144fcce [svn] My patch "persistent connection tweaks". 
Published in <sxshf531qhj.fsf@florida.arsdigita.de>.

(Applied with the addition of correct calculation for the
length of the request.)
 2000-11-19 15:42:13 -08:00
hniksic
b0b1c815c1 [svn] A bunch of new features: 
- use mmap() to read whole files in core instead of allocating memory
  and read'ing it.

- use a new, more general, HTML parser (html-parse.c) and interface to
  it from Wget (html-url.c).

- respect <meta name=robots content=nofollow> (easy with the new HTML
  parser).

- use hash tables instead of linked lists in places where the lists
  were used to facilitate mappings.

- rewrite the code in host.c to be more readable and faster (hash
  tables instead of home-grown lists.)

- make convert_links properly convert partial URLs to complete ones
  for those URLs that have *not* been downloaded.

- use HTTP persistent connections where available.  very
  simple-minded, caches the last connection to the server.

Published in <sxshf533d5r.fsf@florida.arsdigita.de>.
 2000-11-19 12:50:10 -08:00
hniksic
0e2b74ce3b [svn] Commit "minor fixes". 2000-11-06 13:24:57 -08:00
hniksic
eef4a668b7 [svn] Update copyright blurbs with the year 2000. 2000-11-01 17:50:03 -08:00
hniksic
c6575c6382 [svn] Another contributed fix... 2000-11-01 17:06:57 -08:00
hniksic
b7a8c6d3f5 [svn] Gracefully handle opt.downloaded overflowing. 
Published in <sxsd7gfnv17.fsf@florida.arsdigita.de>.
 2000-11-01 15:17:31 -08:00
hniksic
0dd418242a [svn] Committed my patches from <sxsbsw16sbu.fsf@florida.arsdigita.de> 
and <sxsvgu824xk.fsf@florida.arsdigita.de>.
 2000-10-31 11:25:32 -08:00
dan
b3e2c0ff97 [svn] Implemented and documented new -E / --html-extension / html_extension option. 2000-10-19 22:55:46 -07:00
hniksic
ee6065f581 [svn] Committed my patch from <dpd7mj3sap.fsf@mraz.iskon.hr>. 2000-05-19 00:37:22 -07:00
hniksic
6b4a85888e [svn] Commit several fixes. 2000-04-12 06:23:35 -07:00
hniksic
bc7060a81d [svn] More old fixes. 2000-03-31 06:14:58 -08:00
dan
3a8c75cac4 [svn] Dan Berger's query string patch is totally bogus. If you have two different 
URLs, gen_page.cgi?page1 and get_page.cgi?page2, they'll both be saved as
get_page.cgi and the second will overwrite the first.  Also, parameters to
implicit CGIs, like "http://www.host.com/db/?2000-03-02" cause the URLs to be
printed with trailing garbage characters, and could seg fault.  I'm not sure
what Dan had in mind with this patch (no explanatory comments), but I'm removing
it for now.  If he can rewrite it so it doesn't break stuff, okay.
 2000-03-02 14:48:07 -08:00
dan
d2e1d7fe9d [svn] Hrvoje didn't regenerate the .info files after changing wget.texi. 
Got rid of newly-introduced nested-if warnings in ftp.c and http.c.  Fixed
apparently completely untested code in main.c that was trying to provide --wait
/ --waitretry backwards compatibility, but had multiple fundamental bugs.
 2000-03-02 13:17:47 -08:00
hniksic
5d8cfbd904 [svn] Applied contributed patches (see ChangeLog for details.) 2000-03-02 06:45:37 -08:00
hniksic
2b2fd2924a [svn] Added user-contributed patches. 2000-03-02 06:16:12 -08:00
hniksic
cf35e2009d [svn] Applied contributed patch. 2000-03-02 05:34:05 -08:00
dan
4331c39c9a [svn] Implemented the item I formerly had in the TODO: When -K and -N are used 
together, we compare local file X.orig (if extant) against server file X.
Previously -k and -N were worthless in combination because the local converted
files always differed from the server versions.
 2000-03-01 22:33:48 -08:00
kwget
31d6616c48 [svn] Initial revision 1999-12-01 23:42:23 -08:00