1
0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Commit Graph

28 Commits

Author SHA1 Message Date
hniksic
95c647eb44 [svn] Split off non-URL related stuff from url.c to convert.c. 2003-09-21 15:47:14 -07:00
hniksic
101f896e47 [svn] Minor fixes and cosmetic changes.
(uri_merge): Get rid of uri_merge_1.
(uri_merge): Merge "foo//", "bar" as "foo//bar", not "foo///bar",
i.e. don't add an extra slash merely because BASE ends with two
slashes.
(parse_credentials): Renamed from parse_uname.  Rewrittern in
standard [beg, end) calling style.
(url_skip_credentials): Renamed from url_skip_uname.  Made static.
(url_skip_credentials): Include # and ; as terminators.  Old code
would mistakenly consider "http://foo.com#hniksic@iskon.hr" to
contain a username.
(url_skip_scheme): Removed because it was unused.
(url_has_scheme): Require "scheme" to be at least one char long.
2003-09-19 17:05:36 -07:00
hniksic
0a3697ad65 [svn] New mechanism for quoting file names.
Published in <m3smmzt4px.fsf@hniksic.iskon.hr>.
2003-09-14 15:04:13 -07:00
hniksic
b2be7522c7 [svn] Update the license to include the OpenSSL exception. 2002-05-17 19:16:36 -07:00
abbotti
e863a6323b [svn] New function schemes_are_similar_p to test enumerated scheme codes for
similarity (SCHEME_HTTP and SCHEME_HTTPS are similar).  Use it in recur.c
(download_child_p).  Fixes a bug that caused -H option to be ignored when
child scheme different to parent scheme.
Published in <agn4eu8apduek7magfu9bfe63gto8i7cdh@farscape.privy.mev.co.uk>.
2002-05-16 10:22:24 -07:00
hniksic
5390ada318 [svn] Support FWTK-style proxies.
Pbublished in <sxslmbsxptu.fsf@florida.arsdigita.de>.
2002-04-12 20:04:47 -07:00
hniksic
0620ada923 [svn] Fix OpenSSL PRNG seeding.
Published in <sxs7ks1noc4.fsf@florida.arsdigita.de>.
2001-12-05 17:13:31 -08:00
hniksic
f4d019a423 [svn] Correctly convert links in <meta http-equiv=Refresh content="...">.
Published in <sxsadx3wp49.fsf@florida.arsdigita.de>.
2001-11-30 20:18:51 -08:00
hniksic
a4db28e20f [svn] Ignore -np when in -p mode.
Published in <sxsg06w2c52.fsf@florida.arsdigita.de>.
2001-11-30 13:17:53 -08:00
hniksic
df05e7ff10 [svn] Handle <base href=...> when converting links.
Published in <sxsadxaae3t.fsf@florida.arsdigita.de>.
2001-11-25 10:40:55 -08:00
hniksic
222e9465b7 [svn] Implemented breadth-first retrieval.
Published in <sxsherjczw2.fsf@florida.arsdigita.de>.
2001-11-24 19:10:34 -08:00
hniksic
d5be8ecca4 [svn] Rewrite parsing and handling of URLs.
Published in <sxs4rnnlklo.fsf@florida.arsdigita.de>.
2001-11-21 16:24:28 -08:00
hniksic
e8e8797873 [svn] Rewrite shorthand URLs in a step separate from parsing.
Published in <sxspu6f7ecz.fsf@florida.arsdigita.de>.
2001-11-18 17:14:14 -08:00
hniksic
f178e6c613 [svn] Clean up handling of schemes.
Published in <sxswv0n7h7s.fsf@florida.arsdigita.de>.
2001-11-18 16:12:05 -08:00
hniksic
0c42479322 [svn] Applied Edward Sabol's patch from
<200106131813.f5DIDss1294858@alderaan.gsfc.nasa.gov>.
It fixes a memory leak in url_equal, and comments it out,
as it's unused.
2001-11-16 08:49:19 -08:00
hniksic
0b056d1720 [svn] Update copyright notices. 2001-05-27 12:35:15 -07:00
hniksic
61bb00adc0 [svn] Various url.c-related changes.
Published in <sxsvgo8nmub.fsf@florida.arsdigita.de>.

* retr.c (retrieve_url): Call uri_merge, not url_concat.
* html-url.c (collect_tags_mapper): Call uri_merge, not
url_concat.
* url.c (mkstruct): Use encode_string instead of xstrdup followed
by URL_CLEANSE.
(path_simplify_with_kludge): Deleted.
(contains_unsafe): Deleted.
(construct): Renamed to uri_merge_1.
(url_concat): Renamed to uri_merge.
* url.c (str_url): Use encode_string instead of the unnecessary
CLEANDUP.
(encode_string_maybe): New function, returns input string if no
encoding is needed.
(encode_string): Call encode_string_maybe to do the dirty work,
xstrdup if no work needed.
* wget.h (XDIGIT_TO_xchar): Define here.
* url.c (decode_string): Use new name.
(encode_string): Ditto.
* http.c (XDIGIT_TO_xchar): Rename HEXD2asc to XDIGIT_TO_xchar.
(dump_hash): Use new name.
* wget.h: Rename ASC2HEXD and HEXD2ASC to XCHAR_TO_XDIGIT and
XDIGIT_TO_XCHAR respectively.
2001-04-13 21:11:35 -07:00
hniksic
8a0e9e765e [svn] Minor -Wall-induced fixes. Also, skip_url is removed.
Published in <sxs8zl5v5cw.fsf@florida.arsdigita.de>.
2001-04-12 20:39:23 -07:00
janp
5014d32c3a [svn] Skip `:port' in the host header if it is the DEFAULT_HTTPS_PORT when
using SSL. Patch submitted by Hack Kampbjorn <hack@hackdata.com>.
2001-03-08 15:11:03 -08:00
hniksic
6e598c81e3 [svn] Committed a bunch of different tweaks of mine.
Published in <sxsr9463wrx.fsf@florida.arsdigita.de>.
2000-11-20 18:06:36 -08:00
hniksic
b0b1c815c1 [svn] A bunch of new features:
- use mmap() to read whole files in core instead of allocating memory
  and read'ing it.

- use a new, more general, HTML parser (html-parse.c) and interface to
  it from Wget (html-url.c).

- respect <meta name=robots content=nofollow> (easy with the new HTML
  parser).

- use hash tables instead of linked lists in places where the lists
  were used to facilitate mappings.

- rewrite the code in host.c to be more readable and faster (hash
  tables instead of home-grown lists.)

- make convert_links properly convert partial URLs to complete ones
  for those URLs that have *not* been downloaded.

- use HTTP persistent connections where available.  very
  simple-minded, caches the last connection to the server.

Published in <sxshf533d5r.fsf@florida.arsdigita.de>.
2000-11-19 12:50:10 -08:00
hniksic
0dd418242a [svn] Committed my patches from <sxsbsw16sbu.fsf@florida.arsdigita.de>
and <sxsvgu824xk.fsf@florida.arsdigita.de>.
2000-10-31 11:25:32 -08:00
dan
b3e2c0ff97 [svn] Implemented and documented new -E / --html-extension / html_extension option. 2000-10-19 22:55:46 -07:00
dan
7931200609 [svn] * *.{gmo,po,pot}: Regenerated after modifying wget --help output.
* ftp.c (ftp_retrieve_list): Use new INFINITE_RECURSION #define.

* html.c: htmlfindurl() now takes final `dash_p_leaf_HTML' parameter.
Wrapped some > 80-column lines.  When -p is specified and we're at a
leaf node, do not traverse <A>, <AREA>, or <LINK> tags other than
<LINK REL="stylesheet">.

* html.h (htmlfindurl): Now takes final `dash_p_leaf_HTML' parameter.

* init.c: Added new -p / --page-requisites / page_requisites option.

* main.c (print_help): Clarified that -l inf and -l 0 both allow
infinite recursion.  Changed the unhelpful --mirrior description
to simply give the options it's equivalent to.  Added new -p option.
(main): Added some comments; handle new -p / --page-requisites.

* options.h (struct options): Added new page_requisites field.

* recur.c: Changed "URL-s" to "URLs" and "HTML-s" to "HTMLs".
Calculate and pass down new `dash_p_leaf_HTML' parameter to
get_urls_html().  Use new INFINITE_RECURSION #define.

* retr.c: Changed "URL-s" to "URLs".  get_urls_html() now takes
final `dash_p_leaf_HTML' parameter.

* url.c: get_urls_html() and htmlfindurl() now take final
`dash_p_leaf_HTML' parameter.

* url.h (get_urls_html): Now takes final `dash_p_leaf_HTML' parameter.

* wget.h: Added some comments and new INFINITE_RECURSION #define.

* wget.texi (Recursive Retrieval Options): Documented new -p option.
2000-08-30 04:26:21 -07:00
hniksic
2b2fd2924a [svn] Added user-contributed patches. 2000-03-02 06:16:12 -08:00
dan
4331c39c9a [svn] Implemented the item I formerly had in the TODO: When -K and -N are used
together, we compare local file X.orig (if extant) against server file X.
Previously -k and -N were worthless in combination because the local converted
files always differed from the server versions.
2000-03-01 22:33:48 -08:00
dan
e5408e7db8 [svn] Implemented new -K / --backup-converted / backup_converted = on option. 2000-02-29 16:17:23 -08:00
kwget
31d6616c48 [svn] Initial revision 1999-12-01 23:42:23 -08:00