moparisthebest/wget
1
0
Fork 0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Code Issues Releases Wiki Activity

Support libpsl for cookie domain checking

Browse Source
This commit is contained in:
Darshit Shah 2014-05-30 22:10:12 +05:30
parent 8624553a31
commit 854ebbf4dd
7 changed files with 67 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
ChangeLog
View File

@ -1,3 +1,8 @@
2014-05-30 Darshit Shah <darnir@gmail.com>
* configure.ac: Allow compilation without libpsl.
* README.checkout: Add libpsl as a dependency.
2014-05-24 Giuseppe Scrivano <gscrivan@redhat.com> 2014-05-24 Giuseppe Scrivano <gscrivan@redhat.com>
* gnulib: update module. * gnulib: update module.

2
NEWS
View File

@ -8,6 +8,8 @@ Please send GNU Wget bug reports to <bug-wget@gnu.org>.
* Changes in Wget X.Y.Z * Changes in Wget X.Y.Z
** Use libpsl for verifying cookie domains
** Default progress bar output changed ** Default progress bar output changed
** Introduce --show-progress to force display the progress bar ** Introduce --show-progress to force display the progress bar

44
README.checkout
View File

@ -4,8 +4,8 @@ Compiling From Repository Sources
does not contain automatically-generated files, even when these are does not contain automatically-generated files, even when these are
normally present in the distribution tarballs. Therefore, to build GNU normally present in the distribution tarballs. Therefore, to build GNU
Wget from the sources in the repository, you'll need to have one or Wget from the sources in the repository, you'll need to have one or
more of the following (note that gettext, OpenSSL, libidn and libiconv more of the following (note that gettext, OpenSSL, libidn, libiconv
are not absolutely required): and libpsl are not absolutely required):
* [20]autoconf (currently, GNU Wget requires version 2.61). This is * [20]autoconf (currently, GNU Wget requires version 2.61). This is
needed to generate the configure script from configure.in. This is needed to generate the configure script from configure.in. This is
@ -68,6 +68,9 @@ Compiling From Repository Sources
* [34]git is used to fetch gnulib files trough the bootstrap.sh script. * [34]git is used to fetch gnulib files trough the bootstrap.sh script.
* [35]libpsl is required for using a public suffix list to check for valid
cookie domains.
For those who might be confused as to what to do once they check out For those who might be confused as to what to do once they check out
the source code, considering configure and Makefile do not yet exist at the source code, considering configure and Makefile do not yet exist at
that point, a shell script called bootstrap.sh has been provided. After that point, a shell script called bootstrap.sh has been provided. After
@ -96,21 +99,21 @@ Compiling From Repository Sources
Originally written by Hrvoje Niksic <hniksic@xemacs.org>. Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
* [35]Edit * [36]Edit
* [36]Comments * [37]Comments
* [37]Info * [38]Info
* [38]Attachments * [39]Attachments
* More Actions: * More Actions:
[Raw Text................] Do [Raw Text................] Do
* [39]MoinMoin Powered * [40]MoinMoin Powered
* [40]Python Powered * [41]Python Powered
* [41]GPL licensed * [42]GPL licensed
* [42]Valid HTML 4.01 * [43]Valid HTML 4.01
__________________________________________________________________ __________________________________________________________________
All content © 2007 Free Software Foundation. For terms of use, All content © 2007 Free Software Foundation. For terms of use,
redistribution, and modification, please see the [43]WikiLicense page. redistribution, and modification, please see the [44]WikiLicense page.
References References
@ -129,12 +132,13 @@ References
32. http://www.gnu.org/software/libidn/ 32. http://www.gnu.org/software/libidn/
33. http://www.gnu.org/software/libiconv/ 33. http://www.gnu.org/software/libiconv/
34. http://git-scm.com/ 34. http://git-scm.com/
35. http://wget.addictivecode.org/CompilingRepoSources?action=edit&editor=text 35. https://github.com/rockdaboot/libpsl
36. http://wget.addictivecode.org/CompilingRepoSources 36. http://wget.addictivecode.org/CompilingRepoSources?action=edit&editor=text
37. http://wget.addictivecode.org/CompilingRepoSources?action=info 37. http://wget.addictivecode.org/CompilingRepoSources
38. http://wget.addictivecode.org/CompilingRepoSources?action=AttachFile 38. http://wget.addictivecode.org/CompilingRepoSources?action=info
39. http://moinmo.in/ 39. http://wget.addictivecode.org/CompilingRepoSources?action=AttachFile
40. http://moinmo.in/Python 40. http://moinmo.in/
41. http://moinmo.in/GPL 41. http://moinmo.in/Python
42. http://validator.w3.org/check?uri=referer 42. http://moinmo.in/GPL
43. http://wget.addictivecode.org/WikiLicense 43. http://validator.w3.org/check?uri=referer
44. http://wget.addictivecode.org/WikiLicense

11
configure.ac
View File

@ -61,6 +61,10 @@ dnl
dnl Process features. dnl Process features.
dnl dnl
AC_ARG_WITH(libpsl,
AS_HELP_STRING([--without-libpsl],
[disable support for libpsl cookie checking.]))
AC_ARG_WITH(ssl, AC_ARG_WITH(ssl,
[[ --without-ssl disable SSL autodetection [[ --without-ssl disable SSL autodetection
--with-ssl={gnutls,openssl} specify the SSL backend. GNU TLS is the default.]]) --with-ssl={gnutls,openssl} specify the SSL backend. GNU TLS is the default.]])
@ -237,6 +241,11 @@ dnl
dnl Checks for libraries. dnl Checks for libraries.
dnl dnl
AS_IF([test x"$with_libpsl" != xno], [
with_libpsl=yes
AC_CHECK_LIB([psl], [psl_builtin])
])
AS_IF([test x"$with_zlib" != xno], [ AS_IF([test x"$with_zlib" != xno], [
with_zlib=yes with_zlib=yes
AC_CHECK_LIB(z, compress) AC_CHECK_LIB(z, compress)
@ -358,6 +367,7 @@ else
fi fi
fi fi
dnl ********************************************************************** dnl **********************************************************************
dnl Checks for IPv6 dnl Checks for IPv6
dnl ********************************************************************** dnl **********************************************************************
@ -580,6 +590,7 @@ AC_MSG_NOTICE([Summary of build options:
Libs: $LIBS Libs: $LIBS
SSL: $with_ssl SSL: $with_ssl
Zlib: $with_zlib Zlib: $with_zlib
PSL: $with_libpsl
Digest: $ENABLE_DIGEST Digest: $ENABLE_DIGEST
NTLM: $ENABLE_NTLM NTLM: $ENABLE_NTLM
OPIE: $ENABLE_OPIE OPIE: $ENABLE_OPIE

6
src/ChangeLog
View File

@ -1,3 +1,8 @@
2014-05-30 Darshit Shah <darnir@gmail.com>
* cookies.c (check_domain_match): Use libpsl to check if the cookie domain
is valid. Also remove unneeded test for numeric addresses.
2014-05-30 Darshit Shah <darnir@gmail.com> 2014-05-30 Darshit Shah <darnir@gmail.com>
* connect.{c,h}, convert.{c,h}, cookies.{c,h}, ftp-ls.c, ftp.h, gettext.h, * connect.{c,h}, convert.{c,h}, cookies.{c,h}, ftp-ls.c, ftp.h, gettext.h,
@ -22,7 +27,6 @@
needs to modify the string. needs to modify the string.
(bar_set_params): Add support for noscroll parameter to bar. (bar_set_params): Add support for noscroll parameter to bar.
2014-05-03 Tim Ruehsen <tim.ruehsen@gmx.de> 2014-05-03 Tim Ruehsen <tim.ruehsen@gmx.de>
* ftp-ls.c (ftp_parse_vms_ls): Explicitly typecast strlen's output * ftp-ls.c (ftp_parse_vms_ls): Explicitly typecast strlen's output

1
src/build_info.c.in
View File

@ -7,6 +7,7 @@ large-file SIZEOF_OFF_T >= 8
nls defined ENABLE_NLS nls defined ENABLE_NLS
ntlm defined ENABLE_NTLM ntlm defined ENABLE_NTLM
opie defined ENABLE_OPIE opie defined ENABLE_OPIE
psl defined HAVE_LIBPSL
ssl choice: ssl choice:
openssl defined HAVE_LIBSSL || defined HAVE_LIBSSL32 openssl defined HAVE_LIBSSL || defined HAVE_LIBSSL32

24
src/cookies.c
View File

@ -51,6 +51,7 @@ as that of the covered work. */
#include <assert.h> #include <assert.h>
#include <errno.h> #include <errno.h>
#include <time.h> #include <time.h>
#include <libpsl.h>
#include "utils.h" #include "utils.h"
#include "hash.h" #include "hash.h"
#include "cookies.h" #include "cookies.h"
@ -503,14 +504,27 @@ numeric_address_p (const char *addr)
static bool static bool
check_domain_match (const char *cookie_domain, const char *host) check_domain_match (const char *cookie_domain, const char *host)
{ {
#ifdef HAVE_LIBPSL
DEBUGP (("cdm: 1")); DEBUGP (("cdm: 1"));
const psl_ctx_t *psl;
int is_acceptable;
/* Numeric address requires exact match. It also requires HOST to if (!(psl = psl_builtin()))
be an IP address. */ {
if (numeric_address_p (cookie_domain)) DEBUGP (("\nlibpsl not built with a public suffix list. "
return 0 == strcmp (cookie_domain, host); "Falling back to simple heuristics.\n"));
goto no_psl;
}
DEBUGP ((" 2")); is_acceptable = psl_is_cookie_domain_acceptable (psl, host, cookie_domain);
return true ? (is_acceptable == 1) : false;
no_psl:
#endif
/* For efficiency make some elementary checks first */
DEBUGP (("cdm: 2"));
/* For the sake of efficiency, check for exact match first. */ /* For the sake of efficiency, check for exact match first. */
if (0 == strcasecmp (cookie_domain, host)) if (0 == strcasecmp (cookie_domain, host))