moparisthebest
/
wget
mirror of https://github.com/moparisthebest/wget
1
0
Fork 0
Code Issues Releases Wiki Activity

[svn] recur.c (parse_robots): Applied Edward J. Sabol's patch for Guan Yang's reported 

problem with "User-agent:<space>*<space>" lines.
This commit is contained in:
dan 2000-03-02 13:28:59 -08:00
parent d2e1d7fe9d
commit 03e5e4fe4d
1 changed files with 5 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
src/recur.c
View File

@ -709,13 +709,13 @@ parse_robots (const char *robots_filename)
*str++ = '\0';
/* Look for the string beginning... */
for (; *str && ISSPACE (*str); str++);
/* Look for comments and kill them off. */
/* Look for comments or trailing spaces and kill them off. */
for (p = str; *p; p++)
if (*p && ISSPACE (*p) && *(p + 1) == '#')
if (*p && ISSPACE (*p) && ((*(p + 1) == '#') || (*(p + 1) == '\0')))
{
/* We have found a shell-style comment `<sp>+ #'. Now
rewind to the beginning of the spaces and place '\0'
there. */
/* We have found either a shell-style comment `<sp>+#' or some
trailing spaces. Now rewind to the beginning of the spaces
and place '\0' there. */
while (p > str && ISSPACE (*p))
--p;
if (p == str)