moparisthebest/wget
1
0
Fork 0
mirror of https://github.com/moparisthebest/wget synced 2024-07-03 16:38:41 -04:00
Code Issues Releases Wiki Activity

Add support for chunks to the WARC outputter.

Browse Source
This commit is contained in:
Gijs van Tulder 2012-01-28 14:09:29 +01:00 committed by Giuseppe Scrivano
parent 586ade4fb1
commit 6d67d793f5
2 changed files with 21 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
src/ChangeLog
View File

@ -1,3 +1,9 @@
2012-01-27 Gijs van Tulder <gvtulder@gmail.com>
* retr.c (fd_read_body): If the response is chunked, the chunk
headers are now written to the WARC file, making the WARC file
an exact copy of the HTTP response.
2012-01-27 Gijs van Tulder <gvtulder@gmail.com>
* retr.c (fd_read_body): Fix a memory leak with chunked responses.

13
src/retr.c
View File

@ -213,6 +213,9 @@ write_data (FILE *out, FILE *out2, const char *buf, int bufsize,
the data is stored to ELAPSED.
If OUT2 is non-NULL, the contents is also written to OUT2.
OUT2 will get an exact copy of the response: if this is a chunked
response, everything -- including the chunk headers -- is written
to OUT2. (OUT will only get the unchunked response.)
The function exits and returns the amount of data read. In case of
error while reading data, -1 is returned. In case of error while
@ -305,6 +308,8 @@ fd_read_body (int fd, FILE *out, wgint toread, wgint startpos,
ret = -1;
break;
}
else if (out2 != NULL)
fwrite (line, 1, strlen (line), out2);
remaining_chunk_size = strtol (line, &endl, 16);
xfree (line);
@ -316,7 +321,11 @@ fd_read_body (int fd, FILE *out, wgint toread, wgint startpos,
if (line == NULL)
ret = -1;
else
{
if (out2 != NULL)
fwrite (line, 1, strlen (line), out2);
xfree (line);
}
break;
}
}
@ -384,10 +393,14 @@ fd_read_body (int fd, FILE *out, wgint toread, wgint startpos,
break;
}
else
{
if (out2 != NULL)
fwrite (line, 1, strlen (line), out2);
xfree (line);
}
}
}
}
if (opt.limit_rate)
limit_bandwidth (ret, timer);