Commit Graph

666 Commits

Author SHA1 Message Date
Sangamkar 74f4782319
libcurl: stop reading from paused transfers
In the transfer loop it would previously not acknwledge the pause bit
and continue until drained or loop ended.

Closes #3240
2018-11-06 19:07:17 +01:00
Daniel Stenberg 2c5ec339ea
Curl_follow: accept non-supported schemes for "fake" redirects
When not actually following the redirect and the target URL is only
stored for later retrieval, curl always accepted "non-supported"
schemes. This was a regression from 46e164069d.

Reported-by: Brad King
Fixes #3210
Closes #3215
2018-11-02 09:50:44 +01:00
Gisle Vanem eda0998894
Fix for compiling with lwIP (3)
lwIP on Windows does not have a WSAIoctl() function. 
But it do have a SO_SNDBUF option to lwip_setsockopt(). But it currently does nothing.
2018-10-23 12:55:07 +02:00
Daniel Stenberg 6535b9303d
Curl_follow: return better errors on URL problems
... by making the converter function global and accessible.

Closes #3153
2018-10-23 11:43:41 +02:00
Daniel Stenberg ca10fae6fc
Curl_follow: remove remaining free(newurl)
Follow-up to 05564e750e. This function no longer frees the passed-in
URL.

Reported-by: Michael Kaufmann
Bug: 05564e750e (commitcomm)
ent-30985666
2018-10-23 11:43:41 +02:00
Daniel Stenberg 05564e750e
multi: avoid double-free
Curl_follow() no longer frees the string. Make sure it happens in the
caller function, like we normally handle allocations.

This bug was introduced with the use of the URL API internally, it has
never been in a release version

Reported-by: Dario Weißer
Closes #3149
2018-10-19 15:29:31 +02:00
Michael Kaufmann 6afe70a00b Curl_follow: Always free the passed new URL
Closes #3124
2018-10-13 13:18:51 +02:00
Daniel Gustafsson 12d833fa1e transfer: fix typo in comment 2018-10-10 23:50:13 +02:00
Daniel Stenberg ef695fc301
Curl_retry_request: fix memory leak
Detected by OSS-Fuzz

Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10648
Closes #3042
2018-09-24 16:38:11 +02:00
Daniel Stenberg 46e164069d
url: use the URL API internally as well
... to make it a truly unified URL parser.

Closes #3017
2018-09-22 11:58:10 +02:00
Daniel Stenberg fb30ac5a2d
URL-API
See header file and man pages for API. All documented API details work
and are tested in the 1560 test case.

Closes #2842
2018-09-08 15:36:11 +02:00
Daniel Stenberg 2825f46d95
CURLOPT_UPLOAD_BUFFERSIZE: set upload buffer size
This is step 3 of #2888.

Fixes #2888
Closes #2896
2018-09-06 10:53:39 +02:00
Daniel Stenberg 1a890997a4
all: s/int/size_t cleanup
Assisted-by: Rikard Falkeborn

Closes #2922
2018-09-01 10:40:42 +02:00
Marcel Raad 15ed9f87e3
curl-compilers: enable -Wimplicit-fallthrough=4 for GCC
This enables level 4 instead of the default level 3, which of the
currently used comments only allows /* FALLTHROUGH */ to silence the
warning.

Closes https://github.com/curl/curl/pull/2747
2018-08-21 18:53:45 +02:00
Daniel Stenberg 8c80a9d1e2
upload: change default UPLOAD_BUFSIZE to 64KB
To make uploads significantly faster in some circumstances.

Part 2 of #2888
Closes #2892
2018-08-18 15:48:52 +02:00
Daniel Stenberg e6e9b006f7
upload: allocate upload buffer on-demand
Saves 16KB on the easy handle for operations that don't need that
buffer.

Part 1 of #2888
2018-08-18 15:48:08 +02:00
Daniel Stenberg d38b4737fd
windows: follow up to the buffer-tuning 1ba1dba7
Somehow I didn't include the amended version of the previous fix. This
is the missing piece.

Pointed-out-by: Viktor Szakats
2018-08-08 14:43:26 +02:00
Daniel Jelinski 1ba1dba76a
windows: implement send buffer tuning
Significantly enhances upload performance on modern Windows versions.

Bug: https://curl.haxx.se/mail/lib-2018-07/0080.html
Closes #2762
Fixes #2224
2018-08-08 13:38:50 +02:00
Harry Sintonen 1836d59ed8
HTTP: Don't attempt to needlessly decompress redirect body
This change fixes a regression where redirect body would needlessly be
decompressed even though it was to be ignored anyway. As it happens this
causes secondary issues since there appears to be a bug in apache2 that
it in certain conditions generates a corrupt zlib response. The
regression was created by commit:
dbcced8e32

Discovered-by: Harry Sintonen
Closes #2798
2018-07-31 17:10:22 +02:00
Daniel Stenberg 5ae22272d1
Curl_debug: remove dead printhost code
The struct field is never set (since 5e0d9aea3) so remove the use of it
and remove the connectdata pointer from the prototype.

Reported-by: Tejas
Bug: https://curl.haxx.se/mail/lib-2018-06/0054.html
Closes #2647
2018-06-12 16:00:27 +02:00
Marian Klymov c45360d463
cppcheck: fix warnings
- Get rid of variable that was generating false positive warning
(unitialized)

- Fix issues in tests

- Reduce scope of several variables all over

etc

Closes #2631
2018-06-11 11:14:48 +02:00
Daniel Stenberg fe6b78b42d
setup_transfer: deal with both sockets being -1
Detected by Coverity; CID 1435559.  Follow-up to f8d608f38d. It would
index the array with -1 if neither index was a socket.
2018-05-11 23:56:37 +02:00
Daniel Stenberg f8d608f38d
transfer: don't unset writesockfd on setup of multiplexed conns
Curl_setup_transfer() can be called to setup a new individual transfer
over a multiplexed connection so it shouldn't unset writesockfd.

Bug: #2520
Closes #2549
2018-05-04 22:30:32 +02:00
Stephan Mühlstrasser 7f41432c19
URL: fix ASCII dependency in strcpy_url and strlen_url
Commit 3c630f9b0a partially reverted the
changes from commit dd7521bcc1 because of
the problem that strcpy_url() was modified unilaterally without also
modifying strlen_url(). As a consequence strcpy_url() was again
depending on ASCII encoding.

This change fixes strlen_url() and strcpy_url() in parallel to use a
common host-encoding independent criterion for deciding whether an URL
character must be %-escaped.

Closes #2535
2018-05-03 15:19:20 +02:00
Daniel Stenberg 3c630f9b0a
strcpy_url: only %-encode values >= 0x80
OSS-Fuzz detected

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8000

Broke in dd7521bcc1
2018-04-25 07:56:01 +02:00
Stephan Mühlstrasser dd7521bcc1
ctype: restore character classification for non-ASCII platforms
With commit 4272a0b0fc curl-speficic
character classification macros and functions were introduced in
curl_ctype.[ch] to avoid dependencies on the locale. This broke curl on
non-ASCII, e.g. EBCDIC platforms. This change restores the previous set
of character classification macros when CURL_DOES_CONVERSIONS is
defined.

Closes #2494
2018-04-24 14:36:06 +02:00
Daniel Stenberg d122df5972
http2: handle GOAWAY properly
When receiving REFUSED_STREAM, mark the connection for close and retry
streams accordingly on another/fresh connection.

Reported-by: Terry Wu
Fixes #2416
Fixes #1618
Closes #2510
2018-04-20 22:19:23 +02:00
Daniel Stenberg a9a7b606c2
transfer: make HTTP without headers count correct body size
This is what "HTTP/0.9" basically looks like.

Reported on IRC

Closes #2382
2018-03-15 10:56:27 +01:00
Daniel Stenberg d52dc4760f
readwrite: make sure excess reads don't go beyond buffer end
CVE-2018-1000122
Bug: https://curl.haxx.se/docs/adv_2018-b047.html

Detected by OSS-fuzz
2018-03-12 07:47:07 +01:00
Daniel Stenberg 8123560d44
HTTP: allow "header;" to replace an internal header with a blank one
Reported-by: Michael Kaufmann
Fixes #2357
Closes #2362
2018-03-11 11:46:10 +01:00
Björn Stenberg b46cfbc068
TODO fixed: Detect when called from within callbacks
Closes #2302
2018-02-15 09:36:03 +01:00
Patrick Monnerat dbcced8e32 HTTP: support multiple Content-Encodings
This is implemented as an output streaming stack of unencoders, the last
calling the client write procedure.

New test 230 checks this feature.

Bug: https://github.com/curl/curl/pull/2002
Reported-By: Daniel Bankhead
2017-11-05 15:09:48 +01:00
Jay Satiro 979d2877be transfer: Fix chunked-encoding upload bug
- When uploading via chunked-encoding don't compare file size to bytes
  sent to determine whether the upload has finished.

Chunked-encoding adds its own overhead which why the bytes sent is not
equal to the file size. Prior to this change if a file was uploaded in
chunked-encoding and its size was known it was possible that the upload
could end prematurely without sending the final few chunks. That would
result in a server hang waiting for the remaining data, likely followed
by a disconnect.

The scope of this bug is limited to some arbitrary file sizes which have
not been determined. One size that triggers the bug is 475020.

Bug: https://github.com/curl/curl/issues/2001
Reported-by: moohoorama@users.noreply.github.com

Closes https://github.com/curl/curl/pull/2010
2017-10-26 14:34:46 -04:00
Max Dymond 7b11c5dbe6
wildcards: don't use with non-supported protocols
Fixes timeouts in the fuzzing tests for non-FTP protocols.

Closes #2016
2017-10-26 13:34:45 +02:00
Daniel Stenberg 5d543fe906 time: rename Curl_tvnow to Curl_now
... since the 'tv' stood for timeval and this function does not return a
timeval struct anymore.

Also, cleaned up the Curl_timediff*() functions to avoid typecasts and
clean up the descriptive comments.

Closes #2011
2017-10-25 18:48:05 +02:00
Daniel Stenberg b9d25f9a6b timediff: return timediff_t from the time diff functions
... to cater for systems with unsigned time_t variables.

- Renamed the functions to curlx_timediff and Curl_timediff_us.

- Added overflow protection for both of them in either direction for
  both 32 bit and 64 bit time_ts

- Reprefixed the curlx_time functions to use Curl_*

Reported-by: Peter Piekarski
Fixes #2004
Closes #2005
2017-10-25 09:54:37 +02:00
Daniel Stenberg 93843c372f
conversions: fix several compiler warnings 2017-09-15 16:58:35 +02:00
Daniel Stenberg 87501e57f1
code style: remove wrong uses of multiple spaces
Closes #1878
2017-09-12 13:54:54 +02:00
Daniel Stenberg e5743f08e7
code style: use spaces around pluses 2017-09-11 09:29:50 +02:00
Daniel Stenberg 6b84438d9a
code style: use spaces around equals signs 2017-09-11 09:29:50 +02:00
Daniel Stenberg 02eb6184ad
Curl_checkheaders: make it available for IMAP and SMTP too
... not only HTTP uses this now.

Closes #1875
2017-09-11 00:26:17 +02:00
Daniel Stenberg 9926357b42 mime: unified to use the typedef'd mime structs everywhere
... and slightly edited to follow our code style better.
2017-09-05 17:33:16 +01:00
Patrick Monnerat ce0881edee mime: new MIME API.
Available in HTTP, SMTP and IMAP.
Deprecates the FORM API.
See CURLOPT_MIMEPOST.
Lib code and associated documentation.
2017-09-02 17:47:10 +01:00
Ryan Winograd 43d036e724
progress: Track total times following redirects
Update the progress timers `t_nslookup`, `t_connect`, `t_appconnect`,
`t_pretransfer`, and `t_starttransfer` to track the total times for
these activities when a redirect is followed. Previously, only the times
for the most recent request would be tracked.

Related changes:

  - Rename `Curl_pgrsResetTimesSizes` to `Curl_pgrsResetTransferSizes`
    now that the function only resets transfer sizes and no longer
    modifies any of the progress timers.

  - Add a bool to the `Progress` struct that is used to prevent
    double-counting `t_starttransfer` times.

Added test case 1399.

Fixes #522 and Known Bug 1.8
Closes #1602
Reported-by: joshhe on github
2017-08-15 18:58:31 +02:00
Salah-Eddin Shaban d6ecb2c851
redirect: skip URL encoding for host names
This fixes redirects to IDN URLs

Fixes #1441
Closes #1762
Reported by: David Lord
2017-08-13 00:03:54 +02:00
Daniel Stenberg b3786f61dc url: make the original string get used on subsequent transfers
... since CURLOPT_URL should follow the same rules as other options:
they remain set until changed or cleared.

Added test 1551 to verify.

Fixes #1631
Closes #1632
Reported-by: Pavel Rochnyak
2017-07-03 23:52:10 +02:00
Daniel Stenberg e54db1b053 transfer: init the infilesize from the postfields...
... with a strlen() if no size was set, and do this in the pretransfer
function so that the info is set early. Otherwise, the default strlen()
done on the POSTFIELDS data never sets state.infilesize.

Reported-by: Vincas Razma
Bug: #1294
2017-05-30 11:39:42 +02:00
Daniel Stenberg bba59073c5 redirect: store the "would redirect to" URL when max redirs is reached
Test 1261 added to verify.

Reported-by: Lloyd Fournier

Fixes #1489
Closes #1497
2017-05-23 23:27:58 +02:00
Daniel Stenberg 31b39c40cf multi: use a fixed array of timers instead of malloc
... since the total amount is low this is faster, easier and reduces
memory overhead.

Also, Curl_expire_done() can now mark an expire timeout as done so that
it never times out.

Closes #1472
2017-05-10 12:55:36 +02:00
Daniel Stenberg e9fd794a61 multi: assign IDs to all timers and make each timer singleton
A) reduces the timeout lists drastically

 B) prevents a lot of superfluous loops for timers that expires "in vain"
    when it has actually already been extended to fire later on
2017-05-10 11:02:47 +02:00