1
0
mirror of https://github.com/moparisthebest/curl synced 2024-10-31 15:45:12 -04:00
Commit Graph

678 Commits

Author SHA1 Message Date
Richy Kim
6b7616690e CURLOPT_BUFFERSIZE: support enlarging receive buffer
Replace use of fixed macro BUFSIZE to define the size of the receive
buffer.  Reappropriate CURLOPT_BUFFERSIZE to include enlarging receive
buffer size.  Upon setting, resize buffer if larger than the current
default size up to a MAX_BUFSIZE (512KB). This can benefit protocols
like SFTP.

Closes #1222
2017-01-19 23:38:04 +01:00
Alessandro Ghedini
1ad1a0d186 http: print correct HTTP string in verbose output when using HTTP/2
Before:
```
 % src/curl https://sigsegv.ninja/ -v --http2
...
> GET / HTTP/1.1
> Host: sigsegv.ninja
> User-Agent: curl/7.52.2-DEV
> Accept: */*
>
...
```

After:
```
 % src/curl https://sigsegv.ninja/ -v --http2
...
> GET / HTTP/2
> Host: sigsegv.ninja
> User-Agent: curl/7.52.2-DEV
> Accept: */*
>
```
2017-01-14 12:50:51 +00:00
Daniel Stenberg
9aa259a139 http: remove "Curl_http_done: called premature" message
... it only confuses people.
2016-12-23 23:32:43 +01:00
Daniel Stenberg
1c3e8bbfed checksrc: warn for assignments within if() expressions
... they're already frowned upon in our source code style guide, this
now enforces the rule harder.
2016-12-14 01:29:44 +01:00
Daniel Stenberg
b228d2952b checksrc: stricter no-space-before-paren enforcement
In order to make the code style more uniform everywhere
2016-12-13 23:39:11 +01:00
Alex Rousskov
cb4e2be7c6 proxy: Support HTTPS proxy and SOCKS+HTTP(s)
* HTTPS proxies:

An HTTPS proxy receives all transactions over an SSL/TLS connection.
Once a secure connection with the proxy is established, the user agent
uses the proxy as usual, including sending CONNECT requests to instruct
the proxy to establish a [usually secure] TCP tunnel with an origin
server. HTTPS proxies protect nearly all aspects of user-proxy
communications as opposed to HTTP proxies that receive all requests
(including CONNECT requests) in vulnerable clear text.

With HTTPS proxies, it is possible to have two concurrent _nested_
SSL/TLS sessions: the "outer" one between the user agent and the proxy
and the "inner" one between the user agent and the origin server
(through the proxy). This change adds supports for such nested sessions
as well.

A secure connection with a proxy requires its own set of the usual SSL
options (their actual descriptions differ and need polishing, see TODO):

  --proxy-cacert FILE        CA certificate to verify peer against
  --proxy-capath DIR         CA directory to verify peer against
  --proxy-cert CERT[:PASSWD] Client certificate file and password
  --proxy-cert-type TYPE     Certificate file type (DER/PEM/ENG)
  --proxy-ciphers LIST       SSL ciphers to use
  --proxy-crlfile FILE       Get a CRL list in PEM format from the file
  --proxy-insecure           Allow connections to proxies with bad certs
  --proxy-key KEY            Private key file name
  --proxy-key-type TYPE      Private key file type (DER/PEM/ENG)
  --proxy-pass PASS          Pass phrase for the private key
  --proxy-ssl-allow-beast    Allow security flaw to improve interop
  --proxy-sslv2              Use SSLv2
  --proxy-sslv3              Use SSLv3
  --proxy-tlsv1              Use TLSv1
  --proxy-tlsuser USER       TLS username
  --proxy-tlspassword STRING TLS password
  --proxy-tlsauthtype STRING TLS authentication type (default SRP)

All --proxy-foo options are independent from their --foo counterparts,
except --proxy-crlfile which defaults to --crlfile and --proxy-capath
which defaults to --capath.

Curl now also supports %{proxy_ssl_verify_result} --write-out variable,
similar to the existing %{ssl_verify_result} variable.

Supported backends: OpenSSL, GnuTLS, and NSS.

* A SOCKS proxy + HTTP/HTTPS proxy combination:

If both --socks* and --proxy options are given, Curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS
proxy.

TODO: Update documentation for the new APIs and --proxy-* options.
Look for "Added in 7.XXX" marks.
2016-11-24 23:41:44 +01:00
Daniel Stenberg
0649433da5 realloc: use Curl_saferealloc to avoid common mistakes
Discussed: https://curl.haxx.se/mail/lib-2016-11/0087.html
2016-11-11 10:03:48 +01:00
Daniel Stenberg
811a693b80 strcasecompare: all case insensitive string compares ignore locale now
We had some confusions on when each function was used. We should not act
differently on different locales anyway.
2016-10-31 08:46:35 +01:00
Daniel Stenberg
502acba2af strcasecompare: is the new name for strequal()
... to make it less likely that we forget that the function actually
does case insentive compares. Also replaced several invokes of the
function with a plain strcmp when case sensitivity is not an issue (like
comparing with "-").
2016-10-31 08:46:35 +01:00
Daniel Stenberg
c5be3d7267 cookies: getlist() now holds deep copies of all cookies
Previously it only held references to them, which was reckless as the
thread lock was released so the cookies could get modified by other
handles that share the same cookie jar over the share interface.

CVE-2016-8623

Bug: https://curl.haxx.se/docs/adv_20161102I.html
Reported-by: Cure53
2016-10-31 08:46:35 +01:00
Michael Kaufmann
e9e5366193 New libcurl option to keep sending on error
Add the new option CURLOPT_KEEP_SENDING_ON_ERROR to control whether
sending the request body shall be completed when the server responds
early with an error status code.

This is suitable for manual NTLM authentication.

Reviewed-by: Jay Satiro

Closes https://github.com/curl/curl/pull/904
2016-09-22 22:22:31 +02:00
Daniel Stenberg
d4c5a91722 http: accept "Transfer-Encoding: chunked" for HTTP/2 as well
... but don't send the actual header over the wire as it isn't accepted.
Chunked uploading is still triggered using this method.

Fixes #1013
Fixes #662
2016-09-16 09:00:20 +02:00
Daniel Stenberg
3533def3d5 http2: make sure stream errors don't needlessly close the connection
With HTTP/2 each transfer is made in an indivial logical stream over the
connection, making most previous errors that caused the connection to get
forced-closed now instead just kill the stream and not the connection.

Fixes #941
2016-08-28 16:44:49 +02:00
Michael Kaufmann
7bda07b046 HTTP: stop parsing headers when switching to unknown protocols
- unknown protocols probably won't send more headers (e.g. WebSocket)
- improved comments and moved them to the correct case statements

Closes #899
2016-08-25 14:49:39 +02:00
Steve Holme
317795d1bf http.c: Remove duplicate (authp->avail & CURLAUTH_DIGEST) check
From commit 2708d4259b.
2016-08-21 10:25:45 +01:00
Steve Holme
c6b869ec79 http.c: Corrected indentation change from commit 2708d4259b
Made by Visual Studio's auto-correct feature and missed by me in my own
code reviews!
2016-08-20 19:34:54 +01:00
Steve Holme
2708d4259b http: Added calls to Curl_auth_is_<mechansism>_supported()
Hooked up the HTTP authentication layer to query the new 'is mechanism
supported' functions when deciding what mechanism to use.

As per commit 00417fd66c existing functionality is maintained for now.
2016-08-20 19:15:13 +01:00
Daniel Stenberg
93b0d907d5 Revert "Proxy-Connection: stop sending this header by default"
This reverts commit 113f04e664.
2016-08-16 08:36:04 +02:00
Michael Kaufmann
38685f86c8 cleanup: minor code cleanup in Curl_http_readwrite_headers()
- the expression of an 'if' was always true
- a 'while' contained a condition that was always true
- use 'if(k->exp100 > EXP100_SEND_DATA)' instead of 'if(k->exp100)'
- fixed a typo

Closes #889
2016-06-28 20:48:44 +02:00
Daniel Stenberg
434f8d0389 internals: rename the SessionHandle struct to Curl_easy 2016-06-22 10:28:41 +02:00
Michael Kaufmann
0bdec5e01d cleanup: fix method names in code comments
Closes #887
2016-06-21 13:11:40 +02:00
Jay Satiro
1aa899ff38 http: Fix HTTP/2 connection reuse
- Change the parser to not require a minor version for HTTP/2.

HTTP/2 connection reuse broke when we changed from HTTP/2.0 to HTTP/2
in 8243a95 because the parser still expected a minor version.

Bug: https://github.com/curl/curl/issues/855
Reported-by: Andrew Robbins, Frank Gevaerts
2016-06-05 03:13:32 -04:00
Daniel Stenberg
f6767f5435 TLS: move the ALPN/NPN enable bits to the connection
Only protocols that actually have a protocol registered for ALPN and NPN
should try to get that negotiated in the TLS handshake. That is only
HTTPS (well, http/1.1 and http/2) right now. Previously ALPN and NPN
would wrongly be used in all handshakes if libcurl was built with it
enabled.

Reported-by: Jay Satiro

Fixes #789
2016-05-09 15:30:25 -04:00
Daniel Stenberg
96eb9a862b http: make sure a blank header overrides accept_decoding
Reported-by: rcanavan
Assisted-by: Isaac Boukris
Closes #785
2016-05-02 11:24:11 +02:00
Daniel Stenberg
4f45240bc8 lib: include curl_printf.h as one of the last headers
curl_printf.h defines printf to curl_mprintf, etc. This can cause
problems with external headers which may use
__attribute__((format(printf, ...))) markers etc.

To avoid that they cause problems with system includes, we include
curl_printf.h after any system headers. That makes the three last
headers to always be, and we keep them in this order:

 curl_printf.h
 curl_memory.h
 memdebug.h

None of them include system headers, they all do funny #defines.

Reported-by: David Benjamin

Fixes #743
2016-04-29 22:32:49 +02:00
Marcel Raad
c2b3f264cb CONNECT_ONLY: don't close connection on GSS 401/407 reponses
Previously, connections were closed immediately before the user had a
chance to extract the socket when the proxy required Negotiate
authentication.

This regression was brought in with the security fix in commit
79b9d5f1a4

Closes #655
2016-04-26 23:01:56 +02:00
Michael Kaufmann
cd8d236245 news: CURLOPT_CONNECT_TO and --connect-to
Makes curl connect to the given host+port instead of the host+port found
in the URL.
2016-04-17 23:50:59 +02:00
Daniel Stenberg
a71012c03e code: style updates 2016-04-03 22:38:36 +02:00
Diego Bes
324a97ecf8 http2: support "prior knowledge", no upgrade from HTTP/1.1
Supports HTTP/2 over clear TCP

- Optimize switching to HTTP/2 by removing calls to init and setup
before switching. Switching will eventually call setup and setup calls
init.

- Supports new version to “force” the use of HTTP/2 over clean TCP

- Add common line parameter “--http2-prior-knowledge” to the Curl
  command line tool.
2016-03-31 22:23:11 +02:00
Daniel Stenberg
541027876d http: remove ((expression)) double parentheses 2016-03-31 10:10:58 +02:00
Daniel Stenberg
b22a954736 Curl_add_buffer_send: avoid possible NULL dereference
... as we check for a NULL pointer below, we move the derefence to after
the check. Detected by PVS Studio.

Reported-by: Alexis La Goutte
2016-03-31 10:10:58 +02:00
Daniel Stenberg
575e885db0 multi: turn Curl_done into file local multi_done
... as it now is used by multi.c only.
2016-03-30 07:52:28 +02:00
Steve Holme
f0bdd72c10 http_ntlm: Renamed from curl_ntlm.[c|h]
Renamed the header and source files for this module as they are HTTP
specific and as such, they should use the naming convention as other
HTTP authentication source files do - this revert commit 260ee6b7bf.

Note: We could also rename curl_ntlm_wb.[c|h], however, the Winbind
code needs separating from the HTTP protocol and migrating into the
vauth directory, thus adding support for Winbind to the SASL based
protocols such as IMAP, POP3 and SMTP.
2016-03-27 17:58:50 +01:00
Steve Holme
ad5e9bfd5d vauth: Moved the Negotiate authentication code to the new vauth directory
Part 1 of 2 - Moved the SSPI based Negotiate authentication code.
2016-03-26 17:21:22 +00:00
Steve Holme
96fff9f6b5 http: Minor update based on CODE_STYLE guidelines 2016-03-24 06:35:23 +00:00
Steve Holme
9351383745 connect/ntlm/http: Fixed compilation warnings when verbose strings disabled
warning C4189: 'data': local variable is initialized but not referenced
2016-03-20 17:51:06 +00:00
Daniel Stenberg
eb083e0d39 http2: don't decompress gzip decoding automatically
At one point during the development of HTTP/2, the commit 133cdd29ea
introduced automatic decompression of Content-Encoding as that was what
the spec said then. Now however, HTTP/2 should work the same way as
HTTP/1 in this regard.

Reported-by: Kazuho Oku

Closes #661
2016-02-17 08:25:40 +01:00
Tatsuhiro Tsujikawa
b080a7cd06 http: Don't break the header into chunks if HTTP/2
nghttp2 callback deals with TLS layer and therefore the header does not
need to be broken into chunks.

Bug: https://github.com/curl/curl/issues/659
Reported-by: Kazuho Oku
2016-02-16 14:49:12 -05:00
Daniel Stenberg
113f04e664 Proxy-Connection: stop sending this header by default
RFC 7230 says we should stop. Firefox already stopped.

Bug: https://github.com/curl/curl/issues/633
Reported-By: Brad Fitzpatrick

Closes #633
2016-02-08 11:09:40 +01:00
Daniel Stenberg
4af40b3646 URLs: change all http:// URLs to https:// 2016-02-03 00:19:02 +01:00
Tatsuhiro Tsujikawa
15cb03ad84 http2: Support trailer fields
This commit adds trailer support in HTTP/2.  In HTTP/1.1, chunked
encoding must be used to send trialer fields.  HTTP/2 deprecated any
trandfer-encoding, including chunked.  But trailer fields are now
always available.

Since trailer fields are relatively rare these days (gRPC uses them
extensively though), allocating buffer for trailer fields is done when
we detect that HEADERS frame containing trailer fields is started.  We
use Curl_add_buffer_* functions to buffer all trailers, just like we
do for regular header fields.  And then deliver them when stream is
closed.  We have to be careful here so that all data are delivered to
upper layer before sending trailers to the application.

We can deliver trailer field one by one using NGHTTP2_ERR_PAUSE
mechanism, but current method is far more simple.

Another possibility is use chunked encoding internally for HTTP/2
traffic.  I have not tested it, but it could add another overhead.

Closes #564
2015-12-15 23:47:46 +01:00
Daniel Stenberg
4bcc532de5 http: add libcurl option to allow HTTP/2 for HTTPS only
... and stick to 1.1 for HTTP. This is in line with what browsers do and
should have very little risk.
2015-12-13 09:24:51 +01:00
Daniel Stenberg
034317d2af fix b6d5cb40d7 2015-12-09 00:27:04 +01:00
Christian Stewart
b6d5cb40d7 build: fix compilation error with CURL_DISABLE_VERBOSE_STRINGS
With curl disable verbose strings in http.c the compilation fails due to
the data variable being undefined later on in the function.

Closes #558
2015-12-08 18:12:16 +01:00
Daniel Stenberg
cd2b73b3ed timecond: do not add if-modified-since without timecondition
The RTSP code path didn't skip adding the if-modified-since for certain
RTSP code paths, even if CURLOPT_TIMECONDITION was set to
CURL_TIMECOND_NONE.

Also, an unknown non-zero CURLOPT_TIMECONDITION value no longer equals
CURL_TIMECOND_IFMODSINCE.

Bug: http://stackoverflow.com/questions/33903982/curl-timecond-none-doesnt-work-how-to-remove-if-modified-since-header
2015-11-25 11:38:10 +01:00
Daniel Stenberg
c341311a0e Revert "cleanup: general removal of TODO (and similar) comments"
This reverts commit 64e959ffe3.

Feedback-by: Dan Fandrich
URL: http://curl.haxx.se/mail/lib-2015-11/0062.html
2015-11-24 09:36:45 +01:00
Daniel Stenberg
b4c39010b5 http2: http_done: don't free already-freed push headers
The push headers are freed after the push callback has been invoked,
meaning this code should only free the headers if the callback was never
invoked and thus the headers weren't freed at that time.

Reported-by: Davey Shafik
2015-11-17 09:47:58 +01:00
Daniel Stenberg
64e959ffe3 cleanup: general removal of TODO (and similar) comments
They tend to never get updated anyway so they're frequently inaccurate
and we never go back to revisit them anyway. We document issues to work
on properly in KNOWN_BUGS and TODO instead.
2015-11-13 16:15:26 +01:00
Jonas Minnberg
fe7590f729 vtls: added support for mbedTLS
closes #496
2015-10-20 07:57:24 +02:00
Daniel Stenberg
c6aedf680f fread_func: move callback pointer from set to state struct
... and assign it from the set.fread_func_set pointer in the
Curl_init_CONNECT function. This A) avoids that we have code that
assigns fields in the 'set' struct (which we always knew was bad) and
more importantly B) it makes it impossibly to accidentally leave the
wrong value for when the handle is re-used etc.

Introducing a state-init functionality in multi.c, so that we can set a
specific function to get called when we enter a state. The
Curl_init_CONNECT is thus called when switching to the CONNECT state.

Bug: https://github.com/bagder/curl/issues/346

Closes #346
2015-10-15 23:32:19 +02:00