Commit Graph

4968 Commits

Author SHA1 Message Date
Paul Groke 8324dc8b1a
dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries
Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix
(similar to the existing '-' prefix for removing entries) to add
DNS cache entries that will time out just like entries that are added
by libcurl itself.

Append " (non-permanent)" to info log message in case a non-permanent
entry is added.

Adjust relevant comments to reflect the new behavior.

Adjust documentation.

Extend unit1607 to test the new functionality.

Closes #6294
2020-12-29 17:49:36 +01:00
Denis Laxalde dc7bd62c67
docs: remove redundant "better" in --fail help
Closes #6385
2020-12-29 17:08:41 +01:00
Kevin Ushey 0be8818f6e
curl.1: fix typo microsft -> microsoft
Closes #6380
2020-12-27 11:07:46 +01:00
XhmikosR aadc75626b
misc: assorted typo fixes
Closes #6375
2020-12-26 23:54:25 +01:00
Daniel Stenberg ec424f311a
KNOWN_BUGS: 6.10 curl never completes Negotiate over HTTP
Closes #5235
Closes #6370
2020-12-26 16:11:17 +01:00
Daniel Stenberg a66df02864
INSTALL: update the list known OSes and CPU archs curl has run on
Closes #6366
2020-12-25 16:06:49 +01:00
Daniel Stenberg 64e6e54f4b
curl_easy_pause.3: add multiplexed pause effects
and generally refresh and update. Remove details for ancient versions.

Reviewed-by: Jay Satiro
Closes #6360
2020-12-22 23:08:03 +01:00
Jay Satiro f0ba3d5e1b curl_easy_pause.3: fix man page reference
Follow-up to ac9a724 from earlier today.

Ref: https://github.com/curl/curl/pull/6359
2020-12-22 16:51:19 -05:00
Daniel Stenberg 4d1a620891
EXPERIMENTAL: add the Hyper backend to the list
... of current experimental features in curl.
2020-12-22 17:17:59 +01:00
Daniel Stenberg ac9a7245fc
speedcheck: exclude paused transfers
Paused transfers should not be stopped due to slow speed even when
CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is
now reset when the transfer is unpaused - as otherwise it would easily
just trigger immediately after unpausing.

Reported-by: Harry Sintonen
Fixes #6358
Closes #6359
2020-12-22 13:51:07 +01:00
Daniel Stenberg 44c5e3901c
cmdline-opts/gen.pl: return hard on errors
... as the warnings tend to go unnoticed otherwise!

Closes #6354
2020-12-21 22:39:26 +01:00
Daniel Stenberg a93c647de8
examples/libtest: add .checksrc to dist
... so that (auto)builds from tarballs also get the correct instructions.

Fixes #6176
Closes #6353
2020-12-21 17:06:06 +01:00
Daniel Stenberg 7a90ddf88f
curl: add variables to --write-out
In particular, these ones can help a user to create its own error
message when one or transfers fail.

writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg'

onerror - lets a user only show the rest on non-zero exit codes

url - the input URL used for this transfer

urlnum - the numerical URL counter (0 indexed) for this transfer

exitcode - the numerical exit code for the transfer

errormsg - obvious

Reported-by: Earnestly on github
Fixes #6199
Closes #6207
2020-12-21 16:38:48 +01:00
Matthias Gatto e2b2afbeea
docs: add AWS HTTP v4 Signature 2020-12-21 16:28:03 +01:00
Matthias Gatto 08e8455ddd
http: introduce AWS HTTP v4 Signature
It is a security process for HTTP.

It doesn't seems to be standard, but it is used by some cloud providers.

Aws:
https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
Outscale:
https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request
GCP (I didn't test that this code work with GCP though):
https://cloud.google.com/storage/docs/access-control/signing-urls-manually

most of the code is in lib/http_v4_signature.c

Information require by the algorithm:
- The URL
- Current time
-  some prefix that are append to some of the signature parameters.

The data extracted from the URL are: the URI, the region,
the host and the API type

example:
https://api.eu-west-2.outscale.com/api/latest/ReadNets
        ~~~ ~~~~~~~~               ~~~~~~~~~~~~~~~~~~~
        ^       ^                          ^
       /         \                        URI
   API type     region

Small description of the algorithm:
- make canonical header using content type, the host, and the date
- hash the post data
- make canonical_request using custom request, the URI,
  the get data, the canonical header, the signed header
  and post data hash
- hash canonical_request
- make str_to_sign using one of the prefix pass in parameter,
  the date, the credential scope and the canonical_request hash
- compute hmac from date, using secret key as key.
- compute hmac from region, using above hmac as key
- compute hmac from api_type, using above hmac as key
- compute hmac from request_type, using above hmac as key
- compute hmac from str_to_sign using above hmac as key
- create Authorization header using above hmac, prefix pass in parameter,
  the date, and above hash

Signed-off-by: Matthias Gatto <matthias.gatto@outscale.com>

Closes #5703
2020-12-21 16:27:50 +01:00
Daniel Stenberg a7696c7343
curl: add --create-file-mode [mode]
This option sets the (octal) mode to use for the remote file when one is
created, using the SFTP, SCP or FILE protocols. When not set, the
default is 0644.

Closes #6244
2020-12-21 10:52:41 +01:00
Daniel Stenberg 6e2392f177
KNOWN_BUGS: Remote recursive folder creation with SFTP
Closes #5204
2020-12-20 22:49:59 +01:00
Jay Satiro 4cc115a85b KNOWN_BUGS: Secure Transport disabling hostname validation also disables SNI
That behavior is a limitation of Apple's Secure Transport.

Reported-by: Cory Benfield
Reported-by: Ian Spence
Confirmed-by: Nick Zitzmann

Ref: https://github.com/curl/curl/issues/998

Closes https://github.com/curl/curl/issues/6347
Closes https://github.com/curl/curl/pull/6348
2020-12-20 15:58:26 -05:00
Daniel Stenberg ccbdbe13c4
TODO: alt-svc should fallback if alt-svc doesn't work
Closes #4908
2020-12-18 23:39:22 +01:00
Daniel Stenberg 9211cb2034
version: include hyper version 2020-12-18 09:58:03 +01:00
Daniel Stenberg 8a113ba93c
docs: add HYPER.md 2020-12-18 09:58:03 +01:00
Daniel Stenberg f25112074d
TODO: Prevent terminal injection when writing to terminal
Closes #6150
2020-12-16 23:38:20 +01:00
Daniel Stenberg 1451f4d2e1
examples: remove superfluous asterisk uses
... for function pointers. Breaks in ancient compilers.
2020-12-16 09:24:16 +01:00
Daniel Stenberg 98b6aa14d9
URL-SYNTAX: add gophers details 2020-12-15 12:58:19 +01:00
Daniel Stenberg be8c94da78
TODO: Package curl for Windows in a signed installer
Closes #5424
2020-12-15 12:02:14 +01:00
Daniel Stenberg c4d88f89a9
BUG-BOUNTY: minor language update
... and remove the wording about entries from before 2019 as the "within
12 months" is still there and covers that.

Closes #6318
2020-12-15 08:57:07 +01:00
Jay Satiro 9f85b986a9 KNOWN_BUGS: SHA-256 digest not supported in Windows SSPI builds
Closes https://github.com/curl/curl/issues/6302
2020-12-14 01:08:15 -05:00
Daniel Stenberg 78af8b68cf
URL-SYNTAX: add default port numbers and IDNA details
Closes #6316
2020-12-13 23:20:36 +01:00
Daniel Stenberg 7ba2577988
URL-SYNTAX: mention how FILE:// access can access network on windows
Closes #6314
2020-12-13 12:24:57 +01:00
Jay Satiro b90c23d580 URL-SYNTAX: Document default SMTP port 25
Note that ports 25 and 587 are common ports for smtp, the former being
the default.

Closes https://github.com/curl/curl/pull/6310
2020-12-12 19:09:42 -05:00
Daniel Stenberg a0f0c7149f
CURLOPT_URL.3: remove scheme specific details
... that are now found in URL-SYNTAX.md

Closes #6307
2020-12-12 22:46:37 +01:00
Dan Fandrich 2a264d494e docs: Fix some typos
[skip ci]
2020-12-12 09:59:28 -08:00
Daniel Stenberg c29db0303d
URL-SYNTAX: mention all supported schemes
Closes #6311
2020-12-12 16:37:16 +01:00
Douglas R. Reno 940d414980
URL-SYNTAX.md: minor language improvements
Closes #6308
2020-12-12 11:19:10 +01:00
Daniel Stenberg ea0916d41b
docs/URL-SYNTAX: the URL syntax curl accepts and works with
Closes #6285
2020-12-11 23:57:46 +01:00
0xflotus 5253444090
docs: enable syntax highlighting in several docs files
... for better readability

Closes #6286
2020-12-11 18:06:41 +01:00
Daniel Stenberg e052859759
RELEASE-NOTES: synced
for 7.74.0
2020-12-09 07:38:24 +01:00
Daniel Stenberg 0611fded46
VERSIONS: refreshed
We always use the patch number these days: all releases are
"major.minor.patch"
2020-12-07 13:23:04 +01:00
Daniel Stenberg ec9cc725d5
ftp: CURLOPT_FTP_SKIP_PASV_IP by default
The command line tool also independently sets --ftp-skip-pasv-ip by
default.

Ten test cases updated to adapt the modified --libcurl output.

Bug: https://curl.se/docs/CVE-2020-8284.html
CVE-2020-8284

Reported-by: Varnavas Papaioannou
2020-12-07 08:38:05 +01:00
Daniel Stenberg 6703eb2f4c
SECURITY-PROCESS: disclose on hackerone
Once a vulnerability has been published, the hackerone issue should be
disclosed. For tranparency.

Closes #6275
2020-12-03 22:29:34 +01:00
Daniel Gustafsson 41b3b830f1 docs: fix typos and markup in ETag manpage sections
Reported-by: emanruse on github
Fixes #6273
2020-12-03 13:25:42 +01:00
Daniel Stenberg 221c9da9af
NEW-PROTOCOL: document what needs to be done to add one
Closes #6263
2020-12-01 10:18:46 +01:00
Daniel Stenberg 020aa0131b
docs/INTERNALS: remove reference to Curl_sendf()
The function has been removed from common usage. Also removed comment in
gopher.c that still referenced it.

Reported-by: Rikard Falkeborn
Fixes #6242
Closes #6243
2020-11-24 13:17:25 +01:00
Rikard Falkeborn 77b2f702c4
examples: update .gitignore
Add files that are generated by 'make examples' and remove some that
have been renamed.

The commits that renamed the programs are e9625c5bc6 (imap.c and
simplesmtp.c were renamed to imap-fetch.c and smtp-send.c) and
ad39e7ec01 (pop3slist.c and pop3s.c were renamed to pop3-list.c and
pop3-ssl.c).

Closes #6240
2020-11-23 23:09:33 +01:00
Daniel Stenberg a95a6ce6b8
urldata: remove 'void *protop' and create the union 'p'
... to avoid the use of 'void *' for the protocol specific structs done
per transfer.

Closes #6238
2020-11-23 16:16:16 +01:00
Daiki Ueno 898fca27cd
http3: use the master branch of GnuTLS for testing
Closes #6235
2020-11-22 16:40:05 +01:00
Daniel Stenberg 7fa6d5e383
KNOWN_BUGS: curl with wolfSSL lacks support for renegotiation
Closes #5839
2020-11-22 00:06:24 +01:00
Daniel Stenberg 10818dc7cb
KNOWN_BUGS: wakeup socket disconnect causes havoc
Closes #6132
Closes #6133
2020-11-22 00:01:29 +01:00
Cristian Morales Vega f21cc62832
cmake: make CURL_ZLIB a tri-state variable
By differentiating between ON and AUTO it can make a missing zlib
library a hard error when CURL_ZLIB=ON is used.

Reviewed-by: Jakub Zakrzewski
Closes #6221
Fixes #6173
2020-11-19 13:38:47 +01:00
Daniel Stenberg 33a0b7ad73
KNOWN_BUGS: cmake: libspsl is not supported
Closes #6214
2020-11-18 22:51:08 +01:00
Daniel Stenberg bf9e14159c
KNOWN_BUGS: cmake autodetects cert paths when cross-compiling
Closes #6178
2020-11-18 22:51:08 +01:00
Daniel Stenberg 529423a270
KNOWN_BUGS: cmake build doesn't fail if zlib not found
Closes #6173
2020-11-18 22:51:08 +01:00
Daniel Stenberg 192099333b
KNOWN_BUGS: cmake libcurl.pc uses absolute library paths
Closes #6169
2020-11-18 22:51:08 +01:00
Daniel Stenberg ad4b608263
KNOWN_BUGS: cmake: generated .pc file contains strange entries
Closes #6167
2020-11-18 22:51:08 +01:00
Daniel Stenberg 943cc00aad
KNOWN_BUGS: cmake uses -lpthread instead of Threads::Threads
Closes #6166
2020-11-18 22:51:08 +01:00
Daniel Stenberg cfd42e3283
KNOWN_BUGS: cmake build in Linux links libcurl to libdl
Closes #6165
2020-11-18 22:51:08 +01:00
Daniel Stenberg 94979161c8
KNOWN_BUGS: make a new section for cmake topics
Closes #6219
2020-11-18 22:50:58 +01:00
Daniel Stenberg 03822c3a6d
CURLOPT_HSTS.3: document the file format
Closes #6205
2020-11-13 23:28:58 +01:00
Daniel Stenberg 71ec4e7d76
httpput-postfields.c: new example doing PUT with POSTFIELDS
Proposed-by: Jeroen Ooms
Ref: #6186
Closes #6188
2020-11-13 09:20:57 +01:00
Daniel Stenberg 8b151cb944
docs: document the 8MB input string limit
for curl_easy_escape and curl_easy_setopt()

The limit is there to catch mistakes and abuse. It is meant to be large
enough to allow virtually all "fine" use cases.

Reported-by: Marc Schlatter
Fixes #6190
Closes #6191
2020-11-09 17:28:45 +01:00
Daniel Stenberg 5ee44566b6
THANKS-filter: ignore autobuild links 2020-11-09 10:15:15 +01:00
Daniel Stenberg fa6bbbe167
examples/httpput: remove use of CURLOPT_PUT
It is deprecated and unnecessary since it already sets CURLOPT_UPLOAD.

Reported-by: Jeroen Ooms
Fixes #6186
Closes #6187
2020-11-09 08:25:14 +01:00
Daniel Stenberg 65bc682524
FAQ: remove "Why is there a HTTP/1.1 in my HTTP/2 request?"
This hasn't been the case for a while now, remove.
2020-11-06 09:28:49 +01:00
Daniel Stenberg 3864ad37e1
FAQ: refresh "Why do I get "certificate verify failed"
Add more details, remove references to ancient curl version.
2020-11-06 09:16:06 +01:00
Daniel Stenberg 490879ea89
FAQ: refreshed
- remove a few ancient questions
 - add configure with static libs question
 - updated wording in several places
 - lowercased curl

Closes #6177
2020-11-05 16:54:12 +01:00
Daniel Gustafsson afddaa6dec examples: fix comment syntax
Commit ac0a88fd2 accidentally added a stray character outside of the
comment which broke compilation. Fix by removing.

Reported-by:  autobuild https://curl.se/dev/log.cgi?id=20201105084306-12742
2020-11-05 09:54:12 +01:00
Daniel Gustafsson afbf7d260c docs: Fix various typos in documentation
Closes #6171
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
2020-11-05 09:36:24 +01:00
Daniel Stenberg ac0a88fd25
copyright: fix year ranges
Follow-up from 4d2f800677
2020-11-05 08:22:10 +01:00
Daniel Stenberg fbbbf5aae0
HISTORY: the new domain 2020-11-04 23:59:53 +01:00
Daniel Stenberg 4d2f800677
curl.se: new home
Closes #6172
2020-11-04 23:59:47 +01:00
Daniel Stenberg 0673cb4d2d
KNOWN_BUGS: FTPS with Schannel times out file list operation
Reported-by: bobmitchell1956 on github
Closes #5284
2020-11-04 23:56:12 +01:00
Daniel Stenberg ce980c255e
KNOWN_BUGS: SMB tests fail with Python 2
Reported-by: Jay Satiro
Closes #5983
2020-11-04 23:49:31 +01:00
Daniel Stenberg 5cb475de99
KNOWN_BUGS: LDAPS with NSS is slow
Reported-by: nosajsnikta on github
Closes #5874
2020-11-04 23:43:43 +01:00
Daniel Stenberg 2cfc4ed983
hsts: add read/write callbacks
- read/write callback options
- man pages for the 4 new setopts
- test 1915 verifies the callbacks

Closes #5896
2020-11-03 16:08:48 +01:00
Daniel Stenberg 7385610d0c
hsts: add support for Strict-Transport-Security
- enable in the build (configure)
- header parsing
- host name lookup
- unit tests for the above
- CI build
- CURL_VERSION_HSTS bit
- curl_version_info support
- curl -V output
- curl-config --features
- CURLOPT_HSTS_CTRL
- man page for CURLOPT_HSTS_CTRL
- curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl)
- man page for --hsts
- save cache to disk
- load cache from disk
- CURLOPT_HSTS
- man page for CURLOPT_HSTS
- added docs/HSTS.md
- fixed --version docs
- adjusted curl_easy_duphandle

Closes #5896
2020-11-03 16:08:42 +01:00
Daniel Stenberg b8895509a0
header.d: fix syntax mistake
follow-up from 1144886f38
2020-11-02 10:31:02 +01:00
Daniel Stenberg 1144886f38
header.d: mention the "Transfer-Encoding: chunked" handling
Ref: #6144
Closes #6148
2020-10-31 23:45:36 +01:00
Daniel Stenberg 5106f1dc40
curl.1: add an "OUTPUT" section at the top of the manpage
Explain the basic concepts behind curl output.

Inspired by #6124

Closes #6134
2020-10-29 09:29:41 +01:00
Jay Satiro 9f4c1c0cce CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo
Reported-by: Rui LIU

Closes https://github.com/curl/curl/issues/6131
2020-10-26 17:18:44 -04:00
Jay Satiro b1ff27995b range.d: fix typo
Follow-up to 15ae039 from earlier today.
2020-10-26 17:18:37 -04:00
José Joaquín Atria 15ae039883
range.d: clarify that curl will not parse multipart responses
Closes #6127
Fixes #6124
2020-10-26 11:02:49 +01:00
Daniel Stenberg 96450a1a33
alt-svc: enable by default
Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported.

alt-svc support in curl is no longer considered experimental

Closes #5868
2020-10-25 23:08:54 +01:00
Daniel Stenberg 4bfca0a807
libssh2: require version 1.0 or later
... and simplify the code accordingly. libssh2 version 1.0 was released
in April 2009.

Closes #6116
2020-10-22 16:45:40 +02:00
Daniel Stenberg 141e23d789
KNOWN_BUGS: mention the individual cmake issues
... to make them easier to refer to and address separately and
one-by-one.
2020-10-21 08:25:19 +02:00
Daniel Stenberg 65fb4d16a3
curl_url_set.3: fix typo in the RETURN VALUE section
Reported-by: Basuke Suzuki
Fixes #6102
2020-10-18 23:02:57 +02:00
Daniel Stenberg 6f85968678
CURLOPT_NOBODY.3: fix typo
Reported-by: Basuke Suzuki
Fixes #6097
2020-10-16 23:02:49 +02:00
Daniel Stenberg bfd35d5eda
CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well 2020-10-16 15:36:01 +02:00
Zenju 6d1a05b0bf
CURLOPT_TCP_NODELAY.3: fix comment in example code
Closes #6096
2020-10-16 15:06:51 +02:00
Viktor Szakats 769c9a4eec
Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3
Approved-by: Daniel Stenberg
Closes #6092
2020-10-16 07:15:19 +00:00
Daniel Stenberg 621e147ca7
docs/FEATURE: convert to markdown
... and clean it up a bit.

Closes #6067
2020-10-15 15:47:38 +02:00
Daniel Stenberg 2cbd2a2abf
THANKS: from 7.73.0 and .mailmap fixes 2020-10-14 07:50:16 +02:00
Daniel Stenberg a4c26b0abe
HISTORY: curl verifies SSL certs by default since version 7.10 2020-10-09 17:29:33 +02:00
Emil Engler 639c6bfcfa
--help: move two options from the misc category
The cmdline opts delegation and suppress-connect-headers
fit better into auth and proxy rather than misc.

Follow-up to aa8777f63f
Closes #6038
2020-10-03 22:59:50 +02:00
Samanta Navarro 2ead0ca482
docs/opts: fix typos in two manual pages
Closes #6039
2020-10-03 22:53:16 +02:00
Emil Engler 6603917c3a
TODO: Add OpenBSD libtool notice
See #5862
Closes #6030
2020-09-30 22:48:31 +02:00
Daniel Stenberg 7b9e3c4b4c
examples/README: convert to markdown
Closes #6028
2020-09-30 22:45:29 +02:00
Daniel Gustafsson 2aac895fb6 src: Consistently spell whitespace without whitespace
Whitespace is spelled without a space between white and space, so
make sure to consistently spell it that way across the codebase.

Closes #6023
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
2020-09-30 21:10:14 +02:00
Daniel Gustafsson 021f2c25fd MANUAL: update examples to resolve without redirects
www.netscape.com is redirecting to a cookie consent form on Aol, and
cool.haxx.se isn't responding to FTP anymore. Replace with examples
that resolves in case users try out the commands when reading the
manual.

Closes #6024
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Emil Engler <me@emilengler.com>
2020-09-30 21:05:14 +02:00
Daniel Stenberg 025b20971c
HISTORY: add some 2020 events 2020-09-30 13:44:32 +02:00
Daniel Stenberg cbe7fad20d
ECH: renamed from ESNI in docs and configure
Encrypted Client Hello (ECH) is the current name.

Closes #6022
2020-09-29 11:23:23 +02:00