The internal buffer in infof() is limited to 2048 bytes of payload plus
an additional byte for NULL termination. Servers with very long error
messages can however cause truncation of the string, which currently
isn't very clear, and leads to badly formatted output.
This appends a "...\n" (or just "..." in case the format didn't with a
newline char) marker to the end of the string to clearly show
that it has been truncated.
Also include a unittest covering infof() to try and catch any bugs
introduced in this quite important function.
Closes#3216
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
The function identifying a leading "scheme" part of the URL considered a few
letters ending with a colon to be a scheme, making something like "short:80"
to become an unknown scheme instead of a short host name and a port number.
Extended test 1560 to verify.
Reported-by: Hagai Auro
Fixes#3220Closes#3223
When not actually following the redirect and the target URL is only
stored for later retrieval, curl always accepted "non-supported"
schemes. This was a regression from 46e164069d.
Reported-by: Brad King
Fixes#3210Closes#3215
As has been outlined in the DEPRECATE.md document, the axTLS code has
been disabled for 6 months and is hereby removed.
Use a better supported TLS library!
Assisted-by: Daniel Gustafsson
Closes#3194
Classic MinGW has neither InitializeCriticalSectionEx nor
GetTickCount64, independent of the target Windows version.
Closes https://github.com/curl/curl/pull/3113
Now FILE transfers send headers to the header callback like HTTP and
other protocols. Also made curl_easy_getinfo(...CURLINFO_PROTOCOL...)
work for FILE in the callbacks.
Makes "curl -i file://.." and "curl -I file://.." work like before
again. Applied the bold header logic to them too.
Regression from c1c2762 (7.61.0)
Reported-by: Shaun Jackman
Fixes#3083Closes#3101
The LD_PRELOAD functionality doesn't exist on macOS, so skip any tests
requiring it.
Fixes#2394Closes#3106
Reported-by: Github user @jakirkham
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
To make it only send one DoH request and avoid the race condition that
could lead to the requests getting sent in reversed order and thus
making it hard to compare in the test case.
Fixes#3107Closes#3108
runtests.pl support running a range of tests, like "44 to 127". Starting
now, the code makes sure that even such given ranges will ignore tests
that are marked as disabled.
Disabled tests can still be run by explictly specifying that test
number.
Closes#3075
The value in question is coming directly from `gnutls-serv`, so it cannot
be modified freely.
Reported-by: Marcel Raad
Ref: 6ae6b2a533 (commitcomment-30621004)
This fixes potential out-of-buffer access on "file:./" URL
$ valgrind curl "file:./"
==24516== Memcheck, a memory error detector
==24516== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==24516== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==24516== Command: /home/even/install-curl-git/bin/curl file:./
==24516==
==24516== Conditional jump or move depends on uninitialised value(s)
==24516== at 0x4C31F9C: strcmp (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24516== by 0x4EBB315: seturl (urlapi.c:801)
==24516== by 0x4EBB568: parseurl (urlapi.c:861)
==24516== by 0x4EBC509: curl_url_set (urlapi.c:1199)
==24516== by 0x4E644C6: parseurlandfillconn (url.c:2044)
==24516== by 0x4E67AEF: create_conn (url.c:3613)
==24516== by 0x4E68A4F: Curl_connect (url.c:4119)
==24516== by 0x4E7F0A4: multi_runsingle (multi.c:1440)
==24516== by 0x4E808E5: curl_multi_perform (multi.c:2173)
==24516== by 0x4E7558C: easy_transfer (easy.c:686)
==24516== by 0x4E75801: easy_perform (easy.c:779)
==24516== by 0x4E75868: curl_easy_perform (easy.c:798)
Was originally spotted by
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10637
Credit to OSS-Fuzz
Closes#3039
- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few indentations/line continuations
in manual examples
Closes https://github.com/curl/curl/pull/3037
In order for this API to fully work for libcurl itself, it now offers a
CURLU_GUESS_SCHEME flag that makes it "guess" scheme based on the host
name prefix just like libcurl always did. If there's no known prefix, it
will guess "http://".
Separately, it relaxes the check of the host name so that IDN host names
can be passed in as well.
Both these changes are necessary for libcurl itself to use this API.
Assisted-by: Daniel Gustafsson
Closes#3018
The previous test certificates contained RSA keys of only 1024 bits.
However, RSA claims that 1024-bit RSA keys are likely to become
crackable some time before 2010. The NIST recommends at least 2048-bit
keys for RSA for now.
Better use full 2048 also for testing.
Closes#2973
Add functionality so that protocols can do custom keepalive on their
connections, when an external API function is called.
Add docs for the new options in 7.62.0
Closes#1641
... and add "MAILINDEX".
As described in #2789, this is a suggested solution. Changing UID=xx to
actually get mail with UID xx and add "MAILINDEX" to get a mail with a
special index in the mail box (old behavior). So MAILINDEX=1 gives the
first non deleted mail in the mail box.
Fixes#2789Closes#2815
Transparently. The related curl_multi_setopt() options all still returns
OK when pipelining is selected.
To re-enable the support, the single line change in lib/multi.c needs to
be reverted.
See docs/DEPRECATE.md
Closes#2705
According to RFC6265 section 5.4, cookies with equal path lengths
SHOULD be sorted by creation-time (earlier first). This adds a
creation-time record to the cookie struct in order to make cookie
sorting more deterministic. The creation-time is defined as the
order of the cookies in the jar, the first cookie read fro the
jar being the oldest. The creation-time is thus not serialized
into the jar. Also remove the strcmp() matching in the sorting as
there is no lexicographic ordering in RFC6265. Existing tests are
updated to match.
Closes#2524
All these tests failed on Windows because something like
sftp://%HOSTIP:%SSHPORT%PWD/
expanded to
sftp://127.0.0.1:1234c:/msys64/home/bla/curl
and then curl complained about the port number ending with a letter.
Use the original POSIX path instead of the Windows path created in
checksystem to fix this.
Closes https://github.com/curl/curl/pull/2920
Since GOPHER support was added in curl `?' character was automatically
translated to `%09' (`\t').
However, this behaviour does not seems documented in RFC 4266 and for
search selectors it is documented to directly use `%09' in the URL.
Apart that several gopher servers in the current gopherspace have CGI
support where `?' is used as part of the selector and translating it to
`%09' often leads to surprising results.
Closes#2910
Modifying the locale with environment variables doesn't work for native
Windows applications. Just disable the test in this case if the decimal
separator is something different than a point. Use a precheck with a
small C program to achieve that.
Closes https://github.com/curl/curl/pull/2786
This warning used to be enabled only for clang as it's a bit stricter
on GCC. Silence the remaining occurrences and enable it on GCC too.
Closes https://github.com/curl/curl/pull/2747
... simply because this is usually a sign of the user having omitted the
file name and the next option is instead "eaten" by the parser as a file
name.
Add test1268 to verify
Closes#2885
Deal with tiny "HTTP/0.9" (header-less) responses by checking the
status-line early, even before a full "HTTP/" is received to allow
detecting 0.9 properly.
Test 1266 and 1267 added to verify.
Fixes#2420Closes#2872
Previously, the macro TEST_HANG_TIMEOUT was unused, but since there is
looping going on, we might as well add timing instead of removing it.
Closes#2853
This allows the use of PKCS#11 URI for certificates and keys without
setting the corresponding type as "ENG" and the engine as "pkcs11"
explicitly. If a PKCS#11 URI is provided for certificate, key,
proxy_certificate or proxy_key, the corresponding type is set as "ENG"
if not provided and the engine is set to "pkcs11" if not provided.
Acked-by: Nikos Mavrogiannopoulos
Closes#2333
Use standard CMake variable BUILD_SHARED_LIBS instead of introducing
custom option CURL_STATICLIB.
Use '-DBUILD_SHARED_LIBS=%SHARED%' in appveyor.yml.
Reviewed-by: Sergei Nikulov
Closes#2755
Turns out that since we're using the native fnmatch function now when
available, and they simply disagree on a huge number of test patterns
that make it hard to test this function like this...
Fixes#2825
Otherwise, LF line endings are converted to CRLF on Windows,
but no conversion is done for the reply, so the test case fails.
Closes https://github.com/curl/curl/pull/2776
It was previously erroneously skipped in some situations.
libtest/libntlmconnect.c wrongly depended on wrong behavior (that it
would get a zero timeout) when no handles are "running" in a multi
handle. That behavior is no longer present with this fix. Now libcurl
will always return a -1 timeout when all handles are completed.
Closes#2733
When the application just started the transfer and then stops it while
the name resolve in the background thread hasn't completed, we need to
wait for the resolve to complete and then cleanup data accordingly.
Enabled test 1553 again and added test 1590 to also check when the host
name resolves successfully.
Detected by OSS-fuzz.
Closes#1968
- Get rid of variable that was generating false positive warning
(unitialized)
- Fix issues in tests
- Reduce scope of several variables all over
etc
Closes#2631
Since 467da3af0, lib1521.c is generated instead of checked in. According
to the commit message, the intention was to remove it from the tarball
as well. However, it is still present when running make dist. To remove
it, add it to nodist_lib1521_SOURCES. This also means there is no need
for the manually added dist-rule in the Makefile.
Also update CMakelists.txt to handle the fact that we now may have
nodist_SOURCES.
If configure detects fnmatch to be available, use that instead of our
custom one for FTP wildcard pattern matching. For standard compliance,
to reduce our footprint and to use already well tested and well
exercised code.
A POSIX fnmatch behaves slightly different than the internal function
for a few test patterns currently and the macOS one yet slightly
different. Test case 1307 is adjusted for these differences.
Closes#2626
A non-escaped bracket ([) is for a character group - as documented. It
will *not* match an individual bracket anymore. Test case 1307 updated
accordingly to match.
Problem detected by OSS-Fuzz, although this fix is probably not a final
fix for the notorious timeout issues.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8525Closes#2614
This avoids appending error data to already existing good data.
Test 92 is updated to match this change.
New test 1156 checks all combinations of --range/--resume, --fail,
Content-Range header and http status code 200/416.
Fixes#1163
Reported-By: Ithubg on github
Closes#2578
The feature is only enabled if the output is believed to be a tty.
-J: There's some minor differences and improvements in -J handling, as
now J should work with -i and it actually creates a file first using the
initial name and then *renames* that to the one found in
Content-Disposition (if any).
-i: only shows headers for HTTP transfers now (as documented).
Previously it would also show for pieces of the transfer that were HTTP
(for example when doing FTP over a HTTP proxy).
-i: now shows trailers as well. Previously they were not shown at all.
--libcurl: the CURLOPT_HEADER is no longer set, as the header output is
now done in the header callback.
The previous limit of 5 can still end up in situation that takes a very
long time and consumes a lot of CPU.
If there is still a rare use case for this, a user can provide their own
fnmatch callback for a version that allows a larger set of wildcards.
This commit was triggered by yet another OSS-Fuzz timeout due to this.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8369Closes#2587
Provide a set of new timers that return the time intervals using integer
number of microseconds instead of floats.
The new info names are as following:
CURLINFO_APPCONNECT_TIME_T
CURLINFO_CONNECT_TIME_T
CURLINFO_NAMELOOKUP_TIME_T
CURLINFO_PRETRANSFER_TIME_T
CURLINFO_REDIRECT_TIME_T
CURLINFO_STARTTRANSFER_TIME_T
CURLINFO_TOTAL_TIME_T
Closes#2495
RFC 6265 section 4.2.1 does not set restrictions on cookie names.
This is a follow-up to commit 7f7fcd0.
Also explicitly check proper syntax of cookie name/value pair.
New test 1155 checks that cookie names are not reserved words.
Reported-By: anshnd at github
Fixes#2564Closes#2566
... and make test 1026 rely on that feature so that --disable-manual
builds don't cause test failures.
Reported-by: Max Dymond and Anders Roxell
Fixes#2533Closes#2540
This extends the INDENTATION case to also handle 'else' statements
and require proper indentation on the following line. Also fixes the
offending cases found in the codebase.
Closes#2532
- Move verify_certificate functionality in schannel.c into a new
file called schannel_verify.c. Additionally, some structure defintions
from schannel.c have been moved to schannel.h to allow them to be
used in schannel_verify.c.
- Make verify_certificate functionality for Schannel available on
all versions of Windows instead of just Windows CE. verify_certificate
will be invoked on Windows CE or when the user specifies
CURLOPT_CAINFO and CURLOPT_SSL_VERIFYPEER.
- In verify_certificate, create a custom certificate chain engine that
exclusively trusts the certificate store backed by the CURLOPT_CAINFO
file.
- doc updates of --cacert/CAINFO support for schannel
- Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
when available. This implements a TODO in schannel.c to improve
handling of multiple SANs in a certificate. In particular, all SANs
will now be searched instead of just the first name.
- Update tool_operate.c to not search for the curl-ca-bundle.crt file
when using Schannel to maintain backward compatibility. Previously,
any curl-ca-bundle.crt file found in that search would have been
ignored by Schannel. But, with CAINFO support, the file found by
that search would have been used as the certificate store and
could cause issues for any users that have curl-ca-bundle.crt in
the search path.
- Update url.c to not set the build time CURL_CA_BUNDLE if the selected
SSL backend is Schannel. We allow setting CA location for schannel
only when explicitly specified by the user via CURLOPT_CAINFO /
--cacert.
- Add new test cases 3000 and 3001. These test cases check that the first
and last SAN, respectively, matches the connection hostname. New test
certificates have been added for these cases. For 3000, the certificate
prefix is Server-localhost-firstSAN and for 3001, the certificate
prefix is Server-localhost-secondSAN.
- Remove TODO 15.2 (Add support for custom server certificate
validation), this commit addresses it.
Closes https://github.com/curl/curl/pull/1325
When a zeroed out allocation is required, use calloc() rather than
malloc() followed by an explicit memset(). The result will be the
same, but using calloc() everywhere increases consistency in the
codebase and avoids the risk of subtle bugs when code is injected
between malloc and memset by accident.
Closes https://github.com/curl/curl/pull/2497
unit1309 and vtls/gtls: error: arithmetic on a null pointer treated as a
cast from integer to pointer is a GNU extension
Reported-by: Rikard Falkeborn
Fixes#2466Closes#2468
curl 7.57.0 and up interpret this according to Appendix E.3.2 of RFC
8089 but then returns an error saying this is unimplemented. This is
actually a regression in behavior on both Windows and Unix.
Before curl 7.57.0 this URL was treated as a path of "//foo/bar" and
then passed to the relevant OS API. This means that the behavior of this
case is actually OS dependent.
The Unix path resolution rules say that the OS must handle swallowing
the extra "/" and so this path is the same as "/foo/bar"
The Windows path resolution rules say that this is a UNC path and
automatically handles the SMB access for the program. So curl on Windows
was already doing Appendix E.3.2 without any special code in curl.
Regression
Closes#2438
This fixes a segfault occurring when a name of the (invalid) form "domain..tld"
is processed.
test46 updated to cover this case.
Follow-up to commit c990ead.
Ref: https://github.com/curl/curl/pull/2440
This patch adds CURLOPT_DNS_SHUFFLE_ADDRESSES to explicitly request
shuffling of IP addresses returned for a hostname when there is more
than one. This is useful when the application knows that a round robin
approach is appropriate and is willing to accept the consequences of
potentially discarding some preference order returned by the system's
implementation.
Closes#1694
Detected by Coverity Analysis:
Error: IDENTIFIER_TYPO:
curl-7.58.0/tests/python_dependencies/impacket/spnego.py:229: identifier_typo: Using "SuportedMech" appears to be a typo:
* Identifier "SuportedMech" is only known to be referenced here, or in copies of this code.
* Identifier "SupportedMech" is referenced elsewhere at least 4 times.
curl-7.58.0/tests/python_dependencies/impacket/smbserver.py:2651: identifier_use: Example 1: Using identifier "SupportedMech".
curl-7.58.0/tests/python_dependencies/impacket/smbserver.py:2308: identifier_use: Example 2: Using identifier "SupportedMech".
curl-7.58.0/tests/python_dependencies/impacket/spnego.py:252: identifier_use: Example 3: Using identifier "SupportedMech" (2 total uses in this function).
curl-7.58.0/tests/python_dependencies/impacket/spnego.py:229: remediation: Should identifier "SuportedMech" be replaced by "SupportedMech"?
Closes#2379
Refuse to operate when given path components featuring byte values lower
than 32.
Previously, inserting a %00 sequence early in the directory part when
using the 'singlecwd' ftp method could make curl write a zero byte
outside of the allocated buffer.
Test case 340 verifies.
CVE-2018-1000120
Reported-by: Duy Phan Thanh
Bug: https://curl.haxx.se/docs/adv_2018-9cd6.html
When targeting x64, MinGW-w64 complains about conversions between
32-bit long and 64-bit pointers. Fix this by reusing the
GNUTLS_POINTER_TO_SOCKET_CAST / GNUTLS_SOCKET_TO_POINTER_CAST logic
from gtls.c, moving it to warnless.h as CURLX_POINTER_TO_INTEGER_CAST /
CURLX_INTEGER_TO_POINTER_CAST.
Closes https://github.com/curl/curl/pull/2341
- Add new option CURLOPT_RESOLVER_START_FUNCTION to set a callback that
will be called every time before a new resolve request is started
(ie before a host is resolved) with a pointer to backend-specific
resolver data. Currently this is only useful for ares.
- Add new option CURLOPT_RESOLVER_START_DATA to set a user pointer to
pass to the resolver start callback.
Closes https://github.com/curl/curl/pull/2311
This enables users to preresolve but still take advantage of happy
eyeballs and trying multiple addresses if some are not connecting.
Ref: https://github.com/curl/curl/pull/2260
Test 319 checks proper raw mode data with non-chunked gzip
transfer-encoded server data.
Test 326 checks raw mode with chunked server data.
Bug: #2303Closes#2308
RFC 5321 4.1.1.4 specifies the CRLF terminating the DATA command
should be taken into account when chasing the <CRLF>.<CRLF> end marker.
Thus a leading dot character in data is also subject to escaping.
Tests 911 and test server are adapted to this situation.
New tests 951 and 952 check proper handling of initial dot in data.
Closes#2304
Whenever an expected pattern syntax rule cannot be matched, the
character starting the rule loses its special meaning and the parsing
is resumed:
- backslash at the end of pattern string matches itself.
- Error in [:keyword:] results in set containing :\[dekorwy.
Unit test 1307 updated for this new situation.
Closes#2273
