A common set of functions instead of many separate implementations for
creating buffers that can grow when appending data to them. Existing
functionality has been ported over.
In my early basic testing, the total number of allocations seem at
roughly the same amount as before, possibly a few less.
See docs/DYNBUF.md for a description of the API.
Closes#5300
OpenSSH for Windows requires paths in the format of /C:/
instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/
Reviewed-by: Daniel Stenberg
Closes#5298
Several tests referred to specific server ports even when the test
didn't actually use that server or specify that it's needed. In such
cases, the test harness substitutes the text "[not running]" as the port
number which causes many such tests to fail due to the inability to
parse the URL. These tests are changed to use %NOLISTENPORT which will
always be substituted correctly.
More connection cache accesses are protected by locks.
CONNCACHE_* is a beter prefix for the connection cache lock macros.
Curl_attach_connnection: now called as soon as there's a connection
struct available and before the connection is added to the connection
cache.
Curl_disconnect: now assumes that the connection is already removed from
the connection cache.
Ref: #4915Closes#5009
Prior to this change if there was a 303 reply to a PUT request then
the subsequent request to respond to that redirect would also be a PUT.
It was determined that was most likely incorrect based on the language
of the RFCs. Basically 303 means "see other" resource, which implies it
is most likely not the same resource, therefore we should not try to PUT
to that different resource.
Refer to the discussions in #5237 and #5248 for more information.
Fixes https://github.com/curl/curl/issues/5237
Closes https://github.com/curl/curl/pull/5248
The purpose with this variable is to provide a port number that is
reasonably likely to not have a listener on the local host so that tests
can try connect failures against it. It uses port 47 - "reserved"
according to IANA.
Updated six tests to use it instead of the previous different ports.
Assisted-by: Emil Engler
Closes#5270
The runtests script now always performs variable replacement on the
entire test source file before the test gets executed, and saves the
updated version in a temporary file (log/test[num]) so that all test
case readers/servers can use that version (if present) and thus enjoy
the powers of test case variable substitution.
This is necessary to allow complete port number freedom.
Test 309 is updated to work with a non-fixed port number thanks to this.
- Ignore intermediate progress updates.
- Support locales that use a character other than period as decimal
separator (eg 100,0%).
test1148 checks that the progress finishes at 100% and has the right
bar width. Prior to this change the test assumed that the only progress
reported for such a quick transfer was 100%, however in rare instances
(like in the CI where transfer time can slow considerably) there may be
intermediate updates. For example, below is stderrlog1148 from a failed
CI run with explicit \r and \n added (it is one line; broken up so that
it's easier to understand).
\r
\r################################## 48.3%
\r######################################################################## 100.0%
\n
Closes https://github.com/curl/curl/pull/5194
test1294: "split request" being when the entire request isn't sent in
the first go, and the remainder is sent in the PERFORM state. A GET
request is otherwise not sending anything during PERFORM.
test1295: same kind of split but with POST
Closes#5197
This change inverse the order of processing for the --etag-compare and
--etag-save option to process first --etag-compare. This in turn allows
to use the same file name to compare and save an etag.
The original behavior of not failing if the etag file does not exists is
conserved.
Fixes#5179Closes#5180
Load long values correctly (e.g. for http_code).
Use curl_off_t (not long) for:
- size_download (CURLINFO_SIZE_DOWNLOAD_T)
- size_upload (CURLINFO_SIZE_UPLOAD_T)
The unit for these values is bytes/second, not microseconds:
- speed_download (CURLINFO_SPEED_DOWNLOAD_T)
- speed_upload (CURLINFO_SPEED_UPLOAD_T)
Fixes#5131Closes#5152
Reported by the new script 'scripts/copyright.pl'. The script has a
regex whitelist for the files that don't need copyright headers.
Removed three (mostly usesless) README files from docs/
Closes#5141
- send more data to make problems more obvious
- don't start the data with minus, it makes diffs harder to read
- skip the headers in the stdout comparison
- save to a file name to also verify 'filename_effective'
Ref: #5131
Update smbserver.py and negtelnetserver.py to be compatible with
Python 3 while staying backwards-compatible to support Python 2.
Fix string encoding and handling of echoed and transferred data.
Tested with both Python 2.7.17 and Python 3.7.7
Reported-by: Daniel Stenberg
Assisted-by: Kamil Dudka
Reviewed-by: Marcel Raad
Fixes#5104Closes#5110
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
--ssl-revoke-best-effort to allow a "best effort" revocation check.
A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.
---
When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:
Unknown error (0x80092012) - The revocation function was unable
to check revocation for the certificate.
Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.
This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.
In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.
Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).
To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.
Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.
The test 2070 was added to verify this behavior, adapted from 310.
Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes https://github.com/curl/curl/pull/4981
Makes curl_easy_getinfo() of "variable" numerical content instead return
the number set in the env variable `CURL_TIME`.
Makes curl_version() of "variable" textual content. This guarantees a
stable version string which can be tested against. Environment variable
`CURL_VERSION` defines the content.
Assisted-by: Mathias Gumz
The test uses SRP to "a server not supporting it" but modern stunnel
versions will silently accept it and remain happy. The test is therefore
faulty.
I haven't figured out how to make stunnel explicitly reject SRP-using
connects.
Reported-by: Marc Hörsken
Fixes#5105Closes#5113
Users of the SMB tests will have to install impacket manually.
Reasoning: our in-tree version of impacket was quite outdated
and only compatible with Python 2 which is already end-of-life.
Upgrading to Python 3 and a compatible impacket version would
require to import additional Python-only and CPython-extension
dependencies. This would have hindered portability enormously.
Closes#5094
When extracting a <section> <part> and there's no </part> before
</section>, this now outputs an error and returns a wrong string to
make users spot the mistake.
Ref: #5070Closes#5071
New test 666 checks this is effective.
As upload buffer size is significant in this kind of tests, shorten it
in similar test 652.
Fixes#4860Closes#4833
Reported-by: RuurdBeerstra on github
Input buffer filling may delay the data sending if data reads are slow.
To overcome this problem, file and callback data reads do not accumulate
in buffer anymore. All other data (memory data and mime framing) are
considered as fast and still concatenated in buffer.
As this may highly impact performance in terms of data overhead, an early
end of part data check is added to spare a read call.
When encoding a part's data, an encoder may require more bytes than made
available by a single read. In this case, the above rule does not apply
and reads are performed until the encoder is able to deliver some data.
Tests 643, 644, 645, 650 and 654 have been adapted to the output data
changes, with test data size reduced to avoid the boredom of long lists of
1-byte chunks in verification data.
New test 667 checks mimepost using single-byte read callback with encoder.
New test 668 checks the end of part data early detection.
Fixes#4826
Reported-by: MrdUkk on github
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.
Fixes#4813
Reported-by: MrdUkk on github
New test 666 checks this is effective.
As upload buffer size is significant in this kind of tests, shorten it
in similar test 652.
Fixes#4860
Reported-by: RuurdBeerstra on github
Input buffer filling may delay the data sending if data reads are slow.
To overcome this problem, file and callback data reads do not accumulate
in buffer anymore. All other data (memory data and mime framing) are
considered as fast and still concatenated in buffer.
As this may highly impact performance in terms of data overhead, an early
end of part data check is added to spare a read call.
When encoding a part's data, an encoder may require more bytes than made
available by a single read. In this case, the above rule does not apply
and reads are performed until the encoder is able to deliver some data.
Tests 643, 644, 645, 650 and 654 have been adapted to the output data
changes, with test data size reduced to avoid the boredom of long lists of
1-byte chunks in verification data.
New test 664 checks mimepost using single-byte read callback with encoder.
New test 665 checks the end of part data early detection.
Fixes#4826
Reported-by: MrdUkk on github
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.
Fixes#4813
Reported-by: MrdUkk on github
Closes#4833
This reduces the HTTP/2 window size to 32 MB since libcurl might have to
buffer up to this amount of data in memory and yet we don't want it set
lower to potentially impact tranfer performance on high speed networks.
Requires nghttp2 commit b3f85e2daa629
(https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end
up in the next release after 1.40.0.
Fixes#4939Closes#4940
It is still possible to override the executable to run during the test,
using the <tool> tag, but this patch removes the requirement that the
tag must be present for unit tests.
It also removes the possibility of human error when existing test cases
are used as the basis for new tests, as recently witnessed in 81c37124.
Reviewed-by: Daniel Stenberg
Closes#4976
When doing a request with a body + Expect: 100-continue and the server
responds with a 417, the same request will be retried immediately
without the Expect: header.
Added test 357 to verify.
Also added a control instruction to tell the sws test server to not read
the request body if Expect: is present, which the new test 357 uses.
Reported-by: bramus on github
Fixes#4949Closes#4964
Note: The RCPT TO command isn't required to advertise to the server that
it contains UTF-8 characters, instead the server is told that a mail may
contain UTF-8 in any envelope command via the MAIL command.
Support the SMTPUTF8 extension when sending mailbox information in the
MAIL command (FROM and AUTH parameters). Non-ASCII domain names will
be ACE encoded, if IDN is supported, whilst non-ASCII characters in
the local address part are passed to the server.
Reported-by: ygthien on github
Fixes#4828
The alt-svc cache survives a call to curl_easy_reset fine, but the file
name to use for saving the cache was cleared. Now the alt-svc cache has
a copy of the file name to survive handle resets.
Added test 1908 to verify.
Reported-by: Craig Andrews
Fixes#4898Closes#4902
RFC 7616 section 3.4 (The Authorization Header Field) states that "For
historical reasons, a sender MUST NOT generate the quoted string syntax
for the following parameters: algorithm, qop, and nc". This removes the
quoting for the algorithm parameter.
Reviewed-by: Steve Holme
Closes#4890
... as this is already done much earlier in the URL parser.
Also add test case 894 that verifies that pop3 with an encodedd CR in
the user name is rejected.
Closes#4887
Previously it was stored in a global state which contributed to
curl_global_init's thread unsafety. This boolean is now instead figured
out in curl_multi_init() and stored in the multi handle. Less effective,
but thread safe.
Closes#4851
- Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection
errors.
Prior to this change CURLE_FAILED_INIT was used, but that was not
correct.
Closes https://github.com/curl/curl/pull/4754
It was removed for output containing ' =' via `s/ =.*//`. With classic
MinGW, this made lines with `free()` end with CRLF, but lines with e.g.
`malloc()` end with only LF. The tests expect LF only.
Closes https://github.com/curl/curl/pull/4788
Previously it would end up with an uninitialized memory buffer that
would lead to a crash or junk getting output.
Added test 1271 to verify.
Reported-by: Brian Carpenter
Closes#4786
Prior to this change tests that required NTLM feature did not require
SSL feature.
There are pending changes to cmake builds that will allow enabling NTLM
in non-SSL builds in Windows. In that case the NTLM auth strings created
are different from what is expected by the NTLM tests and they fail:
"The issue with NTLM is that previous non-SSL builds would not enable
NTLM and so the NTLM tests would be skipped."
Assisted-by: marc-groundctl@users.noreply.github.com
Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729
Closes https://github.com/curl/curl/pull/4768
Even if the initial request line wasn't found. With the fix to 1455, the
test number is now detected correctly.
(Problem found when running tests in random order.)
Closes#4744
On my current Debian Unstable with libidn2 2.2.0, I get an error if
LC_ALL is set to blank. Then curl errors out with:
curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8
Closes#4738
Fixup the test to instead not compare the port number. It sometimes
caused problems like this:
"curl: (45) bind failed with errno 98: Address already in use"
Closes#4733
