Several tests referred to specific server ports even when the test
didn't actually use that server or specify that it's needed. In such
cases, the test harness substitutes the text "[not running]" as the port
number which causes many such tests to fail due to the inability to
parse the URL. These tests are changed to use %NOLISTENPORT which will
always be substituted correctly.
More connection cache accesses are protected by locks.
CONNCACHE_* is a beter prefix for the connection cache lock macros.
Curl_attach_connnection: now called as soon as there's a connection
struct available and before the connection is added to the connection
cache.
Curl_disconnect: now assumes that the connection is already removed from
the connection cache.
Ref: #4915Closes#5009
As stunnel is an external tool and it has no specific option to export
the actually used port number when asked to listen to 0, runtests
instead iterates over ten randomly picked high number ports and sticks
to the first one stunnel can listen to.
Closes#5267
These were commented out in e9dd099870 when Makefile.inc was included
instead. 11 years have passed since then and the commented code is of
course very outdated. Remove it to avoid confusion.
Closes#5311
Since the http2 test server is a mere proxy that needs to know about the
dynamic port the HTTP server is using, it too needs to get restarted
when the http server is killed.
A regression caused by 80d6515.
Fixes#5289Closes#5291
Prior to this change if there was a 303 reply to a PUT request then
the subsequent request to respond to that redirect would also be a PUT.
It was determined that was most likely incorrect based on the language
of the RFCs. Basically 303 means "see other" resource, which implies it
is most likely not the same resource, therefore we should not try to PUT
to that different resource.
Refer to the discussions in #5237 and #5248 for more information.
Fixes https://github.com/curl/curl/issues/5237
Closes https://github.com/curl/curl/pull/5248
The purpose with this variable is to provide a port number that is
reasonably likely to not have a listener on the local host so that tests
can try connect failures against it. It uses port 47 - "reserved"
according to IANA.
Updated six tests to use it instead of the previous different ports.
Assisted-by: Emil Engler
Closes#5270
This fixes the error: 'for' loop initial declaration used outside C99
mode by declaring the loop increment variable in the beginning of the
block instead of inside the for loop.
Fixes#5254
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic
ports
Test 842-845 are unfortunately a bit hard to move over to this concept
right now and require "default port" still...
The runtests script now always performs variable replacement on the
entire test source file before the test gets executed, and saves the
updated version in a temporary file (log/test[num]) so that all test
case readers/servers can use that version (if present) and thus enjoy
the powers of test case variable substitution.
This is necessary to allow complete port number freedom.
Test 309 is updated to work with a non-fixed port number thanks to this.
Since the Windows versions of both programs would write Windows
PIDs to their pidfiles which we cannot handle, we need to use
our known perl.exe Cygwin/msys PID together with exec() in order
to tie the spawned processes to the existance of our perl.exe
The perl.exe that is executing secureserver.pl and sshserver.pl
has a Cygwin/msys PID, because it is started inside Cygwin/msys.
Related to #5188
- Ignore intermediate progress updates.
- Support locales that use a character other than period as decimal
separator (eg 100,0%).
test1148 checks that the progress finishes at 100% and has the right
bar width. Prior to this change the test assumed that the only progress
reported for such a quick transfer was 100%, however in rare instances
(like in the CI where transfer time can slow considerably) there may be
intermediate updates. For example, below is stderrlog1148 from a failed
CI run with explicit \r and \n added (it is one line; broken up so that
it's easier to understand).
\r
\r################################## 48.3%
\r######################################################################## 100.0%
\n
Closes https://github.com/curl/curl/pull/5194
With this bit set, my mac successfully resolves "ip6-localhost" when in
fact there is no such host known to my machine! That in turn made test
241 wrongly execute and fail.
Closes#5202
test1294: "split request" being when the entire request isn't sent in
the first go, and the remainder is sent in the PERFORM state. A GET
request is otherwise not sending anything during PERFORM.
test1295: same kind of split but with POST
Closes#5197
This change inverse the order of processing for the --etag-compare and
--etag-save option to process first --etag-compare. This in turn allows
to use the same file name to compare and save an etag.
The original behavior of not failing if the etag file does not exists is
conserved.
Fixes#5179Closes#5180
Replace the incomplete workaround regarding FD_CLOSE
only signalling once by instead doing a pre-check with
standard select and storing the result for later use.
select keeps triggering on closed sockets on Windows while
WSAEventSelect fires only once with data still available.
By doing the pre-check we do not run in a deadlock
due to waiting forever for another FD_CLOSE event.
Fix race-condition of waiting threads finishing while events are
already being processed which lead to invalid or skipped events.
Use mutex to check for one event at a time or do post-processing.
In addition to mutex-based locking use specific event as signal.
Closes#5156
Load long values correctly (e.g. for http_code).
Use curl_off_t (not long) for:
- size_download (CURLINFO_SIZE_DOWNLOAD_T)
- size_upload (CURLINFO_SIZE_UPLOAD_T)
The unit for these values is bytes/second, not microseconds:
- speed_download (CURLINFO_SPEED_DOWNLOAD_T)
- speed_upload (CURLINFO_SPEED_UPLOAD_T)
Fixes#5131Closes#5152
Reported by the new script 'scripts/copyright.pl'. The script has a
regex whitelist for the files that don't need copyright headers.
Removed three (mostly usesless) README files from docs/
Closes#5141
- send more data to make problems more obvious
- don't start the data with minus, it makes diffs harder to read
- skip the headers in the stdout comparison
- save to a file name to also verify 'filename_effective'
Ref: #5131
Update smbserver.py and negtelnetserver.py to be compatible with
Python 3 while staying backwards-compatible to support Python 2.
Fix string encoding and handling of echoed and transferred data.
Tested with both Python 2.7.17 and Python 3.7.7
Reported-by: Daniel Stenberg
Assisted-by: Kamil Dudka
Reviewed-by: Marcel Raad
Fixes#5104Closes#5110
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
--ssl-revoke-best-effort to allow a "best effort" revocation check.
A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.
---
When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:
Unknown error (0x80092012) - The revocation function was unable
to check revocation for the certificate.
Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.
This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.
In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.
Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).
To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.
Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.
The test 2070 was added to verify this behavior, adapted from 310.
Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes https://github.com/curl/curl/pull/4981
Makes curl_easy_getinfo() of "variable" numerical content instead return
the number set in the env variable `CURL_TIME`.
Makes curl_version() of "variable" textual content. This guarantees a
stable version string which can be tested against. Environment variable
`CURL_VERSION` defines the content.
Assisted-by: Mathias Gumz
The test uses SRP to "a server not supporting it" but modern stunnel
versions will silently accept it and remain happy. The test is therefore
faulty.
I haven't figured out how to make stunnel explicitly reject SRP-using
connects.
Reported-by: Marc Hörsken
Fixes#5105Closes#5113
Users of the SMB tests will have to install impacket manually.
Reasoning: our in-tree version of impacket was quite outdated
and only compatible with Python 2 which is already end-of-life.
Upgrading to Python 3 and a compatible impacket version would
require to import additional Python-only and CPython-extension
dependencies. This would have hindered portability enormously.
Closes#5094
When extracting a <section> <part> and there's no </part> before
</section>, this now outputs an error and returns a wrong string to
make users spot the mistake.
Ref: #5070Closes#5071
Even though the existing code can be fixed to run on Python 3, the
tests will fail due to the Unicode transition the protocol is invalid.
Follow up to ee63837Closes#5085
Recent gcc warns when byte count of strncpy() equals the destination
buffer size. Since the destination buffer is previously cleared and
the source string is always shorter, reducing the byte count by one
silents the warning without affecting the result.
Closes#5059
When using maximum code optimization level (-O3), valgrind wrongly
detects uses of uninitialized values in strcmp().
Preset buffers with all zeroes to avoid that.
This test does A LOT of *wakeup() calls and then calls curl_multi_poll()
twice. The first *poll() is then expected to return early and the second
not - as the first is supposed to drain the socketpair pipe.
It turns out however that when given "excessive" amounts of writes to
the pipe, some operating systems (the Solaris based are known) will
return EAGAIN before the pipe is drained, which in our test case causes
the second *poll() call to also abort early.
This change attempts to avoid the OS-specific behaviors in the test by
reducing the amount of wakeup calls from 1234567 to 10.
Reported-by: Andy Fiddaman
Fixes#5037Closes#5058
New test 666 checks this is effective.
As upload buffer size is significant in this kind of tests, shorten it
in similar test 652.
Fixes#4860Closes#4833
Reported-by: RuurdBeerstra on github
Input buffer filling may delay the data sending if data reads are slow.
To overcome this problem, file and callback data reads do not accumulate
in buffer anymore. All other data (memory data and mime framing) are
considered as fast and still concatenated in buffer.
As this may highly impact performance in terms of data overhead, an early
end of part data check is added to spare a read call.
When encoding a part's data, an encoder may require more bytes than made
available by a single read. In this case, the above rule does not apply
and reads are performed until the encoder is able to deliver some data.
Tests 643, 644, 645, 650 and 654 have been adapted to the output data
changes, with test data size reduced to avoid the boredom of long lists of
1-byte chunks in verification data.
New test 667 checks mimepost using single-byte read callback with encoder.
New test 668 checks the end of part data early detection.
Fixes#4826
Reported-by: MrdUkk on github
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.
Fixes#4813
Reported-by: MrdUkk on github
New test 666 checks this is effective.
As upload buffer size is significant in this kind of tests, shorten it
in similar test 652.
Fixes#4860
Reported-by: RuurdBeerstra on github
Input buffer filling may delay the data sending if data reads are slow.
To overcome this problem, file and callback data reads do not accumulate
in buffer anymore. All other data (memory data and mime framing) are
considered as fast and still concatenated in buffer.
As this may highly impact performance in terms of data overhead, an early
end of part data check is added to spare a read call.
When encoding a part's data, an encoder may require more bytes than made
available by a single read. In this case, the above rule does not apply
and reads are performed until the encoder is able to deliver some data.
Tests 643, 644, 645, 650 and 654 have been adapted to the output data
changes, with test data size reduced to avoid the boredom of long lists of
1-byte chunks in verification data.
New test 664 checks mimepost using single-byte read callback with encoder.
New test 665 checks the end of part data early detection.
Fixes#4826
Reported-by: MrdUkk on github
In case a read callback returns a status (pause, abort, eof,
error) instead of a byte count, drain the bytes read so far but
remember this status for further processing.
Takes care of not losing data when pausing, and properly resume a
paused mime structure when requested.
New tests 670-673 check unpausing cases, with easy or multi
interface and mime or form api.
Fixes#4813
Reported-by: MrdUkk on github
Closes#4833
This reduces the HTTP/2 window size to 32 MB since libcurl might have to
buffer up to this amount of data in memory and yet we don't want it set
lower to potentially impact tranfer performance on high speed networks.
Requires nghttp2 commit b3f85e2daa629
(https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end
up in the next release after 1.40.0.
Fixes#4939Closes#4940
It is still possible to override the executable to run during the test,
using the <tool> tag, but this patch removes the requirement that the
tag must be present for unit tests.
It also removes the possibility of human error when existing test cases
are used as the basis for new tests, as recently witnessed in 81c37124.
Reviewed-by: Daniel Stenberg
Closes#4976
When doing a request with a body + Expect: 100-continue and the server
responds with a 417, the same request will be retried immediately
without the Expect: header.
Added test 357 to verify.
Also added a control instruction to tell the sws test server to not read
the request body if Expect: is present, which the new test 357 uses.
Reported-by: bramus on github
Fixes#4949Closes#4964
Note: The RCPT TO command isn't required to advertise to the server that
it contains UTF-8 characters, instead the server is told that a mail may
contain UTF-8 in any envelope command via the MAIL command.
Support the SMTPUTF8 extension when sending mailbox information in the
MAIL command (FROM and AUTH parameters). Non-ASCII domain names will
be ACE encoded, if IDN is supported, whilst non-ASCII characters in
the local address part are passed to the server.
Reported-by: ygthien on github
Fixes#4828
