* runtests.pl:
- Fix out-of-tree build under CMake when srcdir is not set. Default
srcdir to the location of runtests.pl.
- Add a hack to allow CMake to use the TFLAGS option as documented
in tests/README and used in scripts/travis/script.sh.
* Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie
support (no one should care, it is already EOL.).
* Remove CTest since it defines its own 'test' target with no tests
since all unittests are already broken and not built by default.
* Add new test targets based on the options from Makefile.am. Since
new test targets are rarely added, I opted for duplicating the
runtests.pl options as opposed to creating a new Makefile.inc file.
Use top-level target names (test-x) instead of x-test since that is
used by CI and others.
Closes#5358
Remove many one second sleeps that were done *after* each newly started
test server already has been verified. They should not have any purpose
there.
Closes#5323
OpenSSH for Windows requires paths in the format of /C:/
instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/
Reviewed-by: Daniel Stenberg
Closes#5298
As stunnel is an external tool and it has no specific option to export
the actually used port number when asked to listen to 0, runtests
instead iterates over ten randomly picked high number ports and sticks
to the first one stunnel can listen to.
Closes#5267
Since the http2 test server is a mere proxy that needs to know about the
dynamic port the HTTP server is using, it too needs to get restarted
when the http server is killed.
A regression caused by 80d6515.
Fixes#5289Closes#5291
The purpose with this variable is to provide a port number that is
reasonably likely to not have a listener on the local host so that tests
can try connect failures against it. It uses port 47 - "reserved"
according to IANA.
Updated six tests to use it instead of the previous different ports.
Assisted-by: Emil Engler
Closes#5270
FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic
ports
Test 842-845 are unfortunately a bit hard to move over to this concept
right now and require "default port" still...
The runtests script now always performs variable replacement on the
entire test source file before the test gets executed, and saves the
updated version in a temporary file (log/test[num]) so that all test
case readers/servers can use that version (if present) and thus enjoy
the powers of test case variable substitution.
This is necessary to allow complete port number freedom.
Test 309 is updated to work with a non-fixed port number thanks to this.
- Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and
--ssl-revoke-best-effort to allow a "best effort" revocation check.
A best effort revocation check ignores errors that the revocation check
was unable to take place. The reasoning is described in detail below and
discussed further in the PR.
---
When running e.g. with Fiddler, the schannel backend fails with an
unhelpful error message:
Unknown error (0x80092012) - The revocation function was unable
to check revocation for the certificate.
Sadly, many enterprise users who are stuck behind MITM proxies suffer
the very same problem.
This has been discussed in plenty of issues:
https://github.com/curl/curl/issues/3727,
https://github.com/curl/curl/issues/264, for example.
In the latter, a Microsoft Edge developer even made the case that the
common behavior is to ignore issues when a certificate has no recorded
distribution point for revocation lists, or when the server is offline.
This is also known as "best effort" strategy and addresses the Fiddler
issue.
Unfortunately, this strategy was not chosen as the default for schannel
(and is therefore a backend-specific behavior: OpenSSL seems to happily
ignore the offline servers and missing distribution points).
To maintain backward-compatibility, we therefore add a new flag
(`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option
(`--ssl-revoke-best-effort`) to select the new behavior.
Due to the many related issues Git for Windows and GitHub Desktop, the
plan is to make this behavior the default in these software packages.
The test 2070 was added to verify this behavior, adapted from 310.
Based-on-work-by: georgeok <giorgos.n.oikonomou@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes https://github.com/curl/curl/pull/4981
It is still possible to override the executable to run during the test,
using the <tool> tag, but this patch removes the requirement that the
tag must be present for unit tests.
It also removes the possibility of human error when existing test cases
are used as the basis for new tests, as recently witnessed in 81c37124.
Reviewed-by: Daniel Stenberg
Closes#4976
When using randomized features of runtests (-R and --shallow) it is
useful to have a fixed random seed to make sure for example extra
commits in a branch or a rebase won't change the seed that would make
repeated runs work differently.
As it is also useful to change seed sometimes, the default seed is now
determined based on the current month (and first line curl -V
output). When the month changes, so will the random seed.
The specific seed is also shown in the standard test suite top header
and it can be set explictly with the new --seed=[num] option so that the
exact order of a previous run can be achieved.
Closes#4734
When set, shallow mode limits runtests -t to make no more than NUM fails
per test case. If more are found, it will randomly discard entries until
the number is right. The random seed can also be set.
This is particularly useful when running MANY tests as then most torture
failures will already fail the same functions over and over and make the
total operation painfully tedious.
Closes#4699
This enables the use of Windows Subsystem for Linux (WSL) to run the
testsuite against Windows binaries while using Linux servers.
This commit introduces the following environment variables:
- CURL_TEST_EXE_EXT: set the executable extension for all components
- CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only
- CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only
Later testcurl.pl could be adjusted to make use of those variables.
- CURL_TEST_EXE_EXT_SRV: set it for the test servers only
(This is one of several commits to support use of WSL for the tests.)
Closes https://github.com/curl/curl/pull/3899
