Added feature in runtests.pl to select tests based on key word.

This commit is contained in:
Dan Fandrich 2008-07-27 02:34:27 +00:00
parent 432945e422
commit b6b03c8ab9
5 changed files with 32 additions and 8 deletions

View File

@ -21,6 +21,11 @@ Daniel Stenberg (26 Jul 2008)
Daniel Fandrich (26 Jul 2008)
- Added test 1044 to test large file support in ftp with -I.
- Eliminate a unnecessary socket creation in Curl_getaddrinfo for an IPv4
address in an IPv6 capable libcurl.
- Added feature in runtests.pl to select tests based on key word.
Daniel Fandrich (23 Jul 2008)
- Changed the long logfile elision code in runtests.pl to properly handle
lines ending in \r.

View File

@ -17,6 +17,7 @@ This release includes the following changes:
o Added --remote-name-all
o Now builds for the INTEGRITY operating system
o Added CURLINFO_APPCONNECT_TIME
o Added test selection by key word in runtests.pl
This release includes the following bugfixes:

View File

@ -28,8 +28,11 @@ Main sections are 'info', 'reply', 'client' and 'verify'.
<info>
<keywords>
A newline-separated list of keywords describing what this test case uses and
tests. Try to use an already used keyword. These keywords will be used for
statistical/informational purposes.
tests. Try to use an already used keyword. These keywords will be used for
statistical/informational purposes and for choosing or skipping classes
of tests. "Keywords" must begin with an alphabetic character, "-", "["
or "{" and may actually consist of multiple words separated by spaces
which are treated together as a single identifier.
</keywords>
</info>

View File

@ -25,7 +25,7 @@
.SH NAME
runtests.pl \- run one or more test cases
.SH SYNOPSIS
.B runtests.pl [options] [test number]
.B runtests.pl [options] [test number] [!test number] [key word] [!key word]
.SH DESCRIPTION
\fIruntests.pl\fP runs one, several or all the existing test cases in curl's
test suite. It is often called from the root Makefile of the curl package with
@ -39,7 +39,10 @@ the numbers with a leading exclamation point, like "!66".
.P
It is also possible to specify tests to skip based on a key word describing
the test. These are specified with a leading exclamation point and the
key word or phrase, like "!HTTP NTLM auth".
key word or phrase, like "!HTTP NTLM auth". Likewise, tests to run can
be specified simply by specifying the unadorned key words, like "FTPS".
Remember that the exclamation marks and spaces will need to be quoted somehow
when entered at many command shells.
.SH OPTIONS
.IP "-a"
Continue running the rest of the test cases even if one test fails. By

View File

@ -45,7 +45,7 @@
# to do with cookies, those that set environment variables, or those that
# do more than touch the file system in a <precheck> or <postcheck>
# section). These can be added to the $TESTCASES line below,
# e.g. $TESTCASES="!8 !31 !63..."
# e.g. $TESTCASES="!8 !31 !63 !cookies..."
#
# Finally, to properly support -g and -n, checktestcmd needs to change
# to check the remote system's PATH, and the places in the code where
@ -197,6 +197,7 @@ my $skipped=0; # number of tests skipped; reported in main loop
my %skipped; # skipped{reason}=counter, reasons for skip
my @teststat; # teststat[testnum]=reason, reasons for skip
my %disabled_keywords; # key words of tests to skip
my %enabled_keywords; # key words of tests to run
my $sshdid; # for socks server, ssh daemon version id
my $sshdvernum; # for socks server, ssh daemon version number
@ -1820,13 +1821,20 @@ sub singletest {
if(!$why) {
my @keywords = getpart("info", "keywords");
my $match;
my $k;
for $k (@keywords) {
chomp $k;
if ($disabled_keywords{$k}) {
$why = "disabled by keyword";
} elsif ($enabled_keywords{$k}) {
$match = 1;
}
}
if(!$why && !$match && %enabled_keywords) {
$why = "disabled by missing keyword";
}
}
if(!$why) {
@ -2781,7 +2789,7 @@ while(@ARGV) {
elsif($ARGV[0] eq "-h") {
# show help text
print <<EOHELP
Usage: runtests.pl [options] [test number(s)]
Usage: runtests.pl [options] [test selection(s)]
-a continue even if a test fails
-bN use base port number N for test servers (default $base)
-c path use this curl executable
@ -2797,7 +2805,8 @@ Usage: runtests.pl [options] [test number(s)]
-v verbose output
[num] like "5 6 9" or " 5 to 22 " to run those tests only
[!num] like "!5 !6 !9" to disable those tests
[!keyword] like "!cookies !IPv6" to disable tests with those key words
[keyword] like "IPv6" to select only tests containing the key word
[!keyword] like "!cookies" to disable any tests containing the key word
EOHELP
;
exit;
@ -2824,6 +2833,9 @@ EOHELP
elsif($ARGV[0] =~ /^!(.+)/) {
$disabled_keywords{$1}=$1;
}
elsif($ARGV[0] =~ /^([-[{a-zA-Z].*)/) {
$enabled_keywords{$1}=$1;
}
else {
print "Unknown option: $ARGV[0]\n";
exit;
@ -2940,7 +2952,7 @@ if ( $TESTCASES eq "all") {
for(@cmds) {
$_ =~ s/[a-z\/\.]*//g;
}
# the the numbers from low to high
# sort the numbers from low to high
foreach my $n (sort { $a <=> $b } @cmds) {
if($disabled{$n}) {
# skip disabled test cases