linux/tools/testing/selftests
Jakub Kicinski 65177e47d3 testing: kselftest_harness: add filtering and enumerating tests
As the number of test cases and length of execution grows it's
useful to select only a subset of tests. In TLS for instance we
have a matrix of variants for different crypto protocols and
during development mostly care about testing a handful.
This is quicker and makes reading output easier.

This patch adds argument parsing to kselftest_harness.

It supports a couple of ways to filter things, I could not come
up with one way which will cover all cases.

The first and simplest switch is -r which takes the name of
a test to run (can be specified multiple times). For example:

  $ ./my_test -r some.test.name -r some.other.name

will run tests some.test.name and some.other.name (where "some"
is the fixture, "test" and "other" and "name is the test.)

Then there is a handful of group filtering options. f/v/t for
filtering by fixture/variant/test. They have both positive
(match -> run) and negative versions (match -> skip).
If user specifies any positive option we assume the default
is not to run the tests. If only negative options are set
we assume the tests are supposed to be run by default.

  Usage: ./tools/testing/selftests/net/tls [-h|-l] [-t|-T|-v|-V|-f|-F|-r name]
	-h       print help
	-l       list all tests

	-t name  include test
	-T name  exclude test
	-v name  include variant
	-V name  exclude variant
	-f name  include fixture
	-F name  exclude fixture
	-r name  run specified test

  Test filter options can be specified multiple times. The filtering stops
  at the first match. For example to include all tests from variant 'bla'
  but not test 'foo' specify '-T foo -v bla'.

Here we can request for example all tests from fixture "foo" to run:

 ./my_test -f foo

or to skip variants var1 and var2:

 ./my_test -V var1 -V var2

Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-01-26 16:00:41 -07:00
..
alsa kselftest/alsa: Add more coverage of sample rates and channel counts 2022-12-01 20:02:14 +01:00
amd-pstate selftests: amd-pstate: Trigger gitsource benchmark and test cpus 2022-11-01 03:22:37 -06:00
arm64 Merge branch 'for-next/selftests' into for-next/core 2022-12-06 11:25:43 +00:00
bpf selftests/bpf: add a test for iter/task_vma for short-lived processes 2022-12-28 14:11:48 -08:00
breakpoints
capabilities
cgroup MM patches for 6.2-rc1. 2022-12-13 19:29:45 -08:00
clone3
core
cpu-hotplug selftests/cpu-hotplug: Add log info when test success 2022-10-05 11:05:18 -06:00
cpufreq
damon selftests/damon: test removed scheme sysfs dir access bug 2022-12-11 18:12:15 -08:00
dma
dmabuf-heaps
drivers selftests: devlink: add a warning for interfaces coming up 2022-12-16 10:21:37 +00:00
efivarfs selftests/efivarfs: Add checking of the test return value 2022-11-22 09:09:04 -07:00
exec
filesystems
firmware
fpu
ftrace Trace probes updates for 6.2: 2022-12-21 18:57:24 -08:00
futex selftests/futex: fix build for clang 2022-10-18 14:13:11 -06:00
gpio selftests: gpio: Use "grep -E" instead of "egrep" 2022-11-25 10:57:23 -07:00
ia64
intel_pstate selftests/intel_pstate: fix build for ARCH=x86_64 2022-10-18 14:13:19 -06:00
iommu iommufd: Add a selftest 2022-11-30 20:16:49 -04:00
ipc
ir
kcmp
kexec selftests/kexec: fix build for ARCH=x86_64 2022-10-18 14:13:25 -06:00
kmod testing: use the copyleft-next-0.3.1 SPDX tag 2022-11-08 15:44:02 +01:00
kselftest
kvm KVM: selftests: restore special vmmcall code layout needed by the harness 2022-12-28 07:18:01 -05:00
landlock Landlock updates for v6.2-rc1 2022-12-13 09:14:50 -08:00
lib
livepatch Merge branch 'for-6.1/sysfs-patched-object' into for-linus 2022-10-05 13:00:03 +02:00
lkdtm
locking
media_tests
membarrier
memfd
memory-hotplug selftests/memory-hotplug: Remove the redundant warning information 2022-10-18 14:21:18 -06:00
mincore
mount
mount_setattr
move_mount_set_group
mqueue
nci NFC: nci: Extend virtual NCI deinit test 2022-11-21 10:49:58 +00:00
net selftests/net: toeplitz: fix race on tpacket_v3 block close 2023-01-19 09:27:15 -08:00
netfilter selftests: netfilter: fix transaction test script timeout handling 2023-01-11 19:17:40 +01:00
nolibc selftests/nolibc: Always rebuild the sysroot when running a test 2022-10-28 15:17:22 -07:00
nsfs
ntb
openat2
perf_events selftests/perf_events: Add a SIGTRAP stress test with disables 2022-10-17 16:32:06 +02:00
pid_namespace
pidfd selftests/pidfd_test: Remove the erroneous ',' 2022-11-02 03:09:57 -06:00
powerpc selftests/powerpc: Fix resource leaks 2022-12-05 21:39:15 +11:00
prctl
proc proc: fix PIE proc-empty-vm, proc-pid-vm tests 2023-01-11 16:14:22 -08:00
pstore
ptp
ptrace
rcutorture torture: Make torture.sh create a properly formatted log file 2022-10-20 15:29:08 -07:00
resctrl
rlimits
rseq
rtc selftests: rtc: skip when RTC is not present 2022-11-15 13:30:51 -07:00
safesetid
sched
seccomp selftests/seccomp: Check CAP_SYS_ADMIN capability in the test mode_filter_without_nnp 2022-12-02 11:32:53 -08:00
sgx
sigaltstack
size
sparc64
splice selftests: splice_read: Fix sysfs read cases 2022-11-29 17:28:31 -07:00
static_keys
sync
syscall_user_dispatch
sysctl testing: use the copyleft-next-0.3.1 SPDX tag 2022-11-08 15:44:02 +01:00
tc-testing selftests: tc-testing: Add matchJSON to tdc 2022-10-26 20:22:33 -07:00
tdx selftests/tdx: Test TDX attestation GetReport support 2022-11-17 11:04:28 -08:00
timens selftests/timens: add a test for vfork+exit 2022-10-25 15:15:52 -07:00
timers
tmpfs
tpm2 selftests/tpm2: Split async tests call to separate shell script runner 2022-11-29 17:28:31 -07:00
uevent
user
user_events tracing/user_events: Use bits vs bytes for enabled status page data 2022-09-29 10:17:37 -04:00
vDSO selftests/vDSO: Add riscv getcpu & gettimeofday test 2022-11-03 03:28:01 -06:00
vm MM patches for 6.2-rc1. 2022-12-13 19:29:45 -08:00
watchdog selftests/watchdog: Fix spelling mistake "Temeprature" -> "Temperature" 2022-10-27 02:53:37 -06:00
wireguard random: use random.trust_{bootloader,cpu} command line option only 2022-11-18 02:18:10 +01:00
x86
zram
.gitignore
gen_kselftest_tar.sh
kselftest_deps.sh selftests: kselftest_deps: Use "grep -E" instead of "egrep" 2022-11-25 10:55:52 -07:00
kselftest_harness.h testing: kselftest_harness: add filtering and enumerating tests 2023-01-26 16:00:41 -07:00
kselftest_install.sh
kselftest_module.h
kselftest.h
lib.mk kselftest: Fix error message for unconfigured LLVM builds 2023-01-12 13:38:04 -07:00
Makefile iommufd for 6.2 2022-12-14 09:15:43 -08:00
run_kselftest.sh