Verify that the BPF verifier rejects a non-SCX struct_ops program
(tcp_congestion_ops) that attempts to call an SCX kfunc (scx_bpf_kick_cpu).
The test expects the load to fail with -EACCES from scx_kfunc_context_filter.
Signed-off-by: Cheng-Yang Chou <yphbchou0911@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Conflict in kernel/sched/ext.c init_sched_ext_class() between:
415cb193bb ("sched_ext: Fix SCX_KICK_WAIT deadlock by deferring wait
to balance callback")
which adds cpus_to_sync cpumask allocation, and:
84b1a0ea0b ("sched_ext: Implement scx_bpf_dsq_reenq() for user DSQs")
8c1b9453fd ("sched_ext: Convert deferred_reenq_locals from llist to
regular list")
which add deferred_reenq init code at the same location. Both are
independent additions. Include both.
Signed-off-by: Tejun Heo <tj@kernel.org>
Add a test that creates a 3-CPU kick_wait cycle (A->B->C->A). A BPF
scheduler kicks the next CPU in the ring with SCX_KICK_WAIT on every
enqueue while userspace workers generate continuous scheduling churn via
sched_yield(). Without the preceding fix, this hangs the machine within seconds.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Christian Loehle <christian.loehle@arm.com>
Tested-by: Christian Loehle <christian.loehle@arm.com>
This reverts commit c50dcf5331.
The tests are superficial, likely AI-generated slop, and flaky. They
don't add actual value and just churn the selftests.
Signed-off-by: Tejun Heo <tj@kernel.org>
Add three selftests covering features introduced in v7.1:
- dsq_reenq: Verify scx_bpf_dsq_reenq() on user DSQs triggers
ops.enqueue() with SCX_ENQ_REENQ and SCX_TASK_REENQ_KFUNC in
p->scx.flags.
- enq_immed: Verify SCX_OPS_ALWAYS_ENQ_IMMED slow path where tasks
dispatched to a busy CPU's local DSQ are re-enqueued through
ops.enqueue() with SCX_TASK_REENQ_IMMED.
- consume_immed: Verify SCX_ENQ_IMMED via the consume path using
scx_bpf_dsq_move_to_local___v2() with explicit SCX_ENQ_IMMED.
All three tests skip gracefully on kernels that predate the required
features by checking availability via __COMPAT_has_ksym() /
__COMPAT_read_enum() before loading.
Signed-off-by: zhidao su <suzhidao@xiaomi.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Similar to commit 835a507535 ("selftests/bpf: Add -fms-extensions to
bpf build flags") and commit 639f58a0f4 ("bpftool: Fix build warnings
due to MS extensions")
Fix "declaration does not declare anything" warning by using
-fms-extensions and -Wno-microsoft-anon-tag flags to build bpf programs
that #include "vmlinux.h"
Signed-off-by: Zhao Mengmeng <zhaomengmeng@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Add a new kselftest to validate that the new ops.dequeue() semantics
work correctly for all task lifecycle scenarios, including the
distinction between terminal DSQs (where BPF scheduler is done with the
task), user DSQs (where BPF scheduler manages the task lifecycle) and
BPF data structures, regardless of which event performs the dispatch.
The test validates the following scenarios:
- From ops.select_cpu():
- scenario 0 (local DSQ): tasks dispatched to the local DSQ bypass
the BPF scheduler entirely; they never enter BPF custody, so
ops.dequeue() is not called,
- scenario 1 (global DSQ): tasks dispatched to SCX_DSQ_GLOBAL also
bypass the BPF scheduler, like the local DSQ; ops.dequeue() is
not called,
- scenario 2 (user DSQ): tasks dispatched to user DSQs from
ops.select_cpu(): tasks enter BPF scheduler's custody with full
enqueue/dequeue lifecycle tracking and state machine validation,
expects 1:1 enqueue/dequeue pairing,
- From ops.enqueue():
- scenario 3 (local DSQ): same behavior as scenario 0,
- scenario 4 (global DSQ): same behavior as scenario 1,
- scenario 5 (user DSQ): same behavior as scenario 2,
- scenario 6 (BPF internal queue): tasks are stored in a BPF queue
from ops.enqueue() and consumed from ops.dispatch(); similarly to
scenario 5, tasks enter BPF scheduler's custody with full
lifecycle tracking and 1:1 enqueue/dequeue validation.
This verifies that:
- terminal DSQ dispatch (local, global) don't trigger ops.dequeue(),
- tasks dispatched to user DSQs, either from ops.select_cpu() or
ops.enqueue(), enter BPF scheduler's custody and have exact 1:1
enqueue/dequeue pairing,
- tasks stored to internal BPF data structures from ops.enqueue() enter
BPF scheduler's custody and have exact 1:1 enqueue/dequeue pairing,
- dispatch dequeues have no flags (normal workflow),
- property change dequeues have the %SCX_DEQ_SCHED_CHANGE flag set,
- no duplicate enqueues or invalid state transitions are happening.
Cc: Tejun Heo <tj@kernel.org>
Cc: Emil Tsalapatis <emil@etsalapatis.com>
Cc: Kuba Piecuch <jpiecuch@google.com>
Reviewed-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Add a new kselftest to verify that the total_bw value in
/sys/kernel/debug/sched/debug remains consistent across all CPUs
under different sched_ext BPF program states:
1. Before a BPF scheduler is loaded
2. While a BPF scheduler is loaded and active
3. After a BPF scheduler is unloaded
The test runs CPU stress threads to ensure DL server bandwidth
values stabilize before checking consistency. This helps catch
potential issues with DL server bandwidth accounting during
sched_ext transitions.
Co-developed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Christian Loehle <christian.loehle@arm.com>
Link: https://patch.msgid.link/20260126100050.3854740-8-arighi@nvidia.com
Add a selftest to validate the correct behavior of the deadline server
for the ext_sched_class.
Co-developed-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Tested-by: Christian Loehle <christian.loehle@arm.com>
Link: https://patch.msgid.link/20260126100050.3854740-7-arighi@nvidia.com
This commit adds two tests. The first is the most basic unit test:
make sure an empty queue peeks as empty, and when we put one element
in the queue, make sure peek returns that element.
However, even this simple test is a little complicated by the different
behavior of scx_bpf_dsq_insert in different calling contexts:
- insert is for direct dispatch in enqueue
- insert is delayed when called from select_cpu
In this case we split the insert and the peek that verifies the
result between enqueue/dispatch.
Note: An alternative would be to call `scx_bpf_dsq_move_to_local` on an
empty queue, which in turn calls `flush_dispatch_buf`, in order to flush
the buffered insert. Unfortunately, this is not viable within the
enqueue path, as it attempts a voluntary context switch within an RCU
read-side critical section.
The second test is a stress test that performs many peeks on all DSQs
and records the observed tasks.
Signed-off-by: Ryan Newton <newton@meta.com>
Reviewed-by: Christian Loehle <christian.loehle@arm.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
With commit 08699d20467b6 ("sched_ext: idle: Consolidate default idle
CPU selection kfuncs") allowing scx_bpf_select_cpu_dfl() to be invoked
from multiple contexts, update the test to validate that the kfunc
behaves correctly when used from ops.enqueue() and via BPF test_run.
Additionally, rename the test to enq_select_cpu, dropping "fails" from
the name, as the logic has now been inverted.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Add a selftest to validate the behavior of the built-in idle CPU
selection policy applied to a subset of allowed CPUs, using
scx_bpf_select_cpu_and().
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
Add a selftest to validate the behavior of the NUMA-aware scheduler
functionalities, including idle CPU selection within nodes, per-node
DSQs and CPU to node mapping.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
The sched_ext selftests is missing proper cross-compilation support, a
proper target entry, and out-of-tree build support.
When building the kselftest suite, e.g.:
make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- \
TARGETS=sched_ext SKIP_TARGETS="" O=/output/foo \
-C tools/testing/selftests install
or:
make ARCH=arm64 LLVM=1 TARGETS=sched_ext SKIP_TARGETS="" \
O=/output/foo -C tools/testing/selftests install
The expectation is that the sched_ext is included, cross-built, the
correct toolchain is picked up, and placed into /output/foo.
In contrast to the BPF selftests, the sched_ext suite does not use
bpftool at test run-time, so it is sufficient to build bpftool for the
build host only.
Add ARCH, CROSS_COMPILE, OUTPUT, and TARGETS support to the sched_ext
selftest. Also, remove some variables that were unused by the
Makefile.
Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Reviewed-by: Shuah Khan <skhan@linuxfoundation.org>
Acked-by: David Vernet <void@manifault.com>
Tested-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
