github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

sched_ext: Fix use-after-free of a local reenq node left on an inactive cpu

Browse Source 
A local DSQ reenq is queued on the rq through a per-sched node and run from
deferred work. When the reenq is requested from dispatch, the work is a
balance callback queued once dispatch is done. A cpu going down installs
balance push early in its teardown, and queue_balance_callback() refuses new
callbacks from then on. ops.dispatch() keeps running on that cpu until the
rq goes offline after an RCU grace period, so a reenq it requests in that
window may never run. The IMMED handling in dispatch_one() requests one the
same way.

A reenq requested from elsewhere for a cpu that is already offline goes
through irq_work_queue_on(), which warns, and its work doesn't run until the
cpu comes back.

Either way the node stays on the rq's list. When the scheduler is freed,
scx_sched_free_rcu_work() warns and frees the pcpu with the node still
linked, and the list is walked again once the cpu comes back.

An inactive cpu's local DSQ holds only tasks that are bound to it or are
being moved off, so the reenq is pointless. Ignore reenqs of an inactive
cpu's local DSQ.

The test is lockless. The hotplug thread of a cpu going down runs on that
cpu, and it marks the cpu inactive before it installs balance push. A
dispatch on the cpu runs to completion before that thread can run there, so
a dispatch that saw the cpu active has queued its callback by then. Requests
from outside dispatch run from the wakeup path or through irq_work, which
balance push doesn't affect. The grace period after the cpu goes inactive
waits for any requester that saw it active, so that irq_work is queued while
the cpu is online.

Fixes: 238eba8c21 ("sched_ext: Use schedule_deferred_locked() in schedule_dsq_reenq()")
Fixes: 76edc2761a ("sched_ext: Use irq_work_queue_on() in schedule_deferred()")
Cc: stable@vger.kernel.org # v7.1+
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Tested-by: Andrea Righi <arighi@nvidia.com>
This commit is contained in:
Tejun Heo 2026-10-15 07:18:05 -10:00
parent 2935062be7
commit 43e1ef14a4
1 changed files with 14 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
kernel/sched/ext/ext.c
View File

@ -1102,6 +1102,18 @@ void schedule_dsq_reenq(struct scx_sched *sch, struct scx_dispatch_q *dsq,
if (dsq->id == SCX_DSQ_LOCAL) {
rq = container_of(dsq, struct rq, scx.local_dsq);
/*
* An inactive cpu's local DSQ holds only tasks that are bound
* to it or are being moved off, so a reenq of it is pointless.
* Once balance push is installed, queue_balance_callback() also
* drops the callback that would run a reenq requested from
* dispatch, and irq_work_queue_on() can't reach an offline cpu.
* The reenq's node would then stay on this rq's list after
* @sch's pcpu is freed.
*/
if (unlikely(!cpu_active(cpu_of(rq))))
return;
/*
* A sub-sched lacking baseline access on the target cid has no
* business triggering IPIs. The lockless test is fine: slipping
@ -10020,7 +10032,8 @@ __bpf_kfunc struct task_struct *scx_bpf_dsq_peek(u64 dsq_id,
* - Local DSQs (%SCX_DSQ_LOCAL or %SCX_DSQ_LOCAL_ON | $cpu)
* - User DSQs
*
* Re-enqueues are performed asynchronously. Can be called from anywhere.
* Re-enqueues are performed asynchronously. Can be called from anywhere. A
* re-enqueue of an inactive cpu's local DSQ is ignored.
*
* %SCX_DSQ_LOCAL resolves to the local DSQ of the rq the current scheduler
* operation is locked to - e.g. the rq being dispatched for in ops.dispatch() -