github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-23 13:14:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

sched_ext: Use @prev's scheduler for the keep decisions in dispatch_one()

Browse Source 
dispatch_one() tests ops flags and bypass state against the root scheduler
in both places where it decides to keep running @prev: the early keep of a
@prev with slice left tests the root's bypass state, and the keep-last at
the end tests the root's SCX_OPS_ENQ_LAST and bypass state. Both are
properties of the scheduler @prev belongs to, and put_prev_task_scx(), which
acts on the outcome, reads them from that scheduler. When @prev belongs to a
sub-scheduler the two sides disagree.

The keep-last case is visible. The root set SCX_OPS_ENQ_LAST, so a lone
@prev of a sub-scheduler is not kept and is enqueued with SCX_ENQ_LAST to a
sub-scheduler that never opted in. This trips the WARN_ON_ONCE in
put_prev_task_scx() for the missing flag, and the sub-scheduler queues the
task like any other and triggers no follow-up scheduling event, which can
lead to stalls.

Test SCX_OPS_ENQ_LAST and bypass state on @prev's sched in both places and
charge SCX_EV_DISPATCH_KEEP_LAST to it. Read the sched at each decision, as
the dispatch in between can drop the rq lock.

Fixes: 88234b075c ("sched_ext: Introduce scx_task_sched[_rcu]()")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
This commit is contained in:
Tejun Heo 2026-09-05 06:09:56 -10:00
parent 3265ef0b67
commit 90f19b2816
1 changed files with 14 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
kernel/sched/ext/ext.c
View File

@ -2955,7 +2955,7 @@ static enum scx_dsp_verdict dispatch_one(struct rq *rq, struct task_struct *prev
* test.
*/
if ((prev->scx.flags & SCX_TASK_QUEUED) && prev->scx.slice &&
!scx_bypassing(root_sch, cpu)) {
!scx_bypassing(scx_task_sched(prev), cpu)) {
verdict = SCX_DSP_PREV;
goto has_tasks;
}
@ -2972,15 +2972,20 @@ static enum scx_dsp_verdict dispatch_one(struct rq *rq, struct task_struct *prev
goto has_tasks;
/*
* Didn't find another task to run. Keep running @prev unless
* %SCX_OPS_ENQ_LAST is in effect.
* Didn't find another task to run. Keep running @prev unless its own
* scheduler set %SCX_OPS_ENQ_LAST and takes the enqueue instead, see
* put_prev_task_scx(). Read the scheduler here as the dispatch above
* may have dropped the rq lock while @prev changed class or scheduler.
*/
if ((prev->scx.flags & SCX_TASK_QUEUED) &&
(!(root_sch->ops.flags & SCX_OPS_ENQ_LAST) || scx_bypassing(root_sch, cpu)) &&
scx_task_can_stay_on_cpu(rq, prev)) {
__scx_add_event(root_sch, SCX_EV_DISPATCH_KEEP_LAST, 1);
verdict = SCX_DSP_PREV;
goto has_tasks;
if (prev->scx.flags & SCX_TASK_QUEUED) {
struct scx_sched *prev_sch = scx_task_sched(prev);
if ((!(prev_sch->ops.flags & SCX_OPS_ENQ_LAST) ||
scx_bypassing(prev_sch, cpu)) && scx_task_can_stay_on_cpu(rq, prev)) {
__scx_add_event(prev_sch, SCX_EV_DISPATCH_KEEP_LAST, 1);
verdict = SCX_DSP_PREV;
goto has_tasks;
}
}
rq->scx.flags &= ~SCX_RQ_IN_DISPATCH;
return SCX_DSP_NONE;