github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-10-23 23:51:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

sched/core: Remove redundant core_sched_seq

Browse Source 
core_sched_seq records whether an rq has consumed the pick made by the
last core-wide selection.

However, rq->core_pick already carries the same per-rq state. A core-wide
selection leaves core_pick populated only for siblings which still need to
consume their picks. It is cleared when the current CPU consumes its pick,
when a sibling is already running the selected task, or when a pending pick
is consumed through the fastpath. The CPU offline path clears it as well.

core_task_seq and core_pick_seq serve a separate purpose. core_task_seq
changes when the task set changes and for each new core-wide pick, while
core_pick_seq records the task sequence on which the selection was made.
Their equality therefore establishes that a pending core_pick is still
valid.

Consequently, once core_pick_seq == core_task_seq, a non-NULL core_pick is
sufficient to tell that this rq still has a pick to consume.
core_sched_seq duplicates that pending/consumed state.

Remove core_sched_seq and use core_pick directly as the pending marker.

Testing included an instrumented comparison of the old and new pending
predicates under core-scheduling stress. Additional guest tests exercised
repeated core-cookie lifecycles, forced idle, and CPU hotplug under load.
No predicate divergence or kernel warning was observed.

Signed-off-by: Hui Su <sh_def@163.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260915163138.2973969-1-sh_def@163.com
This commit is contained in:
Hui Su 2026-09-16 01:31:38 +09:00 committed by Peter Zijlstra
parent 819224e506
commit fbbc63fed0
2 changed files with 4 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
kernel/sched/core.c
View File

@ -6276,10 +6276,7 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
* selection. In this case, do a core-wide selection.
*/
if (rq->core->core_pick_seq == rq->core->core_task_seq &&
rq->core->core_pick_seq != rq->core_sched_seq &&
rq->core_pick) {
WRITE_ONCE(rq->core_sched_seq, rq->core->core_pick_seq);
next = rq->core_pick;
rq->dl_server = rq->core_dl_server;
rq->core_pick = NULL;
@ -6311,11 +6308,13 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
}
/*
* core->core_task_seq, core->core_pick_seq, rq->core_sched_seq
* core->core_task_seq, core->core_pick_seq
*
* @task_seq guards the task state ({en,de}queues)
* @pick_seq is the @task_seq we did a selection on
* @sched_seq is the @pick_seq we scheduled
*
* Once a core-wide selection is committed, a non-NULL core_pick denotes
* a pick which still needs to be consumed on this CPU.
*
* However, preemptions can cause multiple picks on the same task set.
* 'Fix' this by also increasing @task_seq for every pick.
@ -6422,7 +6421,6 @@ pick_next_task(struct rq *rq, struct rq_flags *rf)
rq->core->core_pick_seq = rq->core->core_task_seq;
next = rq->core_pick;
rq->core_sched_seq = rq->core->core_pick_seq;
/* Something should have been selected for current CPU */
WARN_ON_ONCE(!next);

1
kernel/sched/sched.h
View File

@ -1371,7 +1371,6 @@ struct rq {
struct task_struct *core_pick;
struct sched_dl_entity *core_dl_server;
unsigned int core_enabled;
unsigned int core_sched_seq;
struct rb_root core_tree;
/* shared state -- careful with sched_core_cpu_deactivate() */