sched_ext: Refuse extending a kernel slice

A rescuee can keep running after its grant is served, on a cpu its scheduler
has lost. The write rules deny a slice extension only where the scheduler
lacks SCX_CAP_BASE on the cpu, and a scheduler holding BASE but not ENQ can
have a task under rescue there:

 1. the scheduler extends the rescuee's slice
 2. BASE is revoked. The cap-revoke eviction skips the rescuee.
 3. the grant is served and scx_rescue_charge() ends the rescue
 4. the task runs out the extension on the cpu its scheduler lost, outside
    the rescue's bandwidth

A scheduler shortens or zeroes a rescuee's slice to preempt it, and
scx_rescue_keep() resumes the grant afterwards. An extension serves no such
purpose.

Refuse extending a kernel slice in both scheduler write paths, the direct
write in scx_bpf_task_set_slice() and the out-of-band application, counted
in SCX_EV_SLICE_DENIED like an extension without BASE. Shortening stays
allowed.

scx_rescue_keep()'s restore of the remainder raises the slice from zero.
Like the kernel's other slice writes, it goes through scx_set_task_slice()
and not the scheduler write paths.

v2:
- scx_rescue_admit()'s kerneldoc no longer mentions a served-time snapshot.
- SCX_EV_SLICE_DENIED's comment lists every refusal it counts.
- scx_bpf_task_set_slice()'s kerneldoc says that shortening is refused on a
  protected slice.

Fixes: 5fd501744b ("sched_ext: Add bandwidth-limited rescue execution for stranded tasks")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
This commit is contained in:
Tejun Heo 2026-10-26 14:17:41 -10:00
parent 98889f2cae
commit 8197fa420d
4 changed files with 29 additions and 19 deletions

View File

@ -113,7 +113,7 @@ enum scx_ent_flags {
* alone.
* - Reenqueue scans skip the task.
* - put_prev_task_scx() keeps a preempted task on the local DSQ.
* - Scheduler slice writes and preemption still apply.
* - Schedulers may shorten the slice, never extend it.
*/
SCX_TASK_KERNEL_SLICE = 1 << 8, /* slice owned by the kernel */

View File

@ -1385,7 +1385,8 @@ enum scx_slice_oob_consts {
* p->scx.slice_oob, applied under @p's rq lock at the next slice consideration.
*
* While %SCX_TASK_PROTECTED is set, every scheduler-reachable slice update is
* refused. See set_task_slice_keep_oob().
* refused. See set_task_slice_keep_oob(). A kernel slice, see
* %SCX_TASK_KERNEL_SLICE, can be shortened but not extended.
*
* dsq_vtime orders the next PRIQ insertion and has no running-side consumer, so
* scx_bpf_task_set_dsq_vtime() writes it directly. Fork-time init and direct
@ -1511,9 +1512,9 @@ static void set_task_slice_oob(struct scx_sched *sch, struct task_struct *p, u64
/*
* Apply a pending out-of-band slice request under @rq's lock. A request whose
* packed id no longer matches @p's current owner is dropped. An extension needs
* baseline cpu access on @p's cid, shortening is always allowed, and a
* protected slice refuses both. %SCX_EV_SLICE_DENIED counts the denials. See
* the write rules above.
* baseline cpu access on @p's cid and is refused on a kernel slice. Shortening
* is always allowed. A protected slice refuses both. %SCX_EV_SLICE_DENIED
* counts the denials. See the write rules above.
*/
static void apply_task_slice_oob(struct rq *rq, struct task_struct *p)
{
@ -1537,7 +1538,8 @@ static void apply_task_slice_oob(struct rq *rq, struct task_struct *p)
slice = dur == SCX_SLICE_OOB_DUR_MASK ? SCX_SLICE_INF : dur;
if (slice > p->scx.slice &&
unlikely(scx_missing_caps(scx_task_sched(p), cpu_of(rq), SCX_CAP_BASE))) {
unlikely(scx_task_has_kernel_slice(p) ||
scx_missing_caps(scx_task_sched(p), cpu_of(rq), SCX_CAP_BASE))) {
__scx_add_event(scx_task_sched(p), SCX_EV_SLICE_DENIED, 1);
return;
}
@ -10427,10 +10429,12 @@ __bpf_kfunc_start_defs();
* rules above for the ownership model.
*
* Raising the slice is honored only while the scheduler holds %SCX_CAP_BASE on
* @p's cpu, otherwise it is counted in %SCX_EV_SLICE_DENIED. Shortening is
* always allowed. On the stashed path the slice is packed into an atomic64_t
* with the scheduler id and a flag bit, so a slice too large to fit is clamped
* and counted in %SCX_EV_SLICE_CLAMPED. %SCX_SLICE_INF is preserved.
* @p's cpu, and never on a kernel slice, see %SCX_TASK_KERNEL_SLICE. Shortening
* is honored unless @p is protected, see %SCX_TASK_PROTECTED, which refuses
* every write. A refused write is counted in %SCX_EV_SLICE_DENIED. On the
* stashed path the slice is packed into an atomic64_t with the scheduler id and
* a flag bit, so a slice too large to fit is clamped and counted in
* %SCX_EV_SLICE_CLAMPED. %SCX_SLICE_INF is preserved.
*
* Return %true on success, %false if @p is not on the calling scheduler.
*/
@ -10468,9 +10472,13 @@ __bpf_kfunc bool scx_bpf_task_set_slice(struct task_struct *p, u64 slice,
return true;
}
/* under the rq lock: apply now, extensions gated on baseline access */
/*
* @p's rq is locked, so apply now. An extension needs baseline access
* and is refused on a kernel slice.
*/
if (slice > p->scx.slice &&
unlikely(scx_missing_caps(sch, cpu_of(locked_rq), SCX_CAP_BASE))) {
unlikely(scx_task_has_kernel_slice(p) ||
scx_missing_caps(sch, cpu_of(locked_rq), SCX_CAP_BASE))) {
__scx_add_event(sch, SCX_EV_SLICE_DENIED, 1);
return true;
}

View File

@ -1343,8 +1343,10 @@ struct scx_event_stats {
s64 SCX_EV_SLICE_CLAMPED;
/*
* The number of times a slice extension was denied because the
* scheduler lacked baseline cpu access on the task's cpu.
* The number of times a slice write was refused. A raise is refused
* without baseline cpu access on the task's cpu and on a kernel slice.
* Every write is refused on a protected slice, a preemption's zeroing
* included.
*/
s64 SCX_EV_SLICE_DENIED;

View File

@ -616,11 +616,11 @@ static void scx_rescue_timer_arm(struct rq *rq)
* @p: task being admitted, off any DSQ
* @slice: CPU time to grant
*
* The slice is a kernel slice, see %SCX_TASK_KERNEL_SLICE. The schedulers keep
* their normal control over @p and may preempt or reslice it. @slice is
* measured on served CPU time, so neither shortens the rescue, see
* scx_rescue_charge() and scx_rescue_keep(). Prolonged denial escalates into
* protected execution, see scx_rescue_timerfn().
* The slice is a kernel slice, see %SCX_TASK_KERNEL_SLICE. The schedulers may
* preempt @p or shorten its slice, never extend it. @slice is measured on
* served CPU time, so neither shortens the rescue, see scx_rescue_charge() and
* scx_rescue_keep(). Prolonged denial escalates into protected execution, see
* scx_rescue_timerfn().
*/
static void scx_rescue_admit(struct rq *rq, struct task_struct *p, s64 slice)
{