github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

io_uring/cancel: rearm local work wake up before sleep

Browse Source 
io_uring_try_cancel_requests() sets ->cq_wait_nr to 1 so that the next
io_req_local_work_add() wakes the cancelling task. The add that takes the
count to zero is the only one that wakes, later ones see a count <= 0 and
return. io_uring_cancel_generic() however only becomes
TASK_INTERRUPTIBLE much later, so the one wake up can be spent on a
running task. That is fine as long as the entry of that add is still
queued, as the io_local_work_pending() check then skips the sleep. It is
not if the add was delayed after queueing its entry:

CPU0 (submitter in exec or exit)      CPU1 (io_req_local_work_add)
                                      mpscq_push(req A)
io_uring_try_cancel_requests()
  io_run_local_work()  // runs req A
io_uring_try_cancel_requests()
  atomic_set(&ctx->cq_wait_nr, 1)
                                      atomic_read(&ctx->cq_wait_nr) == 1
                                      count taken, wake_up_state()
                                        // task is running, no-op
prepare_to_wait(TASK_INTERRUPTIBLE)
io_local_work_pending() == false
schedule()
                                      mpscq_push(req B)
                                      cq_wait_nr <= 0, return

Only the sleeping task can run req B, so it is stuck in execve() until a
signal happens to wake it. An exiting task doesn't even get that.

Arm the count again once the task state is set and before the work list
is checked, for the rings whose local work this task is the one to run.

Fixes: d46ab2c98a ("io_uring: switch local task_work to a mpscq")
Assisted-by: LLM
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Link: https://patch.msgid.link/20261015-b4-io-uring-fixes-v1-8-c1556f5d7bf5@toxicpanda.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Josef Bacik 2026-10-15 19:30:32 +00:00 committed by Jens Axboe
parent 8257f6ec45
commit 76a82c666a
1 changed files with 12 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
io_uring/cancel.c
View File

@ -694,8 +694,18 @@ __cold void io_uring_cancel_generic(bool cancel_all, struct io_sq_data *sqd)
io_uring_drop_tctx_refs(current);
/* only skip the sleep for local work this task may run */
xa_for_each(&tctx->xa, index, node) {
if (io_local_work_pending(node->ctx) &&
io_allowed_defer_tw_run(node->ctx))
if (!(node->ctx->flags & IORING_SETUP_DEFER_TASKRUN) ||
!io_allowed_defer_tw_run(node->ctx))
continue;
/*
* The wake up io_uring_try_cancel_requests() asked for
* may have been issued while we were still running. Ask
* again now that it can't be missed.
*/
atomic_set(&node->ctx->cq_wait_nr, 1);
/* pairs with the xchg() in mpscq_push() */
smp_mb();
if (io_local_work_pending(node->ctx))
goto end_wait;
}
/*