github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-06 21:58:01 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

io_uring: defer foreign io-wq enqueues to task_work

Browse Source 
An IOPOLL ring without DEFER_TASKRUN can be reaped by a task other than
the one that submitted the I/O. If a polled request is marked
REQ_F_REISSUE, io_free_batch_list() calls io_queue_iowq() with the
original tctx. Ordinary task exit can tear down tctx->io_wq after
draining only the tracked requests. The live request pins the task, but
not its io-wq, so the foreign task can race io_wq_put_and_exit() and
enqueue through a freed or NULL queue.

Forward foreign userspace enqueues to the owning task before accessing
tctx->io_wq. Use a dedicated task_work callback so reissues still go
straight to io-wq, and fail them if task_work is running in cancellation
mode. Preserve the existing PF_KTHREAD cancellation path. This also
covers the other callers of io_queue_iowq() without changing the
worker-owned enqueue path.

Signed-off-by: Kyle Zeng <kylebot@openai.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Kyle Zeng 2026-10-25 02:43:10 +00:00 committed by Jens Axboe
parent 37e425460b
commit 3de16279b4
1 changed files with 23 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
io_uring/io_uring.c
View File

@ -423,6 +423,17 @@ static __always_inline bool io_assign_file(struct io_kiocb *req,
return !!req->file;
}
static void io_req_queue_iowq_tw(struct io_tw_req tw_req, io_tw_token_t tw)
{
struct io_kiocb *req = tw_req.req;
io_tw_lock(req->ctx, tw);
if (unlikely(tw.cancel))
io_req_defer_failed(req, -ECANCELED);
else
io_queue_iowq(req);
}
void io_queue_iowq(struct io_kiocb *req)
{
struct io_uring_task *tctx = req->tctx;
@ -430,10 +441,17 @@ void io_queue_iowq(struct io_kiocb *req)
BUG_ON(!tctx);
lockdep_assert_held(&req->ctx->uring_lock);
if ((current->flags & PF_KTHREAD) || !tctx->io_wq) {
io_req_task_queue_fail(req, -ECANCELED);
if (current->flags & PF_KTHREAD)
goto cancel;
/* Only the submitter can safely access its io-wq during task exit. */
if (unlikely(current != tctx->task)) {
req->io_task_work.func = io_req_queue_iowq_tw;
io_req_task_work_add(req);
return;
}
if (!tctx->io_wq)
goto cancel;
/*
* The file is looked up as late as possible, as it may be the result
@ -462,6 +480,9 @@ void io_queue_iowq(struct io_kiocb *req)
trace_io_uring_queue_async_work(req, io_wq_is_hashed(&req->work));
io_wq_enqueue(tctx->io_wq, &req->work);
return;
cancel:
io_req_task_queue_fail(req, -ECANCELED);
}
unsigned io_linked_nr(struct io_kiocb *req)