github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

io_uring/rsrc: charge buffer arrays and nodes to memcg

Browse Source 
The io_mapped_ubuf of a registered buffer has one bio_vec per page,
unless the pages coalesce into huge pages. Huge pages only count against
RLIMIT_MEMLOCK once per ring, so a buffer that maps the same 2MB folio
over and over in a way that defeats coalescing gets a full bvec array
per registration while only the one folio counts as locked. 1GB of that
registered in 100 slots took several times a 64MB memory.max, with 2.6MB
charged.

Allocate the io_mapped_ubuf and the io_rsrc_node with
GFP_KERNEL_ACCOUNT. An object reused from the ring caches stays charged
to whoever allocated it first, same as the io_kiocb cache.

Fixes: de2939388b ("io_uring: improve registered buffer accounting for huge pages")
Signed-off-by: Chris Mason <mason@kernel.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This commit is contained in:
Chris Mason 2026-10-19 16:57:42 +00:00 committed by Jens Axboe
parent 47c68f9250
commit 075d5837ba
1 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
io_uring/rsrc.c
View File

@ -162,8 +162,9 @@ static struct io_mapped_ubuf *io_alloc_imu(struct io_ring_ctx *ctx,
int nr_bvecs)
{
if (nr_bvecs <= IO_CACHED_BVECS_SEGS)
return io_cache_alloc(&ctx->imu_cache, GFP_KERNEL);
return kvmalloc_flex(struct io_mapped_ubuf, bvec, nr_bvecs);
return io_cache_alloc(&ctx->imu_cache, GFP_KERNEL_ACCOUNT);
return kvmalloc_flex(struct io_mapped_ubuf, bvec, nr_bvecs,
GFP_KERNEL_ACCOUNT);
}
static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
@ -246,7 +247,7 @@ struct io_rsrc_node *io_rsrc_node_alloc(struct io_ring_ctx *ctx, int type)
{
struct io_rsrc_node *node;
node = io_cache_alloc(&ctx->node_cache, GFP_KERNEL);
node = io_cache_alloc(&ctx->node_cache, GFP_KERNEL_ACCOUNT);
if (node) {
node->type = type;
node->refs = 1;