linux/io_uring
Pavel Begunkov 61cfadaae6 io_uring/zcrx: consolidate dma syncing
Split refilling into two steps, first allocate niovs, and then do DMA
sync for them. This way dma synchronisation code can be better
optimised. E.g. we don't need to call dma_dev_need_sync() for each every
niov, and maybe we can coalesce sync for adjacent netmems in the future
as well.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://patch.msgid.link/19f2d50baa62ff2e0c6cd56dd7c394cab728c567.1774261953.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2026-04-01 10:21:12 -06:00
..
advise.c io_uring: finish IOU_OK -> IOU_COMPLETE transition 2025-05-21 08:41:16 -06:00
advise.h
alloc_cache.c io_uring: add alloc_cache.c 2025-01-28 15:10:40 -07:00
alloc_cache.h io_uring: Add SPDX id lines to remaining source files 2026-02-04 07:23:45 -07:00
bpf_filter.c Merge branch 'io_uring-7.0' into for-7.1/io_uring 2026-03-14 08:57:15 -06:00
bpf_filter.h io_uring: allow registration of per-task restrictions 2026-02-06 07:29:19 -07:00
bpf-ops.c io_uring/bpf-ops: implement bpf ops registration 2026-03-16 16:15:00 -06:00
bpf-ops.h io_uring/bpf-ops: implement bpf ops registration 2026-03-16 16:15:00 -06:00
cancel.c for-7.0/io_uring-20260206 2026-02-09 17:22:00 -08:00
cancel.h io_uring/cancel: de-unionize file and user_data in struct io_cancel_data 2026-02-16 14:16:27 -07:00
cmd_net.c io_uring/cmd_net: split ioctl code out of io_uring_cmd_sock() 2026-03-09 07:21:53 -06:00
epoll.c io_uring: finish IOU_OK -> IOU_COMPLETE transition 2025-05-21 08:41:16 -06:00
epoll.h io_uring/epoll: add support for IORING_OP_EPOLL_WAIT 2025-02-20 07:59:56 -07:00
eventfd.c io_uring: switch struct io_ring_ctx internal bitfields to flags 2026-03-16 15:32:59 -06:00
eventfd.h io_uring: Add SPDX id lines to remaining source files 2026-02-04 07:23:45 -07:00
fdinfo.c io_uring/fdinfo: be a bit nicer when looping a lot of SQEs/CQEs 2026-02-03 10:58:32 -07:00
fdinfo.h
filetable.c io_uring/filetable: clamp alloc_hint to the configured alloc range 2026-02-11 15:20:44 -07:00
filetable.h io_uring: Trim out unused includes 2026-01-05 17:08:20 -07:00
fs.c non-consuming variants of do_{unlinkat,rmdir}() 2026-01-16 12:51:50 -05:00
fs.h
futex.c treewide: Replace kmalloc with kmalloc_obj for non-scalar types 2026-02-21 01:02:28 -08:00
futex.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
io_uring.c io_uring/zcrx: return back two step unregistration 2026-04-01 10:21:12 -06:00
io_uring.h io_uring: mark known and harmless racy ctx->int_flags uses 2026-03-16 15:33:10 -06:00
io-wq.c Convert 'alloc_obj' family to use the new default GFP_KERNEL argument 2026-02-21 17:09:51 -08:00
io-wq.h io_uring: Add SPDX id lines to remaining source files 2026-02-04 07:23:45 -07:00
kbuf.c io_uring/kbuf: use 'ctx' consistently 2026-03-17 14:03:54 -06:00
kbuf.h io_uring/kbuf: remove obsolete buf_nr_pages and update comments 2025-11-20 13:23:05 -07:00
Kconfig io_uring/bpf-ops: implement loop_step with BPF struct_ops 2026-03-16 16:15:00 -06:00
loop.c io_uring: introduce callback driven main loop 2026-03-16 16:15:00 -06:00
loop.h io_uring: introduce callback driven main loop 2026-03-16 16:15:00 -06:00
Makefile io_uring/bpf-ops: implement loop_step with BPF struct_ops 2026-03-16 16:15:00 -06:00
memmap.c treewide: Replace kmalloc with kmalloc_obj for non-scalar types 2026-02-21 01:02:28 -08:00
memmap.h io_uring: Add SPDX id lines to remaining source files 2026-02-04 07:23:45 -07:00
mock_file.c Convert 'alloc_obj' family to use the new default GFP_KERNEL argument 2026-02-21 17:09:51 -08:00
msg_ring.c io_uring: switch struct io_ring_ctx internal bitfields to flags 2026-03-16 15:32:59 -06:00
msg_ring.h io_uring/msg_ring: Drop custom destructor 2024-12-27 10:08:21 -07:00
napi.c net: use napi_id_valid helper 2025-02-17 16:43:04 -08:00
napi.h io_uring/napi: add static napi tracking strategy 2024-11-06 13:55:38 -07:00
net.c io_uring/net: use 'ctx' consistently 2026-03-17 14:35:00 -06:00
net.h io_uring/zctx: unify zerocopy issue variants 2026-03-09 07:21:54 -06:00
nop.c io_uring/nop: add support for IORING_SETUP_CQE_MIXED 2025-08-27 11:24:15 -06:00
nop.h
notif.c io_uring: Add SPDX id lines to remaining source files 2026-02-04 07:23:45 -07:00
notif.h io_uring/notif: implement notification stacking 2024-04-22 19:31:18 -06:00
opdef.c io_uring: remove iopoll_queue from struct io_issue_def 2026-03-16 16:14:14 -06:00
opdef.h io_uring: remove iopoll_queue from struct io_issue_def 2026-03-16 16:14:14 -06:00
openclose.c io_uring/openclose: fix io_pipe_fixed() slot tracking for specific slots 2026-02-11 20:31:21 -07:00
openclose.h io_uring/bpf_filter: allow filtering on contents of struct open_how 2026-01-27 11:10:46 -07:00
poll.c io_uring/poll: cache req->apoll_events 2026-03-17 14:34:51 -06:00
poll.h io_uring: add wrapper type for io_req_tw_func_t arg 2025-11-03 08:31:26 -07:00
query.c io_uring/zcrx: declare some constants for query 2026-03-09 07:21:54 -06:00
query.h io_uring/query: drop unused io_handle_query_entry() ctx arg 2025-11-26 09:37:10 -07:00
refs.h io_uring: Add SPDX id lines to remaining source files 2026-02-04 07:23:45 -07:00
register.c io_uring: switch struct io_ring_ctx internal bitfields to flags 2026-03-16 15:32:59 -06:00
register.h io_uring: temporarily disable registered waits 2024-11-15 09:58:34 -07:00
rsrc.c io_uring: switch struct io_ring_ctx internal bitfields to flags 2026-03-16 15:32:59 -06:00
rsrc.h io_uring/rsrc: replace reg buffer bit field with flags 2026-02-10 05:26:15 -07:00
rw.c io_uring/rw: use cached file rather than req->file 2026-03-17 14:35:00 -06:00
rw.h io_uring: add wrapper type for io_req_tw_func_t arg 2025-11-03 08:31:26 -07:00
slist.h io_uring: Add SPDX id lines to remaining source files 2026-02-04 07:23:45 -07:00
splice.c io_uring: don't include filetable.h in io_uring.h 2025-09-08 13:20:46 -06:00
splice.h io_uring/splice: open code 2nd direct file assignment 2024-10-29 13:43:28 -06:00
sqpoll.c Convert 'alloc_obj' family to use the new default GFP_KERNEL argument 2026-02-21 17:09:51 -08:00
sqpoll.h io_uring/sqpoll: switch away from getrusage() for CPU accounting 2025-10-22 10:51:20 -06:00
statx.c allow incomplete imports of filenames 2026-01-13 15:18:07 -05:00
statx.h
sync.c io_uring/sync: validate passed in offset 2026-01-21 11:50:59 -07:00
sync.h
tctx.c io_uring: switch struct io_ring_ctx internal bitfields to flags 2026-03-16 15:32:59 -06:00
tctx.h
timeout.c io_uring: switch struct io_ring_ctx internal bitfields to flags 2026-03-16 15:32:59 -06:00
timeout.h io_uring/timeout: migrate reqs from ts64 to ktime 2026-03-09 07:21:54 -06:00
truncate.c io_uring: finish IOU_OK -> IOU_COMPLETE transition 2025-05-21 08:41:16 -06:00
truncate.h io_uring: add support for ftruncate 2024-02-09 09:04:39 -07:00
tw.c io_uring: mark known and harmless racy ctx->int_flags uses 2026-03-16 15:33:10 -06:00
tw.h io_uring: split out CQ waiting code into wait.c 2026-01-22 09:21:16 -07:00
uring_cmd.c io_uring/uring_cmd: allow non-iopoll cmds with IORING_SETUP_IOPOLL 2026-03-16 16:14:14 -06:00
uring_cmd.h io_uring/cmd: remove struct io_uring_cmd_data 2025-07-18 12:34:56 -06:00
wait.c io_uring: split out CQ waiting code into wait.c 2026-01-22 09:21:16 -07:00
wait.h io_uring: introduce callback driven main loop 2026-03-16 16:15:00 -06:00
waitid.c io_uring/waitid: fix KCSAN warning on io_waitid->head 2026-01-19 19:55:30 -07:00
waitid.h io_uring: move cancelations to be io_uring_task based 2024-11-06 13:55:38 -07:00
xattr.c Convert 'alloc_obj' family to use the new default GFP_KERNEL argument 2026-02-21 17:09:51 -08:00
xattr.h
zcrx.c io_uring/zcrx: consolidate dma syncing 2026-04-01 10:21:12 -06:00
zcrx.h io_uring/zcrx: add a struct for refill queue 2026-04-01 10:21:12 -06:00