-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmrjrZ0QHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpl3xEACV0gok/rxtzKB/z2gedIdA1NjY488cNdH3
UP+YpoTNm4GE/A9bfmTLWjGaQlxmDSdlM8cmQzAKTjHNiGNFK/Pv+fqWzaz38Aet
tUrxby2748CC3oin4BVxdTLqgSg35Yaw2S/zM/Zsxt3eMHtOdWZZYo8FKkvdf7fp
DR166vphOGQFu598Allrd4OTXj+TLd6EFlZ7WSN7fq2a8193XuIvfH5HRkDEUbVB
GA1wlltke/zildACho1MldBDkQYeGGGVk/J/1V5P8sB7TRrzyiRbO8RyfvwSR6OH
ChNeVXA/iLjwJA9HVX7+q+VnVQRK/EgIVZZoum+BG9WO9ZcUaelxgF3ZIdYc0Pj0
MDwg6iL4oWZKAqdb1jCYpjOqT3avp+oSQSD9sGpnzSmta2sMtQyulo00UGMBDuBP
xx8zzeggbuJymFXrCTAZ7Ek1EUNZh3qcWgdUOhxHV+R0s9ovyZszOdGV4dLNm0wX
aBO375d4O9FutZRzUA5vxEHbBue+S3maPkW2txn4WlnJNGRDIN/GyKoY6c5oyogy
ApYQ2Gqlbvf7VOPiYVADhycVNUzcolf8EBOXu8zFvMTA8aXLwXalrIaEurnhpE51
EQaPIZFXH4/dj6noHpgvYaAGFAFXpSSSXVIkG8a0CuhkDjrf6vU+dbt4PVhWP3xT
BSFU97FGbw==
=i457
-----END PGP SIGNATURE-----
Merge tag 'block-7.4-20261029' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull more block updates from Jens Axboe:
"Bigger than usual for an end-of-merge-window pull request, main reason
being that the MD side I had missed a pull on, and both NVMe and block
got a good pile of LLM driven fixes where some of them may as well
just go in now rather than wait a full cycle. This contains:
- NVMe fixes via Keith:
- nvme-tcp fallout from the parallel queue allocation: initialize
pf_cache_lock and the header digest before requests are set up,
and don't dereference a NULL queue->ctrl on reconnect
- nvme-fabrics queue mapping fixes: reset io_queues on reconnect,
limit each queue type to the online CPUs, and clear a poll map
with no poll queues
- nvme-tcp io_work priority inversion with send_mutex held
(Seiji), error recovery on read_sock failure and skipping TLS
control records while waiting for the ICResp (Martin B), TLS
handshake timeout validation, and an address attribute race
- nvme-rdma NULL metadata_sgl dereference with T10-PI
- nvme-pci early boot hang with the quirks parameter
- nvme-multipath ANA log bounds underflow, and an nvme-fc
discovery reference leak
- nvmet hardening: rdma inline SGL out-of-bounds write and short
connect requests, fc use-after-free in LS create connection and
teardown on admin queue failure, pci-epf transport error
reporting and SGL length checks, zns NUMD wrap, fc stale read
data, auth state and status fixes, ports list and qid_max
locking, rdma double puts and irq-safe locking, and a double
free on metadata SGL allocation failure
- nvmet fixes from Josef: big discard length truncation, NULL
derefs for PCI endpoint targets and debugfs host_traddr, stop
listening when a tcp port is removed, and don't BUG() in
nvme-loop when a scan runs beside a reset
- MD fixes via Yu Kuai:
- raid5 validation: reject too few disks, invalid reshape disks, a
per-device size below one chunk, and an inconsistent reshape
state from the superblock instead of BUG(), plus journal payload
size checks
- raid5 error path and lifetime fixes: extra_page leak and NULL
derefs in free_conf/setup_conf, draining released_stripes before
destroying the cache, RCU for the big_stripe_tree lookup, and
per-array io_unit slab names
- md core: reject v0.90 arrays with too many devices, validate
bblog_size, take reconfig_mutex when enabling PPL, fix the
mddev->private NULL deref when stopping an array under IO, and
don't hand out the array before mddev->kobj is added
- raid10 missing barrier wakeup, raid0 device count validation, a
bogus mddev_unlock() in lbs_store(), and shared helpers
- blk-mq fixes from Josef:
- don't trust the bio cookie in blk_mq_poll()
- keep tag sharing fair after a queue update
- don't bring back flags of a completed request from the timeout
handler
- honor a restarted timer
- initialize the dma state for P2P
- quiesce a queue joining a quiesced tag set
- set up and exit hctxs under tag_list_lock
- free the tag bitmaps in the SRCU callback
- blk-cgroup and friends from Josef:
- keep the old blkg while reassociating a bio
- free struct ioc after a grace period
- don't truncate v1 iops limits
- fix root io.stat on 32-bit
- don't free a blkg with its punt work queued
- flush iolatency enable_work after going offline
- pass dead group I/O counts on and make reset_stats reset them
- don't enter reclaim under rq_qos_mutex
- Other core fixes from Josef:
- a disk use-after-free in sync_bdevs()
- don't losing the status of a failed flush request
- 64-bit sector math for mac partitions
- apply the segment size to the integrity buffer
- blk-iocost: abort a weight transfer on invalid donation weights,
and publish hweights as one atomic value
- Fix a NULL deref and use-after-free in __blk_crypto_evict_key(),
reachable via dm-inlinecrypt
- Fix recursive i_rwsem locking in blkdev_write_iter(), from two
fixes for the same race landing in the window
- Advance the iterator after copy-in for SG_DXFER_TO_FROM_DEV, fixing
an infinite loop in blk_rq_map_user_iov()
- Check the open mode for LOOP_SET_STATUS in the compat ioctl
- rnbd-srv use-after-free and kobject fixes
- drbd timer and req_lock fixes
- zloop cleanups and error path fixes
- cdrom header and kernel-doc cleanups"
* tag 'block-7.4-20261029' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (109 commits)
blk-crypto: fix NULL deref and UAF in __blk_crypto_evict_key()
zloop: check that the zone backing file support ->{read,write}_iter
zloop: fput file after failed zone init
zloop: share more code for opening a zone backing file
zloop: remove a duplicate call to zloop_get_block_size
block: fix kernel-doc warnings in blk-map.c and blk-mq-dma.c
blk-iocost: publish hweights as one atomic value
blk-iocost: abort weight transfer on invalid donation weights
rnbd-srv: publish the sess_dev at the end of OPEN
rnbd-srv: fix use-after-free on two CLOSE messages
block/rnbd-srv: drop the sess_dev kobject unlocked
block/rnbd-srv: don't free the device on a sysfs error
drbd: take req_lock to skip the initial sync as peer
drbd: stop the start_resync timer of a dying device
block: apply the segment size to the integrity buffer
blk-mq: free the bitmaps of tags in the SRCU callback
block: partitions/mac: compute the sectors in 64 bits
blk-mq: set up a queue under tag_list_lock
blk-mq: exit the hctxs of a queue under tag_list_lock
blk-mq: quiesce a queue that joins a quiesced tag set
...
-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmrbkFsQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpiHpD/9MdThoGJTZ3mKgaZPd7lOgkxWL51CY/ULB
X9zzyfHP9hpXxwXqpSGm8XhN7cmSDXGaCYvhVPS2YQQv0458s1UGMWDi8PJNulFS
6M8ng6Hex2z/Uq+u6vMkruz+hH76c3ajo6kGOhJnWye+/Rq1bqqFcYBWMpLBE/yj
lp8JGG92HyMnsIo5gzBHbUoNEc+X0+jz88mk0JRZoONJD4cNb1S1bW5v+uAk0Nvc
/qPE9AXU2DnngvbzhfKGza/4Lkx9WKBXFmxJSiG9ZH6GJn3yndBNBi7RAODku+/E
i+SiukHPDhn3M5cWZuJ8ilTpVWRCnnsbhavM1PN2tvcW6IfZrAz+m321+JVkzKu0
cLUDvmz2WS0+PcodXGn3KcGVYt7vAjtQVZwZLbq2012u5xSvZN/VaeiVDIRr1uC/
hQPiD51ZpsTt/2lAyrFGmLBsskNWc0Xhd6ePX7euNCdhfHj5UuWKcXUiQeeGiUvT
X2JyQAJSY9VZ6eO02w77bta0II+3SSnZlA6xVmE3+Nwv2z6qr3b31gcfMIiap3Tu
jUkKvDuB9Dw9hhgLr1t6BCuaPS3ktyd1F3dBvbs8YXyyeworU2KGtSF7mraJxts8
GCH+DOAR1j29qMi4/bjhxOKQYAKzhWbptCiJgyReL6V+uFO6JxAeWn3qn36mjezp
BuHNTX1A5g==
=hlkK
-----END PGP SIGNATURE-----
Merge tag 'for-7.4/io_uring-dmabuf-20261023' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull io_uring dma-buf support from Jens Axboe:
"This adds native dma-buf support to the I/O path, exposed through
io_uring. It's been in development for a while and should be ready to
go now.
- dma-buf file I/O infrastructure: a dma_buf_io_ctx created through a
new ->init_dma_buf_io_ctx file operation, and a dma_buf_io_map
holding the device DMA mapping. The two are split so a mapping can
be invalidated and re-established underneath a live context, with
the map counting in-flight requests so it can be torn down safely.
- A new iov_iter type for dmabuf maps, and a dma map backed bio type
for the block layer, with raw block device support on top.
- nvme-pci backend implementing dma-buf backed requests, including
SGL support.
- io_uring dmabuf backed registered buffers, built on a reworked
buffer registration structure with an extended update path, regbuf
import flags, an uncloneable flag for buffer types that can't be
cloned, and cloning support for registered dma-bufs"
* tag 'for-7.4/io_uring-dmabuf-20261023' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
io_uring/dma-buf: don't fail fixed buffers without dma-buf support
nvme-pci: detach dma-bufs only on removal
block: reject dma-bufs larger than 4GiB
io_uring/dma-buf: allow cloning registered dma-bufs
dma-buf-io: document dma_buf_io_init_map
dma-buf-io: fix typos in kernel docs
block: fix indention for dma-buf checks
io_uring/rsrc: add dmabuf backed registered buffers
io_uring/rsrc: add regbuf import flags
io_uring/rsrc: add uncloneable regbuf flag
io_uring/rsrc: extend buffer update
io_uring/rsrc: introduce buf registration structure
nvme-pci: add SGL support for the dmabuf path
nvme-pci: implement dma-buf backed requests
nvme-pci: rename nvme_pci_sgl_set_data to nvme_pci_dma_iter_set_sgl
block: add dma-buf support for raw bdev
block: introduce dma map backed bio type
block: always adjust bi_offset on bio_advance_iter
iov_iter: add iterator type for dmabuf maps
dma-buf: introduce initial file I/O infrastructure
Commit 2fec4fd97c ("block: take i_rwsem for the direct I/O write
fallback") takes i_rwsem around blkdev_buffered_write() in the direct
write fallback to close a race with set_blocksize(). Commit
1b1be9ea4e ("block: take the inode lock for buffered write fallback")
fixes the same race by taking i_rwsem inside blkdev_buffered_write()
instead. With both commits applied, the fallback path takes i_rwsem
recursively:
blkdev_write_iter()
inode_lock_shared(bd_inode)
blkdev_buffered_write()
inode_lock_shared(inode)
syzbot reports this as:
WARNING: possible recursive locking detected
syz.0.40/6005 is trying to acquire lock:
ffff888024956f68 (&sb->s_type->i_mutex_key#10){++++}-{4:4}, at: blkdev_buffered_write+0xd7/0x160 block/fops.c:738
but task is already holding lock:
ffff888024956f68 (&sb->s_type->i_mutex_key#10){++++}-{4:4}, at: blkdev_write_iter+0x6c5/0x810 block/fops.c:815
rw_semaphore read locks are not recursive: once a writer is queued,
new readers block behind it. If set_blocksize() calls inode_lock()
between the two acquisitions, it waits for the first read lock to be
released, while the second inode_lock_shared() waits behind
set_blocksize(), and the task deadlocks.
Drop the outer locking in the fallback.
Fixes: 1b1be9ea4e ("block: take the inode lock for buffered write fallback")
Reported-by: syzbot+bd975ef54294bec73f0e@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=bd975ef54294bec73f0e
Cc: Tal Zussman <tz2294@columbia.edu>
Cc: Julian Sun <sunjunchao@bytedance.com>
Assisted-by: LLM
Signed-off-by: Zhenxian Ma <mzx199711@gmail.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20261022062513.77372-1-mzx199711@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmrYsn8QHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgppmpEACRjRrpwbzCY47zpMyXICIFsUTqzs9MHKLh
VZERo2w1Xdpu0GGLZAnNwbBB96pCsob5/Ipq4EaleSObkhHDYjLcev0vnLITf7Hm
Ky5ogvofLmcpklYGP04TewoYko7tV8+KtcP3RyHpQvyTZs63dSvUeJsnZiHacc51
Un9gB9GwbJE9EWOmF8JaHl58+qZVJXKnSnAJH/C17KYn5F9OcGsrbyDXODmE0DMT
F44A0ZTMzDV3IJfPXg9rjc8rD5eTX+/xdbFBVFCXUofi3D3DgWHk6qVTVh6mmACp
l8BfmfjohpJmz9rsT7FqXq5Mey4WxEhepjvat/uE97SR6WDPszruKVHfT4E47jNW
r/hP2ogtOJAP04hklaHYeJy3gaOA90YmhZU7Idu6iOVCUuZgqFgHbKfHRp1zmMMx
PPaenKrAQA0oBb46IsgZA9rAbF6xd8E5/B36dGDe33JYXwQGnJH046IIE28Ytq1R
740vmq+4EJ2mCrhB8SR/7GnndTTWL/RAmhw8PkugWl9YB9uxrcOIppiyjvorKTrD
yEKuLJW9hYNkiCz/tsE/ZOfwLaBP03JPSvx4si4g1nyKUgEF3jxnD55vkXb2kQll
Xxa0YcQP0Bfn0oQWljzS6gmiTIcbwy2jPCu08esbo/rig5tfLjdTZmTH0oZmy4LH
9kq45JFKhA==
=8Czt
-----END PGP SIGNATURE-----
Merge tag 'for-7.4/block-20261021' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Apple M4 (t8132) ANS2 NVMe support
- nvme-multipath sysfs attributes for path_state, iopolicies, and
fail_on_ctrl_loss, plus path state evaluation fixes
- nvme-tcp parallelized I/O queue allocation and startup, and using
the caller's network namespace
- nvmet passthru UAF fix and rework, PR preempt use-after-free
fixes, auth hardening, and traffic based keep alive support
- DH-HMAC-CHAP secret attributes readable by root only
- Various other fixes and cleanups
- swim driver overhaul and fixes, plus floppy fixes, via a pull from
Denis
- Block device buffered and direct I/O rework from Tal: use
iomap_dirty_folio, take i_rwsem for the write fallback and splice
read, honor IOCB_NOWAIT in the buffered read path, and reject atomic
write fallback to buffered I/O
- Large zone revalidation rework: serialize revalidation, handle
capacity changes by dropping and retrying zone write plugs, propagate
read-only and offline conditions, and treat offline and read-only
zones as dead
- ublk robustness work around canceled FETCH rounds and going live over
canceled io commands, dropping the default I/O scheduler, and a batch
of selftest additions
- blk-iocost flush cost support and zone append accounting
- rust block fixes for Send/Sync bounds on GenDisk and TagSet, and
import style cleanups
- Atomic write fixes capping size by the PI buffer constraints and
failing short atomic pins
- Skip the redundant flush for O_DSYNC direct writes, and only use
REQ_FUA when the device supports it
- Store GPT attributes as a raw value
- Remove support for swapping loop fds
- Various other fixes and cleanups all over
* tag 'for-7.4/block-20261021' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (144 commits)
block: restore the iterator when rejecting a misaligned bio
blk-mq: clear stale inactive flag on hctx reuse
block, bfq: initialize active lists for all actuators
zloop: print device that had errors
ublk: pin module while release work is pending
blk-mq: drop the tag_set argument from blk_mq_free_rq_map/tags
blk-mq: share one tags_srcu instance across all tag sets
blk-iocost: fix stale comment in ioc_rqos_throttle()
blk-iocost: account zone append completions in latency stats
blk-iocost: charge zone appends as writes
blk-iocost: add flush cost support with the flushiops model parameter
ublk: use global index for reported zones
selftests: ublk: add test for going live over canceled io commands
selftests: ublk: stop a batch thread when PREP_IO_CMDS fails
selftests: ublk: move the control command helpers into ctrl.c
ublk: refuse to go live after an io command was canceled
ublk: let STOP_DEV cancel the server's commands before its release
ublk: allow a NULL io->cmd in the cancel callback
ublk: reset the FETCH round under ub->mutex
ublk: reset the FETCH round in release also without a disk
...
The dma-buf offset is set in the 32-bit bvec_iter bi_offset. Reject
buffers that can carry an offset that would wrap.
Fixes: 839231d193 ("block: add dma-buf support for raw bdev")
Signed-off-by: Keith Busch <kbusch@kernel.org>
Link: https://patch.msgid.link/20261021184658.644645-2-kbusch@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Please consider pulling these changes from the signed vfs-7.4-rc1.iomap tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCatHrlgAKCRCRxhvAZXjc
okLFAQDtL+joq4F75iD2qaULpmj7kWby23E7u88ccIVUONbpWgD+L0S5lPCd4upL
fGyRMlQOybiRX1uHH13xGPU9wZ2Iggk=
=ZSC6
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.4-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull iomap updates from Christian Brauner:
"This contains the iomap changes for this cycle.
The bulk is lazy bounce buffering for checksummed reads. XFS bounce
buffers every direct read on a device that requires stable pages for
checksumming so that userspace modifying the buffer while the read is
in flight can't corrupt the data and fail the checksum.
That burns DRAM bandwidth and CPU for copies that are almost never
needed and, since the bounce buffer was a bio_vec, produced smaller
and unaligned I/Os with 4k pages, i.e. 1 MB minus 4k.
Now the read goes to the user buffer first and only a checksum failure
allocates a bounce buffer and reads again. Integrity verification
moves out of iomap into the filesystem so it can handle errors its own
way, T10 PI handling for direct I/O moves into ->submit_io() to match
buffered I/O, and a sysfs knob restores the old behaviour for the rare
application that really does modify in-flight buffers.
The block side gains bio_prepare_reissue(), exports
fs_bio_integrity_{alloc,free}(), caps atomic writes by the PI buffer
size and fixes an integer overflow in max_integrity_io_size() and a PI
verification bug with an initial bvec offset. The old read side bounce
buffering in the block layer is removed. This is a branch shared with
the block tree and may also reach you through the block pull request.
Three error handling regressions are fixed:
- A failed ->writeback_submit() left wpc->wb_ctx pointing at an
already completed ioend which was then submitted a second time and
corrupted ip->i_ioend_list on XFS
- iomap_fiemap() lost an iteration error when it flushed the pending
extent and marked a truncated list as complete
- A failed direct I/O bio lost its error when the sub-block tail
zeroing succeeded. An XFS error tag to inject the writeback failure
comes with it.
iomap_dio_hole_iter() respects the block size alignment when
iov_iter_zero() stops short at a page boundary, which btrfs with 8k
blocks on 4k pages hit as an assertion since it disables page faults
during direct I/O. The homegrown deferral of error completions to task
context is replaced with bio_complete_in_task(). Plus an unused
parameter, two typo fixes and a kernel-doc fix"
* tag 'vfs-7.4-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (34 commits)
fs/iomap: document ret parameter of iomap_iter_continue
iomap: follow the alignment requirement for iomap_dio_hole_iter()
iomap: don't lose a failed direct I/O bio's error when zeroing the tail
iomap: don't lose a fiemap iteration error when emitting the last extent
xfs: add an error tag to inject a ->writeback_submit() failure
iomap: don't resubmit an ioend after ->writeback_submit() failed
iomap: use bio_complete_in_task for buffered simple dio read failures
iomap: use bio_complete_in_task for buffered write failures
iomap: use bio_complete_in_task for buffered read failures
block,iomap: remove the old read side bounce buffering support
xfs: log a message at mount time when using integrity protection
xfs: add error injection for lazy bounce buffering
xfs: add support for lazy direct read bounce buffering
iomap,xfs: move integrity verification to the file system
xfs: use BIO_COMPLETE_IN_TASK for bounce buffered read I/Os
iomap: better read bounce buffering support
block,iomap: fix protection information verification with initial bvec offset
xfs: move PI generation into xfs_submit_zoned_bio
iomap,xfs: move T10 PI handling for direct I/O into ->submit_io
iomap: add a IOMAP_IOEND_INTEGRITY flag
...
Commit c0e473a0d2 ("block: fix race between set_blocksize and read
paths") added shared inode locking around ordinary buffered writes,
but missed the direct-I/O fallback path.
Move the lock into blkdev_buffered_write() so both paths serialize
with set_blocksize().
Also correct the comment claiming that this path takes invalidate_lock;
it only takes i_rwsem.
Fixes: c0e473a0d2 ("block: fix race between set_blocksize and read paths")
Signed-off-by: Julian Sun <sunjunchao@bytedance.com>
Link: https://patch.msgid.link/20261015091105.1315605-3-sunjunchao@bytedance.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
blkdev_write_iter() rejects NOWAIT buffered writes, but does not
enforce this restriction when direct I/O falls back to buffered I/O.
Successful fallback waits for writeback, violating NOWAIT semantics.
Reject NOWAIT requests in blkdev_buffered_write() with -EAGAIN.
Fixes: c35fc7a5ab ("block_dev: support RFW_NOWAIT on block device nodes")
Signed-off-by: Julian Sun <sunjunchao@bytedance.com>
Link: https://patch.msgid.link/20261015091105.1315605-2-sunjunchao@bytedance.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add a simple proxy implementation of init_dma_buf_io_ctx() forwarding
the call to a new struct block_device_operations operation. Also reject
dma-buf backed iterators for buffered IO.
Reviewed-by: Christoph Hellwig <hch@lst.de>
[pavel: reject dma-buf without O_DIRECT]
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://patch.msgid.link/c01c4709dc55a12fb0f2fe1ec847b5f1f0c92789.1791975048.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Premapped buffers don't require a generic bio_vec since these have
already been dma mapped. Repurpose the bi_io_vec space to store dmabuf
maps as they are mutually exclusive.
The bio splitting differs from the normal path because it's already
pre-mapped and for the block layer it's just an offset into the dma-buf.
The actual segmentation is only available to the importer driver and not
the block layer, however, it doesn't contain alignment gaps and we don't
have to check it. For the same reason we can't precisely split by the
number of segments, but we use ->min_seg_shift stored in the map to
calculate the minimum number of bytes a request consisting of
lim->max_segments full segments can cover and split by that. It's
stricter and can add extra splitting. E.g. for a {4K, 4G} segmentation,
the min segment size is 4K, and we'll split it into bios of
(4K * lim->max_segments) bytes each, but it should be good enough for
now to cover the most popular use cases.
Suggested-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/39b964cf7510f42755188563da63d3d5acaf2aa9.1791975048.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Since commit 2729a60bbf ("block: don't silently ignore metadata for
sync read/write"), blkdev_direct_IO() sends every IOCB_HAS_METADATA
request to __blkdev_direct_IO(). Remove the now unreachable metadata
mapping in __blkdev_direct_IO_async() and the unmap in its completion
handler.
No functional change.
Assisted-by: Claude:claude-fable-5
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20261009-blkdev-fixes-v6-9-307939c387df@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
blkdev_direct_write() turns an -EBUSY from page cache invalidation into
a 0 return, so an IOCB_ATOMIC write is retried in full through
blkdev_buffered_write(), with no atomicity guarantee.
Skip the buffered fallback in blkdev_write_iter() for IOCB_ATOMIC, as
it already does for IOCB_NOWAIT, so the -EBUSY case returns -EAGAIN and
the caller retries, matching __iomap_dio_rw().
ext4 has the same fallback and only warns in it. For block devices the
fallback can be skipped before any I/O is submitted, so fail early
instead.
Fixes: caf336f81b ("block: Add fops atomic write support")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260802-blkdev-fixes-v1-0-a82fc549fd74%40columbia.edu?part=2
Assisted-by: Claude:claude-fable-5
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: John Garry <john.garry@linux.dev>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20261009-blkdev-fixes-v6-6-307939c387df@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
On a partial page pin, __blkdev_direct_IO_simple() and
__blkdev_direct_IO_async() submit what was pinned with REQ_ATOMIC set
and leave the rest to the buffered fallback, tearing an IOCB_ATOMIC
write.
This can be triggered deterministically. A 16K pwritev2(RWF_ATOMIC)
whose last page is PROT_NONE, on a scsi_debug device with atomic_wr=1,
completes short with only three of the four pages written, violating
RWF_ATOMIC semantics.
Make bio_iov_iter_get_pages() release the pins and return -EINVAL when
a REQ_ATOMIC bio doesn't cover the whole iterator, since an atomic
write is submitted as a single bio and a short one would be torn. That
covers iomap as well, where a partially unmapped buffer could trip the
WARN_ON_ONCE() in iomap_dio_bio_iter_one(). The async block device path
currently sets REQ_ATOMIC after pinning, so set it before, and move
REQ_NOWAIT along with it.
Fixes: caf336f81b ("block: Add fops atomic write support")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260802-blkdev-fixes-v1-0-a82fc549fd74%40columbia.edu?part=2
Assisted-by: Claude:claude-fable-5
Reviewed-by: John Garry <john.garry@linux.dev>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20261009-blkdev-fixes-v6-5-307939c387df@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
blkdev_read_iter() takes inode_lock_shared() unconditionally around
filemap_read(). Unlike blkdev_write_iter(), it does not reject
IOCB_NOWAIT for buffered I/O, so a non-blocking read, or the buffered
tail of a short IOCB_NOWAIT direct read, blocks behind set_blocksize()
holding i_rwsem across sync_blockdev().
A preadv2(RWF_NOWAIT) issued while another thread changes the block
size with a dirty page cache blocks for as long as sync_blockdev()
takes, 4 to 6 seconds on a scsi_debug device with delay=5.
Use inode_trylock_shared() for IOCB_NOWAIT and return the bytes the
direct path already read, or -EAGAIN if none, when the lock is
contended, preserving NOWAIT semantics.
Fixes: c0e473a0d2 ("block: fix race between set_blocksize and read paths")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260802-blkdev-fixes-v1-0-a82fc549fd74%40columbia.edu?part=2
Assisted-by: Claude:claude-fable-5
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20261009-blkdev-fixes-v6-4-307939c387df@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
def_blk_fops wires ->splice_read directly to filemap_splice_read(),
which allocates folios based on mapping_min_folio_order() without any
lock against set_blocksize(). A splice from a block device can race
set_blocksize() raising the minimum folio order and insert a folio that
is too small for the mapping. blkdev_read_iter() wraps filemap_read()
in inode_lock_shared() for this reason, but the splice path was missed.
Splicing from a block device while toggling the block size between 512
bytes and 64K with BLKBSZSET hits this within seconds on a
CONFIG_DEBUG_VM kernel:
page dumped because: VM_BUG_ON_FOLIO(folio_order(folio) < mapping_min_folio_order(mapping))
kernel BUG at mm/filemap.c:858!
Oops: invalid opcode: 0000 [#1] SMP NOPTI
RIP: 0010:__filemap_add_folio+0x51c/0x570
Call Trace:
filemap_add_folio+0x64/0x140
page_cache_ra_order+0x1dd/0x3d0
filemap_get_pages+0x153/0x760
filemap_splice_read+0x13f/0x300
splice_file_to_pipe+0xc0/0xd0
do_splice+0x6a8/0x890
__do_splice+0xb0/0x210
__x64_sys_splice+0x80/0x100
do_syscall_64+0x10e/0x520
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Take inode_lock_shared() around filemap_splice_read(), like the read
path does.
Fixes: 3c20917120 ("block/bdev: enable large folio support for large logical block sizes")
Assisted-by: Claude:claude-fable-5
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20261009-blkdev-fixes-v6-3-307939c387df@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Commit c0e473a0d2 ("block: fix race between set_blocksize and read
paths") closed a race between set_blocksize() and block device I/O: with
large sector size support, set_blocksize() can change i_blkbits and the
mapping's minimum folio order while a concurrent reader still holds a
folio of the old, smaller order, leading to crashes. In particular, it
made blkdev_write_iter() wrap buffered writes in inode_lock_shared().
However, the direct I/O fallback path was missed in that conversion.
blkdev_write_iter() passes blkdev_buffered_write() as an argument to
direct_write_fallback() with no lock held. A direct write that completes
only partially then finishes as a buffered write with no protection.
This can cause a BUG by racing partial direct writes against
ioctl(BLKBSZSET). Writer threads issue O_DIRECT pwritev() with a
two-segment iovec whose second segment is an unreadable PROT_NONE
mapping. The direct path then writes the first segment, fails to pin the
second, and returns short, entering the fallback. A second thread keeps
toggling the second segment's protection so that some fallbacks get past
fault_in_iov_iter_readable() and reach the page cache, a third thread
populates the page cache with folios of the current block size via
pread() and readahead(), and a fourth thread toggles the block size
between 512 bytes and 64K with BLKBSZSET. The minimum folio order only
moves with block sizes above PAGE_SIZE, i.e. with
CONFIG_TRANSPARENT_HUGEPAGE raising BLK_MAX_BLOCK_SIZE to 64K.
On a CONFIG_DEBUG_VM kernel this yields the following BUG:
page dumped because: VM_BUG_ON_FOLIO(folio_order(folio) < mapping_min_folio_order(mapping))
kernel BUG at mm/filemap.c:858!
Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
RIP: 0010:__filemap_add_folio+0x860/0x8d0
Call Trace:
filemap_add_folio+0xc9/0x1f0
__filemap_get_folio_mpol+0x240/0x660
iomap_write_begin+0xa87/0xd70
iomap_file_buffered_write+0x304/0x6a0
blkdev_write_iter+0x255/0x510
do_iter_readv_writev+0x23d/0x3c0
vfs_writev+0x211/0x7d0
do_pwritev+0x121/0x190
do_syscall_64+0x121/0x630
entry_SYSCALL_64_after_hwframe+0x77/0x7f
The same workload also trips WARN_ON_ONCE(pos >= folio_pos(folio) +
fsize) in iomap_trim_folio_range().
Fix this by calling blkdev_buffered_write() in the fallback path under
inode_lock_shared(), matching the plain buffered-write branch. With the
fix the same workload runs clean.
A short IOCB_NOWAIT direct write reaches the same fallback. Taking
i_rwsem there can now block behind set_blocksize(), and the fallback
already blocks on writeback of the data it copied in
direct_write_fallback(). blkdev_write_iter() already rejects a purely
buffered IOCB_NOWAIT write with -EOPNOTSUPP, so do not enter the
fallback for IOCB_NOWAIT at all: return the bytes the direct path
already wrote, or -EAGAIN if none, and let the caller retry.
The reproducer used was written by an LLM, and is available at [1].
[1] https://gist.github.com/tzussman/69d06bc57d42a42989eb038b1b5aeb74
Fixes: 3c20917120 ("block/bdev: enable large folio support for large logical block sizes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260730-blk-dontcache-v7-0-3e8e6850068d%40columbia.edu?part=5
Assisted-by: Claude:claude-fable-5
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20261009-blkdev-fixes-v6-2-307939c387df@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
With CONFIG_BUFFER_HEAD=n, block devices are written back through iomap,
but def_blk_aops uses filemap_dirty_folio, which only sets PG_dirty. It
does not set the per-block dirty bits in the folio's iomap_folio_state,
so iomap_writeback_folio() finds no dirty range, submits no I/O and
clears PG_dirty, resulting in data loss.
Other iomap users set .dirty_folio to iomap_dirty_folio, which marks the
folio's blocks dirty before calling filemap_dirty_folio().
This is only observable with block size < folio size. With a single
block there is no iomap_folio_state to get out of sync and
iomap_writeback_folio() marks the whole folio dirty itself. For a
page-aligned device, this may require using the BLKBSZSET ioctl to set
the block size, which requires CAP_SYS_ADMIN. A device whose size is not
page aligned already gets a sub-page block size from
set_init_blocksize(), so no ioctl and no privilege is needed.
To reproduce, on a device with a sub-page block size, write a known
pattern with O_DIRECT, mmap the same range, store to it, msync() and
fsync(), then read it back with O_DIRECT. A reproducer is available at
[1].
[1] https://gist.github.com/tzussman/18ab05cba4b3fdc79cce0a69d1fd05b4
Fixes: 925c86a19b ("fs: add CONFIG_BUFFER_HEAD")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://sashiko.dev/#/patchset/20260730-blk-dontcache-v7-0-3e8e6850068d%40columbia.edu?part=5
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20261009-blkdev-fixes-v6-1-307939c387df@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
IOCB_HAS_METADATA leaves iocb->private as a uio_meta, while
iocb_bio_iopoll() expects a bio. Since 2729a60bbf, metadata I/O
always uses __blkdev_direct_IO(), which never publishes the bio for
polling, so IOPOLL calls bio_poll() on the uio_meta.
Reproducer: io_uring IOPOLL plus a PI attribute on an integrity device
(scsi_debug dif=1 dix=1). KASAN reports:
Oops: general protection fault, probably for non-canonical
address 0xdffffc0000000004
KASAN: null-ptr-deref in range
[0x0000000000000020-0x0000000000000027]
RIP: 0010:bio_poll+0x9d/0x4b0
Call Trace:
iocb_bio_iopoll+0x5b/0x80
io_uring_classic_poll+0xf3/0x140
io_do_iopoll+0x4e3/0xc00
io_iopoll_check+0x275/0x560
__do_sys_io_uring_enter+0x2e5/0x720
Return -EOPNOTSUPP when HIPRI is combined with user metadata.
Fixes: 2729a60bbf ("block: don't silently ignore metadata for sync read/write")
Assisted-by: LLM
Signed-off-by: Yang Xiuwei <yangxiuwei@kylinos.cn>
Link: https://patch.msgid.link/20260923093732.1504291-1-yangxiuwei@kylinos.cn
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Writes that require additional per-I/O allocations such as when using
file system level protection information can be limited to a maximum
size. Allow passing that to bio_iov_iter_get_pages.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
When a block device does not support FUA natively, the block layer
emulates it by adding a cache flush to every write bio. An O_DSYNC
direct write that spans N bios then costs N flushes, rather than the
single generic_write_sync() issued after the write completes.
Introduce blkdev_dio_fua() to decide when REQ_FUA is set. A synchronous
write can rely on generic_write_sync() when the device lacks FUA, so it
sets REQ_FUA only when bdev_fua() is true. An asynchronous write
completes in blkdev_bio_end_io() and cannot call the blocking
generic_write_sync(), so it keeps REQ_FUA (emulated when needed) to stay
durable.
Suggested-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Zhenxian Ma <mzx199711@gmail.com>
Signed-off-by: Zhenxian Ma <mazhenxian@xiaohongshu.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/tencent_A41247F29AC0971AE96B52964BE4491B440A@qq.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
For an O_DIRECT | O_DSYNC write, dio_bio_write_op() adds REQ_FUA to the
bio, so the data is durable once the direct I/O returns. The
unconditional generic_write_sync() in blkdev_write_iter() then issues a
REQ_PREFLUSH that is redundant.
Skip it when the direct path already provided durability via FUA. A
need_sync flag, clear by default, is set only for buffered writes and
for the buffered fallback after a partial direct write.
Measured on a Seagate ST20000NM007D (20 TB, 7200 rpm, fua=1,
write_cache=write back), Linux v7.2.0-rc7, single-threaded pwrite()
loop opening the raw block device with O_WRONLY | O_DIRECT | O_DSYNC,
4 KiB writes for 60 s:
Sequential 4 KiB writes:
baseline patched
IOPS 119.7 7497.0
avg latency (us) 8357 133
p50 latency (us) 8346 127
p99 latency (us) 8368 395
p99.9 latency (us) 8728 569
Random 4 KiB writes (100 GiB span):
baseline patched
IOPS 156.1 666.4
avg latency (us) 6405 1500
p50 latency (us) 6186 1450
p99 latency (us) 16133 2285
p99.9 latency (us) 17250 9916
Signed-off-by: Zhenxian Ma <mzx199711@gmail.com>
Signed-off-by: Zhenxian Ma <mazhenxian@xiaohongshu.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/f847707d772a9225460406b5aecba030d7209b05.1786782832.git.mzx199711@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmqGD2AQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpomLD/9rr3SIQvdQRTnSCJlw1cFypSrnRhGAx1g8
v1F7UXuf4Abt4GIerbdXmBTx9zZRctpcOC41ReQk3YKkYuukuvna+Z6tBGf5I5+e
3SR3Zkrj2k4zyNC49g0/QkUJkTTS4tlsBFhNJtY3Q5BBIRMRd/sPvZ+cLnD7g/wH
3zM9IaEMk9xu9mF1nGI7LRl1qY0IOr4ynQpqohl5sEL3TQWgIa0r2B0jhGacBAvC
GGDwIwGDrM6oGwCn6mLN78O5odhLyTufIMNy3EnVw7BcDpLApcnJzYYjGzv+ElCM
WLBJwWYj+PYeoenuJuFhbRihbOPg5gZT3Mn8CUhLxUYzwZvIdFGWOjcGffD9sVu/
JO4nB6A+2AG/wgsb2YVKefhN86FoD0F/2hhdEdgZPJE6a+7+xeyyp9IzyQEWd0cg
Seo06fxE4pwF+dczXT3jzU3LXSZb/Jbh9hzu163VMSWbGYDCpDzfipqZGeN6DWE7
nG2NssAQtjtZrgdggYbZXtwecLZhu3cMj3jbhS4wSUzhQFW42yceJstCx4wpRhW9
m9EXjHQnRFMAb2TjUC3FzEncjbxL+uACjcKhTetd1hLbTI5I8KArZXjam1OdJOHV
urdtsru79LanQLAgRwWYPeKCZGNhX7I2o3OsmVDLQgHIDaZguTcPUVQs3wVui7Dk
IDmmrtwzpg==
=G1l4
-----END PGP SIGNATURE-----
Merge tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Enable Clang context analysis for the nvme host driver, adding
context annotations across core, fabrics, rdma, tcp and pci
- nvmet reservation state exposed through a new namespace-level
debugfs directory, plus ABI documentation for the host sysfs and
target configfs interfaces
- nvme-tcp host memory disclosure fixes on the read path: reject a
read that transferred too few bytes, don't accept C2HData based
on blk_rq_payload_bytes() alone, and fix the R2T case for a read
command
- Parallelize nvme-rdma I/O queue allocation and startup (Surabhi)
- Apple nvme fixes and quirks: page aligned admin queue buffers,
destroy the admin queue on removal, and various DMA/NVMMU
correctness fixes
- A large pile of nvmet and host fixes for out-of-bounds reads,
refcount/resource leaks, and NULL derefs across auth, zns,
passthru, pci-epf, rdma and configfs
- Various other fixes and cleanups
- MD updates via Yu Kuai:
- llbitmap reshape support, the large series wiring exact bitmap
mapping and reshape lifecycle through raid5 and raid10, growing
the page cache in place, and remapping checkpointed bits as
reshape progresses
- raid5 fixes for lockless max_nr_stripes and recovery_offset
accesses, a reshape deadlock with more failed devices than max
degraded, and bitmap batch counter consistency
- Atomic write handling for raid1/raid10, and removal of the
REQ_NOWAIT support from raid1/10/456
- raid5-ppl use-after-free fix in ppl_do_flush()
- A batch of smaller fixes across md core and the bitmap code
- s390/dasd ESE full-track write support and the surrounding
infrastructure, plus enabling CONTEXT_ANALYSIS for s390/block
- RWF_DONTCACHE support for block devices, built on new task-context
bio completion infrastructure, and wiring it up for the iomap and
buffer dropbehind writeback paths
- Async io_uring zone reset all, plus zone management command cleanups
allowing REQ_NOWAIT and tightening conventional zone rejection
- Block integrity refactoring: lift BIP_CHECK_FLAGS to the shared
header, handle nogenerate/noverify properly in fs-integrity, and drop
the blk-integrity.h include from bdev.c
- Split out a new blk_plug.h header
- ublk improvements: add UBLK_F_IO_DESC_SIZE, split request validation
from io_desc init, reject non-power-of-2 zone sizes in SET_PARAMS,
and a series of hardening fixes around map/unmap and auto buf reg
- null_blk cleanups and configfs serialization fixes
- nbd queue freeze removal on the setup paths, and a new
pre_defined_connections module parameter for pre-created devices
- blk-cgroup fixes for the race between policy activation and blkg
destruction, and accounting per-cpu stats over possible CPUs across
blk-stat, iolatency, iocost and kyber
- Various dio fixes: leak on metadata mapping error, validate user
space vectors during extraction, and set dma_alignment from the
backing file for loop and zloop direct I/O
- bio cleanups
- Various other fixes and cleanups all over
* tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (241 commits)
nbd: add pre_defined_connections module parameter for pre-created devices
nbd: remove queue freeze for newly created nbd from netlink path
nbd: factor out a nbd_genl_foreach_sock
nbd: skip queue freeze when setting size at device startup
nbd: remove queue freeze in nbd_add_socket
nbd: clear queue limits on disconnect
nbd: disallow NBD_SET_SOCK on an active device
nbd: simplify find_fallback() by removing redundant logic
blk-mq: add missing call to srcu_barrier() in blk_mq_free_tag_set()
block: mtip32xx: synchronize ioctls with device removal
ublk: avoid teardown retry loop on xarray allocation failure
null_blk: fix UBSAN shift-out-of-bounds when zone_size is 0 or overflows
block: don't include blk-integrity.h in bdev.c
xfs: avoid double deferrals for RWF_DONTCACHE writes
loop: Fix recently introduced lock inversion
block: set QUEUE_FLAG_DYING unconditionally in blk_mark_disk_dead()
swim3: Add missing MODULE_DESCRIPTION
selftests: ublk: add SET_PARAMS validation test
selftests: ublk: add helper for SET_PARAMS
ublk: reject non-power-of-2 zone sizes in SET_PARAMS
...
In preparation to supporting dma-buf backed iterators and bios,
introduce bio_iov_iter_set() which attempts to set up the bio directly
from the given iterator. For now, it only supports bvec and expects
users to check the result and fall back to other means if fails, but
later we'll add more types.
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/4686a0e47fc14f3f888967a80d45a6f66044f1e0.1785596451.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Block device buffered reads and writes already pass through
filemap_read() and iomap_file_buffered_write() respectively, both of
which handle IOCB_DONTCACHE. Enable RWF_DONTCACHE for block device files
by setting FOP_DONTCACHE in def_blk_fops.
For CONFIG_BUFFER_HEAD=y, writeback goes through buffer_head's
__bh_submit() which sets BIO_COMPLETE_IN_TASK on dropbehind folios. For
CONFIG_BUFFER_HEAD=n, writeback goes through iomap which handles it via
BIO_COMPLETE_IN_TASK on the ioend bio.
This support is useful for databases that operate on raw block devices,
among other userspace applications.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://patch.msgid.link/20260730-blk-dontcache-v7-5-3e8e6850068d@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The bio-based drivers don't necessarily check the alignment split, and
stacking block drivers don't always handle a misalignment detected after
submitting the bio. Validate user vectors against the device's
dma_alignment as the bio is built from the iov_iter, rejecting
misaligned early with -EINVAL.
Cc: stable@vger.kernel.org
Fixes: 5ff3f74e14 ("block: simplify direct io validity check")
Fixes: 7eac331869 ("iomap: simplify direct io validity check")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Link: https://patch.msgid.link/20260720201057.1862857-6-kbusch@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Convert block iomap_ops to the new ->iomap_next() callback. The callback is
generated with the DEFINE_IOMAP_ITER_NEXT() macro, which wraps the
iomap_iter_next() helper to finish the previous mapping if needed and
produce the next one. No functional changes are intended.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-15-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
A failed integrity mapping holds a dio reference, so we need to go
through the full bio ending in case there were previously submitted
bio's in the sequence.
Fixes: 2729a60bbf ("block: don't silently ignore metadata for sync read/write")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Link: https://patch.msgid.link/20260720201057.1862857-3-kbusch@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
blkdev_iov_iter_get_pages() can return various error values, including
EIO, EFAULT, and ENOMEM. Set the actual reported status so user space
can know why an operation failed.
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Link: https://patch.msgid.link/20260720201057.1862857-2-kbusch@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add a helper that sets bi_status and call bio_endio() as that is a very
common pattern and convert the core block code over to it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Keith Busch <kbusch@kernel.org>
Reviewed-by: Md Haris Iqbal <haris.iqbal@linux.dev>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Link: https://patch.msgid.link/20260528084632.2505277-1-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Remove blkdev_write_begin(), blkdev_write_end(), and their entries in
def_blk_aops. These have been unreachable since commit 487c607df7
("block: use iomap for writes to block devices") switched block device
buffered writes from generic_perform_write() to
iomap_file_buffered_write(), which bypasses aops->write_begin/end.
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260525-blk-write-cleanup-v1-1-391c073e3831@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
Since after commit 12e4e8c7ab ("io_uring/rw: enable bio caches for
IRQ rw"), bio_put is safe for task and irq context, bio_alloc_bioset is
safe for task context and no one calls in irq context, so we can enable
per cpu bio cache by default.
Benchmarked with t/io_uring and ext4+nvme:
taskset -c 6 /root/fio/t/io_uring -p0 -d128 -b4096 -s1 -c1 -F1 -B1 -R1
-X1 -n1 -P1 /mnt/testfile
base IOPS is 562K, patch IOPS is 574K. The CPU usage of bio_alloc_bioset
decrease from 1.42% to 1.22%.
The worst case is allocate bio in CPU A but free in CPU B, still use
t/io_uring and ext4+nvme:
base IOPS is 648K, patch IOPS is 647K.
Also use fio test ext4/xfs with libaio/sync/io_uring on null_blk and
nvme, no obvious performance regression.
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add caller-provided callbacks for read and readahead so that it can be
used generically, especially by filesystems that are not block-based.
In particular, this:
* Modifies the read and readahead interface to take in a
struct iomap_read_folio_ctx that is publicly defined as:
struct iomap_read_folio_ctx {
const struct iomap_read_ops *ops;
struct folio *cur_folio;
struct readahead_control *rac;
void *read_ctx;
};
where struct iomap_read_ops is defined as:
struct iomap_read_ops {
int (*read_folio_range)(const struct iomap_iter *iter,
struct iomap_read_folio_ctx *ctx,
size_t len);
void (*read_submit)(struct iomap_read_folio_ctx *ctx);
};
read_folio_range() reads in the folio range and is required by the
caller to provide. read_submit() is optional and is used for
submitting any pending read requests.
* Modifies existing filesystems that use iomap for read and readahead to
use the new API, through the new statically inlined helpers
iomap_bio_read_folio() and iomap_bio_readahead(). There is no change
in functionality for those filesystems.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Keep bio_iov_iter_get_bdev_pages local with the callers, as blindly
looking at the bdev logical block size is often not the best idea
unless on a block device.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Keith Busch <kbusch@kernel.org>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmjbLCgQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpoY0D/9J+11BC88pBxCrLKv/V2TwCNokRMi0dU3L
r3EUdA46k0oXmvb6ueZqIcfY2e+IX7rdQkaRbh1zRdsNejqHo4548C3ePWGdBAcM
OdNEGfpehO0aD0td1+mK/NxoJMLhbs5QraPanz+SOkGZOKeF+vGCga5PUDivsr5J
16T9yb7i+isENLdAc2RJbZVyAphqHQlo5GHi5ZIKOVi5cNt8GU/q2sQl7NYmGvHd
aq37svvZHFOhLRajP959Fw9WOxEYITewzQ4UYf1FZjUodJUxO+vCnP0ooBQRlyu8
1B4PYWwSE+Vn3GkQE0Om+mzo9AVPOiLmoAWGxdgJBMyEkZndocr46XEslXOufQ1Z
T3Gu19G6jCxcyByNVhjVnaajYKmvSQAy1w75m4XlfqTRm4f9Om+LAJavUk3RuaOL
7lXKQ7Ql1/Tby9Jmf8afjYYXXotNDNku6rz2P3qtOwAA26mNJfgVt0rO+8XGRDe9
ioLbCkTjslYMc/Oh4jSsbrspsVALbaQMq/Dmah8k0EWb4QAHVgCJyGBoff3hOboI
jD6B1enaKOQVgcjWcjm/FjOk3jv2h3v4X26YWQZTvEc/1PnSnST78Zi/ePhzDdmt
sBALUAS37TfTgNMzrhbHl5Zs13k0C0XyANuayuKuo5hlNnC1wbdap+5FZJOmpuOB
YT+VkYnaOA==
=kOmc
-----END PGP SIGNATURE-----
Merge tag 'for-6.18/block-20250929' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block updates from Jens Axboe:
- NVMe pull request via Keith:
- FC target fixes (Daniel)
- Authentication fixes and updates (Martin, Chris)
- Admin controller handling (Kamaljit)
- Target lockdep assertions (Max)
- Keep-alive updates for discovery (Alastair)
- Suspend quirk (Georg)
- MD pull request via Yu:
- Add support for a lockless bitmap.
A key feature for the new bitmap are that the IO fastpath is
lockless. If a user issues lots of write IO to the same bitmap
bit in a short time, only the first write has additional overhead
to update bitmap bit, no additional overhead for the following
writes.
By supporting only resync or recover written data, means in the
case creating new array or replacing with a new disk, there is no
need to do a full disk resync/recovery.
- Switch ->getgeo() and ->bios_param() to using struct gendisk rather
than struct block_device.
- Rust block changes via Andreas. This series adds configuration via
configfs and remote completion to the rnull driver. The series also
includes a set of changes to the rust block device driver API: a few
cleanup patches, and a few features supporting the rnull changes.
The series removes the raw buffer formatting logic from
`kernel::block` and improves the logic available in `kernel::string`
to support the same use as the removed logic.
- floppy arch cleanups
- Reduce the number of dereferencing needed for ublk commands
- Restrict supported sockets for nbd. Mostly done to eliminate a class
of issues perpetually reported by syzbot, by using nonsensical socket
setups.
- A few s390 dasd block fixes
- Fix a few issues around atomic writes
- Improve DMA interation for integrity requests
- Improve how iovecs are treated with regards to O_DIRECT aligment
constraints.
We used to require each segment to adhere to the constraints, now
only the request as a whole needs to.
- Clean up and improve p2p support, enabling use of p2p for metadata
payloads
- Improve locking of request lookup, using SRCU where appropriate
- Use page references properly for brd, avoiding very long RCU sections
- Fix ordering of recursively submitted IOs
- Clean up and improve updating nr_requests for a live device
- Various fixes and cleanups
* tag 'for-6.18/block-20250929' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (164 commits)
s390/dasd: enforce dma_alignment to ensure proper buffer validation
s390/dasd: Return BLK_STS_INVAL for EINVAL from do_dasd_request
ublk: remove redundant zone op check in ublk_setup_iod()
nvme: Use non zero KATO for persistent discovery connections
nvmet: add safety check for subsys lock
nvme-core: use nvme_is_io_ctrl() for I/O controller check
nvme-core: do ioccsz/iorcsz validation only for I/O controllers
nvme-core: add method to check for an I/O controller
blk-cgroup: fix possible deadlock while configuring policy
blk-mq: fix null-ptr-deref in blk_mq_free_tags() from error path
blk-mq: Fix more tag iteration function documentation
selftests: ublk: fix behavior when fio is not installed
ublk: don't access ublk_queue in ublk_unmap_io()
ublk: pass ublk_io to __ublk_complete_rq()
ublk: don't access ublk_queue in ublk_need_complete_req()
ublk: don't access ublk_queue in ublk_check_commit_and_fetch()
ublk: don't pass ublk_queue to ublk_fetch()
ublk: don't access ublk_queue in ublk_config_io_buf()
ublk: don't access ublk_queue in ublk_check_fetch_buf()
ublk: pass q_id and tag to __ublk_check_and_get_req()
...
The block layer checks all the segments for validity later, so no need
for an early check. Just reduce it to a simple position and total length
check, and defer the more invasive segment checks to the block layer.
Signed-off-by: Keith Busch <kbusch@kernel.org>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The block layer tries to align bio vectors to the block device's logical
block size. Some cases don't have a block device, or we may need to
align to something larger, which we can't derive it from the queue
limits. Have the caller specify what they want, or allow any length
alignment if nothing was specified. Since the most common use case
relies on the block device's limits, a helper function is provided.
Signed-off-by: Keith Busch <kbusch@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The block fops don't try to handle metadata for synchronous requests,
probably because the completion handler looks at dio->iocb which is not
valid for synchronous requests.
But silently ignoring metadata (or warning in case of
__blkdev_direct_IO_simple) is a really bad idea as that can cause
silent data corruption if a user ever shows up.
Instead simply handle metadata for synchronous requests as the completion
handler can simply check for bio_integrity() as the block layer default
integrity will already be freed at this point, and thus bio_integrity()
will only return true for user mapped integrity.
Fixes: 3d8b5a22d4 ("block: add support to pass user meta buffer")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/20250819082517.2038819-3-hch@lst.de
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Currently the kernel will happily route io_uring requests with metadata
to file operations that don't support it. Add a FMODE_ flag to guard
that.
Fixes: 4de2ce04c8 ("fs: introduce IOCB_HAS_METADATA for metadata")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/20250819082517.2038819-2-hch@lst.de
Signed-off-by: Christian Brauner <brauner@kernel.org>
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaINCtwAKCRCRxhvAZXjc
ogPuAQChc4tCjlNp+yAwbSmuzWooKTN8PHI6v+3ftjdaKSy9AgD/Yya1i8aBYBA8
9HBtIKGAqvcgNB3por7yN+GJ8fxb/Ag=
=YmLL
-----END PGP SIGNATURE-----
Merge tag 'vfs-6.17-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs iomap updates from Christian Brauner:
- Refactor the iomap writeback code and split the generic and ioend/bio
based writeback code.
There are two methods that define the split between the generic
writeback code, and the implemementation of it, and all knowledge of
ioends and bios now sits below that layer.
- Add fuse iomap support for buffered writes and dirty folio writeback.
This is needed so that granular uptodate and dirty tracking can be
used in fuse when large folios are enabled. This has two big
advantages. For writes, instead of the entire folio needing to be
read into the page cache, only the relevant portions need to be. For
writeback, only the dirty portions need to be written back instead of
the entire folio.
* tag 'vfs-6.17-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
fuse: refactor writeback to use iomap_writepage_ctx inode
fuse: hook into iomap for invalidating and checking partial uptodateness
fuse: use iomap for folio laundering
fuse: use iomap for writeback
fuse: use iomap for buffered writes
iomap: build the writeback code without CONFIG_BLOCK
iomap: add read_folio_range() handler for buffered writes
iomap: improve argument passing to iomap_read_folio_sync
iomap: replace iomap_folio_ops with iomap_write_ops
iomap: export iomap_writeback_folio
iomap: move folio_unlock out of iomap_writeback_folio
iomap: rename iomap_writepage_map to iomap_writeback_folio
iomap: move all ioend handling to ioend.c
iomap: add public helpers for uptodate state manipulation
iomap: hide ioends from the generic writeback code
iomap: refactor the writeback interface
iomap: cleanup the pending writeback tracking in iomap_writepage_map_blocks
iomap: pass more arguments using the iomap writeback context
iomap: header diet
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaINCgQAKCRCRxhvAZXjc
os+nAP9LFHUwWO6EBzHJJGEVjJvvzsbzqeYrRFamYiMc5ulPJwD+KW4RIgJa/MWO
pcYE40CacaekD8rFWwYUyszpgmv6ewc=
=wCwp
-----END PGP SIGNATURE-----
Merge tag 'vfs-6.17-rc1.mmap_prepare' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull mmap_prepare updates from Christian Brauner:
"Last cycle we introduce f_op->mmap_prepare() in c84bf6dd2b ("mm:
introduce new .mmap_prepare() file callback").
This is preferred to the existing f_op->mmap() hook as it does require
a VMA to be established yet, thus allowing the mmap logic to invoke
this hook far, far earlier, prior to inserting a VMA into the virtual
address space, or performing any other heavy handed operations.
This allows for much simpler unwinding on error, and for there to be a
single attempt at merging a VMA rather than having to possibly
reattempt a merge based on potentially altered VMA state.
Far more importantly, it prevents inappropriate manipulation of
incompletely initialised VMA state, which is something that has been
the cause of bugs and complexity in the past.
The intent is to gradually deprecate f_op->mmap, and in that vein this
series coverts the majority of file systems to using f_op->mmap_prepare.
Prerequisite steps are taken - firstly ensuring all checks for mmap
capabilities use the file_has_valid_mmap_hooks() helper rather than
directly checking for f_op->mmap (which is now not a valid check) and
secondly updating daxdev_mapping_supported() to not require a VMA
parameter to allow ext4 and xfs to be converted.
Commit bb666b7c27 ("mm: add mmap_prepare() compatibility layer for
nested file systems") handles the nasty edge-case of nested file
systems like overlayfs, which introduces a compatibility shim to allow
f_op->mmap_prepare() to be invoked from an f_op->mmap() callback.
This allows for nested filesystems to continue to function correctly
with all file systems regardless of which callback is used. Once we
finally convert all file systems, this shim can be removed.
As a result, ecryptfs, fuse, and overlayfs remain unaltered so they
can nest all other file systems.
We additionally do not update resctl - as this requires an update to
remap_pfn_range() (or an alternative to it) which we defer to a later
series, equally we do not update cramfs which needs a mixed mapping
insertion with the same issue, nor do we update procfs, hugetlbfs,
syfs or kernfs all of which require VMAs for internal state and hooks.
We shall return to all of these later"
* tag 'vfs-6.17-rc1.mmap_prepare' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
doc: update porting, vfs documentation to describe mmap_prepare()
fs: replace mmap hook with .mmap_prepare for simple mappings
fs: convert most other generic_file_*mmap() users to .mmap_prepare()
fs: convert simple use of generic_file_*_mmap() to .mmap_prepare()
mm/filemap: introduce generic_file_*_mmap_prepare() helpers
fs/xfs: transition from deprecated .mmap hook to .mmap_prepare
fs/ext4: transition from deprecated .mmap hook to .mmap_prepare
fs/dax: make it possible to check dev dax support without a VMA
fs: consistently use can_mmap_file() helper
mm/nommu: use file_has_valid_mmap_hooks() helper
mm: rename call_mmap/mmap_prepare to vfs_mmap/mmap_prepare
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaINCeQAKCRCRxhvAZXjc
otqEAP9bWFExQtnzrNR+1s4UBfPVDAaTJzDnBWj6z0+Idw9oegEAoxF2ifdCPnR4
t/xWiM4FmSA+9pwvP3U5z3sOReDDsgo=
=WMMB
-----END PGP SIGNATURE-----
Merge tag 'vfs-6.17-rc1.fallocate' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull fallocate updates from Christian Brauner:
"fallocate() currently supports creating preallocated files
efficiently. However, on most filesystems fallocate() will preallocate
blocks in an unwriten state even if FALLOC_FL_ZERO_RANGE is specified.
The extent state must later be converted to a written state when the
user writes data into this range, which can trigger numerous metadata
changes and journal I/O. This may leads to significant write
amplification and performance degradation in synchronous write mode.
At the moment, the only method to avoid this is to create an empty
file and write zero data into it (for example, using 'dd' with a large
block size). However, this method is slow and consumes a considerable
amount of disk bandwidth.
Now that more and more flash-based storage devices are available it is
possible to efficiently write zeros to SSDs using the unmap write
zeroes command if the devices do not write physical zeroes to the
media.
For example, if SCSI SSDs support the UMMAP bit or NVMe SSDs support
the DEAC bit[1], the write zeroes command does not write actual data
to the device, instead, NVMe converts the zeroed range to a
deallocated state, which works fast and consumes almost no disk write
bandwidth.
This series implements the BLK_FEAT_WRITE_ZEROES_UNMAP feature and
BLK_FLAG_WRITE_ZEROES_UNMAP_DISABLED flag for SCSI, NVMe and
device-mapper drivers, and add the FALLOC_FL_WRITE_ZEROES and
STATX_ATTR_WRITE_ZEROES_UNMAP support for ext4 and raw bdev devices.
fallocate() is subsequently extended with the FALLOC_FL_WRITE_ZEROES
flag. FALLOC_FL_WRITE_ZEROES zeroes a specified file range in such a
way that subsequent writes to that range do not require further
changes to the file mapping metadata. This flag is beneficial for
subsequent pure overwriting within this range, as it can save on block
allocation and, consequently, significant metadata changes"
* tag 'vfs-6.17-rc1.fallocate' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
ext4: add FALLOC_FL_WRITE_ZEROES support
block: add FALLOC_FL_WRITE_ZEROES support
block: factor out common part in blkdev_fallocate()
fs: introduce FALLOC_FL_WRITE_ZEROES to fallocate
dm: clear unmap write zeroes limits when disabling write zeroes
scsi: sd: set max_hw_wzeroes_unmap_sectors if device supports SD_ZERO_*_UNMAP
nvmet: set WZDS and DRB if device enables unmap write zeroes operation
nvme: set max_hw_wzeroes_unmap_sectors if device supports DEAC bit
block: introduce max_{hw|user}_wzeroes_unmap_sectors to queue limits
Change the address_space_operations callbacks write_begin() and
write_end() to take struct kiocb * as the first argument instead of
struct file *.
Update all affected function prototypes, implementations, call sites,
and related documentation across VFS, filesystems, and block layer.
Part of a series refactoring address_space_operations write_begin and
write_end callbacks to use struct kiocb for passing write context and
flags.
Signed-off-by: Taotao Chen <chentaotao@didiglobal.com>
Link: https://lore.kernel.org/20250716093559.217344-4-chentaotao@didiglobal.com
Signed-off-by: Christian Brauner <brauner@kernel.org>
The iomap_folio_ops are only used for buffered writes, including the zero
and unshare variants. Rename them to iomap_write_ops to better describe
the usage, and pass them through the call chain like the other operation
specific methods instead of through the iomap.
xfs_iomap_valid grows a IOMAP_HOLE check to keep the existing behavior
that never attached the folio_ops to a iomap representing a hole.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/20250710133343.399917-12-hch@lst.de
Acked-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Replace the ioend pointer in iomap_writeback_ctx with a void *wb_ctx
one to facilitate non-block, non-ioend writeback for use. Rename
the submit_ioend method to writeback_submit and make it mandatory so
that the generic writeback code stops seeing ioends and bios.
Co-developed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/20250710133343.399917-6-hch@lst.de
Acked-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Replace ->map_blocks with a new ->writeback_range, which differs in the
following ways:
- it must also queue up the I/O for writeback, that is called into the
slightly refactored and extended in scope iomap_add_to_ioend for
each region
- can handle only a part of the requested region, that is the retry
loop for partial mappings moves to the caller
- handles cleanup on failures as well, and thus also replaces the
discard_folio method only implemented by XFS.
This will allow to use the iomap writeback code also for file systems
that are not block based like fuse.
Co-developed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/20250710133343.399917-5-hch@lst.de
Acked-by: Damien Le Moal <dlemoal@kernel.org> # zonefs
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Add inode and wpc fields to pass the inode and writeback context that
are needed in the entire writeback call chain, and let the callers
initialize all fields in the writeback context before calling
iomap_writepages to simplify the argument passing.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/20250710133343.399917-3-hch@lst.de
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
block_write_end() looks like it can be used as a ->write_end()
implementation. However, it can't as it does not unlock nor put
the folio. Since it does not use the 'file', 'mapping' nor 'fsdata'
arguments, remove them.
Signed-off-by: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Link: https://lore.kernel.org/20250624132130.1590285-1-willy@infradead.org
Signed-off-by: Christian Brauner <brauner@kernel.org>
Add support for FALLOC_FL_WRITE_ZEROES, if the block device enables the
unmap write zeroes operation, it will issue a write zeroes command.
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Link: https://lore.kernel.org/20250619111806.3546162-9-yi.zhang@huaweicloud.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Martin K. Petersen" <martin.petersen@oracle.com>
Signed-off-by: Christian Brauner <brauner@kernel.org>
