-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmqGD2AQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpomLD/9rr3SIQvdQRTnSCJlw1cFypSrnRhGAx1g8
v1F7UXuf4Abt4GIerbdXmBTx9zZRctpcOC41ReQk3YKkYuukuvna+Z6tBGf5I5+e
3SR3Zkrj2k4zyNC49g0/QkUJkTTS4tlsBFhNJtY3Q5BBIRMRd/sPvZ+cLnD7g/wH
3zM9IaEMk9xu9mF1nGI7LRl1qY0IOr4ynQpqohl5sEL3TQWgIa0r2B0jhGacBAvC
GGDwIwGDrM6oGwCn6mLN78O5odhLyTufIMNy3EnVw7BcDpLApcnJzYYjGzv+ElCM
WLBJwWYj+PYeoenuJuFhbRihbOPg5gZT3Mn8CUhLxUYzwZvIdFGWOjcGffD9sVu/
JO4nB6A+2AG/wgsb2YVKefhN86FoD0F/2hhdEdgZPJE6a+7+xeyyp9IzyQEWd0cg
Seo06fxE4pwF+dczXT3jzU3LXSZb/Jbh9hzu163VMSWbGYDCpDzfipqZGeN6DWE7
nG2NssAQtjtZrgdggYbZXtwecLZhu3cMj3jbhS4wSUzhQFW42yceJstCx4wpRhW9
m9EXjHQnRFMAb2TjUC3FzEncjbxL+uACjcKhTetd1hLbTI5I8KArZXjam1OdJOHV
urdtsru79LanQLAgRwWYPeKCZGNhX7I2o3OsmVDLQgHIDaZguTcPUVQs3wVui7Dk
IDmmrtwzpg==
=G1l4
-----END PGP SIGNATURE-----
Merge tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Enable Clang context analysis for the nvme host driver, adding
context annotations across core, fabrics, rdma, tcp and pci
- nvmet reservation state exposed through a new namespace-level
debugfs directory, plus ABI documentation for the host sysfs and
target configfs interfaces
- nvme-tcp host memory disclosure fixes on the read path: reject a
read that transferred too few bytes, don't accept C2HData based
on blk_rq_payload_bytes() alone, and fix the R2T case for a read
command
- Parallelize nvme-rdma I/O queue allocation and startup (Surabhi)
- Apple nvme fixes and quirks: page aligned admin queue buffers,
destroy the admin queue on removal, and various DMA/NVMMU
correctness fixes
- A large pile of nvmet and host fixes for out-of-bounds reads,
refcount/resource leaks, and NULL derefs across auth, zns,
passthru, pci-epf, rdma and configfs
- Various other fixes and cleanups
- MD updates via Yu Kuai:
- llbitmap reshape support, the large series wiring exact bitmap
mapping and reshape lifecycle through raid5 and raid10, growing
the page cache in place, and remapping checkpointed bits as
reshape progresses
- raid5 fixes for lockless max_nr_stripes and recovery_offset
accesses, a reshape deadlock with more failed devices than max
degraded, and bitmap batch counter consistency
- Atomic write handling for raid1/raid10, and removal of the
REQ_NOWAIT support from raid1/10/456
- raid5-ppl use-after-free fix in ppl_do_flush()
- A batch of smaller fixes across md core and the bitmap code
- s390/dasd ESE full-track write support and the surrounding
infrastructure, plus enabling CONTEXT_ANALYSIS for s390/block
- RWF_DONTCACHE support for block devices, built on new task-context
bio completion infrastructure, and wiring it up for the iomap and
buffer dropbehind writeback paths
- Async io_uring zone reset all, plus zone management command cleanups
allowing REQ_NOWAIT and tightening conventional zone rejection
- Block integrity refactoring: lift BIP_CHECK_FLAGS to the shared
header, handle nogenerate/noverify properly in fs-integrity, and drop
the blk-integrity.h include from bdev.c
- Split out a new blk_plug.h header
- ublk improvements: add UBLK_F_IO_DESC_SIZE, split request validation
from io_desc init, reject non-power-of-2 zone sizes in SET_PARAMS,
and a series of hardening fixes around map/unmap and auto buf reg
- null_blk cleanups and configfs serialization fixes
- nbd queue freeze removal on the setup paths, and a new
pre_defined_connections module parameter for pre-created devices
- blk-cgroup fixes for the race between policy activation and blkg
destruction, and accounting per-cpu stats over possible CPUs across
blk-stat, iolatency, iocost and kyber
- Various dio fixes: leak on metadata mapping error, validate user
space vectors during extraction, and set dma_alignment from the
backing file for loop and zloop direct I/O
- bio cleanups
- Various other fixes and cleanups all over
* tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (241 commits)
nbd: add pre_defined_connections module parameter for pre-created devices
nbd: remove queue freeze for newly created nbd from netlink path
nbd: factor out a nbd_genl_foreach_sock
nbd: skip queue freeze when setting size at device startup
nbd: remove queue freeze in nbd_add_socket
nbd: clear queue limits on disconnect
nbd: disallow NBD_SET_SOCK on an active device
nbd: simplify find_fallback() by removing redundant logic
blk-mq: add missing call to srcu_barrier() in blk_mq_free_tag_set()
block: mtip32xx: synchronize ioctls with device removal
ublk: avoid teardown retry loop on xarray allocation failure
null_blk: fix UBSAN shift-out-of-bounds when zone_size is 0 or overflows
block: don't include blk-integrity.h in bdev.c
xfs: avoid double deferrals for RWF_DONTCACHE writes
loop: Fix recently introduced lock inversion
block: set QUEUE_FLAG_DYING unconditionally in blk_mark_disk_dead()
swim3: Add missing MODULE_DESCRIPTION
selftests: ublk: add SET_PARAMS validation test
selftests: ublk: add helper for SET_PARAMS
ublk: reject non-power-of-2 zone sizes in SET_PARAMS
...
Please consider pulling these changes from the signed vfs-7.3-rc1.misc tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCan7RJQAKCRCRxhvAZXjc
otI6AQCIwshv02UI9fmbTC9pFr8b+v/9IWQwdVULaH61HXgICQD7BE/CktL8nfRs
zCzGS6kBdWgiOA6bdVpgGGLqiMRnQgo=
=Kc+G
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull misc vfs updates from Christian Brauner:
"Bigger cleanups:
- The lockref dead-count handling is tidied up.
The open-coded check for a count below zero as the dead marker
relies on information the caller should not have.
- make put_mnt_ns() leave mounts connected. Destroying a mount
namespace disconnected its mounts from their mount points. So a
file descriptor still open on the parent of a mount point could be
used to peek under it.
Locked mounts were already kept connected to prevent exactly that.
But a mount is only locked when its tree is copied across a user
namespace boundary. So a mount namespace set up by a privileged
component had no locked mounts and its mounts were disconnected.
Passing UMOUNT_CONNECTED keeps every mount connected and prevents
that bug.
- vfs_prepare_mode() passes S_IFDIR for directories. I meant to fix
that ago but didn't get to it. So now someone finally did it.
This kills the exception where the mode could be 0 when a directory
was created whereas every other creation operation passed it
explicitly already.
- move long delayed work for ufs, jffs2, hfsplus, hfs and affs from
the per-cpu system_long_wq to the new unbound system_dfl_long_wq.
None of that work relies on per-cpu state and the work item is
enqueued with queue_delayed_work() whose timer is global anyway. So
it may as well benefit from scheduler task placement.
Smaller fixes and cleanups:
- unlock_buffer() and journal_end_buffer_io_sync() use
clear_and_wake_up_bit()
- the pipe page pools are unified into a single per-pipe pool and the
extra wake_up(rd_wait) is limited to EPOLLET consumers
- eventpoll now computes its timer slack lazily in ep_poll()
- shrink_dcache_for_umount() keeps making progress on busy roots
- excess xarray nodes are freed in clear_inode()
- romfs detects hard link cycles
- the user path of nested backing files is fixed
- pidfd holds exec_update_lock around the namespace ioctl
- non-memcg-aware nr_cached_objects is skipped during memcg slab
shrink
- iomap_write_iter() always returns status
- mangle_path() is renamed to seq_mangle_path()
- inode timestamp accessors are annotated
- new regression test for pipe->poll_usage.
- a few documentation, kernel-doc and selftest fixes"
* tag 'vfs-7.3-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (67 commits)
selftests/namespaces: Fix racy pipe handshake in timens and pidns_separate
selftests/epoll: add a regression test for pipe->poll_usage
pipe: only enable the extra wake_up(rd_wait) for EPOLLET consumers
pidfd: hold exec_update_lock around namespace ioctl
fs: fix user path of nested backing files
fs: remove stale inode_insert5() kernel-doc parameter
fs: fix switch/case indentation in sysfs() syscall
fs: document semantics of kstat::{uid,gid} fields
dcache: keep shrink_dcache_for_umount() making progress on busy roots
seq_file: rename mangle_path to seq_mangle_path
nstree: add/fix struct ns_id_req kernel-doc member fields
dcache: use lockref routines for dead count checks
lockref: tidy up dead count handling
initramfs: fix typo in reserve_initrd_mem comment
fs/pipe: unify the page pools into a single per-pipe pool
fs: annotate inode timestamp accessors
eventpoll: compute timer slack lazily in ep_poll()
selftests/filesystems: add mntns cleanup test
put_mnt_ns(): leave mounts connected
affs: Move long delayed work on system_dfl_long_wq
...
Please consider pulling these changes from the signed vfs-7.3-rc1.iomap tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCan7RJQAKCRCRxhvAZXjc
oqOjAP9eZsO4M7OQpTKPgg3Sq2O9QCVgXBy4SVFba7DrbZHLZAEA/NWE+4udJzeL
j1Xb2VAyMc4S+Ze/O8sMml5Dyh4V5wE=
=34r1
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull iomap updates from Christian Brauner:
"The bulk of this is the conversion of iomap to a single ->iomap_next()
callback and thus finishing the move to an iterator model.
Every iomap operation drove its iteration through a struct iomap_ops
holding ->iomap_begin() and ->iomap_end(). iomap_iter() only ever sees
those as pointers. That means every step of every iteration is an
indirect call.
This collapses both into one ->iomap_next() callback that finishes the
previous mapping and produces the next one. This lets callers inline
the iteration loop and pass its ->iomap_next() as a compile time
constant. That means the compiler can turn it into a direct and hence
inlineable call.
This also allows future callers to express custom logic to drive the
iteration forward better. xfs, btrfs, ext4, ext2, erofs, f2fs, gfs2,
hpfs, fuse, exfat, zonefs, ntfs, ntfs3 and the block device mapping
are all converted. No functional changes are intended.
This also adds a simple direct I/O path for small reads. On Gen5 NVMe
the __iomap_dio_rw() dominates 4K random reads. The same single-core
io_uring poll mode workload reaches ~3.2M IOPS against the raw block
device but only ~1.92M through ext4 or XFS.
__iomap_dio_rw(), iomap_iter(), iomap_dio_bio_iter() and kfree() were
at the top of the profile. The new path is very lightweight if no
special behavior is requested. The bio comes from a dedicated bioset
and laid out so the whole request is a single cacheline aligned
allocation. Completion runs inline.
That takes ext4 from 1.92M to 2.19M IOPS in the original workload. fio
shows around:
- 4% at libaio queue depths of 64 and up
- around 5% for io_uring
- up to 10% for io_uring poll mode at depth 256
on both ext4 and xfs.
A few other patches:
- iomap_folio_mark_uptodate() lets a filesystem that writes into the
page cache outside the iomap read and write paths keep iomap's
internal uptodate bitmap in sync, which fuse needs for
server-pushed notify stores before it can enable large folios;
- two fixes for iomap_bio_read_folio_range_sync(): a potential crash
when device integrity behavior is changed and a missing
bio_uninit().
- a folio batch release fix on iomap callback failures
- FGP_NOFS is dropped from iomap_get_folio()
- documentation fix"
* tag 'vfs-7.3-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (29 commits)
iomap: iomap_bio_read_folio_range_sync is missing a call to bio_uninit
iomap: don't free integrity payload that doesn't exist
docs: fix grammatical error in iomap docs
exfat: convert iomap ops to ->iomap_next()
fuse: convert iomap ops to ->iomap_next()
hpfs: convert iomap ops to ->iomap_next()
gfs2: convert iomap ops to ->iomap_next()
f2fs: convert iomap ops to ->iomap_next()
block: convert iomap ops to ->iomap_next()
ext2: convert iomap ops to ->iomap_next()
zonefs: convert iomap ops to ->iomap_next()
erofs: convert iomap ops to ->iomap_next()
ext4: convert iomap ops to ->iomap_next()
ntfs: convert iomap ops to ->iomap_next()
ntfs3: convert iomap ops to ->iomap_next()
btrfs: convert iomap ops to ->iomap_next()
xfs: convert iomap ops to ->iomap_next()
iomap: add ->iomap_next()
iomap: use GFP_NOWAIT when application for iomap_dio_simple allocations
iomap: decouple simple direct I/O reads from iomap_dio_rw
...
Which could leak blkg references.
Fixes: c03cea4214 ("iomap: add initial support for writes without buffer heads")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260804124404.737145-3-hch@lst.de
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Anuj Gupta <anuj20.g@samsung.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
fs_bio_integrity_alloc might not allocate a bio integrity payload if PI
verification is disabled on the block device. Check for that case before
calling fs_bio_integrity_free in iomap_bio_read_folio_range_sync to
avoid a NULL pointer dereferences.
Make the branch cover the PI verification as well - while
fs_bio_integrity_verify works without an integrity payload, it requires
one to actually do useful work.
Fixes: 0b10a37052 ("iomap: support T10 protection information")
Cc: stable@vger.kernel.org # v7.1
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Anuj Gupta <anuj20.g@samsung.com>
Reviewed-by: Kanchan Joshi <joshi.k@samsung.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Link: https://patch.msgid.link/20260804124404.737145-2-hch@lst.de
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Set BIO_COMPLETE_IN_TASK on iomap writeback bios when a dropbehind folio
is added. This ensures that bi_end_io runs in task context, where
folio_end_dropbehind() can safely invalidate folios.
With the bio layer now handling task-context deferral generically,
IOMAP_IOEND_DONTCACHE is no longer needed, as XFS no longer needs to
route DONTCACHE ioends through its completion workqueue. Remove the flag
and its NOMERGE entry.
Without the NOMERGE, regular I/Os that get merged with a dropbehind
folio will also have their completion deferred to task context.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
Link: https://patch.msgid.link/20260730-blk-dontcache-v7-3-3e8e6850068d@columbia.edu
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The bio-based drivers don't necessarily check the alignment split, and
stacking block drivers don't always handle a misalignment detected after
submitting the bio. Validate user vectors against the device's
dma_alignment as the bio is built from the iov_iter, rejecting
misaligned early with -EINVAL.
Cc: stable@vger.kernel.org
Fixes: 5ff3f74e14 ("block: simplify direct io validity check")
Fixes: 7eac331869 ("iomap: simplify direct io validity check")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Link: https://patch.msgid.link/20260720201057.1862857-6-kbusch@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Have one ->iomap_next() callback instead of ->iomap_begin() and
->iomap_end(). ->iomap_next() finishes the previous mapping if needed,
and produces the next mapping.
Collapsing to a single callback lets a performance-critical caller
inline its iteration loop and pass its ->iomap_next() function as a
compile-time constant, so the compiler can devirtualize that callback
into a direct call instead of an indirect call through a function
pointer.
iomap_iter() uses ->iomap_next() when the filesystem provides that
callback and otherwise falls back to the ->iomap_begin()/->iomap_end()
path, so filesystems can be converted one at a time.
Suggested-by: Christoph Hellwig <hch@lst.de>
Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-6-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
For non-blocking iocbs we should avoid blocking allocation where
possible, so switch to a GFP_NOWAIT allocation here.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Fengnan Chang <changfengnan@bytedance.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260729192737.3190206-5-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The pending iomap_iter_next conversion creates performance issues for the
new simple direct I/O read fast path, because it assumes a model where
the iterator must be advanced at the end, which the direct I/O read fast
path tries to avoid.
Side step this by splitting the simple path from iomap_dio_rw, and
require the file systems to call into it explicitly, and pass only a
->begin callback. This allows to drop various checks for incompatible
features while creating a requirement for the file system to only call
the simple path for cases that it can handle.
As a side-benefit we can now inline the initial part of the simple
direct I/O read fast path and let the compiler convert the indirect call
to ->begin into a direct call.
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Fengnan Chang <changfengnan@bytedance.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260729192737.3190206-4-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
In preparation for changing iomap to use an in-iter (->iomap_next())
model, move the iomap_iter() logic out into the new iomap_iter_next()
helper function.
iomap_iter_next() is added as an inlined helper so it can be called
directly by ->iomap_next() implementations where the begin()/end()
callbacks can be direct calls.
The DEFINE_IOMAP_ITER_NEXT() and DEFINE_IOMAP_ITER_NEXT_END() macros are
also provided to generate the boilerplate ->iomap_next() wrapper
functions that simply forward to iomap_iter_next() with the appropriate
begin/end callbacks. DEFINE_IOMAP_ITER_NEXT() is for the common case
where there is no end() callback. DEFINE_IOMAP_ITER_NEXT_END() is for
the case where there is an explicit end() callback.
No functional change intended. The one would-be behavioral difference is
that on the iomap_end() error path (ret < 0 && !advanced), the old code
returned with iter.status left as the caller's last value whereas the
new code zeroes it, but this is not observable in practice as there are
no in-tree callers that read iter.status after the iteration loop.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Fengnan Chang <changfengnan@bytedance.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260729192737.3190206-3-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
A sashiko review of an unrelated patch points out that the folio
batch mechanism used for iomap zero range fails to release the batch
in a couple error scenarios. If either calls to ->iomap_end() or
->iomap_begin() fail, the direct return paths bypass the batch
cleanup.
The ->iomap_end() case is not a practical issue at the moment
because there is no user of the mechanism that returns an error from
this path. The ->iomap_begin() case is theoretically possible
because XFS can invoke the fill helper and error out at various
points thereafter. This subtly complicates things because XFS does
not transfer iomap_flags to the iomap data structure in the error
path.
To deal with both of these issues, first make sure to invoke the
cleanup helper in the error path for either fs callback. Second,
update the helper to clear the flag unconditionally and release the
batch so long as it is populated. This more clearly delineates the
purpose of the flag to control the I/O path and not necessarily the
status of the fbatch, so add a comment around this as well.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 395ed1ef00 ("iomap: optional zero range dirty folio processing")
Signed-off-by: Brian Foster <bfoster@redhat.com>
Link: https://patch.msgid.link/20260729192737.3190206-2-joannelkoong@gmail.com
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Add an exported helper iomap_folio_mark_uptodate() to mark a folio as
uptodate and update its uptodate bitmap if the folio has iomap state
data attached.
This is needed because there are some filesystems (eg fuse) that have
paths outside of conventional iomap calls that need to mark a folio as
uptodate (eg writing server-pushed data directly into the page cache)
and need the iomap-internal uptodate bitmap to be in sync with the
uptodate state of the folio.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260707220450.1200943-3-joannelkoong@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
When running 4K random read workloads on high-performance Gen5 NVMe
SSDs, the software overhead in the iomap direct I/O path
(__iomap_dio_rw) becomes a significant bottleneck.
Using io_uring with poll mode for a 4K randread test on a raw block
device:
taskset -c 30 ./t/io_uring -p1 -d512 -b4096 -s32 -c32 -F1 -B1 -R1 -X1
-n1 -P1 /dev/nvme10n1
Result: ~3.2M IOPS
Running the exact same workload on ext4 and XFS:
taskset -c 30 ./t/io_uring -p1 -d512 -b4096 -s32 -c32 -F1 -B1 -R1 -X1
-n1 -P1 /mnt/testfile
Result: ~1.92M IOPS
Profiling the ext4 workload reveals that a significant portion of CPU
time is spent on memory allocation and the iomap state machine
iteration:
5.33% [kernel] [k] __iomap_dio_rw
3.26% [kernel] [k] iomap_iter
2.37% [kernel] [k] iomap_dio_bio_iter
2.35% [kernel] [k] kfree
1.33% [kernel] [k] iomap_dio_complete
Introduce a simple dio path to reduce the overhead of iomap. It is
triggered when the request satisfies all of:
- a READ request whose I/O size is <= inode blocksize (fits in a single
block, no splits);
- no custom iomap_dio_ops (dops) registered by the filesystem;
- no caller-accumulated residual (done_before == 0);
- none of IOMAP_DIO_FORCE_WAIT / IOMAP_DIO_PARTIAL / IOMAP_DIO_BOUNCE
set, the range is within i_size, and the inode is not encrypted.
The bio is allocated from a dedicated bioset whose front_pad embeds
struct iomap_dio_simple, so the whole request lives in a single
cacheline-aligned allocation and no separate struct iomap_dio is
needed. Completion is handled inline from ->bi_end_io for the common
success case, and only punted to the s_dio_done_wq workqueue on error.
After this optimization, the heavy generic functions disappear from the
profile, replaced by a single streamlined execution path:
4.83% [kernel] [k] iomap_dio_simple
With this patch, 4K random read IOPS on ext4 increases from 1.92M to
2.19M in the original single-core io_uring poll-mode workload.
Below are the test results using fio:
fs workload qd simple=0 simple=1 gain
ext4 libaio 1 18,740 18,761 +0.11%
ext4 libaio 64 462,850 480,587 +3.83%
ext4 libaio 128 459,498 478,824 +4.21%
ext4 libaio 256 459,938 480,156 +4.40%
ext4 io_uring 1 18,836 18,880 +0.24%
ext4 io_uring 64 568,193 600,625 +5.71%
ext4 io_uring 128 570,998 602,148 +5.46%
ext4 io_uring 256 572,052 602,536 +5.33%
ext4 io_uring_poll 1 19,283 19,272 -0.06%
ext4 io_uring_poll 64 989,735 1,013,342 +2.39%
ext4 io_uring_poll 128 1,467,336 1,538,444 +4.85%
ext4 io_uring_poll 256 1,663,498 1,830,842 +10.06%
xfs libaio 1 18,764 18,776 +0.06%
xfs libaio 64 462,408 480,860 +3.99%
xfs libaio 128 461,280 480,819 +4.24%
xfs libaio 256 461,626 480,190 +4.02%
xfs io_uring 1 18,871 18,903 +0.17%
xfs io_uring 64 570,383 597,399 +4.74%
xfs io_uring 128 568,290 597,370 +5.12%
xfs io_uring 256 570,616 598,775 +4.93%
xfs io_uring_poll 1 19,211 19,315 +0.54%
xfs io_uring_poll 64 989,726 1,008,455 +1.89%
xfs io_uring_poll 128 1,430,426 1,513,064 +5.78%
xfs io_uring_poll 256 1,587,339 1,742,220 +9.76%
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260701033253.46420-4-changfengnan@bytedance.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Change should_report_dio_fserror() to take an error code instead of the
full struct iomap_dio, decoupling it for reuse by the upcoming simple
direct I/O path.
No functional change.
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260701033253.46420-3-changfengnan@bytedance.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Extract the alignment computation from iomap_dio_bio_iter() into a
standalone helper so the upcoming simple direct I/O path can reuse it
without requiring a struct iomap_dio.
No functional change.
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260701033253.46420-2-changfengnan@bytedance.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
FGP_NOFS is legacy; filesystems should be using memalloc_nofs_save/restore
instead. We have it here in iomap because it was buried in
grab_cache_page_write_begin() and we didn't want to change this behaviour
as part of the folio transition.
I have tested this with XFS and see no issues. Other filesystems (cc'd)
may need to make adjustments. Please test with lockdep enabled.
Cc: Darrick J. Wong <djwong@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Cc: Sungjong Seo <sj1557.seo@samsung.com>
Cc: Yuezhang Mo <yuezhang.mo@sony.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>
Cc: Andreas Gruenbacher <agruenba@redhat.com>
Cc: Hyunchul Lee <hyc.lee@gmail.com>
Cc: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Cc: Carlos Maiolino <cem@kernel.org>
Cc: Damien Le Moal <dlemoal@kernel.org>
Cc: Naohiro Aota <naohiro.aota@wdc.com>
Cc: Johannes Thumshirn <jth@kernel.org>
Cc: linux-xfs@vger.kernel.org
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-block@vger.kernel.org
Cc: fuse-devel@lists.linux.dev
Cc: gfs2@lists.linux.dev
Cc: ntfs3@lists.linux.dev
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260624174228.2015893-1-willy@infradead.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
iomap_split_ioend can split bios that already come from
iomap_ioend_bioset and thus deadlock when the bioset is exhausted.
Add a separate bio_set to avoid this deadlock.
Christian Brauner <brauner@kernel.org> says:
Mark iomap_ioend_split_bioset static as it is only used in ioend.c,
fixing the sparse warning reported by the kernel test robot.
Fixes: 5fcbd555d4 ("iomap: split bios to zone append limits in the submission handlers")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260629125229.3400726-1-hch@lst.de
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Different io_private values indicate distinct completion contexts that
must not be merged together, as this could leak or corrupt the private
data associated with each ioend.
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Link: https://patch.msgid.link/20260713074206.1768006-1-yi.zhang@huaweicloud.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The range alignment strategy differs between ifs_clear_range_dirty() and
ifs_set_range_dirty(). The former rounds inwards to clear only
fully-covered blocks, while the latter rounds outwards to mark any
partially-touched block as dirty. Add comments to document this
asymmetry in block range calculation.
Suggested-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Link: https://patch.msgid.link/20260714082325.325163-6-yi.zhang@huaweicloud.com
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
ifs_set_range_dirty() and ifs_set_range_uptodate() compute last_blk
as (off + len - 1) >> i_blkbits. When off is 0 and len is 0, the
unsigned subtraction underflows to SIZE_MAX, producing a huge
last_blk and nr_blks value that causes bitmap_set() to write far
beyond the ifs->state allocation.
Regarding ifs_set_range_uptodate(), it is temporarily safe because len
cannot be passed in as 0. However, for ifs_set_range_dirty() this is
reachable from __iomap_write_end(): when copy_folio_from_iter_atomic()
returns 0 (e.g. user buffer fault) and the folio is already uptodate,
the guard at the top of __iomap_write_end() does not trigger because
!folio_test_uptodate() is false, and iomap_set_range_dirty() is called
with copied == 0.
Add a !len guard to both functions before the computation, so that a
zero-length range is a no-op.
Fixes: 4ce02c6797 ("iomap: Add per-block dirty state tracking to improve performance")
Cc: stable@vger.kernel.org # v6.6
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Link: https://patch.msgid.link/20260714082325.325163-5-yi.zhang@huaweicloud.com
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The did_zero output parameter was unconditionally set after the loop,
which is incorrect. It should only be set when the zeroing operation
actually completes, not when IOMAP_F_STALE is set or when
IOMAP_F_FOLIO_BATCH is set but !folio causes the loop to break early,
or when iomap_iter_advance() returns an error.
This causes did_zero to be incorrectly set when zeroing a clean
unwritten extent because the loop exits early without actually zeroing
any data.
Fix it by using a local variable to track whether any folio was actually
zeroed, and only set did_zero after the loop if zeroing happened.
Fixes: 98eb8d9502 ("iomap: set did_zero to true when zeroing successfully")
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Link: https://patch.msgid.link/20260714082325.325163-4-yi.zhang@huaweicloud.com
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Current iomap_invalidate_folio() can only invalidate an entire folio. If
we truncate a partial folio on a filesystem where the block size is
smaller than the folio size, it will leave behind dirty bits for the
truncated or punched blocks. During the write-back process, it will
attempt to map the invalid hole range. Fortunately, this has not caused
any real problems so far because the ->writeback_range() function
corrects the length.
However, the implementation of FALLOC_FL_ZERO_RANGE in ext4 depends on
the support for invalidating partial folios. When ext4 partially zeroes
out a dirty and unwritten folio, it does not perform a flush first like
XFS. Therefore, if the dirty bits of the corresponding area cannot be
cleared, the zeroed area after writeback remains in the written state
rather than reverting to the unwritten state. Fix this by supporting
invalidation of partial folios.
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Link: https://patch.msgid.link/20260714082325.325163-3-yi.zhang@huaweicloud.com
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
The block range calculation in ifs_clear_range_dirty() is incorrect when
partially clearing a range in a folio. We cannot clear the dirty bit of
the first block or the last block if the start or end offset is not
blocksize-aligned. This has not yet caused any issues since we always
clear a whole folio in iomap_writeback_folio().
Fix this by rounding up the first block to blocksize alignment, and
calculate the last block by rounding down (using truncation). Correct
the nr_blks calculation accordingly.
Fixes: 4ce02c6797 ("iomap: Add per-block dirty state tracking to improve performance")
Signed-off-by: Zhang Yi <yi.zhang@huawei.com>
Link: https://patch.msgid.link/20260714082325.325163-2-yi.zhang@huaweicloud.com
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Currently the iomap buffered read path tries to build up read context
(i.e. bios for the typical block based case) over multiple iomaps as
long as the sector matches. This does not take into account files
that can map to multiple different devices. While this could be fixed
by a bdev check in iomap_bio_read_folio_range, the building up of I/O
over iomaps actually was a problem for the not yet merged ext2 iomap
port, as that does want to send out I/O at the end of an indirect
block mapped range.
So instead of adding more checks move over to a model where a bio only
spans a single iomap. Change ->submit_read to be called after each
iteration so that the bio based users submit the bio after each iomap.
Fuse is unchanged because the previous commit stopped using ->submit_read
for it.
Fixes: dfeab2e95a ("erofs: add multiple device support")
Reported-by: Kelu Ye <yekelu1@huawei.com>
Reported-by: Yifan Zhao <zhaoyifan28@huawei.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260629121750.3392300-4-hch@lst.de
Tested-by: Yifan Zhao <zhaoyifan28@huawei.com>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Add a iomap_bio_submit_read_endio helper factored out of
iomap_bio_submit_read to that all ->submit_read implementations for
iomap_read_ops that use iomap_bio_read_folio_range can shared the
logic.
Right now that logic is mostly trivial, but already has a bug for XFS
because the XFS version is too trivial: file system integrity validation
needs a workqueue context and thus can't happen from the default iomap
bi_end_io I/O handler. Unfortunately the iomap refactoring just before
fs integrity landed moved code around here and the call go misplaced,
meaning it never got called. The PI information still is verified by
the block layer, but the offloading is less efficient (and the future
userspace interface can't get at it).
Fixes: 0b10a37052 ("iomap: support T10 protection information")
Cc: stable@vger.kernel.org # v7.1
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260629121750.3392300-2-hch@lst.de
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
iomap: fix zero padding data issue in concurrent append writes
changed ioend accounting so that io_size tracks only valid data
within EOF. This trims io_size when a writeback range extends
past end_pos:
ioend->io_size += map_len;
if (ioend->io_offset + ioend->io_size > end_pos)
ioend->io_size = end_pos - ioend->io_offset;
However, if end_pos ends up below ioend->io_offset, the subtraction
becomes negative and is stored in size_t io_size, causing an unsigned
wrap to a huge value. This can happen when writeback continues past
byte-level EOF up to a block-aligned range, or when a concurrent
truncate shrinks the file after end_pos was sampled in
iomap_writeback_handle_eof().
A wrapped io_size can mislead append detection and corrupt
completion-time size handling, since filesystem end_io paths consume
io_size for decisions such as on-disk EOF updates and unwritten/COW
completion ranges.
Fix this by clamping io_size to zero when EOF has moved to or before
the ioend start offset. This preserves the original intent of trimming
io_size to valid in-EOF data while avoiding the underflow.
Fixes: 51d20d1dac ("iomap: fix zero padding data issue in concurrent append writes")
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Morduan Zang <zhangdandan@uniontech.com>
Link: https://patch.msgid.link/9E38E2659B47DC2A+20260624062622.337469-1-zhangdandan@uniontech.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
If bio_iov_iter_get_pages() or the bounce helper succeeds but builds a
short bio, the REQ_ATOMIC size check rejects it before submission. The
old error path only dropped the bio reference, leaving any pages already
attached to the bio unreleased.
Release or unbounce the pages before falling through to out_put_bio on
this error path.
This bug was reported by sashiko:
https://sashiko.dev/#/patchset/20260608073134.95964-1-changfengnan%40bytedance.com
Fixes: 9e0933c21c ("fs: iomap: Atomic write support")
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260612044041.10677-1-changfengnan@bytedance.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
iomap_write_iter() returns either an error code or 0 if partial
progress has been made. The error sanitization was required in the
past because the return value was used by iomap_iter() to determine
how much progress was made, and thus what to pass to ->iomap_end()
and how much to advance the iter.
Now that iter handlers advance the iter incrementally and progress
is separate from return status, this is no longer needed.
iomap_iter() infers partial progress directly from the iter state
and similarly, iomap_file_buffered_write() uses iter.pos to
determine whether to return a short write or an error code.
This also eliminates a minor quirk in the write iteration where if
an error interrupts a partial write, we'd have to loop back into
iomap_write_iter() once more and run into the error a second time
before iomap terminates the operation and returns the error. With
the error code returned directly and separate from write progress,
we can complete the operation and return from iomap_iter()
immediately.
Signed-off-by: Brian Foster <bfoster@redhat.com>
Link: https://patch.msgid.link/20260617114253.635751-1-bfoster@redhat.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Please consider pulling these changes from the signed vfs-7.2-rc1.misc tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaiwLKgAKCRCRxhvAZXjc
ou/zAP9SOUE6n58i0BdhLYw0RA9Ge5tz42e4inSFi4tkgfCrDwEAlxbRHDcMyhWB
dHmx8OW6b5riMrW+lGPMH58RMoTkdQ4=
=3vuc
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.2-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull misc vfs updates from Christian Brauner:
"Features:
- Reduce pipe->mutex contention by pre-allocating pages outside the
lock in anon_pipe_write().
anon_pipe_write() called alloc_page() once per page while holding
pipe->mutex. The allocation can sleep doing direct reclaim and runs
memcg charging, which extends the critical section and stalls any
concurrent reader on the same mutex. Now up to 8 pages are
pre-allocated before the mutex is taken, leftovers are recycled
into the per-pipe tmp_page[] cache before unlock, and any remainder
is released after unlock, keeping the allocator out of the critical
section on both sides. On a writers x readers sweep with 64KB
writes against a 1 MB pipe throughput improves 6-28% and average
write latency drops 5-22%; under memory pressure - when the cost of
holding the mutex across reclaim is highest - throughput improves
21-48% and latency drops 17-33%. The microbenchmark is added to
selftests.
- uaccess/sockptr: fix the ignored_trailing logic in
copy_struct_to_user() to behave as documented and the usize check
in copy_struct_from_sockptr() for user pointers, and add
copy_struct_{from,to}_bounce_buffer() and copy_struct_to_sockptr()
helpers for upcoming users (IPPROTO_SMBDIRECT, IPPROTO_QUIC).
- bpf: add a sleepable bpf_real_inode() kfunc that resolves the real
inode backing a dentry via d_real_inode(). On overlayfs the inode
attached to the dentry doesn't carry the underlying device
information; this is used by the filesystem restriction BPF program
that was merged into systemd.
- docs: add guidelines for submitting new filesystems, motivated by
the maintenance burden abandoned and untestable filesystems impose
on VFS developers, blocking infrastructure work like folio
conversions and iomap migration.
Fixes:
- libfs: set SB_I_NOEXEC and SB_I_NODEV by default in init_pseudo()
and drop the now-redundant assignments in callers. This began as a
one-line dma-buf fix for a path_noexec() warning; a pseudo
filesystem has no reason not to set SB_I_NOEXEC. All init_pseudo()
callers were audited: the only visible effect is on dma-buf where
SB_I_NOEXEC silences the warning.
- Handle set_blocksize() failures in legacy filesystems (bfs, hpfs,
qnx4, jfs, befs, affs, isofs, minix, ntfs3, omfs). Mounting a
device with a sector size > PAGE_SIZE crashed roughly half of them;
the rest had the same missing error handling pattern. Plus a
follow-up releasing the superblock buffer_head when setting the
minix v3 block size fails.
- mount: honour SB_NOUSER in the new mount API.
- fs/fcntl: fix a SOFTIRQ-unsafe lock order in fasync signaling by
switching the process-group paths of send_sigio() and send_sigurg()
from read_lock(&tasklist_lock) to RCU, matching the single-PID
path.
- vfs: add an FS_USERNS_DELEGATABLE flag and set it for NFS, fixing
delegated NFS mounts (fsopen() in a container with the mount
performed by a privileged daemon) that broke when non-init
s_user_ns was tied to FS_USERNS_MOUNT.
- selftests/namespaces: fix a hang in nsid_test where an unreaped
grandchild kept the TAP pipe write-end open, a waitpid(-1) race in
listns_efault_test, and a false FAIL on kernels without listns()
where the tests should SKIP.
- filelock: fix the break_lease() stub signature for
CONFIG_FILE_LOCKING=n.
- init/initramfs_test: wait for the async initramfs unpacking before
running; the test and do_populate_rootfs() share the parser state.
- fs/coredump: reduce redundant log noise in
validate_coredump_safety().
- iomap: pass the correct length to fserror_report_io() in
__iomap_write_begin().
- backing-file: fix the backing_file_open() kerneldoc.
Cleanups:
- initramfs: refactor the cpio hex header parsing to use hex2bin()
instead of the hand-rolled simple_strntoul() which is reverted, and
extend the initramfs KUnit tests to cover header fields with 0x
prefixes.
- Replace __get_free_pages() and friends with kmalloc()/kzalloc()
across quota, proc, ocfs2/dlm, nilfs2, nfs, nfsd, libfs, jfs, jbd2,
isofs, fuse, select, namespace, configfs, binfmt_misc, bfs, and the
do_mounts init code - part of the larger work of replacing page
allocator calls with kmalloc().
- Use clear_and_wake_up_bit() in unlock_buffer() and
journal_end_buffer_io_sync() instead of open-coding the sequence.
- Drop unused VFS exports: unexport drop_super_exclusive(), remove
start_removing_user_path_at(), and fold __start_removing_path()
into start_removing_path().
- fs/read_write: narrow the __kernel_write() export with
EXPORT_SYMBOL_FOR_MODULES().
- vfs: uapi: retire octal and hex constants in favor of (1 << n) for
the O_ flags. Finding a free bit for a new flag across the
architectures was needlessly hard with the mixed bases.
- dcache: add extra sanity checks of dead dentries in dentry_free()
via a new DENTRY_WARN_ONCE() that also prints d_flags.
- iov_iter: use kmemdup_array() in dup_iter() to harden the
allocation against multiplication overflow.
- fs/pipe: write to ->poll_usage only once.
- vfs: remove an always-taken if-branch in find_next_fd().
- dcache: use kmalloc_flex() for struct external_name in __d_alloc().
- namei: use QSTR() instead of QSTR_INIT() in path_pts().
- sync_file_range: delete dead S_ISLNK code.
- Comment fixes: retire a stale comment in fget_task_next() and fix
assorted spelling mistakes"
* tag 'vfs-7.2-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (73 commits)
backing-file: fix backing_file_open() kerneldoc parameter
iomap: pass the correct len to fserror_report_io in __iomap_write_begin
vfs: add FS_USERNS_DELEGATABLE flag and set it for NFS
filelock: fix break_lease() stub signature for CONFIG_FILE_LOCKING=n
vfs: uapi: retire octal and hex numbers in favor of (1 << n) for O_ flags
bpf: add bpf_real_inode() kfunc
fs/read_write: Do not export __kernel_write() to the entire world
libfs: drop redundant SB_I_NOEXEC/SB_I_NODEV in init_pseudo() callers
libfs: set SB_I_NOEXEC and SB_I_NODEV by default in init_pseudo()
mount: honour SB_NOUSER in the new mount API
fs/fcntl: fix SOFTIRQ-unsafe lock order in fasync signaling
selftests/pipe: add pipe_bench microbenchmark
fs/pipe: pre-allocate pages outside pipe->mutex in anon_pipe_write
fs: retire stale comment in fget_task_next()
fs: fix spelling mistakes in comment
bfs: replace get_zeroed_page() with kzalloc()
binfmt_misc: replace __get_free_page() with kmalloc()
configfs: replace __get_free_pages() with kzalloc()
fs/namespace: use __getname() to allocate mntpath buffer
fs/select: replace __get_free_page() with kmalloc()
...
Please consider pulling these changes from the signed vfs-7.2-rc1.iomap tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaiwLKgAKCRCRxhvAZXjc
os7sAPsH6z7HATCG9xOAOmsIwgi7l0u7dMNnU7Xrqs9H3qAjNgD/SJ/Ajw0GMkMP
RxJvze9lvkY9hN9avR9thMVF+yE+fgM=
=TlkI
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.2-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull iomap updates from Christian Brauner:
- Add the vfs infrastructure required to implement fs-verity support
for XFS with a post-EOF merkle tree: fsverity generates and stores a
zero-block hash, and iomap learns to verify data on buffered reads,
to handle fsverity during writeback via the new IOMAP_F_FSVERITY
flag, and to write fsverity metadata through iomap_fsverity_write().
- Skip the memset of the iomap in iomap_iter() once the iteration is
done. In high-IOPS scenarios (4k randread NVMe polling via io_uring)
the pointless memset wasted memory write bandwidth; this improves
IOPS by about 5% on ext4 and xfs.
- Add balance_dirty_pages_ratelimited() to iomap_zero_iter(), aligning
it with iomap_write_iter(). This prepares for the exFAT iomap
conversion where zeroing beyond valid_size can trigger large-scale
zeroing operations that caused memory pressure without throttling.
- Remove the over-strict inline data boundary check. If a filesystem
provides a valid inline_data pointer and length there is no reason to
require that inline data must not cross a page boundary.
- Don't make REQ_POLLED imply REQ_NOWAIT, matching the earlier
equivalent block layer fix: there are valid cases to poll for I/O
completion without REQ_NOWAIT, and REQ_NOWAIT for file system writes
is currently not supported as writes aren't idempotent.
- Introduce IOMAP_F_ZERO_TAIL for filesystems that maintain a separate
valid data length (exFAT, NTFS). For a write starting at or beyond
valid_size, __iomap_write_begin() now zeroes only the tail portion of
the block while preserving valid data before it, instead of leaving
stale data in the page cache. The flag is also added to the iomap
trace event strings.
* tag 'vfs-7.2-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
iomap: Add IOMAP_F_ZERO_TAIL flag to trace event strings
iomap: introduce iomap_fsverity_write() for writing fsverity metadata
iomap: teach iomap to read files with fsverity
iomap: introduce IOMAP_F_FSVERITY and teach writeback to handle fsverity
fsverity: generate and store zero-block hash
iomap: introduce IOMAP_F_ZERO_TAIL flag
iomap: don't make REQ_POLLED imply REQ_NOWAIT
iomap: remove over-strict inline data boundary check
iomap: add dirty page control to iomap_zero_iter
iomap: avoid memset iomap when iter is done
len is size of the (larger) write request, plen is the range for which
the read failed here.
Fixes: a9d573ee88 ("iomap: report file I/O errors to the VFS")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260610050642.1906695-1-hch@lst.de
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Please consider pulling these changes from the signed vfs-7.1-rc7.fixes tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaiQb2wAKCRCRxhvAZXjc
ol4fAP46Ta2gcT8uzJRHaXo3HMWJYY02dK1pXQ4zZDdiQMMSTgEAuaj/mOvx0VeX
prloV/t7TBri6GAiWeE7Wu54d9rJZQ4=
=clMX
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.1-rc7.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs fixes from Christian Brauner:
- Fix error handling in ovl_cache_get()
- Tighten access checks for exited tasks in pidfd_getfd()
- Fix selftests leak in __wait_for_test()
- Limit FUSE_NOTIFY_RETRIEVE to uptodate folios
- Reject fuse_notify() pagecache ops on directories
- Clear JOBCTL_PENDING_MASK for caller in zap_other_threads()
- Fix failure to unlock in nfsd4_create_file()
- Fix pointer arithmetic in qnx6 directory iteration
- Fix UAF due to unlocked ->mnt_ns read in may_decode_fh()
- Avoid potential null folio->mapping deref during iomap error
reporting
* tag 'vfs-7.1-rc7.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
iomap: avoid potential null folio->mapping deref during error reporting
fhandle: fix UAF due to unlocked ->mnt_ns read in may_decode_fh()
fs/qnx6: fix pointer arithmetic in directory iteration
VFS: fix possible failure to unlock in nfsd4_create_file()
signal: clear JOBCTL_PENDING_MASK for caller in zap_other_threads()
fuse: reject fuse_notify() pagecache ops on directories
fuse: limit FUSE_NOTIFY_RETRIEVE to uptodate folios
selftests: harness: fix pidfd leak in __wait_for_test
pidfd: refuse access to tasks that have started exiting harder
ovl: keep err zero after successful ovl_cache_get()
Add IOMAP_F_ZERO_TAIL to the flag string mapping in iomap trace
events. This allows the new flag to be properly displayed in
ftrace output when iomap operations use it.
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Link: https://patch.msgid.link/20260603144031.7370-1-linkinjeon@kernel.org
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
This is just a wrapper around iomap_file_buffered_write() to create
necessary iterator over metadata.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
Link: https://patch.msgid.link/20260520123722.405752-10-aalbersh@kernel.org
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
Obtain fsverity info for folios with file data and fsverity metadata.
Filesystem can pass vi down to ioend and then to fsverity for
verification. This is different from other filesystems ext4, f2fs, btrfs
supporting fsverity, these filesystems don't need fsverity_info for
reading fsverity metadata. While reading merkle tree iomap requires
fsverity info to synthesize hashes for zeroed data block.
fsverity metadata has two kinds of holes - ones in merkle tree and one
after fsverity descriptor.
Merkle tree holes are blocks full of hashes of zeroed data blocks. These
are not stored on the disk but synthesized on the fly. This saves a bit
of space for sparse files. Due to this iomap also need to lookup
fsverity_info for folios with fsverity metadata. ->vi has a hash of the
zeroed data block which will be used to fill the merkle tree block.
The hole past descriptor is interpreted as end of metadata region. As we
don't have EOF here we use this hole as an indication that rest of the
folio is empty. This patch marks rest of the folio beyond fsverity
descriptor as uptodate.
For file data, fsverity needs to verify consistency of the whole file
against the root hash, hashes of holes are included in the merkle tree.
Verify them too.
Issue reading of fsverity merkle tree on the fsverity inodes. This way
metadata will be available at I/O completion time.
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
Link: https://patch.msgid.link/20260520123722.405752-9-aalbersh@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
This flag indicates that I/O is for fsverity metadata.
In the write path skip i_size check and i_size updates as metadata is
past EOF. In writeback don't update i_size and continue writeback if
even folio is beyond EOF. In read path don't zero fsverity folios, again
they are past EOF.
The iomap_block_needs_zeroing() is also called from write path. For
folios of larger order we don't want to zero out pages in the folio as
these could contain other merkle tree blocks. For fsverity, filesystem
will request to read PAGE_SIZE memory regions. For data folios, iomap
will zero the rest of the folio for anything which is beyond EOF. We
don't want this for fsverity folios.
Christian Brauner <brauner@kernel.org> says:
Changed IOMAP_F_FSVERITY from (1U << 10) to (1U << 11) to avoid colliding
with IOMAP_F_ZERO_TAIL, which already uses (1U << 10).
Signed-off-by: Andrey Albershteyn <aalbersh@kernel.org>
Link: https://patch.msgid.link/20260520123722.405752-8-aalbersh@kernel.org
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
In filesystems that maintain a separate Valid Data Length, such as exFAT
and NTFS, a partial write may start at or beyond the current valid_size and
extend it. In this case, the region after the previous valid_size but
within the same filesystem block is considered unwritten.
This patch introduces IOMAP_F_ZERO_TAIL. When this flag is set in iomap,
__iomap_write_begin() will zero only the tail portion while preserving any
valid data before it in the same block.
Without this tail zeroing, stale data in the unwritten portion of the block
can remain in the page cache. Subsequent reads can then return incorrect
contents from that region.
Acked-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Link: https://patch.msgid.link/20260518114705.9601-2-linkinjeon@kernel.org
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
When a buffered read fails, iomap_finish_folio_read() reports the error
with fserror_report_io(folio->mapping->host, ...). This is called after
ifs->read_bytes_pending has been decremented by the bytes attempted to
be read.
For a folio split across multiple read completions, the folio is only
guaranteed to stay locked while read_bytes_pending > 0. Once
iomap_finish_folio_read() decrements read_bytes_pending, another
in-flight read can complete and end the read on the folio, which unlocks
it. This allows truncate logic to run and detach the folio (set
folio->mapping to NULL). The error reporting path then can dereference a
NULL folio->mapping. As reported by Sam Sun, this is the race that can
occur:
CPU0: failed completion CPU1: final completion CPU2: truncate
----------------------- ---------------------- --------------
read_bytes_pending -= len
finished = false
/* preempted before
fserror_report_io() */
read_bytes_pending -= len
finished = true
folio_end_read()
truncate clears
folio->mapping
fserror_report_io(
folio->mapping->host, ...)
^ NULL deref
Fix this by reporting the error first before decrementing
ifs->read_bytes_pending.
Fixes: a9d573ee88 ("iomap: report file I/O errors to the VFS")
Cc: stable@vger.kernel.org
Reported-by: Sam Sun <samsun1006219@gmail.com>
Closes: https://lore.kernel.org/linux-fsdevel/CAEkJfYPhWdd59RKmuNLJg-bkypHz7xiOwaWyNVu3A8CUqQCnvg@mail.gmail.com/
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260604011858.2297561-1-joannelkoong@gmail.com
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
As described in commit 2bc0576925 ("block: don't make REQ_POLLED imply
REQ_NOWAIT"), which fixed the same issue for the block device node, there
are valid cases to poll for I/O completion without REQ_NOWAIT.
Additionally, sing REQ_NOWAIT for file system writes is currently not
supported as file systems writes are not idempotent and would need a
retry of just the bio and not the entire operation to be fully supported.
Switch iomap to set REQ_POLLED and remove the now unused bio_set_polled
helper.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260518062917.506483-1-hch@lst.de
Signed-off-by: Christian Brauner <brauner@kernel.org>
When bouncing for block size > PAGE_SIZE file systems that require
file system block size alignment (e.g. zoned XFS), the bio needs to
be big enough to fit an entire block.
Fixes: 8dd5e7c75d ("block: add helpers to bounce buffer an iov_iter into bios")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Link: https://patch.msgid.link/20260507050153.1298375-2-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The current iomap_inline_data_valid() check ensures that inline data
does not cross a PAGE_SIZE boundary. However, this is an unnecessarily
strict constraint. If a filesystem provides a valid iomap::inline_data
pointer and iomap::length, we should trust that the caller has mapped
sufficient memory for the range, even if it spans across page boundaries.
Removing this check allows filesystems to point directly to their
internal data structures without forced page-alignment or additional
redundant allocations. This remove iomap_inline_data_valid() and
its callers in buffered and direct I/O paths.
Suggested-by: Matthew Wilcox <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Link: https://patch.msgid.link/20260511141151.6021-1-linkinjeon@kernel.org
Signed-off-by: Christian Brauner <brauner@kernel.org>
This patch prepares the iomap framework for exFAT's upcoming migration to
iomap. During testing of the exFAT iomap branch with xfstests generic/299 on
a VM with 8GB RAM and a 40GB disk, system unresponsiveness was observed.
iomap_zero_iter() lacked dirty page throttling, which could cause memory
pressure when exFAT's valid_size mechanism triggers large-scale zeroing
operations during writes beyond valid_size.
Align iomap_zero_iter() with iomap_write_iter() by adding
balance_dirty_pages_ratelimited() to throttle dirty page generation during
large zeroing operations
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Link: https://patch.msgid.link/20260511094007.728011-1-chizhiling@163.com
Reviewed-by: Christoph Hellwig <hch@lst.de>
Cc: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Christian Brauner <brauner@kernel.org>
When iomap_iter() finishes its iteration (returns <= 0), it is no longer
necessary to memset the entire iomap and srcmap structures.
In high-IOPS scenarios (like 4k randread NVMe polling with io_uring),
where the majority of I/Os complete in a single extent map, this wasted
memory write bandwidth, as the caller will just discard the iterator.
Use this command to test:
taskset -c 30 ./t/io_uring -p1 -d512 -b4096 -s32 -c32 -F1 -B1 -R1 -X1
-n1 -P1 /mnt/testfile
IOPS improve about 5% on ext4 and XFS.
However, we MUST still call iomap_iter_reset_iomap() to release the
folio_batch if IOMAP_F_FOLIO_BATCH is set, otherwise we leak page
references. Therefore, split the cleanup logic: always release the
folio_batch, but skip the memset() when ret <= 0.
Signed-off-by: Fengnan Chang <changfengnan@bytedance.com>
Link: https://patch.msgid.link/20260420061630.62077-1-changfengnan@bytedance.com
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <brauner@kernel.org>
- Various cleanups for the interface between fs/crypto/ and
filesystems, from Christoph Hellwig
- Simplify and optimize the implementation of v1 key derivation by
using the AES library instead of the crypto_skcipher API
-----BEGIN PGP SIGNATURE-----
iIoEABYIADIWIQSacvsUNc7UX4ntmEPzXCl4vpKOKwUCadV8xhQcZWJpZ2dlcnNA
a2VybmVsLm9yZwAKCRDzXCl4vpKOK0wSAPsGg/zd0bMiF9dcKKVESAVIePSKFbvx
5e1speATaXTSVAEAnkLLL1PZJJq9HOKpQY8Wkqpzy8Kmt8x53VeI4YW0fQc=
=Rb7e
-----END PGP SIGNATURE-----
Merge tag 'fscrypt-for-linus' of git://git.kernel.org/pub/scm/fs/fscrypt/linux
Pull fscrypt updates from Eric Biggers:
- Various cleanups for the interface between fs/crypto/ and
filesystems, from Christoph Hellwig
- Simplify and optimize the implementation of v1 key derivation by
using the AES library instead of the crypto_skcipher API
* tag 'fscrypt-for-linus' of git://git.kernel.org/pub/scm/fs/fscrypt/linux:
fscrypt: use AES library for v1 key derivation
ext4: use a byte granularity cursor in ext4_mpage_readpages
fscrypt: pass a real sector_t to fscrypt_zeroout_range
fscrypt: pass a byte length to fscrypt_zeroout_range
fscrypt: pass a byte offset to fscrypt_zeroout_range
fscrypt: pass a byte length to fscrypt_zeroout_range_inline_crypt
fscrypt: pass a byte offset to fscrypt_zeroout_range_inline_crypt
fscrypt: pass a byte offset to fscrypt_set_bio_crypt_ctx
fscrypt: pass a byte offset to fscrypt_mergeable_bio
fscrypt: pass a byte offset to fscrypt_generate_dun
fscrypt: move fscrypt_set_bio_crypt_ctx_bh to buffer.c
ext4, fscrypt: merge fscrypt_mergeable_bio_bh into io_submit_need_new_bio
ext4: factor out a io_submit_need_new_bio helper
ext4: open code fscrypt_set_bio_crypt_ctx_bh
ext4: initialize the write hint in io_submit_init_bio
Signed-off-by: Carlos Maiolino <cem@kernel.org>
-----BEGIN PGP SIGNATURE-----
iJUEABMJAB0WIQSmtYVZ/MfVMGUq1GNcsMJ8RxYuYwUCadyo1AAKCRBcsMJ8RxYu
YzqJAYDic0mU0VPFyPVQKGvlEw0QTPbWNrbnLa7POD3pyZ4yhTuroA5016YuMM2m
DouQN6sBgOTZ/kSRH/Rgui6dXmnG5VdR5dZo/ADcTmimfycqwXSgFnxEgTNN4eQE
Uq0drKXU1A==
=l6Ln
-----END PGP SIGNATURE-----
Merge tag 'xfs-merge-7.1' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux
Pull xfs updates from Carlos Maiolino:
"There aren't any new features.
The whole series is just a collection of bug fixes and code
refactoring. There is some new information added a couple new
tracepoints, new data added to mountstats, but no big changes"
* tag 'xfs-merge-7.1' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux: (41 commits)
xfs: fix number of GC bvecs
xfs: untangle the open zones reporting in mountinfo
xfs: expose the number of open zones in sysfs
xfs: reduce special casing for the open GC zone
xfs: streamline GC zone selection
xfs: refactor GC zone selection helpers
xfs: rename xfs_zone_gc_iter_next to xfs_zone_gc_iter_irec
xfs: put the open zone later xfs_open_zone_put
xfs: add a separate tracepoint for stealing an open zone for GC
xfs: delay initial open of the GC zone
xfs: fix a resource leak in xfs_alloc_buftarg()
xfs: handle too many open zones when mounting
xfs: refactor xfs_mount_zones
xfs: fix integer overflow in busy extent sort comparator
xfs: fix integer overflow in deferred intent sort comparators
xfs: fold xfs_setattr_size into xfs_vn_setattr_size
xfs: remove a duplicate assert in xfs_setattr_size
xfs: return default quota limits for IDs without a dquot
xfs: start gc on zonegc_low_space attribute updates
xfs: don't decrement the buffer LRU count for in-use buffers
...
Please consider pulling these changes from the signed vfs-7.1-rc1.kino tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCadjZCgAKCRCRxhvAZXjc
otmnAP4sbsxZQdz2TG2hJuOwnEZOkkxZQOUMc3ERVyZaWXIeTAEA7e5M+8FpoG9n
8ipO76UoaXdGLESrqVdp9EOhLqOW7QY=
=uMeJ
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.1-rc1.kino' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs i_ino updates from Christian Brauner:
"For historical reasons, the inode->i_ino field is an unsigned long,
which means that it's 32 bits on 32 bit architectures. This has caused
a number of filesystems to implement hacks to hash a 64-bit identifier
into a 32-bit field, and deprives us of a universal identifier field
for an inode.
This changes the inode->i_ino field from an unsigned long to a u64.
This shouldn't make any material difference on 64-bit hosts, but
32-bit hosts will see struct inode grow by at least 4 bytes. This
could have effects on slabcache sizes and field alignment.
The bulk of the changes are to format strings and tracepoints, since
the kernel itself doesn't care that much about the i_ino field. The
first patch changes some vfs function arguments, so check that one out
carefully.
With this change, we may be able to shrink some inode structures. For
instance, struct nfs_inode has a fileid field that holds the 64-bit
inode number. With this set of changes, that field could be
eliminated. I'd rather leave that sort of cleanups for later just to
keep this simple"
* tag 'vfs-7.1-rc1.kino' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
nilfs2: fix 64-bit division operations in nilfs_bmap_find_target_in_group()
EVM: add comment describing why ino field is still unsigned long
vfs: remove externs from fs.h on functions modified by i_ino widening
treewide: fix missed i_ino format specifier conversions
ext4: fix signed format specifier in ext4_load_inode trace event
treewide: change inode->i_ino from unsigned long to u64
nilfs2: widen trace event i_ino fields to u64
f2fs: widen trace event i_ino fields to u64
ext4: widen trace event i_ino fields to u64
zonefs: widen trace event i_ino fields to u64
hugetlbfs: widen trace event i_ino fields to u64
ext2: widen trace event i_ino fields to u64
cachefiles: widen trace event i_ino fields to u64
vfs: widen trace event i_ino fields to u64
net: change sock.sk_ino and sock_i_ino() to u64
audit: widen ino fields to u64
vfs: widen inode hash/lookup functions to u64
Please consider pulling these changes from the signed vfs-7.1-rc1.integrity tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCadjZCgAKCRCRxhvAZXjc
ogmqAQCMD0V+pHwBAGwPQYBPc6Tf4LCxqNAmt/kypYsdVWkweQEAxbbeBPUigid5
QeO0zCunikgIjMyJDXINpUPgsLYvyAo=
=q2L6
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.1-rc1.integrity' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs integrity updates from Christian Brauner:
"This adds support to generate and verify integrity information (aka
T10 PI) in the file system, instead of the automatic below the covers
support that is currently used.
The implementation is based on refactoring the existing block layer PI
code to be reusable for this use case, and then adding relatively
small wrappers for the file system use case. These are then used in
iomap to implement the semantics, and wired up in XFS with a small
amount of glue code.
Compared to the baseline this does not change performance for writes,
but increases read performance up to 15% for 4k I/O, with the benefit
decreasing with larger I/O sizes as even the baseline maxes out the
device quickly on my older enterprise SSD"
* tag 'vfs-7.1-rc1.integrity' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
xfs: support T10 protection information
iomap: support T10 protection information
iomap: support ioends for buffered reads
iomap: add a bioset pointer to iomap_read_folio_ops
ntfs3: remove copy and pasted iomap code
iomap: allow file systems to hook into buffered read bio submission
iomap: only call into ->submit_read when there is a read_ctx
iomap: pass the iomap_iter to ->submit_read
iomap: refactor iomap_bio_read_folio_range
block: pass a maxlen argument to bio_iov_iter_bounce
block: add fs_bio_integrity helpers
block: make max_integrity_io_size public
block: prepare generation / verification helpers for fs usage
block: add a bdev_has_integrity_csum helper
block: factor out a bio_integrity_setup_default helper
block: factor out a bio_integrity_action helper
Zorro Lang reported the following lockdep splat:
"While running fstests xfs/556 on kernel 7.0.0-rc4+ (HEAD=04a9f1766954),
a lockdep warning was triggered indicating an inconsistent lock state
for sb->s_type->i_lock_key.
"The deadlock might occur because iomap_read_end_io (called from a
hardware interrupt completion path) invokes fserror_report, which then
calls igrab. igrab attempts to acquire the i_lock spinlock. However,
the i_lock is frequently acquired in process context with interrupts
enabled. If an interrupt occurs while a process holds the i_lock, and
that interrupt handler calls fserror_report, the system deadlocks.
"I hit this warning several times by running xfs/556 (mostly) or
generic/648 on xfs. More details refer to below console log."
along with this dmesg, for which I've cleaned up the stacktraces:
run fstests xfs/556 at 2026-03-18 20:05:30
XFS (sda3): Mounting V5 Filesystem 396e9164-c45a-4e05-be9d-b38c2c5c6477
XFS (sda3): Ending clean mount
XFS (sda3): Unmounting Filesystem 396e9164-c45a-4e05-be9d-b38c2c5c6477
XFS (sda3): Mounting V5 Filesystem bf3f89c3-3c45-4650-a9c7-744f39c0191e
XFS (sda3): Ending clean mount
XFS (sda3): Unmounting Filesystem bf3f89c3-3c45-4650-a9c7-744f39c0191e
XFS (dm-0): Mounting V5 Filesystem bf3f89c3-3c45-4650-a9c7-744f39c0191e
XFS (dm-0): Ending clean mount
device-mapper: table: 253:0: adding target device (start sect 209 len 1) caused an alignment inconsistency
device-mapper: table: 253:0: adding target device (start sect 210 len 62914350) caused an alignment inconsistency
buffer_io_error: 6 callbacks suppressed
Buffer I/O error on dev dm-0, logical block 209, async page read
Buffer I/O error on dev dm-0, logical block 209, async page read
XFS (dm-0): Unmounting Filesystem bf3f89c3-3c45-4650-a9c7-744f39c0191e
XFS (dm-0): Mounting V5 Filesystem bf3f89c3-3c45-4650-a9c7-744f39c0191e
XFS (dm-0): Ending clean mount
================================
WARNING: inconsistent lock state
7.0.0-rc4+ #1 Tainted: G S W
--------------------------------
inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage.
od/2368602 [HC1[1]:SC0[0]:HE0:SE1] takes:
ff1100069f2b4a98 (&sb->s_type->i_lock_key#31){?.+.}-{3:3}, at: igrab+0x28/0x1a0
{HARDIRQ-ON-W} state was registered at:
__lock_acquire+0x40d/0xbd0
lock_acquire.part.0+0xbd/0x260
_raw_spin_lock+0x37/0x80
unlock_new_inode+0x66/0x2a0
xfs_iget+0x67b/0x7b0 [xfs]
xfs_mountfs+0xde4/0x1c80 [xfs]
xfs_fs_fill_super+0xe86/0x17a0 [xfs]
get_tree_bdev_flags+0x312/0x590
vfs_get_tree+0x8d/0x2f0
vfs_cmd_create+0xb2/0x240
__do_sys_fsconfig+0x3d8/0x9a0
do_syscall_64+0x13a/0x1520
entry_SYSCALL_64_after_hwframe+0x76/0x7e
irq event stamp: 3118
hardirqs last enabled at (3117): [<ffffffffb54e4ad8>] _raw_spin_unlock_irq+0x28/0x50
hardirqs last disabled at (3118): [<ffffffffb54b84c9>] common_interrupt+0x19/0xe0
softirqs last enabled at (3040): [<ffffffffb290ca28>] handle_softirqs+0x6b8/0x950
softirqs last disabled at (3023): [<ffffffffb290ce4d>] __irq_exit_rcu+0xfd/0x250
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(&sb->s_type->i_lock_key#31);
<Interrupt>
lock(&sb->s_type->i_lock_key#31);
*** DEADLOCK ***
1 lock held by od/2368602:
#0: ff1100069f2b4b58 (&sb->s_type->i_mutex_key#19){++++}-{4:4}, at: xfs_ilock+0x324/0x4b0 [xfs]
stack backtrace:
CPU: 15 UID: 0 PID: 2368602 Comm: od Kdump: loaded Tainted: G S W 7.0.0-rc4+ #1 PREEMPT(full)
Tainted: [S]=CPU_OUT_OF_SPEC, [W]=WARN
Hardware name: Dell Inc. PowerEdge R660/0R5JJC, BIOS 2.1.5 03/14/2024
Call Trace:
<IRQ>
dump_stack_lvl+0x6f/0xb0
print_usage_bug.part.0+0x230/0x2c0
mark_lock_irq+0x3ce/0x5b0
mark_lock+0x1cb/0x3d0
mark_usage+0x109/0x120
__lock_acquire+0x40d/0xbd0
lock_acquire.part.0+0xbd/0x260
_raw_spin_lock+0x37/0x80
igrab+0x28/0x1a0
fserror_report+0x127/0x2d0
iomap_finish_folio_read+0x13c/0x280
iomap_read_end_io+0x10e/0x2c0
clone_endio+0x37e/0x780 [dm_mod]
blk_update_request+0x448/0xf00
scsi_end_request+0x74/0x750
scsi_io_completion+0xe9/0x7c0
_scsih_io_done+0x6ba/0x1ca0 [mpt3sas]
_base_process_reply_queue+0x249/0x15b0 [mpt3sas]
_base_interrupt+0x95/0xe0 [mpt3sas]
__handle_irq_event_percpu+0x1f0/0x780
handle_irq_event+0xa9/0x1c0
handle_edge_irq+0x2ef/0x8a0
__common_interrupt+0xa0/0x170
common_interrupt+0xb7/0xe0
</IRQ>
<TASK>
asm_common_interrupt+0x26/0x40
RIP: 0010:_raw_spin_unlock_irq+0x2e/0x50
Code: 0f 1f 44 00 00 53 48 8b 74 24 08 48 89 fb 48 83 c7 18 e8 b5 73 5e fd 48 89 df e8 ed e2 5e fd e8 08 78 8f fd fb bf 01 00 00 00 <e8> 8d 56 4d fd 65 8b 05 46 d5 1d 03 85 c0 74 06 5b c3 cc cc cc cc
RSP: 0018:ffa0000027d07538 EFLAGS: 00000206
RAX: 0000000000000c2d RBX: ffffffffb6614bc8 RCX: 0000000000000080
RDX: 0000000000000000 RSI: ffffffffb6306a01 RDI: 0000000000000001
RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000
R10: ffffffffb75efc67 R11: 0000000000000001 R12: ff1100015ada0000
R13: 0000000000000083 R14: 0000000000000002 R15: ffffffffb6614c10
folio_wait_bit_common+0x407/0x780
filemap_update_page+0x8e7/0xbd0
filemap_get_pages+0x904/0xc50
filemap_read+0x320/0xc20
xfs_file_buffered_read+0x2aa/0x380 [xfs]
xfs_file_read_iter+0x263/0x4a0 [xfs]
vfs_read+0x6cb/0xb70
ksys_read+0xf9/0x1d0
do_syscall_64+0x13a/0x1520
Zorro's diagnosis makes sense, so the solution is to kick the failed
read handling to a workqueue much like we added for writeback ioends in
commit 294f54f849 ("fserror: fix lockdep complaint when igrabbing
inode").
Cc: Zorro Lang <zlang@redhat.com>
Link: https://lore.kernel.org/linux-xfs/20260319194303.efw4wcu7c4idhthz@doltdoltdolt/
Fixes: a9d573ee88 ("iomap: report file I/O errors to the VFS")
Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
Link: https://patch.msgid.link/20260323210017.GL6223@frogsfrogsfrogs
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Now that the zero range pagecache flush is purely isolated to
providing zeroing correctness in this case, we can remove it and
replace it with the folio batch mechanism that is used for handling
unwritten extents.
This is still slightly odd in that XFS reports a hole vs. a mapping
that reflects the COW fork extents, but that has always been the
case in this situation and so a separate issue. We drop the iomap
warning that assumes the folio batch is always associated with
unwritten mappings, but this is mainly a development assertion as
otherwise the core iomap fbatch code doesn't care much about the
mapping type if it's handed the set of folios to process.
Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Carlos Maiolino <cem@kernel.org>
