-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmqGD2AQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpomLD/9rr3SIQvdQRTnSCJlw1cFypSrnRhGAx1g8
v1F7UXuf4Abt4GIerbdXmBTx9zZRctpcOC41ReQk3YKkYuukuvna+Z6tBGf5I5+e
3SR3Zkrj2k4zyNC49g0/QkUJkTTS4tlsBFhNJtY3Q5BBIRMRd/sPvZ+cLnD7g/wH
3zM9IaEMk9xu9mF1nGI7LRl1qY0IOr4ynQpqohl5sEL3TQWgIa0r2B0jhGacBAvC
GGDwIwGDrM6oGwCn6mLN78O5odhLyTufIMNy3EnVw7BcDpLApcnJzYYjGzv+ElCM
WLBJwWYj+PYeoenuJuFhbRihbOPg5gZT3Mn8CUhLxUYzwZvIdFGWOjcGffD9sVu/
JO4nB6A+2AG/wgsb2YVKefhN86FoD0F/2hhdEdgZPJE6a+7+xeyyp9IzyQEWd0cg
Seo06fxE4pwF+dczXT3jzU3LXSZb/Jbh9hzu163VMSWbGYDCpDzfipqZGeN6DWE7
nG2NssAQtjtZrgdggYbZXtwecLZhu3cMj3jbhS4wSUzhQFW42yceJstCx4wpRhW9
m9EXjHQnRFMAb2TjUC3FzEncjbxL+uACjcKhTetd1hLbTI5I8KArZXjam1OdJOHV
urdtsru79LanQLAgRwWYPeKCZGNhX7I2o3OsmVDLQgHIDaZguTcPUVQs3wVui7Dk
IDmmrtwzpg==
=G1l4
-----END PGP SIGNATURE-----
Merge tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Enable Clang context analysis for the nvme host driver, adding
context annotations across core, fabrics, rdma, tcp and pci
- nvmet reservation state exposed through a new namespace-level
debugfs directory, plus ABI documentation for the host sysfs and
target configfs interfaces
- nvme-tcp host memory disclosure fixes on the read path: reject a
read that transferred too few bytes, don't accept C2HData based
on blk_rq_payload_bytes() alone, and fix the R2T case for a read
command
- Parallelize nvme-rdma I/O queue allocation and startup (Surabhi)
- Apple nvme fixes and quirks: page aligned admin queue buffers,
destroy the admin queue on removal, and various DMA/NVMMU
correctness fixes
- A large pile of nvmet and host fixes for out-of-bounds reads,
refcount/resource leaks, and NULL derefs across auth, zns,
passthru, pci-epf, rdma and configfs
- Various other fixes and cleanups
- MD updates via Yu Kuai:
- llbitmap reshape support, the large series wiring exact bitmap
mapping and reshape lifecycle through raid5 and raid10, growing
the page cache in place, and remapping checkpointed bits as
reshape progresses
- raid5 fixes for lockless max_nr_stripes and recovery_offset
accesses, a reshape deadlock with more failed devices than max
degraded, and bitmap batch counter consistency
- Atomic write handling for raid1/raid10, and removal of the
REQ_NOWAIT support from raid1/10/456
- raid5-ppl use-after-free fix in ppl_do_flush()
- A batch of smaller fixes across md core and the bitmap code
- s390/dasd ESE full-track write support and the surrounding
infrastructure, plus enabling CONTEXT_ANALYSIS for s390/block
- RWF_DONTCACHE support for block devices, built on new task-context
bio completion infrastructure, and wiring it up for the iomap and
buffer dropbehind writeback paths
- Async io_uring zone reset all, plus zone management command cleanups
allowing REQ_NOWAIT and tightening conventional zone rejection
- Block integrity refactoring: lift BIP_CHECK_FLAGS to the shared
header, handle nogenerate/noverify properly in fs-integrity, and drop
the blk-integrity.h include from bdev.c
- Split out a new blk_plug.h header
- ublk improvements: add UBLK_F_IO_DESC_SIZE, split request validation
from io_desc init, reject non-power-of-2 zone sizes in SET_PARAMS,
and a series of hardening fixes around map/unmap and auto buf reg
- null_blk cleanups and configfs serialization fixes
- nbd queue freeze removal on the setup paths, and a new
pre_defined_connections module parameter for pre-created devices
- blk-cgroup fixes for the race between policy activation and blkg
destruction, and accounting per-cpu stats over possible CPUs across
blk-stat, iolatency, iocost and kyber
- Various dio fixes: leak on metadata mapping error, validate user
space vectors during extraction, and set dma_alignment from the
backing file for loop and zloop direct I/O
- bio cleanups
- Various other fixes and cleanups all over
* tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (241 commits)
nbd: add pre_defined_connections module parameter for pre-created devices
nbd: remove queue freeze for newly created nbd from netlink path
nbd: factor out a nbd_genl_foreach_sock
nbd: skip queue freeze when setting size at device startup
nbd: remove queue freeze in nbd_add_socket
nbd: clear queue limits on disconnect
nbd: disallow NBD_SET_SOCK on an active device
nbd: simplify find_fallback() by removing redundant logic
blk-mq: add missing call to srcu_barrier() in blk_mq_free_tag_set()
block: mtip32xx: synchronize ioctls with device removal
ublk: avoid teardown retry loop on xarray allocation failure
null_blk: fix UBSAN shift-out-of-bounds when zone_size is 0 or overflows
block: don't include blk-integrity.h in bdev.c
xfs: avoid double deferrals for RWF_DONTCACHE writes
loop: Fix recently introduced lock inversion
block: set QUEUE_FLAG_DYING unconditionally in blk_mark_disk_dead()
swim3: Add missing MODULE_DESCRIPTION
selftests: ublk: add SET_PARAMS validation test
selftests: ublk: add helper for SET_PARAMS
ublk: reject non-power-of-2 zone sizes in SET_PARAMS
...
-----BEGIN PGP SIGNATURE-----
iQJPBAABCgA5FiEE8rQSAMVO+zA4DBdWxWXV+ddtWDsFAmqEFbYbFIAAAAAABAAO
bWFudTIsMi41KzEuMTIsMiwyAAoJEMVl1fnXbVg7/R0P/jNxH2bHnc3yNLTuSuIA
hj6QNSeTF6Q8bY5HGLbxIURr1npcODXGAFSZK5/vONZWgjSVy8j65gmciOPrD6Aq
YF9zjFY4JuRfltx2E1aOKLyeozq/LPPs/3VBcfTHv2vWiYEabkpvZrNu9Fu7iX7c
9TXSHzuBGp+Ao9faj9i0V/6en7hRxA6NW00FoIxvyANglJqcV35NV68zTuUeY62C
8pFoDpDMV2ePQZW9l4EUsgqsEL6/PF362HmVyWUREYtnfpqJuzTADMV5NH29jXN1
ZHw/P8La7gkHnzaJqDK1Nhs1L2vYLAdtIi5XdCMRCzHfuT2s7Wpo2VuXlpN77bP+
nR7ZMxbg6zAIvxAc246bYEUQ6LNxVTZ4+7gSDJXUbWdpM1zYoHYLPAvelsCvrpq+
2RJgChcEPNPHpQ4TXHLLERaJ6IvYFX1zfH+tU4yOGrapsF9wvVWpliR5dyJOGbj3
RDOmZdV1XHZzstb5muBgigC2Tqh/z4h2Inl3ShucxDpZUUAl7wXNAuogfUV88ioh
/zHqcSJVFYiSFi7m4Ml//TQgbtezUwFPHWdVoUybqHxAiBzC1HZ2/HBAyn93Cd8y
PbNYoF7D3bU7UQ7g+aJj9lbOm4ttamg00R3D3JGfYneufUOJWGAnditQv0xakORG
RXbq24AW0Im6O9zNU+E/iaCT
=rFrX
-----END PGP SIGNATURE-----
Merge tag 'for-7.3-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux
Pull btrfs updates from David Sterba:
"This is the summer edition of btrfs changes, smaller than usual. Yet,
there are performance improvements in various areas or for specific
workloads and some notable changes like removing space cache v1 code
or mount option reduction.
User visible changes:
- free space v1 disabled by default; the v2 (free space tree) is mkfs
default since 5.15, filesystems with v1 still work but could be
slightly slower due to lack of block group caching
- mount option 'rescue=usebackuproot' requires read-only mount, it's
too risky to allow writable mount
- remove standalone mount option 'usebackuproot', deprecated in 5.9
- preserve constraints of NODATASUM and NODATACOW when chattr and
mount options may change the attributes
- remove arbitrary limitation of 4KiB for page size when allowing
block sizes smaller than page
- print messages when pinned block groups affect swap activation
Performance improvements:
- use iomap bounce buffer for direct io instead of a fall back to
buffered io; past correctness vs speed trade-offs dropped
performance to ~50% of theoretical maximum, now it's ~95%,
effectively doubled
- replace xarray with local LRU list for tracking inhibited
extent buffers, restored performance to pre-inhibition state
(relatively ~3x)
- remove unnecessary 1 jiffy delay in "non-SSD" mode with multiple
logging tasks, decrease latency, throughput increased ~5x on sample
workload
- skip hole detection during full fsync for files without holes
and lots of extents, reduce run time ~5x on sample workload
(microsecond ranges)
- reduce locking around extent readahead so it does not slow down
other tasks using an overlapping range
- enhance extent buffer allocation modes to allow NOWAIT semantics
in some cases
Notable fixes:
- write-protect folios during writeback, prevent concurrent mmap
and compress/checksumming/etc undesired interactions
- in zoned mode, handle transient overcommit full instead of going
read-only
- fix possible deadlock between defragmentation and delayed
allocation reservations
- handle remaining iputs at umount time
- fix lockdep warning between device scan locking and log mutex
- add workaround for degenerate RAID56 device count modes (2 and 3)
not supported by the parity calculation library
- restore check that subvolume is not read-only when changing ACLs
- retry reading verity data colliding with up-to-date status changes
Core:
- simplify raid56 stripe handling by using contiguous virtual
allocations
- in zoned mode, fix various metadata write issues in writeback or
unmount
- space reservation fixes
- remove unused data structure members
- more auto-freeing conversions
- error pointer values are printed using %pe format"
* tag 'for-7.3-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux: (72 commits)
btrfs: skip hole detection during full fsync for files without holes
btrfs: add extra ASSERT()s to make sure the folio size is correct
btrfs: use GFP_NOWAIT for tree block readahead
btrfs: enable unlocked NOFAIL retry for eb allocations
btrfs: add struct btrfs_eb_prealloc
btrfs: factor init_extent_buffer from __alloc_extent_buffer
btrfs: qgroup: fix a wrong length calculation in qgroup_free_reserved_data()
btrfs: add validation for extent states
btrfs: use aligned range for locking in reflink
btrfs: use aligned range for locking in extent_fiemap()
btrfs: zoned: don't clobber the extent buffer when zeroing it out
btrfs: zoned: drop stranded dirty metadata buffers at unmount
btrfs: zoned: drop stranded dirty metadata on transaction abort
btrfs: zoned: flush active metadata block group at btree_writepages() start
btrfs: convert reflink.c to use btrfs_inode as parameters
btrfs: use simple booleans for log_commit field in struct btrfs_root
btrfs: check for exit condition after waking in wait_log_commit()
btrfs: move condition for log commit wait into wait_log_commit()
btrfs: remove log batch counter use for fsync
btrfs: stop sleeping for one jiffy in non-ssd mounts during log commit
...
Please consider pulling these changes from the signed vfs-7.3-rc1.super tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCan7RJQAKCRCRxhvAZXjc
oroBAP40BufiChnC6I9Ev79QWF+Pfri1r/3AR4xXRG4HkXQaYQD/YE+Ria3+Bt35
+tWU66cYlQ/+9LfU8DJ3KXm7jPUBiQA=
=hXLH
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc1.super' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs superblock updates from Christian Brauner:
- Make it possible to share a block device between multiple
filesystems.
erofs can mount read-only blob devices shared between many
superblocks, but because we only tracked a single superblock a
freeze, thaw, removal or sync on such a device was never propagated
to all the superblocks using it, and there was no way to find them.
Add an efficient table to lookup all superblocks using a given block
device.
- A bunch of pre-existing fixes fell out of this work:
A block-device freeze racing a btrfs device change could leave the
whole filesystem stuck frozen. A bdev_freeze() issued by "dmsetup
suspend" or an LVM snapshot resolves that holder to freeze the
filesystem. and bdev_thaw() resolves it again to thaw. A freeze
landing while btrfs is adding, removing or replacing a device freezes
the filesystem. The membership change then drops that link. So the
matching thaw could no longer find the superblock.
Forbid freezing a device for the duration of a membership change,
modelled on deny_write_access()/allow_write_access().
* tag 'vfs-7.3-rc1.super' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (24 commits)
super: fix dying superblock warning messages
block: reject block device inodes with i_rdev == 0 in lookup_bdev()
selftests/filesystems: add ustat() coverage
fs: look up the superblock via the device table in user_get_super()
super: make fs_holder_ops private
f2fs: open via dedicated fs bdev helpers
erofs: open via dedicated fs bdev helpers
fs: tolerate per-superblock freeze errors on shared devices
fs: look up superblocks via the device table in fs_holder_ops
ext4: open via dedicated fs bdev helpers
btrfs: open via dedicated fs bdev helpers
xfs: port to fs_bdev_file_open_by_path()
fs: add dedicated block device open helpers for filesystems
fs: maintain a global device-to-superblock table
ocfs2: don't reset s_dev on dismount
ext4: use anonymous devices for KUnit test superblocks
fs, block: move blk_mode_t and fop_flags_t into <linux/types.h>
super: take lock after last reference count
super: convert s_count to refcount_t s_passive
btrfs: deny freezing devices undergoing a replace
...
Please consider pulling these changes from the signed vfs-7.3-rc1.misc tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCan7RJQAKCRCRxhvAZXjc
otI6AQCIwshv02UI9fmbTC9pFr8b+v/9IWQwdVULaH61HXgICQD7BE/CktL8nfRs
zCzGS6kBdWgiOA6bdVpgGGLqiMRnQgo=
=Kc+G
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull misc vfs updates from Christian Brauner:
"Bigger cleanups:
- The lockref dead-count handling is tidied up.
The open-coded check for a count below zero as the dead marker
relies on information the caller should not have.
- make put_mnt_ns() leave mounts connected. Destroying a mount
namespace disconnected its mounts from their mount points. So a
file descriptor still open on the parent of a mount point could be
used to peek under it.
Locked mounts were already kept connected to prevent exactly that.
But a mount is only locked when its tree is copied across a user
namespace boundary. So a mount namespace set up by a privileged
component had no locked mounts and its mounts were disconnected.
Passing UMOUNT_CONNECTED keeps every mount connected and prevents
that bug.
- vfs_prepare_mode() passes S_IFDIR for directories. I meant to fix
that ago but didn't get to it. So now someone finally did it.
This kills the exception where the mode could be 0 when a directory
was created whereas every other creation operation passed it
explicitly already.
- move long delayed work for ufs, jffs2, hfsplus, hfs and affs from
the per-cpu system_long_wq to the new unbound system_dfl_long_wq.
None of that work relies on per-cpu state and the work item is
enqueued with queue_delayed_work() whose timer is global anyway. So
it may as well benefit from scheduler task placement.
Smaller fixes and cleanups:
- unlock_buffer() and journal_end_buffer_io_sync() use
clear_and_wake_up_bit()
- the pipe page pools are unified into a single per-pipe pool and the
extra wake_up(rd_wait) is limited to EPOLLET consumers
- eventpoll now computes its timer slack lazily in ep_poll()
- shrink_dcache_for_umount() keeps making progress on busy roots
- excess xarray nodes are freed in clear_inode()
- romfs detects hard link cycles
- the user path of nested backing files is fixed
- pidfd holds exec_update_lock around the namespace ioctl
- non-memcg-aware nr_cached_objects is skipped during memcg slab
shrink
- iomap_write_iter() always returns status
- mangle_path() is renamed to seq_mangle_path()
- inode timestamp accessors are annotated
- new regression test for pipe->poll_usage.
- a few documentation, kernel-doc and selftest fixes"
* tag 'vfs-7.3-rc1.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (67 commits)
selftests/namespaces: Fix racy pipe handshake in timens and pidns_separate
selftests/epoll: add a regression test for pipe->poll_usage
pipe: only enable the extra wake_up(rd_wait) for EPOLLET consumers
pidfd: hold exec_update_lock around namespace ioctl
fs: fix user path of nested backing files
fs: remove stale inode_insert5() kernel-doc parameter
fs: fix switch/case indentation in sysfs() syscall
fs: document semantics of kstat::{uid,gid} fields
dcache: keep shrink_dcache_for_umount() making progress on busy roots
seq_file: rename mangle_path to seq_mangle_path
nstree: add/fix struct ns_id_req kernel-doc member fields
dcache: use lockref routines for dead count checks
lockref: tidy up dead count handling
initramfs: fix typo in reserve_initrd_mem comment
fs/pipe: unify the page pools into a single per-pipe pool
fs: annotate inode timestamp accessors
eventpoll: compute timer slack lazily in ep_poll()
selftests/filesystems: add mntns cleanup test
put_mnt_ns(): leave mounts connected
affs: Move long delayed work on system_dfl_long_wq
...
Please consider pulling these changes from the signed vfs-7.3-rc1.lookup tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCan7RJQAKCRCRxhvAZXjc
ouj6AQCnBHstCt5RRimywLYU+eTmdKe203zRQYxXpwlYOAzm7wEAm1mIC7/Of6bD
jkiJilcPDcyYcJsqwBw/Vcz5HR+v8wo=
=TX6K
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc1.lookup' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs lookup updates from Christian Brauner:
"This refactors lookup_open() and adds vfs_lookup_open() for nfsd.
mnt_want_write() and parent locking are moved into lookup_open()
itself.
audit_inode_child() is also now called in lookup_open() on failure.
That is the calling convention in vfs_create() and vfs_mkdir(), but
lookup_open() made no such call when atomic_open() should have created
a file and did not. And neither did the regular ->create() path fwiw.
This also contains work to remove the unneeded excl argument from the
->create() inode op"
* tag 'vfs-7.3-rc1.lookup' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
fs/namei.c: fix coding style in atomic_open() and lookup_open()
fs/namei.c: fix kerneldoc of atomic_open() and vfs_lookup_open()
fs/namei.c: update stale comments in lookup_open()
Remove excl arg to ->create inode_operation
fs/namei.c: update kerneldoc of atomic_open()
vfs: call audit_inode_child() in lookup_open() on failure
vfs: move create error && negative dentry case in lookup_open() up
VFS: add vfs_lookup_open() for nfsd
VFS: move delegated_inode retry loop into lookup_open()
VFS: move mnt_want_write() and locking into lookup_open()
Please consider pulling these changes from the signed vfs-7.3-rc1.kthread tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCan7RJQAKCRCRxhvAZXjc
ohHuAQCFeGhxqsb44KhuJaWXewiXbefr6eg31lXmxgSfg0DILAD/Tw/WmpiUbX5M
H+/jh6NmLKnQ/ADjlWFRy3R1K1Il3w4=
=jwPP
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc1.kthread' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull kthread vfs updates from Christian Brauner:
"This stops kernel threads from sharing filesystem state with
userspace. This work is about 3 cycles old and has been in -next
for about that time.
When the kernel boots init_task creates PID 1 and then kthreadd. From
that point every kthread and PID 1 share the same fs_struct. That is
why pivot_root() has to rewrite the fs_struct of all kthreads. The
rewriting exists so that kthreads can use init's filesystem state when
they want to. It also means userspace can move the ground out from
under the kernel.
PID 1 now gets a completely separate fs_struct. All kthreads are
anchored in a private SB_KERNMOUNT instance of nullfs that cannot be
mounted on and cannot be used to follow other mounts. Userspace init
can no longer affect kthread filesystem state and kthreads can no
longer affect userspace fs state without explicit opting in to that.
Path lookup from a kthread now fails by default. It makes it
deliberately hard to offload security sensitive operations into init's
filesystem state from a kthread.
Places that legitimately need to look something up there opt in
through the new scoped_with_init_fs() which temporarily overrides the
caller's fs_struct with init's. usermodehelpers remain the only kernel
tasks that genuinely share init's filesystem state, since they execute
random binaries in the root filesystem (excellent...).
The visible result is that /proc/2/root is a nullfs with an empty
mountinfo while /proc/1/root is the real root"
* tag 'vfs-7.3-rc1.kthread' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (26 commits)
initramfs_test: use test init/exit hooks to override init fs
fs: stop rewriting paths for PF_EXITING | PF_DUMPCORE
fs: stop rewriting kthread fs structs
fs: start all kthreads in nullfs
nullfs: make nullfs multi-instance
devtmpfs: create private mount namespace
fs: add umh argument to struct kernel_clone_args
fs: stop sharing fs_struct between init_task and pid 1
af_unix: use scoped_with_init_fs() for coredump socket lookup
initramfs: use scoped_with_init_fs() for rootfs unpacking
pnfs/blocklayout: use scoped_with_init_fs() for SCSI device lookup
ksmbd: use scoped_with_init_fs() for VFS path operations
ksmbd: use scoped_with_init_fs() for filesystem info path lookup
ksmbd: use scoped_with_init_fs() for share path resolution
fs: use scoped_with_init_fs() for kernel_read_file_from_path_initns()
coredump: use scoped_with_init_fs() for coredump path resolution
btrfs: use scoped_with_init_fs() for update_dev_time()
scsi: target: use scoped_with_init_fs() for APTPL metadata
scsi: target: use scoped_with_init_fs() for ALUA metadata
crypto: ccp: use scoped_with_init_fs() for SEV file access
...
Please consider pulling these changes from the signed vfs-7.3-rc1.iomap tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCan7RJQAKCRCRxhvAZXjc
oqOjAP9eZsO4M7OQpTKPgg3Sq2O9QCVgXBy4SVFba7DrbZHLZAEA/NWE+4udJzeL
j1Xb2VAyMc4S+Ze/O8sMml5Dyh4V5wE=
=34r1
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull iomap updates from Christian Brauner:
"The bulk of this is the conversion of iomap to a single ->iomap_next()
callback and thus finishing the move to an iterator model.
Every iomap operation drove its iteration through a struct iomap_ops
holding ->iomap_begin() and ->iomap_end(). iomap_iter() only ever sees
those as pointers. That means every step of every iteration is an
indirect call.
This collapses both into one ->iomap_next() callback that finishes the
previous mapping and produces the next one. This lets callers inline
the iteration loop and pass its ->iomap_next() as a compile time
constant. That means the compiler can turn it into a direct and hence
inlineable call.
This also allows future callers to express custom logic to drive the
iteration forward better. xfs, btrfs, ext4, ext2, erofs, f2fs, gfs2,
hpfs, fuse, exfat, zonefs, ntfs, ntfs3 and the block device mapping
are all converted. No functional changes are intended.
This also adds a simple direct I/O path for small reads. On Gen5 NVMe
the __iomap_dio_rw() dominates 4K random reads. The same single-core
io_uring poll mode workload reaches ~3.2M IOPS against the raw block
device but only ~1.92M through ext4 or XFS.
__iomap_dio_rw(), iomap_iter(), iomap_dio_bio_iter() and kfree() were
at the top of the profile. The new path is very lightweight if no
special behavior is requested. The bio comes from a dedicated bioset
and laid out so the whole request is a single cacheline aligned
allocation. Completion runs inline.
That takes ext4 from 1.92M to 2.19M IOPS in the original workload. fio
shows around:
- 4% at libaio queue depths of 64 and up
- around 5% for io_uring
- up to 10% for io_uring poll mode at depth 256
on both ext4 and xfs.
A few other patches:
- iomap_folio_mark_uptodate() lets a filesystem that writes into the
page cache outside the iomap read and write paths keep iomap's
internal uptodate bitmap in sync, which fuse needs for
server-pushed notify stores before it can enable large folios;
- two fixes for iomap_bio_read_folio_range_sync(): a potential crash
when device integrity behavior is changed and a missing
bio_uninit().
- a folio batch release fix on iomap callback failures
- FGP_NOFS is dropped from iomap_get_folio()
- documentation fix"
* tag 'vfs-7.3-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (29 commits)
iomap: iomap_bio_read_folio_range_sync is missing a call to bio_uninit
iomap: don't free integrity payload that doesn't exist
docs: fix grammatical error in iomap docs
exfat: convert iomap ops to ->iomap_next()
fuse: convert iomap ops to ->iomap_next()
hpfs: convert iomap ops to ->iomap_next()
gfs2: convert iomap ops to ->iomap_next()
f2fs: convert iomap ops to ->iomap_next()
block: convert iomap ops to ->iomap_next()
ext2: convert iomap ops to ->iomap_next()
zonefs: convert iomap ops to ->iomap_next()
erofs: convert iomap ops to ->iomap_next()
ext4: convert iomap ops to ->iomap_next()
ntfs: convert iomap ops to ->iomap_next()
ntfs3: convert iomap ops to ->iomap_next()
btrfs: convert iomap ops to ->iomap_next()
xfs: convert iomap ops to ->iomap_next()
iomap: add ->iomap_next()
iomap: use GFP_NOWAIT when application for iomap_dio_simple allocations
iomap: decouple simple direct I/O reads from iomap_dio_rw
...
If we the no-holes feature is enabled (a default since btrfs-progs 5.15),
when doing a full fsync we always iterate of all leaves in the subvolume
root that contain file extent items in order to detect holes between them.
This can take a lot of time for files with a large number of extents.
But if we know there are no prealloc extents and the amount of space
(uncompressed space) is greater than or equals to the i_size of the
inode, then we cannot have holes and therefore avoid searching for
them. So skip the search if those conditions are met.
The following test script was used:
$ cat test.sh
#!/bin/bash
MNT=/mnt/nullb0
DEV=/dev/nullb0
umount $MNT &> /dev/null
mkfs.btrfs -f $DEV
mount $DEV $MNT
# 256M gives 64K extents of 4K each.
FILE_SIZE=$((256 * 1024 * 1024))
touch $MNT/foobar
for ((i = 0; i < $FILE_SIZE; i += 8192)); do
xfs_io -c "pwrite -S 0xab $i 4K" $MNT/foobar > /dev/null
done
xfs_io -c "fsync" $MNT/foobar
for ((i = 4096; i < $FILE_SIZE; i += 8192)); do
xfs_io -c "pwrite -S 0xab $i 4K" $MNT/foobar > /dev/null
done
# unmount and mount, clear caches and ensure the next fsync is a
# full sync.
umount $MNT
mount $DEV $MNT
# Do some change to the file in order to fsync.
xfs_io -c "pwrite -S 0xcd 0 4K" $MNT/foobar > /dev/null
T0=$(date +%s%N)
xfs_io -c "fsync" $MNT/foobar
T1=$(date +%s%N)
echo
echo "Took $(( (T1 - T0) / 1000 ))us"
umount $MNT
Before this change:
Took 28721us
After this change:
Took 5453us
That's about 5.3x times faster.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Inspired by the previous crash exposed by generic/795, we want to make
sure every folio from btrfs page cache is properly aligned to block
size.
This is especially important for bs > ps support, as every btrfs
infrastructure, e.g. extent map and extent state, requires strong block
alignment checks.
Furthermore, also output the minimal folio order from the inode mapping,
which is the determining factor during debugging, helping a lot pinning
down the final cause.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
extent_buffer readahead should not be able to painfully stall a
search_slot and hog tree locks by getting stuck in direct reclaim.
If the allocation fails, that is fine, we simply fail to do the
readahead in that case.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Now that we have the btrfs_eb_prealloc struct to carry the allocation
and the "needs prealloc" signal, wire that up between the various
search_slot style callers down into alloc_extent_buffer.
If the prealloc struct indicates that it supports a nowait try, then
alloc_extent_buffer tries to allocate NOWAIT. If that succeeds, great.
Otherwise, we return EAGAIN and signal via the struct that preallocation
is required. The caller then does the allocation and tries again with
the eb, bfs, and folios wired through in the prealloc struct.
If unlock-and-allocate retries are not supported then we just use the
normal gfp flags like before.
Note that there are still two GFP_NOFS allocations, as far as I know,
that happen under the lock and cannot be preallocated:
- the __xa_cmpxchg to insert the eb into the eb xarray
- the xarray allocations for filemap_add_folio to add the folios to
the btree_inode mapping.
The former we could wire up with xa_reserve if we signaled the "prealloc
start" back up to the retry point. However, since there is no concept of
reservation in the filemap xarray, it seemed relatively unhelpful to
bother. These allocations are relatively small cached slab allocations,
so hopefully we can move the needle on reclaim stalls without reserving
them.
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In further preparation for supporting NOFAIL allocations with retries
outside the critical section, add a struct to carry the extent_buffer
and btrfs_folio_state we need to allocate.
Refactor the allocation pathways to use the new struct but with no
functional change. Wire empty prealloc structs in from callers.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In preparation for preallocating extent_buffer data, factor eb
initialization away from specifically allocating it. This allows us to
allocate the eb, bfs, folios, etc. together in the main search_slot code
paths, but still share initialization code with the dummy/test/clone
allocation paths.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In that function, we round down the start position and round up the
ending position.
But during the calculation of @len, we use "round_up(start + len,
sectorsize)", which is the rounded up end position, not the rounded up
length.
Which results a much larger length, and later we are still using
"start + len", which is completely incorrect.
Fix it by declaring a local @aligned_start and @aligned_len and use them
instead.
Fixes: bc42bda223 ("btrfs: qgroup: Fix qgroup reserved space underflow by only freeing reserved ranges")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Extent maps have the extra validation since commit 3f255ece2f ("btrfs:
introduce extra sanity checks for extent maps"), but extent states do
not have a similar check.
Introduce a basic alignment check for the following call sites, so that
we can cover all extent states inserted into the tree:
- insert_state_fast()
- insert_state()
- split_state()
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In btrfs_extent_same_range() and btrfs_clone_files(), the range passed
into btrfs_lock_extent() is not aligned at its end, because we can
reflink until the EOF, which may not be block aligned.
Although this is not a big deal, for the sake of consistency, and to
prepare for the upcoming stricter alignment check, pass an aligned range
end to btrfs_lock_extent() and btrfs_unlock_extent().
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The @end parameter for all extent io tree helpers is inclusive, but
the call site in extent_fiemap() is passing an exclusive end into
btrfs_lock_extent(), which will step into the next block unexpectedly.
Pass the inclusive end into btrfs_lock_extent() and
btrfs_unlock_extent().
Fixes: ac3c0d36a2 ("btrfs: make fiemap more efficient and accurate reporting extent sharedness")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
On a zoned filesystem a freed-but-still-dirty tree block is written out
as zeros (EXTENT_BUFFER_ZONED_ZEROOUT) only to keep the zone write
pointer advancing. btree_csum_one_bio() implemented this by memzeroing
the extent buffer's own folios before submission.
That destroys the in-memory buffer while it may still be referenced. In
particular btrfs_free_tree_block() can run on it afterwards and reads
the header to add a delayed reference; once the header has been zeroed
it frees bytenr 0 and corrupts the extent tree (the
btrfs_header_bytenr(buf) != 0 ASSERT in btrfs_free_tree_block(), or an
"unable to find ref" abort). It is flaky and reproduces under fsstress,
e.g. generic/461 and generic/013.
Write the zeros to disk from the shared zero page instead and leave the
extent buffer content untouched, so any later reference - including the
delayed reference from btrfs_free_tree_block() - still sees a valid
header. end_bbio_meta_write() now clears writeback on the buffer's own
folios, as the bio no longer carries them.
Fixes: aa6313e6ff ("btrfs: zoned: don't clear dirty flag of extent buffer")
Assisted-by: LLM (debugging, commit message)
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
On a zoned filesystem a freed tree block is kept dirty and flagged
EXTENT_BUFFER_ZONED_ZEROOUT so a later writeback zeroes it out and
advances the zone write pointer. Unsynced tree-log updates (e.g. from
rename or link) leave such buffers behind when the log is freed at
commit, and across log generations they can end up ahead of the write
pointer behind a hole, so btree_writepages() can never write them.
During normal operation the space is later reclaimed by a zone reset; at
unmount it is not, and the buffers survive to the final iput() of the
btree inode, which hangs in folio_wait_writeback() once the endio
workqueues are stopped.
They cannot be written back from where they are freed (free_log_tree(),
inside the committing transaction) without deadlocking against that
commit, and they are stale anyway, not referenced by the committed
superblock. Drop their dirty state in close_ctree(), before
btrfs_stop_all_workers().
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
On a zoned filesystem a freed tree block is not cleared but kept dirty
and flagged EXTENT_BUFFER_ZONED_ZEROOUT, so a later writeback zeroes it
out and advances the zone write pointer. A transaction abort turns the
filesystem read-only before that writeback runs, so these buffers stay
dirty and stranded ahead of the write pointer where btree_writepages()
can no longer write them. They survive to the final iput() of the btree
inode at unmount, which submits the write after the endio workqueues are
gone, hanging unmount in folio_wait_writeback().
Clear the dirty state of such buffers when cleaning up the aborted
transaction, where the buffer tree still references all of them.
Assisted-by: LLM (debugging, commit message)
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btree_writepages() writes the btree inode's dirty metadata in ascending
logical address order. On a zoned filesystem only one metadata and one
system block group is active for writing at a time, and
check_bg_is_active() (via btrfs_check_meta_write_pointer()) pivots the
active block group as writeback moves from one block group to the next.
If the active block group sits at a higher logical address than another
block group that also holds dirty metadata, the ascending walk reaches
the lower one first and, to write it, has to finish the active block
group and activate the lower one. It cannot finish a block group that
still has unsent IO, and during WB_SYNC_ALL && !for_sync (commit)
writeback it deliberately refuses to wait for that IO under
fs_info->zoned_meta_io_lock, as that can deadlock. The pivot thus cannot
issue the submission itself either, so it gives up:
btrfs_check_meta_write_pointer() returns -EAGAIN, which
btrfs_write_and_wait_transaction() treats as fatal and aborts the
transaction, forcing the filesystem read-only. This happens
intermittently under metadata-heavy relocation (e.g. fstests btrfs/187).
Flush the active metadata and system block groups at the start of
btree_writepages(), under the fs_info->zoned_meta_io_lock it already
holds, so they have no unsent IO left and the later pivot can finish
them and make forward progress.
Fixes: 13bb483d32 ("btrfs: zoned: activate metadata block group on write time")
Assisted-by: LLM (debugging, commit message)
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Inside reflink.c we still have a lot of functions passing VFS inode
pointers, then internally convert them into btrfs_inode pointers.
For example, inside btrfs_clone(), we have 12 BTRFS_I() call sites,
while only 3 callsites that really require a VFS inode pointer.
Do the cleanup to convert the following functions to pass a btrfs_inode
pointer instead of a vanilla inode pointer:
- btrfs_clone()
- btrfs_extent_same_range()
- clone_finish_inode_update().
Which covers all ad-hoc BTRFS_I() call sites inside reflink.c.
Reviewed-by: Daniel Vacek <neelx@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We are using atomic types for the log_commit array of struct btrfs_root
but all we need is simple booleans. The log_commit array elements are
always protected by the root's log_mutex, both for writes and reads, so
we can use a simple boolean. The use of atomics if from the very early
days of the log tree code where the access to the fields was not protected
by any lock.
So switch to simple booleans, which results in cheaper code and slightly
reduces the object size too.
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We check for the exit condition after we add ourselves to the wait queue
and before we unlock the root's log_mutex, sleep and lock again log_mutex.
This is not incorrect, but it's not optimal since in the first iteration
this is pointless because we already know that root->log_commit[index] is
not zero, so we should check the exit condition only after unlocking
log_mutex, sleeping, waking up and locking again the log_mutex.
So move the check for the exit condition to bottom of the loop, after we
were woken and locked log_mutex again.
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Instead of having every caller check for root->log_commit[] being non-zero
and then call wait_log_commit(), move the check into wait_log_commit() and
have the callers call it unconditionally.
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We have the log batch counter defined per root which is now useless after
the previous patch (titled: "btrfs: stop sleeping for one jiffy in non-ssd
mounts during log commit"). The counter is incremented early in the fsync
path, before and after flushing dellaloc and waiting for writeback, and
then the counter is read during the log sync path. The goal was to wait
for tasks that are about to join a log transaction, so that we could
reduce the amount of IO and log syncing (flush all log tree extent buffers
and write super blocks), but that mechanism does not work since if there
are currently no log writers, btrfs_sync_log() does not unlock the root's
log_mutex, so no new log writers can join the log transaction. Having
concurrent fsync tasks increasing the log_batch counter only makes us loop
unnecessarily in btrfs_sync_log() - that is always true since the previous
patch mentioned above and was true before that patch only when not using
the "-o ssd" mount option (which is activated by default if the filesystem
does not have rotational devices).
So remove the log batch counter. No performance changes were observed
after removing it.
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Joining/starting a log transaction tracks if we ever had more than one task
concurrently logging by setting the flag BTRFS_ROOT_MULTI_LOG_TASKS in the
respective root. Once set, this flag remains for the rest of the lifetime
of the transaction, only cleared when we don't have a log root and need to
create a new one (transaction commits drop log roots).
During log commit, if we are not on a ssd mount (or use the -o nossd mount
option) and the BTRFS_ROOT_MULTI_LOG_TASKS flag is set, we sleep for one
jiffy with the excuse to allow future log writers to join and log inodes
and then commit a larger log transaction to reduce overall IO. However
this is extremely inefficient because:
1) If at some point we had multiple tasks logging concurrently but now
we have only one task at a time, we force it to wait for 1 jiffy;
2) One jiffy can vary between 1ms to 10ms, depending on the kernel
config option CONFIG_HZ, which by default has a value of 250HZ and
that corresponds to 4ms - that is a lot.
This massively reduces the latency of fsyncs for non-ssd mounts, even
on consumer grade spinning disks.
Remove this mechanism to track if we have (or ever had) multiple tasks
logging and wait for 1 jiffy.
The following fio test was used to benchmark:
$ cat fio-buffered-fsync.sh
DEV=/dev/sdj
MNT=/mnt/sdj
MOUNT_OPTIONS=""
MKFS_OPTIONS=""
if [ $# -ne 6 ]; then
echo "Use $0 NUM_JOBS FILE_SIZE IO_SIZE FSYNC_FREQ BLOCK_SIZE [write|randwrite]"
exit 1
fi
NUM_JOBS=$1
FILE_SIZE=$2
IO_SIZE=$3
FSYNC_FREQ=$4
BLOCK_SIZE=$5
WRITE_MODE=$6
if [ "$WRITE_MODE" != "write" ] && [ "$WRITE_MODE" != "randwrite" ]; then
echo "Invalid WRITE_MODE, must be 'write' or 'randwrite'"
exit 1
fi
cat <<EOF > /tmp/fio-job.ini
[writers]
rw=$WRITE_MODE
fsync=$FSYNC_FREQ
fallocate=none
group_reporting=1
direct=0
bs=$BLOCK_SIZE
ioengine=psync
filesize=$FILE_SIZE
io_size=$IO_SIZE
directory=$MNT
numjobs=$NUM_JOBS
EOF
echo
echo "Using config:"
echo
cat /tmp/fio-job.ini
echo
umount $MNT &> /dev/null
mkfs.btrfs -f $MKFS_OPTIONS $DEV
mount $MOUNT_OPTIONS $DEV $MNT
fio /tmp/fio-job.ini
umount $MNT
Running the script as: ./fio-buffered-fsync.sh 8 64M 64M 1 4K randwrite
Before patch:
WRITE: bw=2647KiB/s (2711kB/s), 2647KiB/s-2647KiB/s (2711kB/s-2711kB/s), io=512MiB (537MB), run=198055-198055msec
After patch:
WRITE: bw=14.9MiB/s (15.6MB/s), 14.9MiB/s-14.9MiB/s (15.6MB/s-15.6MB/s), io=512MiB (537MB), run=34471-34471msec
That's about 5.7 times faster.
Reviewed-by: Boris Burkov <boris@bur.io>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The correct path of the "read_policy" module parameter should be
/sys/module/btrfs/parameters/read_policy. Fix it.
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Zenghui Yu <zenghui.yu@linux.dev>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_read_merkle_tree_page() can find a folio in the mapping that is not
uptodate. After taking the folio lock, the current code treats that state
as a read error and returns -EIO.
That can make a previous transient read failure sticky. If the failed read
left a not-uptodate folio in the mapping, later callers find that folio and
fail instead of retrying the read.
Keep the existing page-cache insertion and locking order, but retry the
Merkle item read when a not-uptodate folio is found in the mapping. Also
unlock the folio when read_key_bytes() fails so that a later caller can
lock it and retry the read.
Fixes: 06ed09351b ("btrfs: convert btrfs_read_merkle_tree_page() to use a folio")
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Yichong Chen <chenyichong@uniontech.com>
Signed-off-by: David Sterba <dsterba@suse.com>
During an interrupted mount, I got the following messages:
workqueue: Failed to create a rescuer kthread for wq "btrfs-qgroup-rescan": -EINTR
BTRFS error (device dm-3): open_ctree failed: -12
Workqueue code is outputting a human readable error string, meanwhile
we're still using a numeric error code.
So follow the workqueue code to use "%pe" format, which will
automatically convert an error pointer to the human readable string.
However this is a minor pitfall, if the return value is not an error
code, e.g. a positive number, "%pe" with "ERR_PTR(ret)" will output the
pointer as a hash value, e.g.:
ret=1 %pe out=0000000019414716
ret=-22 %pe out=-EINVAL
So we should not use this "%pe" output for callsites that are known to
return positive values.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
That macro is only utilized 4 times, all inside file.c, while we have
tons of open-coded usages. And since it's a macro, there is no proper
type checks at all.
There isn't much need for such a rarely utilized macro.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Commit f9a48549a1 ("btrfs: inhibit extent buffer writeback to prevent
COW amplification") tracks the extent buffers a transaction handle has
inhibited in a per-handle xarray. Keying the tracking to the transaction
handle is correct, but using an xarray for it causes two problems in
production.
First, a write_iops regression. Every COW calls
btrfs_inhibit_eb_writeback() from btrfs_force_cow_block() and
should_cow_block(), which does an xa_store() keyed by eb->start. The
kernel test robot reported a 22.6% fio.write_iops regression on a
single-task 4k randwrite workload (ftruncate ioengine, buffered IO) on
btrfs. The cost is the per-COW xarray store done on every COW'd block.
Replacing it with a non-allocating fixed buffer recovers the lost
throughput, and that buffer does more per-COW bookkeeping yet still
recovers, so the cost is the xarray operation itself rather than the
extra tracking work.
Second, an unbounded cleanup walk. btrfs_uninhibit_all_eb_writeback()
iterates every eb the handle inhibited with xa_for_each(). A single
handle that COWs a very large number of blocks (inode eviction, or
truncate of a file with many extents, where btrfs_truncate_inode_items()
loops over many search_again descents under one handle) makes that walk
arbitrarily long. It runs in __btrfs_end_transaction() before
num_writers is dropped, so it blocks the committing thread; this shows up
as multi-second stalls and RCU stall reports.
Replace the xarray with a fixed inline array on btrfs_trans_handle,
managed with a CLOCK (second-chance) eviction policy. Inhibiting a buffer
becomes an array append with no allocation and no tree walk, and the
end-of-handle cleanup is bounded by the array size.
The set that actually needs protection is the working set the handle
revisits across search_again descents, the search path frontier, which is
on the order of the tree height. It is not every block the handle ever
COWs. should_cow_block() re-inhibiting an already tracked buffer marks it
referenced, so revisited buffers survive eviction while write-once buffers
are reclaimed first. A small fixed buffer is therefore enough where a
non-evicting array would either overflow or have to grow without bound.
BTRFS_INHIBITED_EBS_SLOTS is 8 and the reference bits pack into a u32.
The CLOCK eviction is what justifies the extra complexity over a plain
non-evicting array. The test workload stresses amplification: it removes
16 heavily fragmented 64 MiB files in one transaction while background
writeback keeps writing out in-use metadata. A re-COW event is a buffer
already COWed in the running transaction that was written back and then
COWed again; the figure below is the ratio of re-COW events to first-COW
events summed across the eviction (n=5, lower is better):
tracking re-COW per first-COW
no inhibition 6.1
non-evicting array, 32 slots 3.8
CLOCK array, 8 slots (this patch) 1.6
unbounded xarray (reverted) 1.4
The non-evicting array fills with write-once buffers and stops covering
the buffers the handle keeps revisiting, so even at four times the slots
it leaves most of the amplification. CLOCK evicts the cold buffers and
keeps the revisited ones, recovering almost all of the unbounded benefit.
The eviction policy, not the buffer size, is what closes the gap.
eb->writeback_inhibitors and the WB_SYNC_ALL bypass in
lock_extent_buffer_for_io() are unchanged, so fsync and commit behavior
are unaffected. A reference is taken on each tracked buffer so it cannot
be freed while the array points at it; eviction drops that reference and
the inhibitor count.
There's another testing report, showing 20% latency improvement on
reflink and deduplication synthetic benchmark. Full detailed report at
https://github.com/lcf0399/linux-regression-evidence/tree/main/btrfs-remap-writeback-inhibition-v2 .
Link: https://lore.kernel.org/all/CANGjgd=fQkHht2PdDi-+EAdzWH7UtxxWhhJ7b80Rr17PbpgxOw@mail.gmail.com/
Reported-by: kernel test robot <oliver.sang@intel.com>
Fixes: f9a48549a1 ("btrfs: inhibit extent buffer writeback to prevent COW amplification")
Closes: https://lore.kernel.org/oe-lkp/202603112240.f7605968-lkp@intel.com
Tested-by: Chengfeng Lin <lin2530632123@gmail.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Sun YangKai <sunk67188@gmail.com>
Signed-off-by: Leo Martins <loemra.dev@gmail.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Avoid allocating the heuristic buckets separately from the workspace,
the lifetime is the same.
The new size of struct heuristic_ws is 2112. SLUB merges same/similar
sized structures for the named caches, so there's a chance such size
already exists on the system, like below:
$ grep 2112 /proc/slabinfo
sighand_cache 593 1335 2112 15 8
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For a filesystem which has btrfs read-only property set to true, all
write operations including acl and xattr should be denied. However, acl
can still be set even if btrfs ro property is true.
This happens because no function on the set_acl code path checks the root
is readonly or not. It was checked in btrfs_setxattr_trans() but got
removed in commit 353c2ea735 ("btrfs: remove redundant readonly root
check in btrfs_setxattr_trans")
That commit didn't check if all the callers properly check the root's
read-only flag. A previous fix is commit b51111271b ("btrfs: check if
root is readonly while setting security xattr").
Always check if the root is read-only before performing the set acl
operation.
Fixes: 353c2ea735 ("btrfs: remove redundant readonly root check in btrfs_setxattr_trans")
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Sun YangKai <sunyangkai@fnnas.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
During relocation recovery, each fs root gets a reference to its relocation
root. If loading or adding a later root fails, or if the first transaction
commit fails, btrfs_recover_relocation() jumps to out_unset before
merge_reloc_roots() and clean_dirty_subvols().
put_reloc_control() drops the list-owned relocation root references, but it
does not clear fs_root->reloc_root or drop the references owned by those
pointers. Mount cleanup only drops them when BTRFS_FS_ERROR is set, so an
error such as -ENOMEM while processing a later root can leave references
behind.
Keep temporary references to the fs roots associated during recovery. On
failure, clear their reloc_root pointers and drop the corresponding
references. Once the first transaction commit succeeds, drop only the
temporary fs root references and let the normal merge and cleanup paths
handle the relocation roots.
Fault injection on a pending-relocation image confirmed the cleanup gap.
With an injected first-commit failure, 25 fs roots had reloc_root set with
fs_error=0. With this fix, the same failure path drops that count to 0
before mount fails.
Fixes: f44deb7442 ("btrfs: hold a ref on the root->reloc_root")
CC: stable@vger.kernel.org
Signed-off-by: Guanghui Yang <3497809730@qq.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add sctx NULL check in the for loop condition of the sort_clone_roots
cleanup path for consistency with the else branch.
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Hongling Zeng <zenghongling@kylinos.cn>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Recently kernel RAID56 lib is trying to remove the unexpected
single-data-RAID56 (2 disks RAID5 or 3 disk RAID5) support, meanwhile
btrfs still supports such setup, which means in the long run btrfs has
to handle such corner case by ourselves.
Thankfully single-data-RAID56 is really RAID1/RAID1C3, since data and
P/Q stripes all match each other, rotation also makes no difference.
This patch will disguise those single-data-RAID56 chunks as
RAID1/RAID1C3 chunks.
This is done at two timings:
- Chunk read
- Chunk allocation
This is done by introducing btrfs_chunk_map::on_disk_type member, which
stores the type read from the on-disk metadata.
Meanwhile btrfs_chunk_map::type is calculated using on_disk_type.
For most profiles @type matches @on_disk_type, but for
single-data-RAID56, the @type will be RAID1/RAID1C3.
This method has a minimal impact on the fs, all other operations like
scrub and read-repair, are all based on the chunk map type, so the
disguise method will require no extra modification to those call sites.
Although there are still some locations that are checking against
block_group->flags, e.g. scrub. Those call sites will still get extra
limits assuming the bg is RAID56. But it should not cause any extra
problem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In the function fill_dummy_bgs(), bg->flags is assigned twice.
Just remove the second assignment.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Those two members are read from on-disk metadata, but never utilized.
And for new chunks we always set those members to BTRFS_STRIPE_LEN
anyway.
Thus there is no need to keep them inside btrfs_chunk_map.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[FALSE ALERTS]
There is a bug report that the warning inside
invalidate_and_check_btree_folios() got triggered during btrfs/298:
BTRFS info (device sdd): first mount of filesystem f9bf732a-a19b-44b9-99a7-614ddff168e2
BTRFS info (device sdd): using crc32c checksum algorithm
BTRFS error (device sdd): failed to find fsid cb2fdb42-b638-4f2f-badd-4127467ba674 when attempting to open seed devices
BTRFS error (device sdd): failed to read chunk tree: -2
------------[ cut here ]------------
WARNING: disk-io.c:3342 at invalidate_and_check_btree_folios+0x260/0x3c0 [btrfs], CPU#4: mount/125993
CPU: 4 UID: 0 PID: 125993 Comm: mount Tainted: G W OE 7.1.0-rc7-custom+ #1 PREEMPT(full)
Hardware name: QEMU KVM Virtual Machine, BIOS edk2-20250812-19.fc42 08/12/2025
Call trace:
invalidate_and_check_btree_folios+0x260/0x3c0 [btrfs] (P)
open_ctree+0x1f50/0x23b0 [btrfs]
btrfs_get_tree+0x89c/0xc48 [btrfs]
vfs_get_tree+0x30/0x110
vfs_cmd_create+0x58/0xe8
__arm64_sys_fsconfig+0x39c/0x518
invoke_syscall.constprop.0+0x48/0x120
el0_svc_common.constprop.0+0x40/0xe8
do_el0_svc+0x24/0x38
el0_svc+0x50/0x310
el0t_64_sync_handler+0xa0/0xe8
el0t_64_sync+0x198/0x1a0
---[ end trace 0000000000000000 ]---
BTRFS warning (device sdd): unable to release extent buffer 365985792 owner 3 gen 17 refs 3 flags 0x5
[CAUSE]
In that invalidate_and_check_btree_folios() we wait for the eb to finish
its read, then check if it's only held by us and the btree inode.
If not, then do a warning as it may be still held, and could cause
problems.
But there is a small window where the check can lead to false alerts:
Thread A (Read endio) | Thread B (Unmount)
----------------------------------+-------------------------------------
end_bbio_meta_read() |
| The eb has one extra ref held |
| by the reader, and has |
| EXTENT_BUFFER_READING flag set | invalidate_and_check_btree_folios()
| | |
|- clear_extent_buffer_reading() | |
| | |- wait_on_bit_io();
| | | The EXTENT_BUFFER_READING flag is
| | | cleared
| | |- if (refcount_read(eb->refs) > 2)
| | The eb is held by the read, us
| | and btree inode, thus it
| | will trigger the warning
|- free_extent_buffer() |
[FIX]
Introduce a helper, free_extent_buffer_clear_reading().
If the new parameter, @clear_reading, is set, we will hold the spinlock
at the beginning of free_extent_buffer_clear_reading() to make sure the
EXTENT_BUFFER_READING flag is cleared inside the same critical section
of decreasing refs.
Now free_extent_buffer() will just call
free_extent_buffer_clear_reading() with @clear_reading set to false, so
no behavior change.
But for end_bbio_meta_read(), it will not clear_extent_buffer_reading()
directly, but pass @clear_reading as true.
Then inside invalidate_and_check_btree_folios(), hold the refs_lock
before reading refs.
So that we eliminate the race window completely.
Reported-by: Su Yue <glass.su@suse.com>
Link: https://lore.kernel.org/linux-btrfs/DC0C775E-13B3-47D9-9AB2-895BB11C029D@suse.com/
Fixes: 83f7e52b7e ("btrfs: warn about extent buffer that can not be released")
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
commit 095be159f3 ("btrfs: unify folio dirty flag clearing") replaced
the folio_clear_dirty_for_io() call in extent_write_cache_pages() with a
plain folio_test_dirty() check. Besides clearing the dirty flag,
folio_clear_dirty_for_io() also calls folio_mkclean(), which write-protects
the shared mmap PTEs mapping the folio. Note that we still do call
folio_clear_dirty_for_io() later in submit_one_sector() when we clear
dirty on the last sector of the folio (the only sector for non-subpage
cases). But we lost this early call in extent_write_cache_pages().
Without the extra write-protection, a process with the file mmap-ed can
modify a sector while it is being used by writeback in a way that
expects a stable folio (checksumming, compressing, copying, etc...)
without faulting, which manifests as a handful of concrete bugs.
1. For large folios or subpage sectorsize, it is possible to submit a bio
which does not cover the whole folio. When this happens, we will have a
bio in flight for a folio that we have *not* called
folio_clear_dirty_for_io() on. If a task with an existing mmap-ed PTE
writes (without faulting..) in this window, it can result in
corruptions. If the write arrives while the checksumming or writing itself
is underway, this can result in an invalid checksum and later corruption
reports on read. If the write arrives after checksumming/writing is done
but before the last sector dirty is cleared, then the write is present
in page cache but doesn't affect the dirty tracking and will be lost
when the folio is fully finished being submitted and the dirty bit
is cleared. This results in losing the write even if fsync() is called.
2. For zoned submissions which are done in batch separate from the main
extent_writepage() loop, we also risk csum violations for those
submissions. Zoned writes are clamped to max_zone_append_size and are
not aligned with folios, so a submission can span two folios. The first
folio being processed in extent_write_cache_pages() will call
extent_write_locked_range() which will submit the partial range of the
next folio, while the rest of that folio could still be dirty. So
clearing dirty on the submitted sectors doesn't call
folio_clear_dirty_for_io() and we have the same issue. Since
extent_write_cache_pages() skips these batch submitted folios (they are
already marked for writeback from submission by the preceding folio), we
must add the extra write protection in lock_delalloc_folios().
3. For inline extents this will subtly risk losing writes that happen
after/while we copy the inline extent but before we clear dirty on
the folio.
4. For folios spanning EOF, mmap could tamper with the zeroed bytes past
EOF and cause them to be persisted where future faults would improperly
see them instead of zeros.
5. Finally, for compressed extents, we risk modifying the folios while we
work on compressing them which will result in corrupted compressed data.
Specifically, in run_delalloc_compressed() we queue up work to do
compress_file_range() in BTRFS_COMPRESSION_CHUNK_SIZE (512K) chunks which
will call btrfs_folio_clamp_clear_dirty() on the range. For non-subpage,
this will always clear the whole folio, safely. For subpage, we risk a
partial clear here as well. In particular, imagine a 2M folio broken up
into 512K chunks of work which might start compression work on one chunk
before all the chunks compress_file_range() workers have gotten far
enough to finish clearing all the dirty bitmaps of the folio and getting
to folio_clear_dirty_for_io(). Large folios on the edges of submission
ranges are similarly at risk to be only partly cleared.
This particular gap was introduced by a second patch in the same series:
commit a4ef54dbb5 ("btrfs: make extent_range_clear_dirty_for_io() to handle sector size < page size cases")
We cannot simply restore the call to folio_clear_dirty_for_io() because
that also drops the dirty flag off the folio which violates invariants
introduced for large folios by
commit 334509ce9d07 ("btrfs: use dirty flag to check if an ordered extent needs to be truncated")
and results in failing to invalidate clean folios past i_size, resulting
in deadlocks.
Therefore, to fix it, leave the existing semantics w.r.t. the folio's
dirty flag (to preserve the correct invalidate behavior) but ensure that
the other aspect of folio_clear_dirty_for_io(), folio_mkclean(), is run
on the folio when we lock it for writeback.
Finally, to help prevent similar regressions in the future, add a debug
warning that triggers at the known corruption sites if we have failed to
write protect the folio.
Assisted-by: LLM (debug, reproduce, research fix, review patch)
Fixes: 095be159f3 ("btrfs: unify folio dirty flag clearing")
Fixes: a4ef54dbb5 ("btrfs: make extent_range_clear_dirty_for_io() to handle sector size < page size cases")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a lockdep report related to device scan:
======================================================
WARNING: possible circular locking dependency detected
7.2.0-20260712.rc2.git0.e3321fa3034d.300.fc44.s390x+debug #1 Not tainted
------------------------------------------------------
(udev-worker)/1653 is trying to acquire lock:
0000006919232220 (&type->i_mutex_dir_key#2){++++}-{3:3}, at: lookup_slow+0x3e/0x70
but task is already holding lock:
00000069238564d8 (&fs_devs->device_list_mutex){+.+.}-{3:3}, at: device_list_add.constprop.0+0x148/0xc60
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #5 (&fs_devs->device_list_mutex){+.+.}-{3:3}:
lock_acquire+0x150/0x3f0
__mutex_lock+0xba/0xdc0
mutex_lock_nested+0x32/0x40
write_all_supers+0x7a/0x670
btrfs_sync_log+0xae6/0xdd0
btrfs_sync_file+0x4fa/0x7a0
__s390x_sys_fsync+0x52/0xa0
__do_syscall+0x172/0x750
system_call+0x72/0x90
-> #4 (&fs_info->tree_log_mutex){+.+.}-{3:3}:
lock_acquire+0x150/0x3f0
__mutex_lock+0xba/0xdc0
mutex_lock_nested+0x32/0x40
btrfs_sync_log+0xaba/0xdd0
btrfs_sync_file+0x4fa/0x7a0
__s390x_sys_fsync+0x52/0xa0
__do_syscall+0x172/0x750
system_call+0x72/0x90
-> #3 (btrfs_trans_num_extwriters){.+.+}-{0:0}:
lock_acquire+0x150/0x3f0
join_transaction+0x108/0x680
start_transaction+0x21a/0x660
btrfs_join_transaction+0x32/0x40
btrfs_dirty_inode+0x52/0xf0
touch_atime+0x90/0xc0
filemap_read+0x446/0x450
vfs_read+0x208/0x370
ksys_read+0x88/0x120
__do_syscall+0x172/0x750
system_call+0x72/0x90
-> #2 (btrfs_trans_num_writers){.+.+}-{0:0}:
reacquire_held_locks+0x14c/0x240
__lock_release.isra.0+0xd8/0x380
lock_release+0xf6/0x270
percpu_up_read+0x28/0xf0
__btrfs_end_transaction+0x178/0x1f0
btrfs_dirty_inode+0x82/0xf0
touch_atime+0x90/0xc0
btrfs_file_mmap_prepare+0x8c/0xa0
__mmap_region+0x214/0x780
mmap_region+0x108/0x160
do_mmap+0x402/0x5a0
vm_mmap_pgoff+0x156/0x230
ksys_mmap_pgoff+0x17e/0x220
__s390x_sys_old_mmap+0xa8/0x140
__do_syscall+0x172/0x750
system_call+0x72/0x90
-> #1 (&mm->mmap_lock){++++}-{3:3}:
lock_acquire+0x150/0x3f0
__might_fault+0x7a/0xa0
filldir64+0x11c/0x210
offset_readdir+0x92/0x200
iterate_dir+0xcc/0x2d0
__do_sys_getdents64+0x7a/0x130
__do_syscall+0x172/0x750
system_call+0x72/0x90
-> #0 (&type->i_mutex_dir_key#2){++++}-{3:3}:
check_prev_add+0x160/0xf40
__lock_acquire+0x12aa/0x15a0
lock_acquire+0x150/0x3f0
down_read+0x5a/0x280
lookup_slow+0x3e/0x70
path_lookupat+0x1f0/0x370
filename_lookup+0xce/0x1f0
kern_path+0x48/0x70
is_same_device+0x146/0x300
device_list_add.constprop.0+0x1be/0xc60
btrfs_scan_one_device+0x13a/0x2f0
btrfs_control_ioctl+0x110/0x1e0
__s390x_sys_ioctl+0xfa/0x130
__do_syscall+0x172/0x750
system_call+0x72/0x90
other info that might help us debug this:
Chain exists of:
&type->i_mutex_dir_key#2 --> &fs_info->tree_log_mutex --> &fs_devs->device_list_mutex
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&fs_devs->device_list_mutex);
lock(&fs_info->tree_log_mutex);
lock(&fs_devs->device_list_mutex);
rlock(&type->i_mutex_dir_key#2);
*** DEADLOCK ***
2 locks held by (udev-worker)/1653:
#0: 0000016c727051c8 (uuid_mutex){+.+.}-{3:3}, at: btrfs_control_ioctl+0x102/0x1e0
#1: 00000069238564d8 (&fs_devs->device_list_mutex){+.+.}-{3:3}, at: device_list_add.constprop.0+0x148/0xc60
stack backtrace:
CPU: 2 UID: 0 PID: 1653 Comm: (udev-worker) Not tainted 7.2.0-20260712.rc2.git0.e3321fa3034d.300.fc44.s390x+debug #1 PREEMPT
Hardware name: IBM 3931 A01 701 (LPAR)
Call Trace:
[<0000016c70680e3e>] dump_stack_lvl+0xae/0x108
[<0000016c7078aa44>] print_circular_bug+0x1a4/0x230
[<0000016c7078ac5c>] check_noncircular+0x18c/0x1b0
[<0000016c7078c030>] check_prev_add+0x160/0xf40
[<0000016c7078fbaa>] __lock_acquire+0x12aa/0x15a0
[<0000016c7078fff0>] lock_acquire+0x150/0x3f0
[<0000016c7180e2fa>] down_read+0x5a/0x280
[<0000016c70b88dde>] lookup_slow+0x3e/0x70
[<0000016c70b8f5d0>] path_lookupat+0x1f0/0x370
[<0000016c70b900ae>] filename_lookup+0xce/0x1f0
[<0000016c70b90218>] kern_path+0x48/0x70
[<0000016c70f5b4a6>] is_same_device+0x146/0x300
[<0000016c70f67cfe>] device_list_add.constprop.0+0x1be/0xc60
[<0000016c70f688da>] btrfs_scan_one_device+0x13a/0x2f0
[<0000016c70ed9cb0>] btrfs_control_ioctl+0x110/0x1e0
[<0000016c70b97d0a>] __s390x_sys_ioctl+0xfa/0x130
[<0000016c718004d2>] __do_syscall+0x172/0x750
[<0000016c718155d2>] system_call+0x72/0x90
[CAUSE]
Btrfs device scan will call is_same_device() with device_list_mutex
held. But is_same_device() will call kern_path() which will do path
resolution and lock the inode.
So device scan has the following lock sequence:
mutex_lock(device_list_mutex) from device_list_add()
|
v
inode_lock_shared() from lookup_slow() during kern_path().
Meanwhile another thread is fsyncing, which has the following
lock sequence:
inode_lock() from btrfs_inode_lock() inside btrfs_direct_write()
|
v
mutex_lock(tree_log_mutex() from btrfs_sync_log(), which is further
triggered from
iomap_dio_complete()->generic_write_sync()->btrfs_sync_file().
|
v
mutex_lock(device_list_mutex) from write_all_supers() inside
btrfs_sync_log().
So the device scan has a reversed lock sequence, compared to the fsync
one, this means we can have the following deadlock:
Device scan | Fsync
----------------------------------------+--------------------------------
device_list_mutex locked |
| inode locked
| try to lock device_list_mutex
try to lock inode |
[FIX]
Instead of a full path lookup, use dev_t to determine if two device
paths are pointing to the same block device.
Inside kernel dev_t is going to uniquely determine a block device, and
the device path lookup is already done by lookup_bdev(), which is done
without device_list_mutex held, thus no reversed locking sequence.
Reported-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Link: https://lore.kernel.org/linux-btrfs/5a9d9847-4ae6-43c4-afdc-6e5fa51d6117@linux.ibm.com/
Fixes: 2e8b6bc0ab ("btrfs: avoid unnecessary device path update for the same device")
Tested-by: Christian Borntraeger <borntraeger@linux.ibm.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The reclaim_mark field in struct btrfs_block_group was a u64 that was
incremented when marking block groups for reclaim during sweeping, but
the actual counter value was never used - only the zero/non-zero state
mattered for determining if a block group needed reclaim.
Convert it to a bool to properly reflect its usage and reduce memory
footprint by 8 bytes. Update assignments to use true/false instead of
increment and zero.
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Sun YangKai <sunk67188@gmail.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The 'idmap' parameter is derived from 'file' that we also pass to
__btrfs_ioctl_snap_create(), assign it inside the function.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently we're using scrub_stripe::folios[] to store all contents of a
stripe.
This means we need all the extra work to handle things like sub-page
cases, and also require larger folios to handle bs > ps cases.
On the other hand, it's not hard to allocate a 64K large folio to cover
the full stripe, getting rid of the cross-page handling.
Furthermore, even if that large folio allocation failed, we can still
use vmalloc() to allocate a virtually contiguous space and still get rid
of cross-page handling.
This patch will go with kvmalloc() to allocate 64K of memory for
the stripe buffer, thus getting rid of all the complex cross-page
handling.
The following aspects can be greatly simplified:
- Checksum verification for both data and metadata
No more per-page iteration, all in one go.
- RAID56 data caching
Just copy the buffer into the RAID56 pages.
- No more kaddr/paddr grabbing
For most cases the virtual address is enough for csum calculation and
io submission.
- Bio assembly
There is already the helper bio_add_vmalloc() to queue vmallocated
memory into a bio.
Although it means we have something else to be concerned about:
- Bio assembly
If the memory is vmallocated, we need to use bio_add_vmalloc()
Otherwise use the existing bio_add_page().
- Read endio
For vmallocated memory, we need to call
invalidate_kernel_vmap_range().
- Scrub bbio bvec size
Since scrub_stripe::buffer is kvmallocated, we also need to enlarge
the scrub bbio, to be able to handle the worst case, where all 64KiB is
allocated by discontiguous 4K physical pages.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently calc_sector_number() is implemented by comparing the first
bvec of the bbio against all blocks inside a scrub_stripe.
This implementation is a little inefficient, and depends on how the
scrub buffer is implemented.
One of the reason implementing such complex function is that, we do not
save the original bvec_iter inside a write btrfs_bio.
Although a read bbio has btrfs_bio::saved_iter to get the original
logical bytenr, it's not implemented for write bios.
On the other hand, since commit 81cea6cd70 ("btrfs: remove
btrfs_bio::fs_info by extracting it from btrfs_bio::inode"), we always
set the btrfs_bio::file_offset as the logical bytenr for scrub, and that
member will not be modified during IO.
So this means we have a stable way to determine the logical bytenr for a
scrub bio, now calc_sector_number() is just as simple as:
return (bbio->file_offset - stripe->logical) >> sectorsize_bits;
Since we're here, also add an ASSERT() to make sure the bbio is inside
the stripe, and change the return type to unsigned int to be extra safe.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For both scrub_repair_read_endio() and scrub_read_endio(), they share
the same bitmap update and bio put. Factor out the common code into a
helper to reduce duplication.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The last user of this macro is removed in commit 001e3fc263 ("btrfs:
scrub: remove scrub_block and scrub_sector structures").
Now that macro is only utilized in an ASSERT(), which no longer makes
much sense.
Just remove it completely.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There's no point in having a label where under it we do nothing but return
a variable. So remove it and directly return where we used to goto.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_dio_iomap_begin() calls btrfs_get_extent(), which returns an
extent map reference that must be dropped on all exit paths.
For direct writes into a NOCOW range, btrfs_get_blocks_direct_write()
keeps using that extent map and asks btrfs_create_dio_extent() to
allocate the ordered extent. If that fails, for example because
btrfs_alloc_ordered_extent() fails, the function returns the error
without dropping the input extent map. The PREALLOC path avoided this by
dropping the input extent map before replacing it with the newly created
one.
Check the error from btrfs_create_dio_extent() before replacing the
map and drop the input extent map on failure.
Fixes: 5f9a8a51d8 ("Btrfs: add semaphore to synchronize direct IO writes with fsync")
CC: stable@vger.kernel.org
Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Shuangpeng Bai <shuangpeng.kernel@gmail.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
