Add a iomap_bio_submit_read_endio helper factored out of
iomap_bio_submit_read to that all ->submit_read implementations for
iomap_read_ops that use iomap_bio_read_folio_range can shared the
logic.
Right now that logic is mostly trivial, but already has a bug for XFS
because the XFS version is too trivial: file system integrity validation
needs a workqueue context and thus can't happen from the default iomap
bi_end_io I/O handler. Unfortunately the iomap refactoring just before
fs integrity landed moved code around here and the call go misplaced,
meaning it never got called. The PI information still is verified by
the block layer, but the offloading is less efficient (and the future
userspace interface can't get at it).
Fixes: 0b10a37052 ("iomap: support T10 protection information")
Cc: stable@vger.kernel.org # v7.1
Signed-off-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260629121750.3392300-2-hch@lst.de
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
In exfat_find_dir_entry(), each TYPE_EXTEND (file name) entry advances the
output pointer by a fixed amount while the loop guard only tracks the
accumulated name length:
if (++order == 2)
uniname = p_uniname->name;
else
uniname += EXFAT_FILE_NAME_LEN;
len = exfat_extract_uni_name(ep, entry_uniname);
name_len += len;
unichar = *(uniname+len);
*(uniname+len) = 0x0;
uniname grows by EXFAT_FILE_NAME_LEN (15) per name entry, but name_len
grows only by the actual extracted length, which is shorter when a name
fragment contains an early NUL. The only guard is
`name_len >= MAX_NAME_LENGTH`, so a crafted directory with many short
name fragments lets uniname run far past the
p_uniname->name[MAX_NAME_LENGTH + 3] buffer while name_len stays small,
causing an out-of-bounds read and write at *(uniname+len).
The sibling extractor exfat_get_uniname_from_ext_entry() already stops
on a short fragment (the lockstep `len != EXFAT_FILE_NAME_LEN` guard
added in commit d42334578e ("exfat: check if filename entries exceeds
max filename length")); exfat_find_dir_entry() never got the
equivalent. Track the per-entry write offset as a count and reject a
fragment once the offset, or the offset plus the extracted length, would
exceed MAX_NAME_LENGTH, before forming the output pointer.
Fixes: ca06197382 ("exfat: add directory operations")
Cc: stable@vger.kernel.org
Suggested-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Bryam Vargas <hexlabsecurity@proton.me>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Commit 07d67f3e9083 ("exfat: add iomap buffered I/O support")
converted exfat buffered I/O to iomap, but did not add a
.swap_activate handler to the address_space_operations.
swapon(2) on an exfat swapfile then fails with EINVAL, which causes
LTP swap tests to fail.
Add exfat_iomap_swap_activate() and hook it into exfat_aops so exfat
uses iomap_swapfile_activate() for swapfile activation.
Fixes: 614f71ca1bdf ("exfat: add iomap buffered I/O support")
Closes: https://lore.kernel.org/all/20260603110212.3020276-1-japo@linux.ibm.com/
Signed-off-by: Jan Polensky <japo@linux.ibm.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Commit 8258ef2800 ("exfat: handle unreconized benign secondary
entries") added cluster freeing for benign secondary entries inside
exfat_remove_entries(). However, exfat_remove_entries() is also called
from the rename and move paths (exfat_rename_file and exfat_move_file),
where the old entry set is being relocated rather than deleted. This
causes benign secondary entries such as vendor extension entries to be
silently destroyed on rename or cross-directory move, violating the
exFAT spec requirement (section 8.2) that implementations preserve
unrecognized benign secondary entries.
Fix this by adding a free_benign parameter to exfat_remove_entries()
so callers can suppress cluster freeing during relocation, and
extending exfat_init_ext_entry() to copy trailing benign secondary
entries from the old entry set into the new one internally. Also
clean up the error paths to delete newly allocated entries on failure.
Fixes: 8258ef2800 ("exfat: handle unreconized benign secondary entries")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/linux-fsdevel/CAG7tbBV--waov7XVu2FHQEc6paR92dufS=em9DW5Kzsrpu3iQg@mail.gmail.com/
Signed-off-by: Rochan Avlur <rochan.avlur@gmail.com>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
exfat_setattr() did not call inode_dio_wait() before performing a size
change, leaving a window where a concurrent in-flight DIO write could be
operating on clusters that the truncate is about to free.
Add inode_dio_wait() before the truncate_setsize()/exfat_truncate()
sequence so that any in-flight DIO completes before cluster freeing
begins.
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Adds exfat_file_llseek() that implements these whence values via
the iomap layer (iomap_seek_hole() and iomap_seek_data()) using the
existing exfat_read_iomap_ops.
Unlike many other modern filesystems, exFAT does not support sparse files
with unallocated clusters (holes). In exFAT, clusters are always fully
allocated once they are written or preallocated. In addition, exFAT
maintains a separate "Valid Data Length" (valid_size) that is distinct
from the logical file size. This affects how holes are reported during
seeking. In exfat_iomap_begin(), ranges where the offset is greater
than or equal to ei->valid_size are mapped as IOMAP_UNWRITTEN, while ranges
below valid_size are mapped as IOMAP_MAPPED. This mapping behavior is used
by the iomap seek functions to correctly report SEEK_HOLE and SEEK_DATA
positions.
- Ranges with offset >= ei->valid_size are mapped as IOMAP_HOLE.
- Ranges with offset < ei->valid_size are mapped as IOMAP_MAPPED.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Add iomap-based direct I/O support to the exfat filesystem. This replaces
the previous exfat_direct_IO() implementation that used
blockdev_direct_IO() with iomap_dio_rw() interface.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Add full buffered I/O support using the iomap framework to the exfat
filesystem. This will replaces the old exfat_get_block(),
exfat_write_begin(), exfat_write_end(), and exfat_block_truncate_page()
with their iomap equivalents. Buffered writes now use
iomap_file_buffered_write(), read uses iomap_bio_read_folio() and
iomap_bio_readahead(), and writeback is handled through iomap_writepages().
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
exfat_cluster_walk() calls brelse(bh) without including the header that
declares the function, causing the following build error:
fs/exfat/exfat_fs.h:542:9: error: implicit declaration of function ‘brelse’ [-Werror=implicit-function-declaration]
Fix this by adding the missing buffer_head.h in exfat_fs.h.
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
This caches the data area start offset in bytes (data_start_bytes)
and introduces a helper function exfat_cluster_to_phys_bytes() to compute
the physical byte position of a given cluster.
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Currently exfat_map_cluster() allocates and returns only one cluster
at a time even when more clusters are needed. This causes multiple
FAT walks and repeated allocation calls during large sequential writes
or when using iomap for writes. This change exfat_map_cluster() and
exfat_alloc_cluster() to be able to allocate multiple contiguous
clusters.
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Add exfat_file_open() to handle file open operation for exFAT.
This change is a preparation step before introducing iomap-based direct
IO support.
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
In preparation for supporting the iomap infrastructure, we need to know
whether a new cluster was allocated or not in exfat_map_cluster().
Add an optional 'bool *balloc' output parameter. When a new cluster is
allocated, *balloc is set to true. Pass NULL from exfat_get_block() to
preserve the existing behavior.
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
The current exFAT driver relies on various macros for unit conversions
between clusters, blocks, sectors, and directory entries. These macros
are structurally unsafe as they lack type enforcement and are prone to
potential integer overflows during bit-shift operations, especially
on 64-bit architectures. Replace all arithmetic macros with static inline
functions to provide strict type checking and explicit casting.
Acked-by: Christoph Hellwig <hch@lst.de>
Acked-by: "Darrick J. Wong" <djwong@kernel.org>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
1) d_splice_alias() handles ERR_PTR() for inode just fine
2) no need to even look for existing aliases in case of directory inodes;
just punt to d_splice_alias(), it'll do the right thing
3) no need to bother with 'd_unhashed(alias)' case - d_find_alias()
would've returned that only in case of a directory, and d_splice_alias()
will handle that just fine on its own.
4) exfat_d_anon_disconn() is entirely pointless now - we only get to
evaluating it in case dentry->d_parent == alias->d_parent and
alias being a non-directory. But in that case IS_ROOT(alias) can't
possibly be true - that would've reqiured alias == alias->d_parent,
i.e alias == dentry->d_parent and dentry->d_parent is guaranteed to
be a directory. So exfat_d_anon_disconn() would always return false
when it's called, which makes && !exfat_d_anon_disconn(alias)
a no-op.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Reviewed-by: Sungjong Seo <sj1557.seo@samsung.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
In exfat_find_dir_entry(), the buffer_head obtained from
exfat_get_dentry() is released with brelse(bh) before the fall-through
TYPE_EXTEND branch reads the directory entry through ep (which points
into bh->b_data):
brelse(bh);
if (entry_type == TYPE_EXTEND) {
...
len = exfat_extract_uni_name(ep, entry_uniname);
...
}
After brelse() drops our reference, nothing guarantees that the
underlying page backing bh->b_data remains valid for the subsequent
exfat_extract_uni_name() read. This is the same pattern fixed in
commit fc961522dd ("exfat: Fix potential use after free in
exfat_load_upcase_table()").
Move brelse(bh) so it runs after ep is no longer dereferenced on
each branch.
Confirmed on QEMU x86_64 with CONFIG_KASAN=y + CONFIG_DEBUG_PAGEALLOC=y
+ CONFIG_PAGE_POISONING=y on linux-next, using a crafted exFAT image
(long filename with same-hash collisions forcing the TYPE_EXTEND path).
With a debug-only invalidate_bdev() inserted between brelse(bh) and
the ep read to make the stale-deref window deterministic, the
unpatched kernel faults:
BUG: KASAN: use-after-free in exfat_find_dir_entry+0x133b/0x15a0
BUG: unable to handle page fault for address: ffff88801a5fa0c2
Oops: 0000 [#1] SMP DEBUG_PAGEALLOC KASAN NOPTI
RIP: 0010:exfat_find_dir_entry+0x1188/0x15a0
With this patch applied, the same instrumented harness completes
cleanly under the same sanitizer stack. I have not reproduced a
crash on an uninstrumented kernel under ordinary reclaim; the
instrumented A/B establishes the lifetime violation and that the
patch closes it, not an unaided triggerability claim.
Fixes: ca06197382 ("exfat: add directory operations")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Michael Bommarito <michael.bommarito@gmail.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
When the size of the upcase table is set to zero in the dentry for any
reason(e.g. corrupted media or misbehaving device), an integer overflow
causes the module to loop indefinitely.
If the size of the upcase table is read zero, do not attempt to load the
table. Instead, fallback to loading the default upcase table. If the
size of the upcase table is zero or no upcase table is found, raise
exfat_fs_error() to mark the volume read-only.
Signed-off-by: David Timber <dxdt@dev.snart.me>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Report exFAT's case sensitivity behavior via the FS_XFLAG_CASEFOLD
flag. exFAT compares names through the volume's upcase table; in
practice that table folds case, and case is preserved at rest.
Acked-by: Namjae Jeon <linkinjeon@kernel.org>
Reviewed-by: Roland Mainz <roland.mainz@nrubsig.org>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Link: https://github.com/exfatprogs/exfatprogs/issues/313
Link: https://patch.msgid.link/20260507-case-sensitivity-v14-4-e62cc8200435@oracle.com
Signed-off-by: Christian Brauner <brauner@kernel.org>
- Implement FALLOC_FL_ALLOCATE_RANGE to add support for preallocating
clusters without zeroing, helping to reduce file fragmentation.
- Add a unified block readahead helper for FAT chain conversion, bitmap
allocation, and directory entry lookups.
- Optimize exfat_chain_cont_cluster() by caching buffer heads to minimize
mark_buffer_dirty() and mirroring overhead during NO_FAT_CHAIN to
FAT_CHAIN conversion.
- Switch to truncate_inode_pages_final() in evict_inode() to prevent
BUG_ON caused by shadow entries during reclaim.
- Fix a 32-bit truncation bug in directory entry calculations by ensuring
proper bitwise coercion.
- Fix sb->s_maxbytes calculation to correctly reflect the maximum possible
volume size for a given cluster size, resolving xfstests generic/213.
- Introduced exfat_cluster_walk() helper to traverse FAT chains by
a specified step, handling both ALLOC_NO_FAT_CHAIN and ALLOC_FAT_CHAIN
modes.
- Introduced exfat_chain_advance() helper to advance an exfat_chain
structure, updating both the current cluster and remaining size.
- Remove dead assignments and fix Smatch warnings.
-----BEGIN PGP SIGNATURE-----
iQJKBAABCgA0FiEE6NzKS6Uv/XAAGHgyZwv7A1FEIQgFAmnbHMUWHGxpbmtpbmpl
b25Aa2VybmVsLm9yZwAKCRBnC/sDUUQhCHXZEACDyb7s63to6V0tdry1D+/Q0hrX
blgUDnD+oGM5J+rFsvmfU+GhUJhe3hIgvLReUpBzrjegdVXCc6wBn+pBzQkjq81A
emJOIjOzyTgwusLuneSMGBZ3oCBJDiREGWqFeoeV1IHZoXvL4oqVr11V4b+8l2ms
Y2FqMljg8AYsgD/Wf3nXPJ1kph/5qpxeKQB4O1RIMEqJp5uoU0byDAtJSjde7oKy
cBRyYeGxcJ0UkQNX3pOfJHe2+Hb2wvCOPWpZ7fw4OfsOYcJMM7IFEKoT2tH67sx9
fo3J/Q74A2Gd0pwyWUwX2xK70uSkoLCR6TxInIgLaKK2IHmgpSHUJa+F5Nz9QFED
PDsBMedt38QXlI4jHNfYTIZuFXgFpIuDJyiphX7tQ72n+JVTiWNnKEF9mRRpRzZ+
2AV6h6YXN0Sd4/EJts+hf1S5admwEKNYpaBXgNTUBD/BbUQcKmvw8JQNk5/tYjU4
9jr4ab1jj1tvWtw9r2Ceu/W71W3l+Ys8gxcJB2DJB1vvo9h23NyHoNwnBMAwmNRv
5ykE7LFFHzwt8QUTMiGlMy5NFpd9cwcVKjzXd+Iogxqa1pDW/lFXrEYxzfvsqPwo
cBNIu2hUf2ESqctAmOWbpCRZwSWMQjh7cUwIHPqNcG2f3JMCzxxVlQC6SUJYr1qv
LoSPVMnQU/ReqFfK6w==
=JG0E
-----END PGP SIGNATURE-----
Merge tag 'exfat-for-7.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/linkinjeon/exfat
Pull exfat updates from Namjae Jeon:
- Implement FALLOC_FL_ALLOCATE_RANGE to add support for preallocating
clusters without zeroing, helping to reduce file fragmentation
- Add a unified block readahead helper for FAT chain conversion, bitmap
allocation, and directory entry lookups
- Optimize exfat_chain_cont_cluster() by caching buffer heads to
minimize mark_buffer_dirty() and mirroring overhead during
NO_FAT_CHAIN to FAT_CHAIN conversion
- Switch to truncate_inode_pages_final() in evict_inode() to prevent
BUG_ON caused by shadow entries during reclaim
- Fix a 32-bit truncation bug in directory entry calculations by
ensuring proper bitwise coercion
- Fix sb->s_maxbytes calculation to correctly reflect the maximum
possible volume size for a given cluster size, resolving xfstests
generic/213
- Introduced exfat_cluster_walk() helper to traverse FAT chains by a
specified step, handling both ALLOC_NO_FAT_CHAIN and ALLOC_FAT_CHAIN
modes
- Introduced exfat_chain_advance() helper to advance an exfat_chain
structure, updating both the current cluster and remaining size
- Remove dead assignments and fix Smatch warnings
* tag 'exfat-for-7.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/linkinjeon/exfat:
exfat: use exfat_chain_advance helper
exfat: introduce exfat_chain_advance helper
exfat: remove NULL cache pointer case in exfat_ent_get
exfat: use exfat_cluster_walk helper
exfat: introduce exfat_cluster_walk helper
exfat: fix incorrect directory checksum after rename to shorter name
exfat: fix s_maxbytes
exfat: fix passing zero to ERR_PTR() in exfat_mkdir()
exfat: fix error handling for FAT table operations
exfat: optimize exfat_chain_cont_cluster with cached buffer heads
exfat: drop redundant sec parameter from exfat_mirror_bh
exfat: use readahead helper in exfat_get_dentry
exfat: use readahead helper in exfat_allocate_bitmap
exfat: add block readahead in exfat_chain_cont_cluster
exfat: add fallocate FALLOC_FL_ALLOCATE_RANGE support
exfat: Fix bitwise operation having different size
exfat: Drop dead assignment of num_clusters
exfat: use truncate_inode_pages_final() at evict_inode()
Introduce exfat_chain_advance() to walk a exfat_chain structure by a
given step, updating both ->dir and ->size fields atomically. This
helper handles both ALLOC_NO_FAT_CHAIN and ALLOC_FAT_CHAIN modes with
proper boundary checking.
Suggested-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Sungjong Seo <sj1557.seo@samsung.com>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Since exfat_get_next_cluster has been updated, no callers pass a NULL
pointer to exfat_ent_get, so remove the handling logic for this case.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Sungjong Seo <sj1557.seo@samsung.com>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Replace the custom exfat_walk_fat_chain() function and open-coded
FAT chain walking logic with the exfat_cluster_walk() helper across
exfat_find_location, __exfat_get_dentry_set, and exfat_map_cluster.
Suggested-by: Sungjong Seo <sj1557.seo@samsung.com>
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Sungjong Seo <sj1557.seo@samsung.com>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Introduce exfat_cluster_walk() to walk the FAT chain by a given step,
handling both ALLOC_NO_FAT_CHAIN and ALLOC_FAT_CHAIN modes. Also
redefine exfat_get_next_cluster as a thin wrapper around it for
backward compatibility.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Sungjong Seo <sj1557.seo@samsung.com>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
When renaming a file in-place to a shorter name, exfat_remove_entries
marks excess entries as DELETED, but es->num_entries is not updated
accordingly. As a result, exfat_update_dir_chksum iterates over the
deleted entries and computes an incorrect checksum.
This does not lead to persistent corruption because mark_inode_dirty()
is called afterward, and __exfat_write_inode later recomputes the
checksum using the correct num_entries value.
Fix by setting es->num_entries = num_entries in exfat_init_ext_entry.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Sungjong Seo <sj1557.seo@samsung.com>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
With fallocate support, xfstest unit generic/213 fails with
QA output created by 213
We should get: fallocate: No space left on device
Strangely, xfs_io sometimes says "Success" when something went wrong
-fallocate: No space left on device
+fallocate: File too large
because sb->s_maxbytes is set to the volume size.
To be in line with other non-extent-based filesystems, set to max volume
size possible with the cluster size of the volume.
Signed-off-by: David Timber <dxdt@dev.snart.me>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
The implementation is now really basic so rename generic_file_fsync()
simple_fsync() and __generic_file_fsync() to simple_fsync_noflush().
Signed-off-by: Jan Kara <jack@suse.cz>
Link: https://patch.msgid.link/20260326095354.16340-56-jack@suse.cz
Signed-off-by: Christian Brauner <brauner@kernel.org>
EXFAT never calls mark_buffer_dirty_inode() and thus
invalidate_inode_buffers() never has anything to evict. Drop the
pointless call.
Signed-off-by: Jan Kara <jack@suse.cz>
Link: https://patch.msgid.link/20260326095354.16340-49-jack@suse.cz
Signed-off-by: Christian Brauner <brauner@kernel.org>
Detected by Smatch.
namei.c:890 exfat_mkdir() warn:
passing zero to 'ERR_PTR'
Signed-off-by: Yang Wen <anmuxixixi@gmail.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Fix three error handling issues in FAT table operations:
1. Fix exfat_update_bh() to properly return errors from sync_dirty_buffer
2. Fix exfat_end_bh() to properly return errors from exfat_update_bh()
and exfat_mirror_bh()
3. Fix ignored return values from exfat_chain_cont_cluster() in inode.c
and namei.c
These fixes ensure that FAT table write errors are properly propagated
to the caller instead of being silently ignored.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
When converting files from NO_FAT_CHAIN to FAT_CHAIN format, profiling
reveals significant time spent in mark_buffer_dirty() and exfat_mirror_bh()
operations. This overhead occurs because each FAT entry modification
triggers a full block dirty marking and mirroring operation.
For consecutive clusters that reside in the same block, optimize by caching
the buffer head and performing dirty marking only once at the end of the
block's modifications.
Performance improvements for converting a 30GB file:
| Cluster Size | Before Patch | After Patch | Speedup |
|--------------|--------------|-------------|---------|
| 512 bytes | 4.243s | 1.866s | 2.27x |
| 4KB | 0.863s | 0.236s | 3.66x |
| 32KB | 0.069s | 0.034s | 2.03x |
| 256KB | 0.012s | 0.006s | 2.00x |
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
The sector offset can be obtained from bh->b_blocknr, so drop the
redundant sec parameter from exfat_mirror_bh(). Also clean up the
function to use exfat_update_bh() helper.
No functional changes.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Replace the custom exfat_dir_readahead() function with the unified
exfat_blk_readahead() helper in exfat_get_dentry(). This removes
the duplicate readahead implementation and uses the common interface,
also reducing code complexity.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Use the newly added exfat_blk_readahead() helper in exfat_allocate_bitmap()
to simplify the code. This eliminates the duplicate inline readahead logic
and uses the unified readahead interface.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
When a file cannot allocate contiguous clusters, exfat converts the file
from NO_FAT_CHAIN to FAT_CHAIN format. For large files, this conversion
process can take a significant amount of time.
Add simple readahead to read all the FAT blocks in advance, as these
blocks are consecutive, significantly improving the conversion performance.
Test in an empty exfat filesystem:
dd if=/dev/zero of=/mnt/file bs=1M count=30k
dd if=/dev/zero of=/mnt/file2 bs=1M count=1
time cat /mnt/file2 >> /mnt/file
| cluster size | before patch | after patch |
| ------------ | ------------ | ----------- |
| 512 | 47.667s | 4.316s |
| 4k | 6.436s | 0.541s |
| 32k | 0.758s | 0.071s |
| 256k | 0.117s | 0.011s |
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Currently, the Linux (ex)FAT drivers do not employ any cluster
allocation strategy to keep fragmentation at bay. As a result, when
multiple processes are competing for new clusters to expand files in
exfat filesystem on Linux simultaneously, the files end up heavily
fragmented. HDDs are most impacted, but this could also have some
negative impact on various forms of flash memory depending on the
type of underlying technology.
For instance, modern digital cameras produce multiple media files for a
single video stream. If the application does not take the fragmentation
issue into account or the system is under memory pressure, the kernel
end up allocating clusters in said files in a interleaved manner.
Demo script:
for (( i = 0; i < 4; i += 1 ));
do
dd if=/dev/urandom iflag=fullblock bs=1M count=64 of=frag-$i &
done
for (( i = 0; i < 4; i += 1 ));
do
wait
done
filefrag frag-*
Result - Linux kernel native exfat, async mount:
780 extents found
740 extents found
809 extents found
712 extents found
Result - Linux kernel native exfat, sync mount:
1852 extents found
1836 extents found
1846 extents found
1881 extents found
Result - Windows XP:
3 extents found
3 extents found
3 extents found
2 extents found
Windows kernel, on the other hand, regardless of the underlying storage
interface or the medium, seems to space out clusters for each file.
Similar strategy has to be employed by Linux fat filesystems for
efficient utilisation of storage backend.
In the meantime, userspace applications like rsync may
use fallocate to combat this issue.
This patch may introduce a regression-like behaviour to some niche
filesystem-agnostic applications that use fallocate and proceed to
non-sequentially write to the file. Examples:
- libtorrent's use of posix_fallocate() and the first fragment from a
peer is near the end of the file
- "Download accelerators" that do partial content requests(HTTP 206)
in multiple threads writing to the same file
The delay incurred in such use cases is documented in WinAPI. Patches
that add the ioctl equivalents to the WinAPI function
SetFileValidData() and `fsutil file queryvaliddata ...` will follow.
Signed-off-by: David Timber <dxdt@dev.snart.me>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
cpos has type loff_t (long long), while s_blocksize has type u32. The
inversion wil happen on u32, the coercion to s64 happens afterwards and
will do 0-left-paddding, resulting in the upper bits getting masked out.
Cast s_blocksize to loff_t before negating it.
Found by static code analysis using Klocwork.
Signed-off-by: Philipp Hahn <phahn-oss@avm.de>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
num_clusters is not used anywhere afterwards. Remove assignment.
Found by static code analysis using Klocwork.
Signed-off-by: Philipp Hahn <phahn-oss@avm.de>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Currently, exfat uses truncate_inode_pages() in exfat_evict_inode().
However, truncate_inode_pages() does not mark the mapping as exiting,
so reclaim may still install shadow entries for the mapping until
the inode teardown completes.
In older kernels like Linux 5.10, if shadow entries are present
at that point,clear_inode() can hit
BUG_ON(inode->i_data.nrexceptional);
To align with VFS eviction semantics and prevent this situation,
switch to truncate_inode_pages_final() in ->evict_inode().
Other filesystems were updated to use truncate_inode_pages_final()
in ->evict_inode() by commit 91b0abe36a ("mm + fs: store shadow
entries in page cache")'.
Signed-off-by: Yang Wen <anmuxixixi@gmail.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
This converts some of the visually simpler cases that have been split
over multiple lines. I only did the ones that are easy to verify the
resulting diff by having just that final GFP_KERNEL argument on the next
line.
Somebody should probably do a proper coccinelle script for this, but for
me the trivial script actually resulted in an assertion failure in the
middle of the script. I probably had made it a bit _too_ trivial.
So after fighting that far a while I decided to just do some of the
syntactically simpler cases with variations of the previous 'sed'
scripts.
The more syntactically complex multi-line cases would mostly really want
whitespace cleanup anyway.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
Add a blank line after variable declarations in fatent.c and file.c.
This improves readability and makes code style more consistent
across the exfat subsystem.
Signed-off-by: William Hansen-Baird <william.hansen.baird@gmail.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Else-branch is unnecessary after return statement in if-branch.
Remove to enhance readability and reduce indentation.
Signed-off-by: William Hansen-Baird <william.hansen.baird@gmail.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
This patch introduces a count parameter to exfat_get_cluster, which
serves as an input parameter for the caller to specify the desired
number of clusters, and as an output parameter to store the length
of consecutive clusters.
This patch can improve read performance by reducing the number of
get_block calls in sequential read scenarios. speacially in small
cluster size.
According to my test data, the performance improvement is
approximately 10% when read FAT_CHAIN file with 512 bytes of
cluster size.
454 MB/s -> 511 MB/s
Suggested-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Change exfat_cache_lookup to return the cluster number of the last
cluster before the next cache (i.e., the end of the current cache range)
or the given 'end' if there is no next cache. This allows the caller to
know whether the next cluster after the current cache is cached.
The function signature is changed to accept an 'end' parameter, which
is the upper bound of the search range. The function now stops early
if it finds a cache that starts within the current cache's tail, meaning
caches are contiguous. The return value is the cluster number at which
the next cache starts (minus one) or the original 'end' if no next cache
is found.
The new behavior is illustrated as follows:
cache: [ccccccc-------ccccccccc]
search: [..................]
return: ^
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
The current cache mechanism does not support reading clusters starting
from a file offset of zero. This patch enables that feature in
preparation for subsequent reads of contiguous clusters from offset zero.
1. support finding clusters with zero offset.
2. allow clusters with zero offset to be cached.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
This patch introduces a parameter 'count' to support fetching multiple
clusters in exfat_map_cluster. The returned 'count' indicates the number
of consecutive clusters, or 0 when the input cluster offset is past EOF.
And the 'count' is also an input parameter for the caller to specify the
required number of clusters.
Only NO_FAT_CHAIN files enable multi-cluster fetching in this patch.
After this patch, the time proportion of exfat_get_block has decreased,
The performance data is as follows:
Cluster size: 512 bytes
Sequential read of a 30GB NO_FAT_CHAIN file:
2.4GB/s -> 2.5 GB/s
proportion of exfat_get_block:
10.8% -> 0.02%
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Yuezhang said: "exfat_map_cluster() is only used for files. The code
in this 'else' block is never executed and can be cleaned up."
Suggested-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
Since exfat_ent_get supports cache buffer head, we can use this option to
reduce sb_bread calls when fetching consecutive entries.
Signed-off-by: Chi Zhiling <chizhiling@kylinos.cn>
Reviewed-by: Yuezhang Mo <Yuezhang.Mo@sony.com>
Signed-off-by: Namjae Jeon <linkinjeon@kernel.org>
