gcc sometimes fails to analyse how two local variables in ntfs_write_bh()
are initialized, as the initialization happens only in the first pass
through the main loop:
fs/ntfs3/fsntfs.c: In function 'ntfs_write_bh':
fs/ntfs3/fsntfs.c:1443:17: error: 'fixup' may be used uninitialized [-Werror=maybe-uninitialized]
1443 | __le16 *fixup;
| ^~~~~
fs/ntfs3/fsntfs.c:1443:17: note: 'fixup' was declared here
1443 | __le16 *fixup;
| ^~~~~
fs/ntfs3/fsntfs.c:1487:30: error: 'sample' may be used uninitialized [-Werror=maybe-uninitialized]
1487 | *ptr = sample;
| ~~~~~^~~~~~~~
fs/ntfs3/fsntfs.c:1444:16: note: 'sample' was declared here
1444 | __le16 sample;
Initializing the two variables to bogus values shuts up the warning and
makes it clear that those cannot be used. I tried rearranging the loop to
move the initialization in front of it, but couldn't quite figure it out.
Fixes: 48d9b57b16 ("fs/ntfs3: add a subset of W=1 warnings for stricter checks")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Added:
improve readahead for bitmap initialization and large directory scans
fsync files by syncing parent inodes
drop of preallocated clusters for sparse and compressed files
zero-fill folios beyond i_valid in ntfs_read_folio()
implement llseek SEEK_DATA/SEEK_HOLE by scanning data runs
implement iomap-based file operations
allow explicit boolean acl/prealloc mount options
a fall-through between switch labels
a delayed-allocation (delalloc) support
Fixed:
check return value of indx_find to avoid infinite loop
initialize new folios before use
an infinite loop in attr_load_runs_range on inconsistent metadata
an infinite loop triggered by zero-sized ATTR_LIST
ntfs_mount_options leak in ntfs_fill_super()
a deadlock in ni_read_folio_cmpr
a circular locking dependency in run_unpack_ex
prevent infinite loops caused by the next valid being the same
restore NULL folio initialization in ntfs_writepages()
a slab-out-of-bounds read in DeleteIndexEntryRoot
Changed:
allow readdir() to finish after directory mutations without rewinddir()
handle attr_set_size() errors when truncating files
make ntfs_writeback_ops static
refactor duplicate kmemdup pattern in do_action()
avoid calling run_get_entry() when run == NULL in ntfs_read_run_nb_ra()
Replaced:
use wait_on_buffer() directly
rename ni_readpage_cmpr into ni_read_folio_cmpr
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEh0DEKNP0I9IjwfWEqbAzH4MkB7YFAmmUuGYACgkQqbAzH4Mk
B7ZEHw/+LI38Sd6kkEJZLz2eMi1oL4FwS604tB9Y2k9KrxiPYbO4u2aEgomaCgTy
QXdeOQWaCJ2Zj0sNVHPoeF9x2neTWu+3DValLxDyL6C+joYYqFPgfkSMTfMOeX+o
Adf0WxGQSJ74Xnxn9dZER+rO50/V6xF0m5E8G9f78+fF+iN6zW8+tqc0wjkbgfNq
2qHJ5pyvN7izkzBV9ZYGt7UeLgoGE7JmKeuw/MyFgqCkh4k9yethK7N2cGYnUFyc
4RGuZloro5K7YlSgtvOufeLWoXNaP1rd8q+/skY1yJsJZbGknEWP11Ph1N67lW3b
VuqcHFKTNvj2fuEm+T+YMpnzRXEAoGNaAocn+sv1Pd6SBuI05xzWhCM+DWxDQWlN
fQhCMphf5FUhRfOLgitXnkgBM9wQAgRrE98+8jPmkaCxYJYviSYeKMkC0QfF09rf
P4Ct1lYbdZNcYjD0EGVISJL3KxQ3XPK26qVqdumOQk+30s85GlAvCfTPoNcwXmxS
xx/gnTFFIGFzyZyyMACRu9EdXZktPlBq70nWUMVfva5aGq0t+rKZSydiwQLCYRHj
NtRtg2O5Qd1QumdpjhQRsX1NC8UU1/VpAnixiS7FMvxo7bw5Ksnk+qiL5Ocao7lC
3Fd/95WYsUJZLkF/J8ayTkhexTyRcl2QZvDzRX00yEseIpQiu2U=
=Y5Q9
-----END PGP SIGNATURE-----
Merge tag 'ntfs3_for_7.0' of https://github.com/Paragon-Software-Group/linux-ntfs3
Pull ntfs3 updates from Konstantin Komarov:
"New code:
- improve readahead for bitmap initialization and large directory scans
- fsync files by syncing parent inodes
- drop of preallocated clusters for sparse and compressed files
- zero-fill folios beyond i_valid in ntfs_read_folio()
- implement llseek SEEK_DATA/SEEK_HOLE by scanning data runs
- implement iomap-based file operations
- allow explicit boolean acl/prealloc mount options
- fall-through between switch labels
- delayed-allocation (delalloc) support
Fixes:
- check return value of indx_find to avoid infinite loop
- initialize new folios before use
- infinite loop in attr_load_runs_range on inconsistent metadata
- infinite loop triggered by zero-sized ATTR_LIST
- ntfs_mount_options leak in ntfs_fill_super()
- deadlock in ni_read_folio_cmpr
- circular locking dependency in run_unpack_ex
- prevent infinite loops caused by the next valid being the same
- restore NULL folio initialization in ntfs_writepages()
- slab-out-of-bounds read in DeleteIndexEntryRoot
Updates:
- allow readdir() to finish after directory mutations without rewinddir()
- handle attr_set_size() errors when truncating files
- make ntfs_writeback_ops static
- refactor duplicate kmemdup pattern in do_action()
- avoid calling run_get_entry() when run == NULL in ntfs_read_run_nb_ra()
Replaced:
- use wait_on_buffer() directly
- rename ni_readpage_cmpr into ni_read_folio_cmpr"
* tag 'ntfs3_for_7.0' of https://github.com/Paragon-Software-Group/linux-ntfs3: (26 commits)
fs/ntfs3: add delayed-allocation (delalloc) support
fs/ntfs3: avoid calling run_get_entry() when run == NULL in ntfs_read_run_nb_ra()
fs/ntfs3: add fall-through between switch labels
fs/ntfs3: allow explicit boolean acl/prealloc mount options
fs/ntfs3: Fix slab-out-of-bounds read in DeleteIndexEntryRoot
ntfs3: Restore NULL folio initialization in ntfs_writepages()
ntfs3: Refactor duplicate kmemdup pattern in do_action()
fs/ntfs3: prevent infinite loops caused by the next valid being the same
fs/ntfs3: make ntfs_writeback_ops static
ntfs3: fix circular locking dependency in run_unpack_ex
fs/ntfs3: implement iomap-based file operations
fs/ntfs3: fix deadlock in ni_read_folio_cmpr
fs/ntfs3: implement llseek SEEK_DATA/SEEK_HOLE by scanning data runs
fs/ntfs3: zero-fill folios beyond i_valid in ntfs_read_folio()
fs/ntfs3: handle attr_set_size() errors when truncating files
fs/ntfs3: drop preallocated clusters for sparse and compressed files
fs/ntfs3: fsync files by syncing parent inodes
fs/ntfs3: fix ntfs_mount_options leak in ntfs_fill_super()
fs/ntfs3: allow readdir() to finish after directory mutations without rewinddir()
fs/ntfs3: improve readahead for bitmap initialization and large directory scans
...
This patch implements delayed allocation (delalloc) in ntfs3 driver.
It introduces an in-memory delayed-runlist (run_da) and the helpers to
track, reserve and later convert those delayed reservations into real
clusters at writeback time. The change keeps on-disk formats untouched and
focuses on pagecache integration, correctness and safe interaction with
fallocate, truncate, and dio/iomap paths.
Key points:
- add run_da (delay-allocated run tree) and bookkeeping for delayed clusters.
- mark ranges as delalloc (DELALLOC_LCN) instead of immediately allocating.
Actual allocation performed later (writeback / attr_set_size_ex / explicit
flush paths).
- direct i/o / iomap paths updated to avoid dio collisions with
delalloc: dio falls back or forces allocation of delayed blocks before
proceeding.
- punch/collapse/truncate/fallocate check and cancel delay-alloc reservations.
Sparse/compressed files handled specially.
- free-space checks updated (ntfs_check_free_space) to account for reserved
delalloc clusters and MFT record budgeting.
- delayed allocations are committed on last writer (file release) and on
explicit allocation flush paths.
Tested-by: syzbot@syzkaller.appspotmail.com
Reported-by: syzbot+2bd8e813c7f767aa9bb1@syzkaller.appspotmail.com
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
When ntfs_read_run_nb_ra() is invoked with run == NULL the code later
assumes run is valid and may call run_get_entry(NULL, ...), and also
uses clen/idx without initializing them. Smatch reported uninitialized
variable warnings and this can lead to undefined behaviour. This patch
fixes it.
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
Closes: https://lore.kernel.org/r/202512230646.v5hrYXL0-lkp@intel.com/
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
... so don't use __getname() there. Switch it (and ntfs_d_hash(), while
we are at it) to kmalloc(PATH_MAX, GFP_NOWAIT). Yes, ntfs_d_hash()
almost certainly can do with smaller allocations, but let ntfs folks
deal with that - keep the allocation size as-is for now.
Stop abusing names_cachep in ntfs, period - various uses of that thing
in there have nothing to do with pathnames; just use k[mz]alloc() and
be done with that. For now let's keep sizes as-in, but AFAICS none of
the users actually want PATH_MAX.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
This patch modifies the ntfs3 driver by replacing the buffer_head-based
operations with the iomap ones.
Implementation details:
- Implements core iomap operations (ntfs_iomap_begin/end) for block mapping:
Proper handling of resident attributes via IOMAP_INLINE.
Support for sparse files through IOMAP_HOLE semantics.
Correct unwritten extent handling for zeroing operations.
- Replaces custom implementations with standardized iomap helpers:
Converts buffered reads to use iomap_read_folio and iomap_readahead.
Implements iomap_file_buffered_write for write operations.
Uses iomap_dio_rw for direct I/O paths.
Migrates zero range operations to iomap_zero_range.
- Preserves special handling paths for compressed files
- Implements proper EOF/valid data size management during writes
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Previously sequential reads operations relied solely on single-page reads,
causing the block layer to perform many synchronous I/O requests,
especially for large volumes or large directories. This patch introduces
explicit readahead via page_cache_sync_readahead() and file_ra_state to
reduce I/O latency and improve sequential throughput.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Replace the use of ntfs_bio_pages with the disk page cache for reading and
writing compressed files. This slightly improves performance when reading
compressed data and simplifies the I/O logic.
When an XPRESS or LZX compressed file is opened for writing, it is now
decompressed into a normal file before modification. A new argument (`int copy`)
is added to ni_read_frame() to handle writing of decompressed and mapped data.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Fix a KMSAN un-init bug found by syzkaller.
ntfs_get_bh() expects a buffer from sb_getblk(), that buffer may not be
uptodate. We do not bring the buffer uptodate before setting it as
uptodate. If the buffer were to not be uptodate, it could mean adding a
buffer with un-init data to the mi record. Attempting to load that record
will trigger KMSAN.
Avoid this by setting the buffer as uptodate, if it’s not already, by
overwriting it.
Reported-by: syzbot+7a2ba6b7b66340cff225@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=7a2ba6b7b66340cff225
Tested-by: syzbot+7a2ba6b7b66340cff225@syzkaller.appspotmail.com
Fixes: 4342306f0f ("fs/ntfs3: Add file operations and implementation")
Signed-off-by: Raphael Pinsonneault-Thibeault <rpthibeault@gmail.com>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Use ntfs_inode field 'ni_bad' to mark inode as bad (if something went wrong)
and to avoid any operations
Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
ntfs_sb_read() was added in 2021 by
commit 82cae269cf ("fs/ntfs3: Add initialization of super block")
but hasn't been used.
Remove it.
Signed-off-by: Dr. David Alan Gilbert <linux@treblig.org>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
To check the length of the volume label, the existing constant
NTFS_LABEL_MAX_LENGTH could be used.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
bitmap_size() is a pretty generic name and one may want to use it for
a generic bitmap API function. At the same time, its logic is
NTFS-specific, as it aligns to the sizeof(u64), not the sizeof(long)
(although it uses ideologically right ALIGN() instead of division).
Add the prefix 'ntfs3_' used for that FS (not just 'ntfs_' to not mix
it with the legacy module) and use generic BITS_TO_U64() while at it.
Suggested-by: Yury Norov <yury.norov@gmail.com> # BITS_TO_U64()
Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Reviewed-by: Yury Norov <yury.norov@gmail.com>
Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Metafile /proc/fs/ntfs3/<dev>/label allows to read/write current ntfs label.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Check functions arguments. Use u8 instead of size_t for ntfs names, more consts and other.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Added error mesages with error codes.
Minor refactoring and code formatting.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Added minor refactoring.
Added and fixed some comments.
In some places, the code has been reformatted to fit into 80 columns.
clang-format-12 was used to format code according kernel's .clang-format.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
sbi->mft.reserved_bitmap is in-memory (not on-disk!) bitmap.
Assumed cpu endian is faster than fixed endian.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
The current volume flags are updated only if VOLUME_FLAG_DIRTY has been changed.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Added null pointer checks in function ntfs_security_init.
Also added le32_to_cpu in functions ntfs_security_init and indx_read.
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Added new function ntfs_check_for_free_space.
Added undo mechanism in attr_data_get_block.
Fixes xfstest generic/083
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
The functions from bitops.h already have _le variants so use them to
prevent invalid reads/writes of the bitmap on big endian systems.
Signed-off-by: Thomas Kühnel <thomas.kuehnel@avm.de>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
When enabled, the windows_names mount option prevents the creation
of files or directories with names not allowed by Windows. Use
the same option name as NTFS-3G for compatibility.
Signed-off-by: Daniel Pinto <danielpinto52@gmail.com>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEh0DEKNP0I9IjwfWEqbAzH4MkB7YFAmL7xrEACgkQqbAzH4Mk
B7Y3iw/+KZLidS4J9Lq+BhoDj21brkrvE+iiBhLClN8LQNn7jwkF+mTpYcGxvm+3
fgrT5YTUt6OBx9N06qjUnKKcwBrOq1xFo6hdlCVHQiZbEUgYsSglE6xVdyt5RICb
AVD6T7uKnHq4cQHe8796h23/Vfd3z2rAetvz5w/6Gd+mOpguLj+sx7QGE7R6o3V/
uvbxj8z7cFkrVy6b7H5gaDUFzAhVxBzZY+2P1CUOg4uUy0YI0PeUbJli8zt/qORP
Mr5mTEeKc1sxWzuDASppjgCPjdQVN+jgy7hQEpC6SLDR5HgtjncCCRE+dA1ZcnQm
PQCG1Xn8CSII8bDCu6Lvr6KtxhRBdG/wb99zpn50wBmb6CzMJGOGmBwrPMMhW8Zo
8ZBYHCY5YgDuXNkFpQrivayrADGaLhmAl1BTjXTDCQU7MoxxFsPO8D/swufvNf3W
5eC5ezQ8FY3sSHRuDvVGHe8djvgsGvfxQAMrbfMJqEBFuPg3EYjJOeRpZK6NUyk4
U31Jtz0hYSuU0dnoEaZFQ23/K7/vl2kile6VlNPApFR+y8OtSARN++Za58xdtg2y
H8XmEbuN/g8XxPXz55Smf4Y8RzaIZ2S56aA19nBqza5o1a6gQUr2SomTHGRLJsFQ
5xLyuUBrZRDxS8jcxa5TTfj7CNFBJkaxtXU8M66vIzXhXcm5+9U=
=iz2l
-----END PGP SIGNATURE-----
Merge tag 'ntfs3_for_6.0' of https://github.com/Paragon-Software-Group/linux-ntfs3
Pull ntfs3 updates from Konstantin Komarov:
- implement FALLOC_FL_INSERT_RANGE
- fix some logic errors
- fixed xfstests (tested on x86_64): generic/064 generic/213
generic/300 generic/361 generic/449 generic/485
- some dead code removed or refactored
* tag 'ntfs3_for_6.0' of https://github.com/Paragon-Software-Group/linux-ntfs3: (39 commits)
fs/ntfs3: uninitialized variable in ntfs_set_acl_ex()
fs/ntfs3: Remove unused function wnd_bits
fs/ntfs3: Make ni_ins_new_attr return error
fs/ntfs3: Create MFT zone only if length is large enough
fs/ntfs3: Refactoring attr_insert_range to restore after errors
fs/ntfs3: Refactoring attr_punch_hole to restore after errors
fs/ntfs3: Refactoring attr_set_size to restore after errors
fs/ntfs3: New function ntfs_bad_inode
fs/ntfs3: Make MFT zone less fragmented
fs/ntfs3: Check possible errors in run_pack in advance
fs/ntfs3: Added comments to frecord functions
fs/ntfs3: Fill duplicate info in ni_add_name
fs/ntfs3: Make static function attr_load_runs
fs/ntfs3: Add new argument is_mft to ntfs_mark_rec_free
fs/ntfs3: Remove unused mi_mark_free
fs/ntfs3: Fix very fragmented case in attr_punch_hole
fs/ntfs3: Fix work with fragmented xattr
fs/ntfs3: Make ntfs_fallocate return -ENOSPC instead of -EFBIG
fs/ntfs3: extend ni_insert_nonresident to return inserted ATTR_LIST_ENTRY
fs/ntfs3: Check reserved size for maximum allowed
...
There are repetitive steps in case of bad inode
This commit wraps them in function
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Improve static type checking by using enum req_op instead of u32 for
block layer request operations.
Cc: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Link: https://lore.kernel.org/r/20220714180729.1065367-60-bvanassche@acm.org
Signed-off-by: Jens Axboe <axboe@kernel.dk>
None of callers check the return value of ntfs_update_mftmirr(), so make
it return void to make code simpler.
Signed-off-by: Pavel Skripkin <paskripkin@gmail.com>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
If ntfs_fill_super() wasn't called then sbi->sb will be equal to NULL.
Code should check this ptr before dereferencing. Syzbot hit this issue
via passing wrong mount param as can be seen from log below
Fail log:
ntfs3: Unknown parameter 'iochvrset'
general protection fault, probably for non-canonical address 0xdffffc0000000003: 0000 [#1] PREEMPT SMP KASAN
KASAN: null-ptr-deref in range [0x0000000000000018-0x000000000000001f]
CPU: 1 PID: 3589 Comm: syz-executor210 Not tainted 5.18.0-rc3-syzkaller-00016-gb253435746d9 #0
...
Call Trace:
<TASK>
put_ntfs+0x1ed/0x2a0 fs/ntfs3/super.c:463
ntfs_fs_free+0x6a/0xe0 fs/ntfs3/super.c:1363
put_fs_context+0x119/0x7a0 fs/fs_context.c:469
do_new_mount+0x2b4/0xad0 fs/namespace.c:3044
do_mount fs/namespace.c:3383 [inline]
__do_sys_mount fs/namespace.c:3591 [inline]
Fixes: 82cae269cf ("fs/ntfs3: Add initialization of super block")
Reported-and-tested-by: syzbot+c95173762127ad76a824@syzkaller.appspotmail.com
Signed-off-by: Pavel Skripkin <paskripkin@gmail.com>
Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
This code triggers a Smatch warning:
fs/ntfs3/fsntfs.c:1606 ntfs_bio_fill_1()
warn: variable dereferenced before check 'bio' (see line 1591)
The "bio" pointer cannot be NULL so there is no need to check.
Originally there was more extensive NULL checking but it was removed
because bio_alloc() will never fail if it is allowed to sleep.
Remove this check as well.
Fixes: 39146b6f66 ("ntfs3: remove ntfs_alloc_bio")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20220128140922.GA29766@kili
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Pass the block_device and operation that we plan to use this bio for to
bio_alloc to optimize the assignment. NULL/0 can be passed, both for the
passthrough case on a raw request_queue and to temporarily avoid
refactoring some nasty code.
Also move the gfp_mask argument after the nr_vecs argument for a much
more logical calling convention matching what most of the kernel does.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
Link: https://lore.kernel.org/r/20220124091107.642561-18-hch@lst.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
