Commit Graph

212 Commits

Author SHA1 Message Date
Bruce Johnston
fb7415f2ab dm vdo: use GFP_NOIO for blkdev_issue_zeroout on format path
GFP_NOWAIT is inappropriate when blkdev_issue_zeroout may sleep and
bio_alloc can fail under pressure; use GFP_NOIO for clear_partition and
vdo_clear_layout zeroout calls.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fixes: fc1d438267 ("dm vdo: save the formatted metadata to disk")
2026-05-04 15:02:25 +02:00
Colin Ian King
a1cf2bd5b6 dm vdo: Fix spelling mistake "postive" -> "positive"
There is a spelling mistake in a vdo_log_error message. Fix it.

Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-30 12:06:08 +02:00
Bruce Johnston
fc1d438267 dm vdo: save the formatted metadata to disk
Add vdo_save_super_block() and vdo_save_geometry_block() to perform
asynchronous writes of the super block and geometry block respectively.
Add vdo_clear_layout() to zero the UDS index's first block, the block
map partition, and the recovery journal partition.

These operations are driven by new phases in the pre-load state machine
(PRE_LOAD_PHASE_FORMAT_*), ensuring that disk writes happen during
pre-resume rather than during dmsetup create.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:19:25 +01:00
Bruce Johnston
defce4e039 dm vdo: add formatting logic and initialization
Add the core formatting logic. The initialization path is updated to
read the geometry block (block 0 on the storage device). If the block
is entirely zeroed, the device is treated as unformatted and
vdo_format() is called. Otherwise, the existing geometry is parsed
and the VDO is loaded as before.

The vdo_format() function initializes the volume geometry and super
block, and marks the VDO as needing it's layout saved to disk.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:19:15 +01:00
Bruce Johnston
9d9c28aa86 dm vdo: add synchronous metadata I/O submission helper
Add vdo_submit_metadata_vio_wait(), a synchronous I/O submission
helper that blocks until completion. This is needed for I/O during
early initialization before work queues are available.

Refactor read_geometry_block() to use it.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:19:05 +01:00
Bruce Johnston
b5d1f45c5f dm vdo: add geometry block structure
Introduce a vdo_geometry_block structure, containing a vio and buffer,
mirroring the existing vdo_super_block structure. Both are now
initialized at VDO startup and freed at shutdown, establishing the
infrastructure needed to read and write the geometry block using the
same mechanisms as the super block.

Refactor read_geometry_block() to use the new structure.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:18:55 +01:00
Bruce Johnston
427bf2c1f7 dm vdo: add geometry block encoding
Add vdo_encode_volume_geometry() to write the geometry block into a
buffer so that it can be written to disk. The corresponding decode
path already exists.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:18:42 +01:00
Bruce Johnston
beced130a3 dm vdo: add upfront validation for logical size
Add a validation check that the logical size passed via the table line
does not exceed MAXIMUM_VDO_LOGICAL_BLOCKS.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:18:25 +01:00
Bruce Johnston
2fb98e4170 dm vdo: add formatting parameters to table line
Extend the dm table line with three new optional parameters:
indexMemory (UDS index memory size), indexSparse (dense vs sparse
index), and slabSize (blocks per allocation slab). These values are
parsed, validated, and stored in the device configuration for use
during formatting.

Rework the slab size constants from the single MAX_VDO_SLAB_BITS into
explicit MIN_VDO_SLAB_BLOCKS, MAX_VDO_SLAB_BLOCKS, and
DEFAULT_VDO_SLAB_BLOCKS values.

Bump the target version from 9.1.0 to 9.2.0 to reflect this table
line change.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:18:08 +01:00
Bruce Johnston
e073bb098a dm vdo: add super block initialization to encodings.c
Add vdo_initialize_component_states() to populate the super block,
computing the space required for the main VDO components on disk.
Those include the slab depot, block map, and recovery journal.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:17:42 +01:00
Bruce Johnston
4b4a8d9560 dm vdo: add geometry block initialization to encodings.c
Add vdo_initialize_volume_geometry() to populate the geometry block,
computing the space required for the two main regions on disk.

Add uds_compute_index_size() to calculate the space required for the
UDS indexer from the UDS configuration.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-26 18:17:31 +01:00
Ken Raeburn
db139186be dm vdo: add __counted_by attribute to a number of structures
This attribute allows the compiler to refine compile-time diagnostics
and run-time sanitizer features with information about the size of the
flexible arrays.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-02 17:03:37 +01:00
Ken Raeburn
9bb388b1a9 dm vdo: remove redundant TYPE argument from vdo_allocate macro
We can infer the type needed from the supplied pointer argument. A
couple invocation sites needed fixing to supply the proper type of
pointer.

Use overflow.h's size_mul, and we can remove the __vdo_do_allocation
wrapper which did the same overflow check.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-02 17:03:28 +01:00
Ken Raeburn
6af58aa3b0 dm vdo: update vdo_allocate_extended to take a field name, no types
All of VDO's "extended" allocations use a flexible array field at the
end of the allocated structure. We can infer the struct type from the
supplied pointer. Replacing the array field type with the field name
lets us use struct_size from overflow.h to compute the size instead of
the local __vdo_do_allocation version.

One allocation of bio structures doesn't conform to this pattern,
since the removal of bi_inline_vecs; directly compute the total size
for that case.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-02 17:03:18 +01:00
Matthew Sakai
b3929b2cc2 dm vdo slab-depot: validate old zone count on load
Verify the old zone count has a valid value before using
it to compute slab summary entry offsets.

Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-02 16:37:59 +01:00
Matthew Sakai
9e809bb1de dm vdo indexer: validate saved zone count
Verify that the loaded zone count is in the valid range
before using it as a loop iterator.

Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-03-02 16:37:43 +01:00
Matthew Sakai
ec8534021a dm vdo encodings: clean up header and version functions
Make several header functions static. Also remove
vdo_is_upgradable_version, which is unused.

Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-01-27 17:00:25 +01:00
Julia Lawall
a8c3ec7d07 dm-vdo: adjust function name reference
There is no function advance_compression_stage().  But
advance_data_vio_compression_stage() does iterate through
the values of the data_vio_compression_stage enum, so it
seems to be what was intended.

Signed-off-by: Julia Lawall <Julia.Lawall@inria.fr>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2026-01-02 15:05:33 +01:00
Linus Torvalds
d358e52546 small dm code refactoring
fix deadlock when reloading a multipath table
 
 don't warn if IMA_DISABLE_HTABLE is not enabled
 
 test for REQ_ATOMIC in dm_accept_partial_bio()
 
 ignore discard return value
 
 MAINTAINERS: add Benjamin Marzinski as a device mapper maintainer
 
 dm-bufio: align write boundary on physical block size
 
 dm-crypt: use the MD5 library directly
 
 dm-crypt: use folio_nr_pages() instead of shift operation
 
 dm-crypt: enable DM_TARGET_ATOMIC_WRITES
 
 dm-verity: remove log message with shash driver name
 
 dm-verity: use the SHA-256 library directly
 
 dm-verity: use 2-way interleaved SHA-256 hashing when supported
 
 dm-verity: expose corrected block count via status
 
 dm-verity: disable recursive forward error correction
 
 dm-verity: remove useless mempool
 
 dm-mpath: enable DM_TARGET_ATOMIC_WRITES
 
 dm-mpath: simplify the setup_scsi_dh code
 
 dm-ebs: mark full buffer dirty even on partial write
 
 dm-vdo: fix kerneldoc warnings
 
 dm-snapshot: fix 'scheduling while atomic' on real-time kernels
 
 dm-raid: fix possible NULL dereference with undefined raid type
 
 dm raid: add documentation for takeover/reshape raid1 -> raid5 table line examples
 
 dm-log-writes: add missing set_freezable() for freezable kthread
 
 dm-pcache: fixes in indexing code
 -----BEGIN PGP SIGNATURE-----
 
 iIoEABYIADIWIQRnH8MwLyZDhyYfesYTAyx9YGnhbQUCaTnD7BQcbXBhdG9ja2FA
 cmVkaGF0LmNvbQAKCRATAyx9YGnhbS3NAQCfc83GS5i7D4NoOjYP7PNElendmc99
 WxsetuFT4dQYKAD/UgistGLf6vQ5LGE8Kp9YWsQ6hb8BgjoKo0mSjwCHQgQ=
 =C9Du
 -----END PGP SIGNATURE-----

Merge tag 'for-6.19/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm

Pull device mapper updates from Mikulas Patocka:

 - convert crypto_shash users to direct crypto library use with simpler
   and faster code and reduced stack usage (Eric Biggers):

     - the dm-verity SHA-256 conversion also teaches it to do two-way
       interleaved hashing for added performance

     - dm-crypt MD5 conversion (used for Loop-AES compatibility)

 - added document for for takeover/reshape raid1 -> raid5 examples (Heinz Mauelshagen)

 - fix dm-vdo kerneldoc warnings (Matthew Sakai)

 - various random fixes and cleanups

* tag 'for-6.19/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (29 commits)
  dm pcache: fix segment info indexing
  dm pcache: fix cache info indexing
  dm-pcache: advance slot index before writing slot
  dm raid: add documentation for takeover/reshape raid1 -> raid5 table line examples
  dm log-writes: Add missing set_freezable() for freezable kthread
  dm-raid: fix possible NULL dereference with undefined raid type
  dm-snapshot: fix 'scheduling while atomic' on real-time kernels
  dm: ignore discard return value
  MAINTAINERS: add Benjamin Marzinski as a device mapper maintainer
  dm-mpath: Simplify the setup_scsi_dh code
  dm vdo: fix kerneldoc warnings
  dm-bufio: align write boundary on physical block size
  dm-crypt: enable DM_TARGET_ATOMIC_WRITES
  dm: test for REQ_ATOMIC in dm_accept_partial_bio()
  dm-verity: remove useless mempool
  dm-verity: disable recursive forward error correction
  dm-ebs: Mark full buffer dirty even on partial write
  dm mpath: enable DM_TARGET_ATOMIC_WRITES
  dm verity fec: Expose corrected block count via status
  dm: Don't warn if IMA_DISABLE_HTABLE is not enabled
  ...
2025-12-11 12:13:29 +09:00
Matthew Sakai
4efe85b0c4 dm vdo: fix kerneldoc warnings
Fix kerneldoc warnings across the dm-vdo target. Also
remove some unhelpful or inaccurate doc comments, and fix
some format inconsistencies that did not produce warnings.

No functional changes.

Suggested-by: Sunday Adelodun <adelodunolaoluwa@yahoo.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-12-10 19:28:22 +01:00
Matthew Wilcox (Oracle)
70e0a80a1f treewide: Remove in_irq()
This old alias for in_hardirq() has been marked as deprecated since
2020; remove the stragglers.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://patch.msgid.link/20251024180654.1691095-1-willy@infradead.org
2025-10-24 21:39:27 +02:00
Linus Torvalds
7dbec0bbc3 dm docs: fix typos
dm, dm-ima, dm-bufio, dm-vdo, dm-raid: small refactoring
 
 dm-error: mark it with DM_TARGET_PASSES_INTEGRITY
 
 dm-pcache: a new target for read/write caching on persistent memory
 
 dm-request-based: fix NULL pointer dereference and quiesce_depth out of sync
 
 dm-linear: optimize REQ_PREFLUSH
 
 dm-vdo: return error on corrupted metadata
 
 dm-integrity: support asynchronous hash interface
 -----BEGIN PGP SIGNATURE-----
 
 iIoEABYIADIWIQRnH8MwLyZDhyYfesYTAyx9YGnhbQUCaN/T8RQcbXBhdG9ja2FA
 cmVkaGF0LmNvbQAKCRATAyx9YGnhbdPWAP9JEpnq09RzwneB/FdCE2WsjInsAaas
 eYqKrkgoatVzOwEAoqEzYb9IUrKMZbFPdWzEA1aXUBCH+UAKhm+G9WG5Vw0=
 =dCXi
 -----END PGP SIGNATURE-----

Merge tag 'for-6.18/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm

Pull device mapper updates from Mikulas Patocka:

 - a new dm-pcache target for read/write caching on persistent memory

 - fix typos in docs

 - misc small refactoring

 - mark dm-error with DM_TARGET_PASSES_INTEGRITY

 - dm-request-based: fix NULL pointer dereference and quiesce_depth out of sync

 - dm-linear: optimize REQ_PREFLUSH

 - dm-vdo: return error on corrupted metadata

 - dm-integrity: support asynchronous hash interface

* tag 'for-6.18/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (27 commits)
  dm raid: use proper md_ro_state enumerators
  dm-integrity: prefer synchronous hash interface
  dm-integrity: enable asynchronous hash interface
  dm-integrity: rename internal_hash
  dm-integrity: add the "offset" argument
  dm-integrity: allocate the recalculate buffer with kmalloc
  dm-integrity: introduce integrity_kmap and integrity_kunmap
  dm-integrity: replace bvec_kmap_local with kmap_local_page
  dm-integrity: use internal variable for digestsize
  dm vdo: return error on corrupted metadata in start_restoring_volume functions
  dm vdo: Update code to use mem_is_zero
  dm: optimize REQ_PREFLUSH with data when using the linear target
  dm-pcache: use int type to store negative error codes
  dm: fix "writen"->"written"
  dm-pcache: cleanup: fix coding style report by checkpatch.pl
  dm-pcache: remove ctrl_lock for pcache_cache_segment
  dm: fix NULL pointer dereference in __dm_suspend()
  dm: fix queue start/stop imbalance under suspend/load/resume races
  dm-pcache: add persistent cache target in device-mapper
  dm error: mark as DM_TARGET_PASSES_INTEGRITY
  ...
2025-10-03 18:48:02 -07:00
Ivan Abramov
9ddf6d3fcb dm vdo: return error on corrupted metadata in start_restoring_volume functions
The return values of VDO_ASSERT calls that validate metadata are not acted
upon.

Return UDS_CORRUPT_DATA in case of an error.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

Fixes: a4eb7e2555 ("dm vdo: implement the volume index")
Signed-off-by: Ivan Abramov <i.abramov@mt-integration.ru>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-09-23 10:49:45 +02:00
Bruce Johnston
e828a1875e dm vdo: Update code to use mem_is_zero
Remove function that would check if data was all zeroes. Use the
built-in kernel function mem_is_zero() instead.

Signed-off-by: Bruce Johnston <bjohnsto@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-09-23 10:49:17 +02:00
Christoph Hellwig
d86eaa0f3c block: remove the bi_inline_vecs variable sized array from struct bio
Bios are embedded into other structures, and at least spare is unhappy
about embedding structures with variable sized arrays.  There's no
real need to the array anyway, we can replace it with a helper pointing
to the memory just behind the bio, and with the previous cleanups there
is very few site doing anything special with it.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: John Garry <john.g.garry@oracle.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2025-09-09 07:31:59 -06:00
Mikulas Patocka
225b2cb640 vdo: omit need_resched() before cond_resched()
There's no need to call need_resched() because cond_resched() will do
nothing if need_resched() returns false.

Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-07-31 15:39:55 +02:00
Ingo Molnar
41cb08555c treewide, timers: Rename from_timer() to timer_container_of()
Move this API to the canonical timer_*() namespace.

[ tglx: Redone against pre rc1 ]

Signed-off-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/all/aB2X0jCKQO56WdMt@gmail.com
2025-06-08 09:07:37 +02:00
Matthew Sakai
3da732687d dm vdo indexer: don't read request structure after enqueuing
The function get_volume_page_protected may place a request on
a queue for another thread to process asynchronously. When this
happens, the volume should not read the request from the original
thread. This can not currently cause problems, due to the way
request processing is handled, but it is not safe in general.

Reviewed-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-05-15 15:54:47 +02:00
Thomas Gleixner
8fa7292fee treewide: Switch/rename to timer_delete[_sync]()
timer_delete[_sync]() replaces del_timer[_sync](). Convert the whole tree
over and remove the historical wrapper inlines.

Conversion was done with coccinelle plus manual fixups where necessary.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2025-04-05 10:30:12 +02:00
Linus Torvalds
5014bebee0 - dm-crypt: switch to using the crc32 library
- dm-verity, dm-integrity, dm-crypt: documentation improvement
 
 - dm-vdo fixes
 
 - dm-stripe: enable inline crypto passthrough
 
 - dm-integrity: set ti->error on memory allocation failure
 
 - dm-bufio: remove unused return value
 
 - dm-verity: do forward error correction on metadata I/O errors
 
 - dm: fix unconditional IO throttle caused by REQ_PREFLUSH
 
 - dm cache: prevent BUG_ON by blocking retries on failed device resumes
 
 - dm cache: support shrinking the origin device
 
 - dm: restrict dm device size to 2^63-512 bytes
 
 - dm-delay: support zoned devices
 
 - dm-verity: support block number limits for different ioprio classes
 
 - dm-integrity: fix non-constant-time tag verification (security bug)
 
 - dm-verity, dm-ebs: fix prefetch-vs-suspend race
 -----BEGIN PGP SIGNATURE-----
 
 iIoEABYIADIWIQRnH8MwLyZDhyYfesYTAyx9YGnhbQUCZ+u7shQcbXBhdG9ja2FA
 cmVkaGF0LmNvbQAKCRATAyx9YGnhbZ0JAQDVhbl77u9jjPWjxJvFodMAqw+KPXGC
 MNzkyzG0lu7oPAEA33vt5pHQtr7F3SJj/sDBuZ+rb5bvUtgxeGqpJOQpTAk=
 =tj00
 -----END PGP SIGNATURE-----

Merge tag 'for-6.15/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm

Pull device mapper updates from Mikulas Patocka:

 - dm-crypt: switch to using the crc32 library

 - dm-verity, dm-integrity, dm-crypt: documentation improvement

 - dm-vdo fixes

 - dm-stripe: enable inline crypto passthrough

 - dm-integrity: set ti->error on memory allocation failure

 - dm-bufio: remove unused return value

 - dm-verity: do forward error correction on metadata I/O errors

 - dm: fix unconditional IO throttle caused by REQ_PREFLUSH

 - dm cache: prevent BUG_ON by blocking retries on failed device resumes

 - dm cache: support shrinking the origin device

 - dm: restrict dm device size to 2^63-512 bytes

 - dm-delay: support zoned devices

 - dm-verity: support block number limits for different ioprio classes

 - dm-integrity: fix non-constant-time tag verification (security bug)

 - dm-verity, dm-ebs: fix prefetch-vs-suspend race

* tag 'for-6.15/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (27 commits)
  dm-ebs: fix prefetch-vs-suspend race
  dm-verity: fix prefetch-vs-suspend race
  dm-integrity: fix non-constant-time tag verification
  dm-verity: support block number limits for different ioprio classes
  dm-delay: support zoned devices
  dm: restrict dm device size to 2^63-512 bytes
  dm cache: support shrinking the origin device
  dm cache: prevent BUG_ON by blocking retries on failed device resumes
  dm vdo indexer: reorder uds_request to reduce padding
  dm: fix unconditional IO throttle caused by REQ_PREFLUSH
  dm vdo: rework processing of loaded refcount byte arrays
  dm vdo: remove remaining ring references
  dm-verity: do forward error correction on metadata I/O errors
  dm-bufio: remove unused return value
  dm-integrity: set ti->error on memory allocation failure
  dm: Enable inline crypto passthrough for striped target
  dm vdo slab-depot: read refcount blocks in large chunks at load time
  dm vdo vio-pool: allow variable-sized metadata vios
  dm vdo vio-pool: support pools with multiple data blocks per vio
  dm vdo vio-pool: add a pool pointer to pooled_vio
  ...
2025-04-02 21:27:59 -07:00
Ken Raeburn
e678900df2 dm vdo indexer: reorder uds_request to reduce padding
Reorder fields and make uds_request_type and uds_zone_message packed,
to squeeze out some space. Use struct_group so the request reset code
no longer needs to care about field order.

On x86_64 this reduces the struct size from 144 to 120, which saves 48
kB (about 12%) per VDO hash zone.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-25 14:15:41 +01:00
Ken Raeburn
dc8f646cd8 dm vdo: rework processing of loaded refcount byte arrays
Clear provisional refcount values and count free/allocated blocks in
one integrated loop. Process 8 aligned bytes at a time instead of
every byte individually.

On an Intel i7-11850H this reduces the CPU time needed to process a
loaded refcount block by a factor of about 5-6. On a large system the
refcount loading may be the largest factor in device startup time.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-24 12:09:44 +01:00
Sweet Tea Dorminy
ff3f7115f4 dm vdo: remove remaining ring references
Lists are the new rings, so update all remaining references to rings to
talk about lists.

Signed-off-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-24 12:09:44 +01:00
Ken Raeburn
36e1b81f59 dm vdo: add missing spin_lock_init
Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
2025-02-24 12:07:02 +01:00
Ken Raeburn
0ce46f4f75 dm vdo slab-depot: read refcount blocks in large chunks at load time
At startup, vdo loads all the reference count data before the device
reports that it is ready. Using a pool of large metadata vios can
improve the startup speed of vdo. The pool of large vios is released
after the device is ready.

During normal operation, reference counts are updated 4kB at a time,
as before.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-03 14:15:30 +01:00
Ken Raeburn
f979da5125 dm vdo vio-pool: allow variable-sized metadata vios
With larger-sized metadata vio pools, vdo will sometimes need to
issue I/O with a smaller size than the allocated size. Since
vio_reset_bio is where the bvec array and I/O size are initialized,
this reset interface must now specify what I/O size to use.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-03 14:15:19 +01:00
Ken Raeburn
979a0fd396 dm vdo vio-pool: support pools with multiple data blocks per vio
Support pools with multiple data blocks per vio

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-03 14:15:08 +01:00
Ken Raeburn
2b515cea77 dm vdo vio-pool: add a pool pointer to pooled_vio
This allows us to simplify the return_vio_to_pool interface.

Also, we don't need to use vdo_forget on local variables or arguments
that are about to go out of scope anyway.

Signed-off-by: Ken Raeburn <raeburn@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-03 14:14:45 +01:00
Matthew Sakai
148a9cec84 dm vdo: remove checks that can not fail
Remove checks that can't fail.

Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-03 14:13:53 +01:00
Chung Chung
f4e99b846c dm vdo indexer: prevent unterminated string warning
Fix array initialization that triggers a warning:

error: initializer-string for array of ‘unsigned char’ is too long
 [-Werror=unterminated-string-initialization]

Signed-off-by: Chung Chung <cchung@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-03 14:11:22 +01:00
Matthew Sakai
3280c9313c dm vdo: use a short static string for thread name prefix
Also remove MODULE_NAME and a BUG_ON check, both unneeded.

This fixes a warning about string truncation in snprintf that
will never happen in practice:

drivers/md/dm-vdo/vdo.c: In function ‘vdo_make’:
drivers/md/dm-vdo/vdo.c:564:5: error: ‘%s’ directive output may be truncated writing up to 55 bytes into a region of size 16 [-Werror=format-truncation=]
    "%s%u", MODULE_NAME, instance);
     ^~
drivers/md/dm-vdo/vdo.c:563:2: note: ‘snprintf’ output between 2 and 66 bytes into a destination of size 16
  snprintf(vdo->thread_name_prefix, sizeof(vdo->thread_name_prefix),
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    "%s%u", MODULE_NAME, instance);
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Reported-by: John Garry <john.g.garry@oracle.com>
Reviewed-by: John Garry <john.g.garry@oracle.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2025-02-03 14:10:37 +01:00
Linus Torvalds
7eef7e306d - Dm: remove unused functions and variables
- Dm-ioctl: rate-limit error messages in syslog
 
 - Dm-persistent-data: fix typo
 
 - Dm-vdo murmurhash: remove u64 alignment requirement
 
 - Dm-vdo: reset bi_ioprio to the default
 
 - Dm: add support for get_unique_id
 
 - Dm thin: Add missing destroy_work_on_stack()
 
 - Dm-bufio: use kmalloc to allocate power-of-two sized buffers
 -----BEGIN PGP SIGNATURE-----
 
 iIoEABYIADIWIQRnH8MwLyZDhyYfesYTAyx9YGnhbQUCZ0SJnBQcbXBhdG9ja2FA
 cmVkaGF0LmNvbQAKCRATAyx9YGnhbV8xAP0dcS6O29X8HzSI8gVIgT8KYCpg3Ms2
 amPDlm/RDewdSQD/Z5+CHlyrlYqnKejAIs7cbZlfxD/0avcg/Kc0h4ijGws=
 =2dtc
 -----END PGP SIGNATURE-----

Merge tag 'for-6.13/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm

Pull device mapper updates from Mikulas Patocka:

 - remove unused functions and variables

 - rate-limit error messages in syslog

 - fix typo

 - remove u64 alignment requirement for murmurhash

 - reset bi_ioprio to the default for dm-vdo

 - add support for get_unique_id

 - Add missing destroy_work_on_stack() to dm-thin

 - use kmalloc to allocate power-of-two sized buffers in bufio

* tag 'for-6.13/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm:
  dm-verity: remove the unused "data_start" variable
  dm-bufio: use kmalloc to allocate power-of-two sized buffers
  dm thin: Add missing destroy_work_on_stack()
  dm: add support for get_unique_id
  dm vdo: fix function doc comment formatting
  dm vdo int-map: remove unused parameters
  dm-vdo: reset bi_ioprio to the default value when the bio is reset
  dm-vdo murmurhash: remove u64 alignment requirement
  dm: Fix typo in error message
  dm ioctl: rate limit a couple of ioctl based error messages
  dm vdo: Remove unused uds_compute_index_size
  dm vdo: Remove unused functions
  dm: zoned: Remove unused functions
  dm: Remove unused dm_table_bio_based
  dm: Remove unused dm_set_md_type
  dm cache: Remove unused functions in bio-prison-v1
  dm cache: Remove unused dm_cache_size
  dm cache: Remove unused dm_cache_dump
  dm cache: Remove unused btracker_nr_writebacks_queued
2024-11-25 18:54:00 -08:00
Matthew Sakai
19ac19e02f dm vdo: fix function doc comment formatting
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2024-11-20 11:38:04 +01:00
Matthew Sakai
7e976b2b9d dm vdo int-map: remove unused parameters
Remove __always_unused parameters from static functions.
Also fix minor formatting issues.

Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202407141607.M3E2XQ0Z-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202409101018.B75pIBKR-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202410011107.U2xbVLRA-lkp@intel.com/
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2024-11-20 11:38:04 +01:00
Susan LeGendre-McGhee
bd7e677c6b dm-vdo: reset bi_ioprio to the default value when the bio is reset
Signed-off-by: Susan LeGendre-McGhee <slegendr@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2024-11-20 11:38:04 +01:00
Susan LeGendre-McGhee
87d76d286c dm-vdo murmurhash: remove u64 alignment requirement
Signed-off-by: Susan LeGendre-McGhee <slegendr@redhat.com>
Signed-off-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2024-11-20 11:38:04 +01:00
Dr. David Alan Gilbert
b0e6210e7e dm vdo: Remove unused uds_compute_index_size
uds_compute_index_size() has been unused since it was added in
commit b46d79bdb8 ("dm vdo: add deduplication index storage interface")

Remove it.

Signed-off-by: Dr. David Alan Gilbert <linux@treblig.org>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2024-11-20 11:38:04 +01:00
Dr. David Alan Gilbert
295815f679 dm vdo: Remove unused functions
get_data_vio_pool_active_discards()
get_data_vio_pool_discard_limit()
get_data_vio_pool_maximum_discards()
set_data_vio_pool_discard_limit()

are all unused since commit
a9da0fb6d8 ("dm vdo: remove all sysfs interfaces")

Remove them.

Signed-off-by: Dr. David Alan Gilbert <linux@treblig.org>
Reviewed-by: Matthew Sakai <msakai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
2024-11-20 11:38:04 +01:00
Kuan-Wei Chiu
d684430207 dm vdo: update min_heap_callbacks to use default builtin swap
Replace the swp function pointer in the min_heap_callbacks of dm-vdo with
NULL, allowing direct usage of the default builtin swap implementation. 
This modification simplifies the code and improves performance by removing
unnecessary function indirection.

Link: https://lkml.kernel.org/r/20241020040200.939973-7-visitorckw@gmail.com
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Ching-Chun (Jim) Huang <jserv@ccns.ncku.edu.tw>
Cc: Coly Li <colyli@suse.de>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: "Liang, Kan" <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Matthew Sakai <msakai@redhat.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-11-05 17:12:36 -08:00
Kuan-Wei Chiu
92a8b224b8 lib/min_heap: introduce non-inline versions of min heap API functions
Patch series "Enhance min heap API with non-inline functions and
optimizations", v2.

Add non-inline versions of the min heap API functions in lib/min_heap.c
and updates all users outside of kernel/events/core.c to use these
non-inline versions.  To mitigate the performance impact of indirect
function calls caused by the non-inline versions of the swap and compare
functions, a builtin swap has been introduced that swaps elements based on
their size.  Additionally, it micro-optimizes the efficiency of the min
heap by pre-scaling the counter, following the same approach as in
lib/sort.c.  Documentation for the min heap API has also been added to the
core-api section.


This patch (of 10):

All current min heap API functions are marked with '__always_inline'. 
However, as the number of users increases, inlining these functions
everywhere leads to a increase in kernel size.

In performance-critical paths, such as when perf events are enabled and
min heap functions are called on every context switch, it is important to
retain the inline versions for optimal performance.  To balance this, the
original inline functions are kept, and additional non-inline versions of
the functions have been added in lib/min_heap.c.

Link: https://lkml.kernel.org/r/20241020040200.939973-1-visitorckw@gmail.com
Link: https://lore.kernel.org/20240522161048.8d8bbc7b153b4ecd92c50666@linux-foundation.org
Link: https://lkml.kernel.org/r/20241020040200.939973-2-visitorckw@gmail.com
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Ching-Chun (Jim) Huang <jserv@ccns.ncku.edu.tw>
Cc: Coly Li <colyli@suse.de>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Kuan-Wei Chiu <visitorckw@gmail.com>
Cc: "Liang, Kan" <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Matthew Sakai <msakai@redhat.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-11-05 17:12:34 -08:00