mirror of
https://github.com/torvalds/linux.git
synced 2026-09-23 21:48:54 +02:00
master
157 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Linus Torvalds
|
ed3b875bea |
mm.git review status for mm-hotfixes-stable..mm-stable
Everything:
Total patches: 501
Reviews/patch: 1.66
Reviewed rate: 70%
Excluding DAMON:
Total patches: 356
Reviews/patch: 2.26
Reviewed rate: 90%
Excluding DAMON and selftests:
Total patches: 329
Reviews/patch: 2.31
Reviewed rate: 92%
Excluding DAMON, selftests and maple_tree:
Total patches: 328
Reviews/patch: 2.31
Reviewed rate: 92%
Summary of patch series in this merge:
- The 2 patch series "mm: drop "sub" prefix from various places" from
Dev Jain implements some page->folio conversion and a naming cleanup.
- The 2 patch series "mm/kasan: remove redundant initialization for
kasan_flag_write_only" from Igor Putko provides some KASAN cleanup work.
- The 2 patch series "mm/filemap: reduce unnecessary xarray lookups"
from Chi Zhiling provides a small speedup in the pagecaache read code.
- The 4 patch series "mm/percpu: Fix possible NOFS/NOIO reclaim
recursion" from Kaitao Cheng improves a few things in the vmalloc code -
mainly the avoidance of GFP_KERNEL allocations when the caller asked for
GFP_NOFS or GFP_NOIO.
- The 3 patch series "mm/kmemleak: avoid soft lockup when scanning task
stacks" from Breno Leitao avoids a soft lockup watchdog trigger from the
kmemleak scanning code in extreme situations.
- The 6 patch series "mm/page_owner: misc cleanups" from Ye Liu is a
collection of unrelated cleanups to the page_owner code. For some
reason lots of people have been working on the page_owner code this
cycle.
- The 4 patch series "mm: convert to walk_page_range_vma() to eliminate
find_vma()" from Kefeng Wang simplifies and accelerates the page walking
library function.
- The 3 patch series "mm/migrate: preparatory cleanups for batch copy
and offload" from Shivank Garg implements cleanups in the migration
code.
- The 4 patch series "mm/page_owner: add per-fd filter infrastructure
for print_mode and NUMA filtering" from Zhen Ni provides per-fd
filtering to page_owner in order to reduce the sometimes vast amount of
output it can produce.
- The 19 patch series "mm: Refactor bootmem gigantic hugepage
allocation" from Muchun Song is a "set of fixes and preparatory cleanups
around bootmem HugeTLB handling, sparse initialization ordering, and
related vmemmap setup".
- The 4 patch series "mm/zsmalloc: reduce lock contention in zs_free()"
from Wenchao Hao reduces lock contention in zs_free(), which dominates
the unmap path under memory pressure on Android (LMK kills) and on x86
servers running zswap-heavy workloads. Up to 1.83x improvement in
microbenchmarking.
- The 2 patch series "move alloc_tag.c file under mm/" from Suren
Baghdasaryan does that.
- The 6 patch series "samples/damon: handle damon_{start,stop}()
failures" from SJ Park fixes improper handling of damon_start(),
damon_stop(), and damon_call() failures across DAMON sample modules to
prevent potential memory leaks, operation disruptions and use-after-free
bugs.
- The 11 patch series "mm/damon/sysfs: kobject_del() directories that
users can create/remove" from SJ Park resolves an issue where delayed
sysfs directory removal under CONFIG_DEBUG_KOBJECT_RELEASE causes
creation failures due to duplicate directory names by adding missing
kobject_del() calls before creating new directories.
- The 3 patch series "mm: cleanup clear_not_present_full_ptes()" from
David Hildenbrand cleans up the core pte handling code.
- The 3 patch series "selftests/damon: misc fixes for test bugs" from
Kunwu Chan fixes several bugs in the DAMON selftests.
- The 2 patch series "selftests/damon: fix memcg_path staging handling"
from Cheng Nie fixes a bug in _damon_sysfs.py for damos_filter
memcg_path setup, and adds a test case for it in sysfs.py.
- The 2 patch series "selftests/damon: test kdamond refresh_ms" from
Ruslan Valiyev introduces selftest coverage for DAMON's refresh_ms sysfs
feature by updating the test control module and verifying that scheme
stats update automatically without manual intervention.
- The 5 patch series "mm/damon: five misc fixups" from Akinobu Mita
contains miscellaneous DAMON fixups.
- The 2 patch series "mm/damon/core: detect internal variation above
max_nr_regions/2" from Jiayuan Chen fixes DAMON's region splitting
behavior when region counts exceed half the maximum budget by
dynamically scaling down the split fraction as the limit approaches,
preventing large regions from staying un-split, and adds corresponding
KUnit test coverage.
- The 6 patch series "mm: preparatory patches for PMD level swap
entries" from Usama Arif refactors and cleans up PMD softleaf helpers,
call sites, and architecture flags to lay the groundwork for a follow-up
series that introduces PMD page table swap entries.
- The 11 patch series "mm/damon: update, optimize, and clean up doc,
tests, and code" from SJ Park updates DAMON design and ABI
documentation, expands unit and selftest coverage, optimizes
damon_commit_target_regions(), and cleans up recently added sysfs
interface code for better readability.
- The 2 patch series "mm/vmpressure: reduce CPU, memory and code
overhead on cgroup v2" from Usama Arif optimizes vmpressure() by
skipping unnecessary work on cgroup v2 for userspace event notifications
and refactors v1-only eventfd handling into mm/memcontrol-v1.c to reduce
memory overhead and code complexity.
- The 10 patch series "selftests/mm: refactor pkey helpers and fix mmap
error handling" from Hongfu Li refactors pkeys shared tracing and
assertion helpers into a common file, unifies protection key selftests
to use consistent diagnostic logging and assertions, and enforces
standardized MAP_FAILED return checks for mmap() calls across the tests.
- The 18 patch series "mm/damon: optimize out nr_accesses_bp" from SJ
Park replaces the error-prone, continuously updated nr_accesses_bp field
in damon_region with an on-demand moving sum function
(damon_nr_accesses_mvsum()), reducing structure memory overhead and
avoiding state corruption bugs.
- The 6 patch series "Open HugeTLB allocation routine for more generic
use" from Ackerley Tng decouples HugeTLB folio allocation from VMA
dependencies by introducing hugetlb_alloc_folio(), enabling subsystems
like guest_memfd to allocate HugeTLB folios without standard VMA
reservations or pseudo-VMAs.
- The 3 patch series "mm/damon: provide pseudo moving sum probe_hits"
from SJ Park integrates DAMON's probe_hits attribute counter into the
pseudo moving sum infrastructure, enabling real-time, online monitoring
without waiting for full aggregation intervals.
- The 18 patch series "mm: Some cleanups for page allocator APIs" from
Brendan Jackman simplifies and refactors the page allocator entry points
and flags by unifying allocation paths, adding internal alloc_flags
arguments, and eliminating redundant __ prefixed alloc_pages variants.
- The 5 patch series "Fix incorrect access of hugetlb pte entries" from
Dev Jain enforces the consistent use of huge_ptep_get() instead of
ptep_get() for HugeTLB entries and fixes an unaligned address issue in
arm64's huge_ptep_get() implementation.
- The 8 patch series "mm/damon: validate all parameters in the core"
from SJ Park consolidates parameter validation into the DAMON core
specifically within damon_start() and damon_commit_ctx() to centralize
error checking, eliminate caller-side redundant checks and to improve
maintenance efficiency.
- The 3 patch series "tools/mm/page_owner_sort: fix filtering and
cleanup issues" from Yichong Chen renames is_need() to filter_record()
for clearer return semantics, fixes per-record allocation memory leaks
and bounds output copies in search_pattern() to address an existing
buffer issue.
- The 4 patch series "memcg: bail out reclaim when memcg is dying" from
Jiayuan Chen mitigates a system-wide stall which occurs when a cgroup is
removed while one of its memory control files is doing synchronous
reclaim.
- The 5 patch series "mm/memory-failure: add panic option for
unrecoverable pages" from Breno Leitao introduces an opt-in
vm.panic_on_unrecoverable_memory_failure sysctl that immediately panics
the kernel on unrecoverable memory errors in kernel-owned pages to
preserve error context and prevent delayed, silent data corruption.
- The 11 patch series "mm/damon: refactor damon_{start,stop,commit}()
for simple error handling" from SJ Park refactors the DAMON core API
functions to guarantee that all contexts are fully stopped when
damon_start(), damon_stop(), or damon_commit() fail, eliminating the
need for complex and error-prone caller-side cleanup code.
- The 5 patch series "Keep tail page private zero at free and folio
split" from Zi Yan adds checks to ensure tail_page->private is zero when
freeing compound or high-order pages and when promoting tail pages
during large folio splits. By validating these fields at free and split
time, it allows the removal of redundant private field clearing inside
prep_compound_tail().
- The 4 patch series "mm: drop redundant lru_add_drain in anon folio
reuse paths" from Barry Song eliminates redundant lru_add_drain() calls
in wp_can_reuse_anon_folio() and do_swap_page() to reduce LRU lock
contention and system overhead.
By validating folio refcounts against the LRU cache before draining
and removing unnecessary drains in the swap path, it achieves up to a
30.5% reduction in drain calls during heavy swap workloads.
- The 3 patch series "mm: clean up folio LRU and swap declarations" from
Jianyue Wu reorganizes folio LRU and swap code by relocating
page-cluster state to mm/swap_state.c, renaming mm/swap.c to mm/folio.c,
and moving MM-internal reclaim declarations into mm/internal.h.
- The 15 patch series "userfaultfd: working set tracking for VM guest
memory" from Kiryl Shutsemau adds userfaultfd support for tracking the
working set of VM guest memory, so a VMM can identify hot pages and
reclaim cold ones to tiered or remote storage.
- The 10 patch series "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part
2)" from David Hildenbrand removes the remaining pieces of
CONFIG_HAVE_BOOTMEM_INFO_NODE, performing some smaller cleanups around
freeing of reserved vmemmap pages on the way.
- The 7 patch series "mm/damon: update probe hits for runtime parameter
commits" from SJ Park ensures that DAMON's probe_hits attribute counter
is properly updated when monitoring intervals are changed at runtime,
matching the behavior of nr_accesses. To achieve this, it refactors and
renames existing helper functions for shared use, applies the updates to
probe_hits, and handles edge cases in damon_probe_hits_mvsum() to
maintain measurement accuracy.
- The 3 patch series "KSM: performance optimizations for rmap_walk_ksm"
from xu xin resolves a severe KSM reverse-mapping performance bottleneck
where thousands of split VMAs sharing a single anon_vma cause extended
lock contention. By adding an interval-filtering check during the rmap
walk, it reduces worst-case anon_vma lock hold times from over 500ms
down to under 2ms, preventing application freezes and latency spikes
under memory pressure.
- The 3 patch series "mm: split a couple of headers from internal.h"
from Mike Rapoport splits declarations related to mm_init, memblock,
vmalloc and sparse into new headers.
- The 2 patch series "KSM: use linear_page_index in collect_procs_ksm()"
from xu xin applies the interval tree optimization from rmap_walk_ksm()
to collect_procs_ksm() to avoid iterating over non-matching VMAs during
KSM memory error handling. It hoists loop-invariant address
initialization and restricts the anon_vma_interval_tree_foreach walk to
a targeted page offset range, reducing redundant checks and improving
lookup efficiency.
- The 3 patch series "selftests/mm: avoid false failures in hugetlb and
KSM tests" from Sayali Patil fixes issues in the hugetlb and KSM MM
selftest categories that can report failures when the prerequisites for
the tests are not satisfied.
- The 19 patch series "mm/damon: introduce data attributes only
monitoring" from SJ Park introduces attribute-weighted region management
in DAMON, allowing users to prioritize specific data attributes (such as
page sizes or cgroups) over or instead of access monitoring.
By assigning weights to attribute probes, DAMON can completely disable
access tracking and adjust monitoring regions based on weighted
probe-hit counters to optimize monitoring quality for attribute-focused
workloads.
- The 8 patch series "mm/hmm: Add mmap lock-drop support for
userfaultfd-backed mappings" from Stanislav Kinsburskii extends
hmm_range_fault() to support userfaultfd-backed regions by allowing the
mmap lock to be dropped during fault handling via a new
hmm_range_fault_locked() helper.
By accepting a locked pointer and signaling retry status when lock
release occurs, it enables page fault resolution in userfaultfd regions
while preserving backward compatibility for existing callers.
- The 33 patch series "mm: make VMA page offset handling more
consistent" from Lorenzo Stoakes cleans up and standardizes how
vma->vm_pgoff is accessed and manipulated across file-backed and
anonymous mappings in the kernel.
It introduces dedicated helper functions such as vma_start_pgoff(),
vma_end_pgoff(), vma_set_pgoff() and linear_page_delta() while renaming
rmap interval tree helpers to better reflect their functionality.
These changes establish a cleaner foundation for future work that will
unify virtual page offset indexing for all anonymous and CoW'd folios.
- The 3 patch series "mm: handle device-private PMDs in walk callbacks"
from Usama Arif addresses kernel panics and state corruption caused by
MM walk callbacks reaching non-present device-private PMD swap entries
created during HMM migrations.
It ensures that functions which acquire pmd_trans_huge_lock() properly
recognize device-private PMDs instead of assuming a present THP or a
standard migration entry.
- The 5 patch series "mm/rmap: Refactor try_to_unmap_one" from Dev Jain
refactors try_to_unmap_one by modularizing Hugetlb, anonymous-lazyfree,
and anonymous-swapbacked logic into dedicated functions, laying the
structural groundwork for batched anonymous large folio unmapping.
- The 4 patch series "Docs/ABI/damon: sysfs ABI document fixes and
additions" from Song Hu fixes typos and fills in missing entries in the
DAMON sysfs ABI document.
- The 10 patch series "dax/kmem: atomic whole-device hotplug via sysfs"
from Gregory Price introduces an atomic sysfs state attribute and
supporting DAX/MM infrastructure to prevent userland races when
offlining and removing entire memory regions.
By adding an unplugged state alongside standard online modes, it
enables whole-device atomic hotplug control while preserving backward
compatibility.
- The 13 patch series "mm: convert more vm_flags_t users to vma_flags_t"
from Lorenzo Stoakes continues transitioning the kernel from the
deprecated vm_flags_t type to vma_flags_t across core memory management
infrastructure.
It replaces legacy type usage in core functions such as do_mmap(),
unmapped area allocation, mm->def_vma_flags, and VMA operations like
mlock, mprotect, and mremap.
- The 2 patch series "Two small patches to clean up mm/mm_slot.h" from
xu xin refactors mm_slot.h by introducing mm_slot_remove() to unify
duplicate slot deletion sequences in khugepaged and KSM. It also adds
code documentation explaining why mm_slot_lookup and mm_slot_insert must
remain as preprocessor macros rather than static inline functions.
- The 10 patch series "mm/damon/core: hide core-private struct fields"
from SJ Park cleans up DAMON core structures by consistently marking
internal-only fields with private: comment tags to prevent improper
direct access from outer layers.
It enforces encapsulation across core structures including
damon_region, damon_target, and damon_ctx and updates DAMON_SYSFS to
interact through approved access APIs instead of exposing raw struct
members.
- The 6 patch series "mm/damon: unurgent fixes for infinite loop, NULL
de-ref and races" from SJ Park addresses potential infinite loops, NULL
dereferences, and race conditions identified in DAMON.
It fixes an infinite loop triggered by extreme user configurations, a
NULL pointer dereference within unit tests and minor monitoring
accuracy degradation caused by subtle runtime races.
- The 2 patch series "mm/page_alloc: fixes for free_pages_nolock() on
RT/UP" from Brendan Jackman fixes an NMI safety flaw in
__free_frozen_pages() where freeing pages on non-SMP or PREEMPT_RT
kernels can bypass can_spin_trylock() checks via non-PCP or isolated
migration paths.
It also resolves potential kernel crashes and privilege escalation
risks triggered when BPF tracing runs in NMI context alongside memory
hotplug or large allocation frees.
- The 4 patch series "mm/page_alloc: couple of followups for recent
cleanups" from Brendan Jackman cleans up and updates page allocator
nomenclature, documentation, and debug assertions.
It aligns internal FPI_ flags with the public "nolock" naming
convention, removes outdated internal implementation details from
high-level page allocator comments, and eliminates obsolete VM_BUG_ON()
assertions in allocation paths.
- The 3 patch series "mm/mseal: further cleanups" from Lorenzo Stoakes
refactors and simplifies the mseal implementation by clarifying API
boundaries and removing unnecessary code complexity.
It replaces generic do_mseal() usage outside the syscall with a
dedicated mseal_mmap_page_zero() helper for MMAP_PAGE_ZERO, eliminates
mm_struct parameters to enforce that sealing applies only to
current->mm, and streamlines overall logic and comments with no
functional changes intended.
- The 4 patch series "mm/vmscan: fix swappiness=max and clean up
per-node proactive reclaim" from Ridong Chen resolves reclaim behavior
bugs and cleans up function parameters across memory reclaim paths.
It fixes swappiness=max in both standard reclaim and MGLRU so
unswappable anonymous memory no longer falls back to evicting page
cache, ensures reclaim_store() returns accurate error codes instead of
collapsing all failures into -EAGAIN, and removes the obsolete gfp_mask
parameter from __node_reclaim().
- The 6 patch series "mm: mincore: misc cleanups" from Kefeng Wang
cleans up and simplifies the mincore code. Most importantly, it removes
the historical special behavior that always reports VM_PFNMAP pages as
non-resident.
- The 2 patch series "mm/huge_memory: drop dead split helper variants"
from Kiryl Shutsemau implements two trivial cleanups in the folio split
API.
- The 7 patch series "mm/damon: fix uninitialized DAMOS field and kunit
exec expectation bugs" from SJ Park resolves minor operational and
testing bugs in DAMON identified by Sashiko. It initializes the
damos->last_applied field to prevent occasional efficiency degradation
and fixes invalid memory accesses in DAMON KUnit tests during test
failure handling.
- The 3 patch series "cleanup for stable_page_flags()" from Jinjiang Tu
cleans up and refactors stable_page_flags() used by /proc/kpageflags
without altering functionality.
It uses BIT_ULL() to prevent shift-overflow warnings on 64-bit flag
bits, converts folio-specific flag checks to standard folio_test_*()
helpers, and removes redundant CONFIG_PAGE_IDLE_FLAG handling.
- The 3 patch series "Batch unmap of uffd-wp file folios" from Dev Jain
extends batched folio unmapping support to file folios within
userfaultfd write-protect (uffd-wp) VMAs by adding batching capabilities
to pte_install_uffd_wp_if_needed().
This removes special-case restrictions on uffd-wp VMAs in
try_to_unmap_one(), significantly simplifying the function's control
flow and complexity.
- The 3 patch series "mm/early_ioremap: clarify and clean up
early_ioremap_reset()" from Sang-Heon Jeon clarifies and cleans up the
architecture-specific usage of __late_set_fixmap() and
__late_clear_fixmap() after early_ioremap_reset().
It adds explicit documentation regarding when early_ioremap_reset()
must be called and removes redundant macro definitions and reset calls
in the RISC-V and ARM64 architectures.
- The 4 patch series "mm: fix reclaim storms in defrag_mode" from
Johannes Weiner addresses severe performance regressions, swap storms,
and spurious OOMs caused by vm.defrag_mode=1 under high memory pressure
in Meta production.
It updates the page allocator slowpath so non-movable allocation
requests actively trigger direct reclaim and direct compaction at
pageblock_order scale, allowing them to claim whole pageblocks rather
than spinning unproductively.
- The 2 patch series "zram: lockmap tweaks" from Sebastian Siewior
optimizes and fixes lockdep tracking for zram devices by consolidating
per-entry lockmaps and isolating lock classes across multiple instances.
It reduces memory overhead by replacing per-entry lockdep_map instances
with a single map per struct zram, and assigns a dynamic lock_class_key
to each instance to prevent false deadlock reports when different zram
devices are backed by distinct filesystems.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCaoUJbQAKCRDdBJ7gKXxA
jqrzAP9WoPU0hiK4qS/kSjhtoZxhjpS5eLSUCy/utKuEvZbfGgEAu1zA+LH+X9Tm
THK5ex4iUZxiFbXpWfLMxE/Q9PmQYQ8=
=QTyb
-----END PGP SIGNATURE-----
Merge tag 'mm-stable-2026-08-18-18-39' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM updates from Andrew Morton:
- "mm: drop "sub" prefix from various places" (Dev Jain)
page->folio conversion and a naming cleanup
- "mm/kasan: remove redundant initialization for kasan_flag_write_only"
(Igor Putko)
KASAN cleanup work
- "mm/filemap: reduce unnecessary xarray lookups" (Chi Zhiling)
Small speedup in the pagecaache read code
- "mm/percpu: Fix possible NOFS/NOIO reclaim recursion" (Kaitao Cheng)
Improve the vmalloc code - mainly the avoidance of GFP_KERNEL
allocations when the caller asked for GFP_NOFS or GFP_NOIO
- "mm/kmemleak: avoid soft lockup when scanning task stacks" (Breno
Leitao)
Avoid a soft lockup watchdog trigger from the kmemleak scanning code
in extreme situations
- "mm/page_owner: misc cleanups" (Ye Liu)
Cleanups to the page_owner code. For some reason lots of people have
been working on the page_owner code this cycle.
- "mm: convert to walk_page_range_vma() to eliminate find_vma()"
(Kefeng Wang)
Simplify and accelerate the page walking library function
- "mm/migrate: preparatory cleanups for batch copy and offload"
(Shivank Garg)
Cleanups in the migration code
- "mm/page_owner: add per-fd filter infrastructure for print_mode and
NUMA filtering" (Zhen Ni)
Per-fd filtering to page_owner in order to reduce the sometimes vast
amount of output it can produce
- "mm: Refactor bootmem gigantic hugepage allocation" (Muchun Song)
Fixes and preparatory cleanups around bootmem HugeTLB handling,
sparse initialization ordering, and related vmemmap setup
- "mm/zsmalloc: reduce lock contention in zs_free()" (Wenchao Hao)
Reduce lock contention in zs_free(), which dominates the unmap path
under memory pressure on Android (LMK kills) and on x86 servers
running zswap-heavy workloads.
Up to 1.83x improvement in microbenchmarking.
- "move alloc_tag.c file under mm/" (Suren Baghdasaryan)
- "samples/damon: handle damon_{start,stop}() failures" (SJ Park)
Fix improper handling of damon_start(), damon_stop(), and
damon_call() failures across DAMON sample modules to prevent
potential memory leaks, operation disruptions and use-after-free
bugs
- "mm/damon/sysfs: kobject_del() directories that users can
create/remove" (SJ Park)
Fix delayed sysfs directory removal under DEBUG_KOBJECT_RELEASE
causeing creation failures due to duplicate directory names by adding
missing kobject_del() calls before creating new directories
- "mm: cleanup clear_not_present_full_ptes()" (David Hildenbrand)
Clean up the core pte handling code
- "selftests/damon: misc fixes for test bugs" (Kunwu Chan)
Fix several bugs in the DAMON selftests
- "selftests/damon: fix memcg_path staging handling" (Cheng Nie)
Fix a bug in _damon_sysfs.py for damos_filter memcg_path setup, and
add a test case for it in sysfs.py.
- "selftests/damon: test kdamond refresh_ms" (Ruslan Valiyev)
Selftest coverage for DAMON's refresh_ms sysfs feature by updating
the test control module and verifying that scheme stats update
automatically without manual intervention
- "mm/damon: five misc fixups" (Akinobu Mita)
Miscellaneous DAMON fixups.
- "mm/damon/core: detect internal variation above max_nr_regions/2"
(Jiayuan Chen)
Fix DAMON's region splitting behavior when region counts exceed half
the maximum budget by dynamically scaling down the split fraction as
the limit approaches, preventing large regions from staying un-split,
and add corresponding KUnit test coverage
- "mm: preparatory patches for PMD level swap entries" (Usama Arif)
Refactor and clean up PMD softleaf helpers, call sites, and
architecture flags to lay the groundwork for a follow-up series that
introduces PMD page table swap entries
- "mm/damon: update, optimize, and clean up doc, tests, and code" (SJ
Park)
Update DAMON design and ABI documentation, expands unit and selftest
coverage, optimize damon_commit_target_regions(), and clean up
recently added sysfs interface code for better readability
- "mm/vmpressure: reduce CPU, memory and code overhead on cgroup v2"
(Usama Arif)
Optimize vmpressure() by skipping unnecessary work on cgroup v2 for
userspace event notifications and refactor v1-only eventfd handling
into mm/memcontrol-v1.c to reduce memory overhead and code complexity
- "selftests/mm: refactor pkey helpers and fix mmap error handling"
(Hongfu Li)
Refactor pkeys shared tracing and assertion helpers into a common
file, unify protection key selftests to use consistent diagnostic
logging and assertions, and enforce standardized MAP_FAILED return
checks for mmap() calls across the tests
- "mm/damon: optimize out nr_accesses_bp" (SJ Park)
Replace the error-prone, continuously updated nr_accesses_bp field in
damon_region with an on-demand moving sum function, reducing
structure memory overhead and avoiding state corruption bugs
- "Open HugeTLB allocation routine for more generic use" (Ackerley Tng)
Decouple HugeTLB folio allocation from VMA dependencies by
introducing hugetlb_alloc_folio(), enabling subsystems like
guest_memfd to allocate HugeTLB folios without standard VMA
reservations or pseudo-VMAs
- "mm/damon: provide pseudo moving sum probe_hits" (SJ Park)
Integrate DAMON's probe_hits attribute counter into the pseudo moving
sum infrastructure, enabling real-time, online monitoring without
waiting for full aggregation intervals
- "mm: Some cleanups for page allocator APIs" (Brendan Jackman)
Simplify and refactor the page allocator entry points and flags by
unifying allocation paths, adding internal alloc_flags arguments, and
eliminating redundant __ prefixed alloc_pages variants.
- "Fix incorrect access of hugetlb pte entries" (Dev Jain)
Enforce the consistent use of huge_ptep_get() instead of ptep_get()
for HugeTLB entries and fixes an unaligned address issue in arm64's
huge_ptep_get() implementation
- "mm/damon: validate all parameters in the core" (SJ Park)
Consolidate parameter validation into the DAMON core specifically
within damon_start() and damon_commit_ctx() to centralize error
checking, eliminate caller-side redundant checks and to improve
maintenance efficiency
- "tools/mm/page_owner_sort: fix filtering and cleanup issues" (Yichong
Chen)
Rename is_need() to filter_record() for clearer return semantics, fix
per-record allocation memory leaks and bound output copies in
search_pattern() to address an existing buffer issue
- "memcg: bail out reclaim when memcg is dying" (Jiayuan Chen)
Mitigate a system-wide stall which occurs when a cgroup is removed
while one of its memory control files is doing synchronous reclaim
- "mm/memory-failure: add panic option for unrecoverable pages" (Breno
Leitao)
Introduce an opt-in vm.panic_on_unrecoverable_memory_failure sysctl
that immediately panics the kernel on unrecoverable memory errors in
kernel-owned pages to preserve error context and prevent delayed,
silent data corruption
- "mm/damon: refactor damon_{start,stop,commit}() for simple error
handling" (SJ Park)
Refactor the DAMON core API functions to guarantee that all contexts
are fully stopped when damon_start(), damon_stop(), or damon_commit()
fail, eliminating the need for complex and error-prone caller-side
cleanup code
- "Keep tail page private zero at free and folio split" (Zi Yan)
Add checks to ensure tail_page->private is zero when freeing compound
or high-order pages and when promoting tail pages during large folio
splits. By validating these fields at free and split time, it allows
the removal of redundant private field clearing inside
prep_compound_tail()
- "mm: drop redundant lru_add_drain in anon folio reuse paths" (Barry
Song)
Eliminate redundant lru_add_drain() calls in
wp_can_reuse_anon_folio() and do_swap_page() to reduce LRU lock
contention and system overhead
By validating folio refcounts against the LRU cache before draining
and removing unnecessary drains in the swap path, it achieves up to a
30.5% reduction in drain calls during heavy swap workloads
- "mm: clean up folio LRU and swap declarations" (Jianyue Wu)
Reorganize folio LRU and swap code by relocating page-cluster state
to mm/swap_state.c, renaming mm/swap.c to mm/folio.c, and moving
MM-internal reclaim declarations into mm/internal.h.
- "userfaultfd: working set tracking for VM guest memory" (Kiryl
Shutsemau)
Add userfaultfd support for tracking the working set of VM guest
memory, so a VMM can identify hot pages and reclaim cold ones to
tiered or remote storage
- "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 2)" (David
Hildenbrand)
Remove the remaining pieces of CONFIG_HAVE_BOOTMEM_INFO_NODE,
performing some smaller cleanups around freeing of reserved vmemmap
pages on the way.
- "mm/damon: update probe hits for runtime parameter commits" (SJ Park)
Ensure that DAMON's probe_hits attribute counter is properly updated
when monitoring intervals are changed at runtime, matching the
behavior of nr_accesses. To achieve this, it refactors and renames
existing helper functions for shared use, applies the updates to
probe_hits, and handles edge cases in damon_probe_hits_mvsum() to
maintain measurement accuracy.
- "KSM: performance optimizations for rmap_walk_ksm" (xu xin)
Resolve a severe KSM reverse-mapping performance bottleneck where
thousands of split VMAs sharing a single anon_vma cause extended lock
contention.
By adding an interval-filtering check during the rmap walk, it
reduces worst-case anon_vma lock hold times from over 500ms down to
under 2ms, preventing application freezes and latency spikes under
memory pressure.
- "mm: split a couple of headers from internal.h" (Mike Rapoport)
Split declarations related to mm_init, memblock, vmalloc and sparse
into new headers
- "KSM: use linear_page_index in collect_procs_ksm()" (xu xin)
Apply the interval tree optimization from rmap_walk_ksm() to
collect_procs_ksm() to avoid iterating over non-matching VMAs during
KSM memory error handling.
It hoists loop-invariant address initialization and restricts the
anon_vma_interval_tree_foreach walk to a targeted page offset range,
reducing redundant checks and improving lookup efficiency.
- "selftests/mm: avoid false failures in hugetlb and KSM tests" (Sayali
Patil)
Fix issues in the hugetlb and KSM MM selftest categories that can
report failures when the prerequisites for the tests are not
satisfied
- "mm/damon: introduce data attributes only monitoring" (SJ Park)
Introduce attribute-weighted region management in DAMON, allowing
users to prioritize specific data attributes (such as page sizes or
cgroups) over or instead of access monitoring.
By assigning weights to attribute probes, DAMON can completely
disable access tracking and adjust monitoring regions based on
weighted probe-hit counters to optimize monitoring quality for
attribute-focused workloads.
- "mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings"
(Stanislav Kinsburskii)
Extend hmm_range_fault() to support userfaultfd-backed regions by
allowing the mmap lock to be dropped during fault handling via a new
hmm_range_fault_locked() helper.
By accepting a locked pointer and signaling retry status when lock
release occurs, it enables page fault resolution in userfaultfd
regions while preserving backward compatibility for existing callers.
- "mm: make VMA page offset handling more consistent" (Lorenzo Stoakes)
Clean up and standardize how vma->vm_pgoff is accessed and
manipulated across file-backed and anonymous mappings in the kernel
It introduces dedicated helper functions such as vma_start_pgoff(),
vma_end_pgoff(), vma_set_pgoff() and linear_page_delta() while
renaming rmap interval tree helpers to better reflect their
functionality.
These changes establish a cleaner foundation for future work that
will unify virtual page offset indexing for all anonymous and CoW'd
folios.
- "mm: handle device-private PMDs in walk callbacks" (Usama Arif)
Address kernel panics and state corruption caused by MM walk
callbacks reaching non-present device-private PMD swap entries
created during HMM migrations
It ensures that functions which acquire pmd_trans_huge_lock()
properly recognize device-private PMDs instead of assuming a present
THP or a standard migration entry.
- "mm/rmap: Refactor try_to_unmap_one" (Dev Jain)
Refactor try_to_unmap_one by modularizing Hugetlb,
anonymous-lazyfree, and anonymous-swapbacked logic into dedicated
functions, laying the structural groundwork for batched anonymous
large folio unmapping.
- "Docs/ABI/damon: sysfs ABI document fixes and additions" (Song Hu)
Fix typos and fills in missing entries in the DAMON sysfs ABI
document
- "dax/kmem: atomic whole-device hotplug via sysfs" (Gregory Price)
Introduce an atomic sysfs state attribute and supporting DAX/MM
infrastructure to prevent userland races when offlining and removing
entire memory regions
By adding an unplugged state alongside standard online modes, it
enables whole-device atomic hotplug control while preserving backward
compatibility.
- "mm: convert more vm_flags_t users to vma_flags_t" (Lorenzo Stoakes)
Continue transitioning the kernel from the deprecated vm_flags_t type
to vma_flags_t across core memory management infrastructure.
It replaces legacy type usage in core functions such as do_mmap(),
unmapped area allocation, mm->def_vma_flags, and VMA operations like
mlock, mprotect, and mremap.
- "Two small patches to clean up mm/mm_slot.h" (xu xin)
Refactor mm_slot.h by introducing mm_slot_remove() to unify duplicate
slot deletion sequences in khugepaged and KSM. It also adds code
documentation explaining why mm_slot_lookup and mm_slot_insert must
remain as preprocessor macros rather than static inline functions.
- "mm/damon/core: hide core-private struct fields" (SJ Park)
Clean up DAMON core structures by consistently marking internal-only
fields with private: comment tags to prevent improper direct access
from outer layers.
It enforces encapsulation across core structures including
damon_region, damon_target, and damon_ctx and updates DAMON_SYSFS to
interact through approved access APIs instead of exposing raw struct
members.
- "mm/damon: unurgent fixes for infinite loop, NULL de-ref and races"
(SJ Park)
Address potential infinite loops, NULL dereferences, and race
conditions identified in DAMON
It fixes an infinite loop triggered by extreme user configurations, a
NULL pointer dereference within unit tests and minor monitoring
accuracy degradation caused by subtle runtime races.
- "mm/page_alloc: fixes for free_pages_nolock() on RT/UP" (Brendan
Jackman)
Fix an NMI safety flaw in __free_frozen_pages() where freeing pages
on non-SMP or PREEMPT_RT kernels can bypass can_spin_trylock() checks
via non-PCP or isolated migration paths.
It also resolves potential kernel crashes and privilege escalation
risks triggered when BPF tracing runs in NMI context alongside memory
hotplug or large allocation frees.
- "mm/page_alloc: couple of followups for recent cleanups" (Brendan
Jackman)
Clean up and update page allocator nomenclature, documentation, and
debug assertions.
It aligns internal FPI_ flags with the public "nolock" naming
convention, removes outdated internal implementation details from
high-level page allocator comments, and eliminates obsolete
VM_BUG_ON() assertions in allocation paths.
- "mm/mseal: further cleanups" (Lorenzo Stoakes)
Refactor and simplify the mseal implementation by clarifying API
boundaries and removing unnecessary code complexity.
It replaces generic do_mseal() usage outside the syscall with a
dedicated mseal_mmap_page_zero() helper for MMAP_PAGE_ZERO,
eliminates mm_struct parameters to enforce that sealing applies only
to current->mm, and streamlines overall logic and comments with no
functional changes intended.
- "mm/vmscan: fix swappiness=max and clean up per-node proactive
reclaim" (Ridong Chen)
Resolve reclaim behavior bugs and clean up function parameters across
memory reclaim paths
It fixes swappiness=max in both standard reclaim and MGLRU so
unswappable anonymous memory no longer falls back to evicting page
cache, ensures reclaim_store() returns accurate error codes instead
of collapsing all failures into -EAGAIN, and removes the obsolete
gfp_mask parameter from __node_reclaim().
- "mm: mincore: misc cleanups" (Kefeng Wang)
Clean up and simplifies the mincore code. Most importantly, it
removes the historical special behavior that always reports VM_PFNMAP
pages as non-resident.
- "mm/huge_memory: drop dead split helper variants" (Kiryl Shutsemau)
Two trivial cleanups in the folio split API
- "mm/damon: fix uninitialized DAMOS field and kunit exec expectation
bugs" (SJ Park)
Resolve minor operational and testing bugs in DAMON identified by
Sashiko. It initializes the damos->last_applied field to prevent
occasional efficiency degradation and fixes invalid memory accesses
in DAMON KUnit tests during test failure handling.
- "cleanup for stable_page_flags()" (Jinjiang Tu)
Clean up and refactor stable_page_flags() used by /proc/kpageflags
without altering functionality.
It uses BIT_ULL() to prevent shift-overflow warnings on 64-bit flag
bits, converts folio-specific flag checks to standard folio_test_*()
helpers, and removes redundant CONFIG_PAGE_IDLE_FLAG handling.
- "Batch unmap of uffd-wp file folios" (Dev Jain)
Extend batched folio unmapping support to file folios within
userfaultfd write-protect (uffd-wp) VMAs by adding batching
capabilities to pte_install_uffd_wp_if_needed().
This removes special-case restrictions on uffd-wp VMAs in
try_to_unmap_one(), significantly simplifying the function's control
flow and complexity.
- "mm/early_ioremap: clarify and clean up early_ioremap_reset()"
(Sang-Heon Jeon)
Clarify and clean up the architecture-specific usage of
__late_set_fixmap() and __late_clear_fixmap() after
early_ioremap_reset()
It adds explicit documentation regarding when early_ioremap_reset()
must be called and removes redundant macro definitions and reset
calls in the RISC-V and ARM64 architectures.
- "mm: fix reclaim storms in defrag_mode" (Johannes Weiner)
Address severe performance regressions, swap storms, and spurious
OOMs caused by vm.defrag_mode=1 under high memory pressure in Meta
production
It updates the page allocator slowpath so non-movable allocation
requests actively trigger direct reclaim and direct compaction at
pageblock_order scale, allowing them to claim whole pageblocks rather
than spinning unproductively.
- "zram: lockmap tweaks" (Sebastian Siewior)
Optimize and fix lockdep tracking for zram devices by consolidating
per-entry lockmaps and isolate lock classes across multiple instances
This reduces memory overhead by replacing per-entry lockdep_map
instances with a single map per struct zram, and assigns a dynamic
lock_class_key to each instance to prevent false deadlock reports
when different zram devices are backed by distinct filesystems.
* tag 'mm-stable-2026-08-18-18-39' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (501 commits)
selftests/mm: thuge-gen: fix test_shmget() for PAGE_SIZE check
selftests/mm: unpoison pages in memory-failure teardown
mm/shmem: downgrade final i_blocks check in shmem_evict_inode() to pr_warn()
mm/khugepaged: replace mutex_lock/mutex_unlock usage with guard macro
mm/zsmalloc: fix release order of locks in zs_page_migrate()
Documentation: zram: remove sections numbering
ksm: stop iterating VMAs when ksm_test_exit returns true
mm: fold userfaultfd_rwp() to false without CONFIG_ARCH_HAS_PTE_PROTNONE
mm/migrate: report RCU-tasks quiescent states in migrate_pages_batch()
zram: use a custom key for each zram object
zram: move lockmap to be per-zram instead per table
selftests/mm: fix gup_longterm EINVAL error message
mm: page_alloc: fix non-movable reclaim storm in defrag_mode
mm: page_alloc: move capture_control to the page allocator
mm: compaction: support non-movable compaction for pageblock requests
mm: page_alloc: __GFP_FS lockdep annotation for direct compaction
hugetlb: evaluate subpool free state while locked
mm/damon: remove trailing semicolons after function definitions
mm/damon/ops-common: prevent migration fallback to non-target nodes
mm/damon: update outdated comment about DAMOS filter handling
...
|
||
|
Linus Torvalds
|
8dcef8882a |
x86/msr updates for v7.3:
- Streamline the x86 MSR handling APIs along the 64-bit
variants, simplifying the interfaces. Removal of the
old APIs is planned for the next cycle, to reduce
churn & integration pain. (Juergen Gross)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqC3qYRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1jSOA//cTRslNJ4t3g2cR8iPDFXsjMmwsvpDJn1
OVqOlyFcQpJCliC37JmRA94B6RufVdoFikOuyshn4SaPix8+lUKZDt+c5N6wfZsI
NJ4zuCHD4k0tDhHCYXihEISBAGUJrHk+6eiNTxDTlCUhCxqLvX1W1L0fiB4R2w0h
/Z1ZfL0Om6QV/V8u7lC8Vio0Uxp12Zjm60I43QBsUTOoD+4poaGGORV+03lnGnaZ
eaq9VRZKrrx6h3K5hS1Ja6j6mWehQzWabn3v5cM15IwYNuVA4g38OKR8ofqk0Shz
DvCILEtjoT7mXfvGoxYTSuEEa5aJoTSplMVbsrRuXHtcwDplnCGxCnYE0pYCfpFD
4xzvBBm15X20M0CFFW3lvJKqyN43M+Wfh/aOG0AT8BmIPKuyzZgc9yJ/QxmS4dYY
5atebSCK+caKr8IXshTTvssEWGF1A6un/0lOfuS2BmdIZuREYxzKWF3sqp3f10Gx
hSm1xB5HRhoQCIFP3PGER8bwwea0UmjI9VWHEkBKfNadqz3OPEZf4PQNiWJ3Q6x0
T7EGOWJlZTzCH2iqGrzPY0g56pPoDwGzjIruAmhc5NMFEOIHy0MKyPw4huWKSSzB
WVpqi8Gnn0HS3dECOnvkDpNM0JXDp9XYSaL5mLs/ZgWPIwJ6fFHbY34xm3A+Cjdp
hGYmT3Mtx1A=
=QBg3
-----END PGP SIGNATURE-----
Merge tag 'x86-msr-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 MSR updates from Ingo Molnar:
- Streamline the x86 MSR handling APIs along the 64-bit variants,
simplifying the interfaces.
Removal of the old APIs is planned for the next cycle, to reduce
churn & integration pain (Juergen Gross)
* tag 'x86-msr-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (21 commits)
x86/mce: Work around build warning after MSR-interface switch
cpufreq: Stop using 32-bit MSR interfaces
x86/featctl: Stop using 32-bit MSR interfaces
KVM/x86: Stop using 32-bit MSR interfaces
x86/mtrr: Stop using 32-bit MSR interfaces
acpi: Stop using 32-bit MSR interfaces
powercap: Stop using 32-bit MSR interfaces
thermal/intel: Stop using 32-bit MSR interfaces
x86/olpc: Stop using 32-bit MSR interfaces
x86/hyperv: Stop using 32-bit MSR interfaces
hwmon: Stop using 32-bit MSR interfaces
EDAC: Stop using 32-bit MSR interfaces
x86/cpu: Stop using 32-bit MSR interfaces
x86/apic: Stop using 32-bit MSR interfaces
x86/resctrl: Stop using 32-bit MSR interfaces
x86/tsc: Stop using 32-bit MSR interfaces
x86/amd: Stop using 32-bit MSR interfaces
x86/pci: Stop using 32-bit MSR interfaces
x86/hygon: Stop using 32-bit MSR interfaces
x86/mce: Stop using 32-bit MSR interfaces
...
|
||
|
Brendan Jackman
|
9b5b281f21 |
x86/virt: use higher-level allocator API
The difference between __alloc_pages_node() and alloc_pages_node() is that the latter allows you to pass NUMA_NO_NODE. The former is going away and the latter works fine here so switch over. No functional change intended. Link: https://lore.kernel.org/20260703-alloc-trylock-v5-10-c87b714e19d3@google.com Signed-off-by: Brendan Jackman <jackmanb@google.com> Reviewed-by: Suren Baghdasaryan <surenb@google.com> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Cc: Thomas Gleixner <tglx@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Assisted-by: Gemini:unknown-version Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Juergen Gross
|
1620ddc0dc |
KVM/x86: Stop using 32-bit MSR interfaces
The 32-bit MSR interfaces rdmsr(), wrmsr() and rdmsr_safe() are planned to be removed. Use the related 64-bit variants instead. No change in functionality intended. Signed-off-by: Juergen Gross <jgross@suse.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Kiryl Shutsemau <kas@kernel.org> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Rick Edgecombe <rick.p.edgecombe@intel.com> Cc: Sean Christopherson <seanjc@google.com> Cc: kvm@vger.kernel.org Link: https://patch.msgid.link/20260629060526.3638272-10-jgross@suse.com |
||
|
Tycho Andersen (AMD)
|
4c2509f3b7 |
x86/virt/sev: Revert "Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN"
Revert |
||
|
Linus Torvalds
|
c98d767b34 |
Arm:
"This is a bit of an odd merge window on the KVM/arm64 front. There is
absolutely no new feature in the pull request. It is purely fixes,
because it is simply becoming too hard to review new stuff when so
many AI-fuelled fixes hit the list".
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX)
that caused the FPU load request.
- Some enhancements about interrupt injection.
- Some bug fixes and other small changes.
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic
PTE updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on
a memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes.
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace.
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root.
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both
of these are preparatory changes for upcoming APX support.
- Clean up KVM's register tracking and storage, primarily to prepare for
APX support, which expands the maximum number of GPRs from 16 to 32.
- Keep a single copy of the PDPTRs rather than two, since architecturally
there is just one.
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in shadow page table handling, where KVM fails to recursively
zap nested EPT/NPT shadow page tables when the nested hypervisor tears
down its own EPT/NPT page tables from the bottom up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in the
generic section; also run with MBEC enabled even for non-nested mode.
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Fix a largely benign bug where KVM TDX would incorrectly state it could
emulate several x2APIC MSRs.
- Use the "safe" WRMSR API when proxying LBR MSR writes as the to-be-written
value is guest controlled and completely unvalidated.
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in the
generic section; also run with GMET enabled even for non-nested mode.
- Fixes and minor cleanups to GHCB handling, on top of the earlier work
already merged into 7.1-rc.
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers.
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
- Don't advertise support for unusable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to follow
the kernel's nomenclature, e.g. for mmu_notifiers.
- Use guard() to cleanup up various KVM+VFIO flows.
- Minor cleanups.
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a gmem
range to multiple memslots, and fix the test that was supposed to ensure
KVM returns -EEXIST.
- Treat memslot binding offsets and sizes as unsigned values to fix a bug
where KVM interprets a large "offset + size" as a negative value and allows
a nonsensical offset.
- Use the inode number instead of the page offset for the NUMA interleaving
index to fix a bug where the effective index would jump by two for
consecutive pages (the caller also adds in the page offset).
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the first
pass, as always waiting only 1ms hid a KVM RISC-V bug as the test reaped the
bitmap before KVM could build up enough state to hit the bug.
- A pile of one-off fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmoypI0UHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroNCSAf8CyuvicojnZGFsNcoY4DZ0whBj5l8
IwVkHaptuyGLSwPAebC/DwtU2NBY4CQdbtLjGmxenf+fSQOtcz1YWKbdY4Nr0hJG
euAb/wO47RqKp1C4WPi7lL2P4XHgmUgD5XZ1qDnRV9gnGddZi3QLLmKgAtNq9Z4m
+pAdGD2/RyZeI0Dx61+5ufE/z8YBlH6GrzgBTxfvvRNKqNDFykBenxYHI7seEfHC
BPz0LxjHDD8BMDUo86qaylZnaeQSmFTGO/eTgvCvabxG6WbM1s0w3K1owmUmP7o3
p9o8274Xh53iK2JYD/qGOtAJXejIwbYKT5bnvUQX0O1EH2+ouvSFapLQ9Q==
=sSqM
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
|
||
|
Linus Torvalds
|
b082086174 |
* Add TDX module update support
* Make kexec and TDX finally place nice together * Put TDX error codes into a single header -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEV76QKkVc4xCGURexaDWVMHDJkrAFAmowVgUACgkQaDWVMHDJ krDwsw/9FKXndYdeSc7P8sJwjSlc9U4PFy/6rdUfnYCP1c2BTeVqNxZvCz+T9E4V Ld4UjjTLCbEq1uQ2UdYugpLnSKSNzf6MEp06Tsq2cRlEcPQBLGaACo+7dnBrUf6g zVUY0lPh1Jlxa+FBzMLf/H1qyUGl/Lv8msmgrC9fHHwQgsPYMgKUU6t72ScRG6FA kUk0crzI1DlSpFFVbXuRIpZSmLFPgyx5TNkteGJrP6q3iR6zYuc30MQsV2GalFrv qrn4cz3EmW0vousqpg7EA6J7okTUem/iswbxbQnpR4AMdJ7NwkKfa/OvHa6UB9UL Nb5gE7lxplKOlJKfeGjrL6ABazpwsk0J4oS9r2GoWDfs30ji3TX5ixVvJ1KCFAFZ dEkS1SazNjAWuDWuptVbu3+4DV34XSUg+kvcCXNTlOHz5KVg5ASzjF0QFvXexjCD BhcIqJ3OI8VtL4AI+HXg+GTpHOUYtGct/Y+5gB16VvoSuX9vNOPBYx2M9Q4Nhb/C U9eM7wqaS6lxX4UGId0ZgrNwHROreTEeGBScn1q1Rm7B6qJgeYo7wrzuKXCyRgcb /evbcZ1DWJTXEORsJobzNiXHQWBtAEs4rLycOKAGx8//kD7vEmy2itslPlEaSgFq jvoAHRNpiE3VW/hItB8WFTgWxiKaIz624gNrMjI2RYI/JSNhKVo= =Y3yy -----END PGP SIGNATURE----- Merge tag 'x86_tdx_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip Pull x86 TDX updates from Dave Hansen: "There are a few cleanups, and some changes that should allow TDX and kexec to coexist nicely. The biggest change, however, is support for updating the TDX module after boot, just like CPU microcode. TDX users really want this because it lets them do security updates without tearing things down and rebooting. - Add TDX module update support - Make kexec and TDX finally place nice together - Put TDX error codes into a single header" * tag 'x86_tdx_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (30 commits) x86/virt/tdx: Document TDX module update x86/virt/tdx: Enable TDX module runtime updates x86/virt/tdx: Refresh TDX module version after update coco/tdx-host: Lock out module updates when reading version x86/virt/seamldr: Add module update locking x86/virt/tdx: Restore TDX module state x86/virt/seamldr: Initialize the newly-installed TDX module x86/virt/seamldr: Install a new TDX module x86/virt/tdx: Reset software states during TDX module shutdown x86/virt/seamldr: Shut down the current TDX module x86/virt/seamldr: Abort updates after a failed step x86/virt/seamldr: Introduce skeleton for TDX module updates x86/virt/seamldr: Allocate and populate a module update request coco/tdx-host: Implement firmware upload sysfs ABI for TDX module updates coco/tdx-host: Don't expose P-SEAMLDR information on CPUs with erratum coco/tdx-host: Expose P-SEAMLDR information via sysfs x86/virt/seamldr: Add a helper to retrieve P-SEAMLDR information x86/virt/seamldr: Introduce a wrapper for P-SEAMLDR SEAMCALLs coco/tdx-host: Expose TDX module version coco/tdx-host: Introduce a "tdx_host" device ... |
||
|
Linus Torvalds
|
c61f479852 |
- Remove redundant GHCB initialization guards in the SEV page state and SVSM
call paths now that the GHCB helpers handle early-boot fallback internally
- Skip SNP initialization in the CCP driver immediately when the preparation
step fails rather than proceeding to an operation that will certainly fail
- Abort SNP preparation and return an error when not all CPUs are online,
since the firmware enforces that every CPU enables SNP and will fail init if
not
- Simplify the VMM communication exception entry path by replacing separate
kernel and user mode macros with a single handler that dispatches based on
the current privilege level
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmowORwACgkQEsHwGGHe
VUqphg//VVXH3Svs2K5+++WQvavXDVwB5B+W4KA5WYYz3I77nmXgb2++aGSptGfy
ckC8ESFwyRWRh6aaOMtWGdbTQryS9e72sqiZEphbAZCe2XwR/A0N60uUktUCGm1W
3ce/VRkeTJI4LTkx3g+VSPTGzDpqfbwwTAJQZt8Ny8kooQCUH+VA7JvyLOYFvyTg
4NtjP2MVTzjE1io9SpZdSAbG/wdn7l7cFI+xC1QXYKZBzHSK5e4vUwnSGd86lrCa
2Q+zVcGcA7w2J6Obx4dNde5Y7AhGmYnhDxbFtNp9iUmG/o75LpVvP+8v7wshSGiL
HjTsRy5tkjrF9MUNXZd6cwFQ9gs6wWAwxXUsPtfs2N8R6Ve0W9aR49MUp1sEFuKm
hNgRLA7idSQ0RUSrL4f3c7Zx8ML6nXM2yRgE+4UTbr5/untHCqYunHelEIy1uay5
2vbqJ5VzT+pfY9qZn+q41AM2E1+ZzLmXFuGPBSOU4G+dU0CoVo+OUrsKZh8Q/4Ff
DajeiTW1+UidZyk2nb+HoX3DIa7jsUBeA3IGAQv8wK31jj3XhKwJTFPwPGkdjcpw
iR35vPzZFh0JLm+AHGrjgLI2+DkWEKl3j6lxI94UxsugaqkywtPZ5XJhJC54YHS9
iOwlpZK37eYy6zITx1+bwfzrW4oSsEZyYHkNgYJhU/RW4MrNJiU=
=Wq1K
-----END PGP SIGNATURE-----
Merge tag 'x86_sev_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 SEV updates from Borislav Petkov:
- Remove redundant GHCB initialization guards in the SEV page state and
SVSM call paths now that the GHCB helpers handle early-boot fallback
internally
- Skip SNP initialization in the CCP driver immediately when the
preparation step fails rather than proceeding to an operation that
will certainly fail
- Abort SNP preparation and return an error when not all CPUs are
online, since the firmware enforces that every CPU enables SNP and
will fail init if not
- Simplify the VMM communication exception entry path by replacing
separate kernel and user mode macros with a single handler that
dispatches based on the current privilege level
* tag 'x86_sev_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/sev: Remove redundant ghcbs_initialized checks around __sev_{get,put}_ghcb()
crypto/ccp: Skip SNP_INIT if preparation fails
x86/sev: Do not initialize SNP if missing CPUs
x86/entry: Zap the #VC entry user and kernel macros
|
||
|
Paolo Bonzini
|
96124f964c |
KVM x86 MMU changes for 7.2
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in the shadow MMU where KVM fails to recursively zap nested TDP
shadow when L1 is tearing its TDP page tables from the bottom up, as KVM's
TDP MMU now does.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorQCsACgkQOlYIJqCj
N/3aAA//Y3Hsd5F4/p2HgI+qYxpskugi4BAInXjCzs92ShOaxxSTEPnzRNCAjDDb
OTpvfJm3ZI2if6A9yYngP7R8tzueMAhHMcCAnkL/+VBsN+ILm4yflwGgs2Tc7pUS
jpd5pioljG16b4B2LC4QM0RsAsfxqjcbLcY1XIFQVObuiSoquv6hjaebbY2IW4n3
IS6QSKRSLZb5qh9IBbWihn7Qyr4E3r5fmoaTptExSb4zKYVMPqpqMnngqYQWfZxg
6lfoh1eMrIU5LdFRgJeifwRzBvrtu2YI2pyih7bzXUe/XTE9AYpIvA1CB+OVy15g
AjDllcYX6B7B5i7uc3trk4eqD75xDnQYRQfnkWj9OuxVKGp7Fcydk1VQHtPLS3K7
HpWB4/KvXbAqGXZltE4qnqycqY64hp5k0B59I/Q+n9hC9hHXjW5etp9e+zZsNSEy
IswGEONdNHszOF0XZE0WE1zsi1D1HW3rg4OANClE+X+z55xqC1evEt7BbvbsDGmX
41sJCyRjc2yClVh5kZhruP9G5w5OT+M8En5fdou5QlgeAWcaGiUXj/e2mF5fV+8p
PtEtih3Q82MsB78Y0jiA8dkRDRkqDqh69I1BCUJJTIdhYO1fTq8t3iV9MGhJz6C1
5TXyA895FsGrp85+IN1Chi09tkCqktlM866mTg0Ph6kPHCWNQpE=
=PUlE
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-mmu-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM x86 MMU changes for 7.2
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in the shadow MMU where KVM fails to recursively zap nested TDP
shadow when L1 is tearing its TDP page tables from the bottom up, as KVM's
TDP MMU now does.
|
||
|
Chao Gao
|
856bc8bb99 |
x86/virt/tdx: Refresh TDX module version after update
The kernel exposes the TDX module version through sysfs so userspace can check update compatibility. That information needs to remain accurate across runtime updates. A runtime update may change the module's update_version, so refresh the cached version right after a successful update. Drop __ro_after_init from tdx_sysinfo because it is now updated at runtime. Do not refresh the rest of tdx_sysinfo, even if some values change across updates. TDX module updates are backward compatible, so existing tdx_sysinfo consumers, such as KVM, can continue to operate without seeing the new values. [ dhansen: trim changelog ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-22-chao.gao@intel.com |
||
|
Dave Hansen
|
6e97c234cd |
x86/virt/seamldr: Add module update locking
TDX metadata like the version number changes during a module update. Add functions to lock out module updates. The current stop_machine() implementation uses worker threads. The scheduler actually does a full, normal context switch over to that thread. preempt_disable() obviously inhibits that context switch and thus, locks out stop_machine() users like the module update. Thanks to Chao for the idea of using preempt_disable(). Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> |
||
|
Chao Gao
|
b344c50a1a |
x86/virt/tdx: Restore TDX module state
After per-CPU initialization, the module is nearly functional. It is in a similar state to TDX initialization before TDH.SYS.CONFIG. At this point, the kernel _could_ just repeat the boot-time sequence, but that would land the new module in a slightly different state than the old module. This would leave old TDs unrunnable, which is not a good outcome. Thankfully, the "handoff" data saved during module shutdown should contain all the information needed to restore the TDX module state to exactly what it was before the update. Restore TDX module state. The TDX module only needs a single copy so only do this on the lead CPU. Restoration errors can theoretically be handled in a few ways. For instance, userspace could try to load a different TDX module version. Or, the kernel could give up on the handoff process and just reinitialize the new module from scratch, which would lose all existing TDs. Simply propagate errors to userspace. Ignore the idea of a TD-destroying reinitialization. It would destroy data like a reboot and if things have gone that wrong a reboot is probably the best option anyway. Note: the location and the format of handoff data is defined by the TDX module. The new module knows where to get handoff data and how to parse it. The kernel does not touch it at all. Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-21-chao.gao@intel.com |
||
|
Chao Gao
|
bf7c0ed2c3 |
x86/virt/seamldr: Initialize the newly-installed TDX module
Continue fleshing out the update process. At this point the new module is sitting in memory but has never been called and is not usable. It is in a similar state to the when the system first boots. Leave the P-SEAMLDR behind. Stop making calls to it. Transition to calling the new TDX module itself to set up both global and per-cpu state. Share tdx_cpu_enable() with the fresh-boot module initialization code. Export it and invoke it on all CPUs. Note: "TDX global initialization" needs to be done once before "TDX per-CPU initialization". It would be a great fit for the new runtime update "is_lead_cpu" logic. But tdx_cpu_enable() already has some logic to do the global initialization properly. Just use it directly to maximize fresh-boot and runtime update code sharing. == Background == The boot-time and post-update initialization flows share the same first steps: - TDX global initialization - TDX per-CPU initialization After that, they diverge: - Fresh boot: Prepare TDMRs/PAMTs Configure the TDX module Configure the global KeyID Initialize TDMRs - Runtime update: Restore TDX module state from handoff data Future changes will consume the handoff data. [ dhansen: major changelog munging ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-20-chao.gao@intel.com |
||
|
Chao Gao
|
2bfb2ef877 |
x86/virt/seamldr: Install a new TDX module
Continue fleshing out the update proces. The old module is shut down and the system is ready for the new module image. Run the SEAMLDR.INSTALL SEAMCALL on all CPUs. Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-19-chao.gao@intel.com |
||
|
Chao Gao
|
65a6542a81 |
x86/virt/tdx: Reset software states during TDX module shutdown
The TDX module requires a one-time global initialization (TDH.SYS.INIT) and per-CPU initialization (TDH.SYS.LP.INIT) before use. These initializations are guarded by software flags to prevent repetition. Reset all software flags guarding the initialization flows to allow the global and per-CPU initializations to be triggered again after updates. [ dhansen: trim down changelog ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-18-chao.gao@intel.com |
||
|
Chao Gao
|
9ea06080a6 |
x86/virt/seamldr: Shut down the current TDX module
The first step of TDX module updates is shutting down the current TDX module. This step also packs state information that needs to be preserved across updates, called "handoff data". This handoff data is consumed by the updated module and stored internally in the SEAM range and hidden from the kernel. Since the handoff data layout may change between modules, the handoff data is versioned. Each module has a native handoff version and provides backward support for several older versions. The complete handoff versioning protocol is complex as it supports both module upgrades and downgrades. See details in "Intel Trust Domain Extensions (Intel TDX) Module Base Architecture Specification", Chapter "Handoff Versioning". Ideally, the kernel needs to retrieve the handoff versions supported by the current module and the new module and select a version supported by both. But since this implementation only supports module upgrades, simply request handoff data from the current module using its highest supported version. That is sufficient for this upgrade-only implementation. Retrieve the module's handoff version from TDX global metadata and add an update step to shut down the module. Module shutdown only needs to run on one CPU. Don't cache the handoff information in tdx_sysinfo. It is used only for module shutdown, and is present only when the TDX module supports updates. Caching it in get_tdx_sys_info() would require extra update-support guards and refreshing the cached value across module updates. [ dhansen: fix up function variables, remove 'cpu'. Return from tdx_module_shutdown() early if handoff call fails. ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Link: https://patch.msgid.link/20260520133909.409394-17-chao.gao@intel.com |
||
|
Chao Gao
|
be4efe63c0 |
x86/virt/seamldr: Abort updates after a failed step
A TDX module update is a multi-step process, and any step can fail. The current update flow continues to later steps after an error. Continuing after a failure can cause the TDX module to enter an unrecoverable state. But certain failures during the initial module shutdown step should simply return an error to userspace, so the update can be retried cleanly. To preserve that recoverability, one option would be to abort the update only for those failures, since they occur before any TDX module state is changed. But special-casing specific failures in specific steps would complicate the do-while() update loop for no benefit. Simply abort update on any failure, at any step. Track failures for each step, stop the update loop once a failure is observed, and do not advance the state machine to the next step. [ dhansen: style nits ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Link: https://lore.kernel.org/linux-coco/aQFmOZCdw64z14cJ@google.com/ # [1] Link: https://patch.msgid.link/20260520133909.409394-16-chao.gao@intel.com |
||
|
Chao Gao
|
ab6be1168c |
x86/virt/seamldr: Introduce skeleton for TDX module updates
tl;dr: Use stop_machine() and a state machine based on the "MULTI_STOP" pattern to implement core TDX module update logic. Long version: TDX module updates require careful synchronization with other TDX operations. The requirements are (#1/#2 reflect current behavior that must be preserved): 1. SEAMCALLs need to be callable from both process and IRQ contexts. 2. SEAMCALLs need to be able to run concurrently across CPUs 3. During updates, only update-related SEAMCALLs are permitted; all other SEAMCALLs shouldn't be called. 4. During updates, all online CPUs must participate in the update work. No single lock primitive satisfies all requirements. For instance, rwlock_t handles #1/#2 but fails #4: CPUs spinning with IRQs disabled cannot be directed to perform update work. Use stop_machine() as it is the only well-understood mechanism that can meet all requirements. And TDX module updates consist of several steps (See Intel Trust Domain Extensions (Intel TDX) Module Base Architecture Specification, Chapter "TD-Preserving TDX module Update"). Ordering requirements between steps mandate lockstep synchronization across all CPUs. multi_cpu_stop() provides a good example of executing a multi-step task in lockstep across CPUs, but it does not synchronize the individual steps inside the callback itself. Implement a similar state machine as the skeleton for TDX module updates. Each state represents one step in the update flow, and the state advances only after all CPUs acknowledge completion of the current step. This acknowledgment mechanism provides the required lockstep execution. The update flow is intentionally simpler than multi_cpu_stop() in two ways: a) use a spinlock to protect the control data instead of atomic_t and explicit memory barriers. b) omit touch_nmi_watchdog() and rcu_momentary_eqs(), which exist there for debugging and are not strictly needed for this update flow Potential alternative to stop_machine() ======================================= An alternative approach is to lock all KVM entry points and kick all vCPUs. Here, KVM entry points refer to KVM VM/vCPU ioctl entry points, implemented in KVM common code (virt/kvm). Adding a locking mechanism there would affect all architectures KVM supports. And to lock only TDX vCPUs, new logic would be needed to identify TDX vCPUs, which the KVM common code currently lacks. This would add significant complexity and maintenance overhead to KVM for this TDX-specific use case, so don't take this approach. [ dhansen: normal changelog/style munging ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Link: https://patch.msgid.link/20260520133909.409394-15-chao.gao@intel.com |
||
|
Chao Gao
|
23a81e6cce |
x86/virt/seamldr: Allocate and populate a module update request
There are two important ABIs here: 'struct tdx_image' - The on-disk and in-memory format for a TDX module update image. 'struct seamldr_params' - The in-memory ABI passed to the TDX module loader. Points to a single 'struct tdx_image' broken up into 4k pages. Userspace supplies the update image in 'struct tdx_image' format. The image consists of a header followed by a sigstruct and the module binary. P-SEAMLDR, however, consumes 'struct seamldr_params' rather than the image directly. Parse the 'struct tdx_image' provided by userspace and populate a matching 'struct seamldr_params'. The 'tdx_image' ABI is versioned. Two public versions exist today: 0x100 and 0x200. This kernel only accepts 0x200. The older 0x100 format is being deprecated and is intentionally not supported here. Future versions of the module might be able to use the same ABIs (user/kernel and kernel/SEAMLDR) but they will not be able to use this kernel code. Reject module images without that specific version. This ensures that the kernel is able to understand the passed-in format. Validate the 'struct tdx_image' header before using it, because the header is consumed solely by the kernel to locate the sigstruct and module within the image. Do not validate the payload itself. The sigstruct and module pages are passed through to P-SEAMLDR, which validates them as part of the update. sigstruct_pages_pa_list currently has only one entry, but it will grow to four pages in the future. Keep it as an array for symmetry with module_pages_pa_list and for extensibility. [ dhansen: normal changelog clarification/munging ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-14-chao.gao@intel.com |
||
|
Chao Gao
|
c3e70c5ee5 |
coco/tdx-host: Implement firmware upload sysfs ABI for TDX module updates
tl;dr: Select fw_upload for doing TDX module updates. The process of selecting among available update images is complicated and nuanced. Punt the selection process out to userspace. One existing userspace implementation today is the script in the Intel TDX Module Binaries repository[1]. Long Version: The kernel supports two primary firmware update mechanisms: 1. request_firmware() - used by microcode, SEV firmware, hundreds of other drivers 2. 'struct fw_upload' - used by CXL, FPGA updates, dozens of others The key difference between is that request_firmware() loads a named file from the filesystem where the filename is kernel-controlled, while fw_upload accepts firmware data directly from userspace. TDX module firmware update selection policy is too complex for the kernel. Leave it to userspace and use fw_upload. Add a skeleton fw_upload implementation to be fleshed out in subsequent patches. Refactor the sysfs visiblity attribute function so it can be used as a more generic flag for the presence of viable runtime update support. Why fw_upload instead of request_firmware()? ============================================ Selecting a TDX module update image is not a simple "load the latest" decision. Userspace needs to choose an image that is compatible with both the platform and the currently running module. Some constraints are hard requirements: a. Module version series are platform-specific. For example, the 1.5.x series runs on Sapphire Rapids but not Granite Rapids, which needs 2.0.x. b. Updates are also constrained by version distance. A 1.5.6 module might permit updates to 1.5.7 but not to 1.5.50. There may also be userspace policy choices: c. Decide the update direction: upgrade or downgrade d. Choose whether to optimize for fewer updates or smaller version steps, for example, 1.2.3=>1.2.5 versus 1.2.3=>1.2.4=>1.2.5. Given that complexity, leave module selection to userspace and use fw_upload. 1. https://github.com/intel/confidential-computing.tdx.tdx-module.binaries/blob/main/version_select_and_load.py [ dhansen: add version script link, add more explanation of code moves, fix some minor whitespace issues ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Link: https://lore.kernel.org/kvm/01fc8946-eb84-46fa-9458-f345dd3f6033@intel.com/ Link: https://patch.msgid.link/20260520133909.409394-13-chao.gao@intel.com |
||
|
Chao Gao
|
5ce9cc5a23 |
coco/tdx-host: Don't expose P-SEAMLDR information on CPUs with erratum
TDX-capable CPUs clobber the current VMCS on P-SEAMLDR calls. Clearing the current VMCS behind KVM's back breaks KVM. Future CPUs will fix this by preserving the current VMCS across P-SEAMLDR calls. A future specification update will describe the VMCS-clearing behavior as an erratum and to state that it does not occur when IA32_VMX_BASIC[60] is set. Add a CPU bug bit and refuse to expose P-SEAMLDR information on affected CPUs. Use a CPU bug bit to stay consistent with X86_BUG_TDX_PW_MCE. As a bonus, the bug bit is visible to userspace, which allows userspace to determine why these sysfs files are not exposed, and it can also be checked by other kernel components in the future if needed. == Alternatives == Two workarounds were considered but both were rejected: 1. Save/restore the current VMCS around P-SEAMLDR calls. This produces ugly assembly code [1] and doesn't play well with #MCE or #NMI if they need to use the current VMCS. 2. Move KVM's VMCS tracking logic to the TDX core code, which would break the boundary between KVM and the TDX core code [2]. [ dhansen: comment and changelog munging. Add seamldr_call() bug check. ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://lore.kernel.org/kvm/fedb3192-e68c-423c-93b2-a4dc2f964148@intel.com/ # [1] Link: https://lore.kernel.org/kvm/aYIXFmT-676oN6j0@google.com/ # [2] Link: https://patch.msgid.link/20260520133909.409394-12-chao.gao@intel.com |
||
|
Chao Gao
|
0988bf698a |
x86/virt/seamldr: Add a helper to retrieve P-SEAMLDR information
P-SEAMLDR reports its state via SEAMLDR.INFO, including its version and the number of remaining runtime updates. This information is useful for userspace. For example, userspace can use the P-SEAMLDR version to determine whether a candidate TDX module is compatible with the running loader, and can use the remaining update count to determine whether another runtime update is still possible. Add a helper to retrieve P-SEAMLDR information in preparation for exposing P-SEAMLDR version and other necessary information to userspace. Export the new kAPI for use by the "tdx_host" device. Note that there are two distinct P-SEAMLDR APIs with similar names: "SEAMLDR.INFO" is metadata about the loader. It's metadata for the update process. "SEAMLDR.SEAMINFO" is metadata about SEAM mode. It is for the module init process, not for the update process. Use SEAMLDR.INFO here. For details, see "Intel Trust Domain Extensions - SEAM Loader (SEAMLDR) Interface Specification". Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-10-chao.gao@intel.com |
||
|
Chao Gao
|
9bc3ce2c5d |
x86/virt/seamldr: Introduce a wrapper for P-SEAMLDR SEAMCALLs
The TDX architecture uses the "SEAMCALL" instruction to communicate with
SEAM mode software. Right now, the only SEAM mode software that the kernel
communicates with is the TDX module. But, there is actually another
component that runs in SEAM mode but it is separate from the TDX module:
the persistent SEAM loader or "P-SEAMLDR". Right now, the only component
that communicates with it is the BIOS which loads the TDX module itself at
boot. But, to support updating the TDX module, the kernel now needs to be
able to talk to it.
P-SEAMLDR SEAMCALLs differ from TDX module SEAMCALLs in areas such as
concurrency requirements.
Add a P-SEAMLDR wrapper to handle these differences and prepare for
implementing concrete functions.
Use seamcall_prerr() (not '_ret') because current P-SEAMLDR calls do not
use any output registers other than RAX.
Note: Despite the similar name, the NP-SEAMLDR ("Non-Persistent")
(ACM) invoked exclusively by the BIOS at boot rather than a component
running in SEAM mode. The kernel cannot call it at runtime. It exposes
no SEAMCALL interface.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://cdrdv2.intel.com/v1/dl/getContent/733582 # [1]
Link: https://patch.msgid.link/20260520133909.409394-9-chao.gao@intel.com
|
||
|
Chao Gao
|
6fe8a33fdb |
coco/tdx-host: Expose TDX module version
For TDX module updates, userspace needs to select compatible update versions based on the current module version. For example, the 1.5.x series runs on Sapphire Rapids but not Granite Rapids, which needs 2.0.x. Updates are also constrained by version distance, so a 1.5.6 module might permit updates to 1.5.7 but not to 1.5.20. Start the process of punting the version selection logic to userspace. Expose the TDX module version in the new faux device. Define TDX_VERSION_FMT macro for the TDX version format since it will be used multiple times. Also convert an existing print statement to use it. == Background == For posterity, here's what other firmware mechanisms do: 1. AMD SEV leverages an existing PCI device for the PSP to expose metadata. TDX uses a faux device as it doesn't have PCI device in its architecture. 2. Microcode uses per-CPU virtual devices to report microcode revisions because CPUs can have different revisions. But, there is only a single TDX module, so exposing the TDX module version through a global TDX faux device is appropriate 3. ARM's CCA implementation isn't in-tree yet, but will likely follow a similar faux device approach, though it's unclear whether they need to expose firmware version information [ dhansen: trim changelog ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ # [1] Link: https://patch.msgid.link/20260520133909.409394-8-chao.gao@intel.com |
||
|
Chao Gao
|
5978335346 |
coco/tdx-host: Introduce a "tdx_host" device
TDX depends on a platform firmware module that runs on the CPU. Unlike other CoCo architectures, TDX has no hardware "device" running the show, just a blob on the CPU. Create a virtual device to anchor interactions with this platform firmware. This lets later code: - expose metadata: TDX module version, seamldr version, to userspace as device attributes - implement firmware uploader APIs (which are tied to a device) to support TDX module runtime updates Use a faux device because the TDX module is singular within the system and has no platform resources. Using a faux device eliminates the need to create a stub bus. The call to tdx_get_sysinfo() ensures that the TDX module is ready to provide services. Note that AMD has a PCI device for the PSP for SEV and ARM CCA will likely have a faux device [1]. Thanks to Dan and Yilun for all the help on this one. [ dhansen: trim changelog ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com> Reviewed-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ # [1] Link: https://patch.msgid.link/20260520133909.409394-7-chao.gao@intel.com |
||
|
Kai Huang
|
2e41297bfa |
x86/virt/tdx: Move low level SEAMCALL helpers out of <asm/tdx.h>
TDX host core code implements three seamcall*() helpers to make SEAMCALLs to the TDX module. Currently, they are implemented in <asm/tdx.h> and are exposed to other kernel code which includes <asm/tdx.h>. However, other than the TDX host core, seamcall*() are not expected to be used by other kernel code directly. For instance, for all SEAMCALLs that are used by KVM, the TDX host core exports a wrapper function for each of them. Move seamcall*() and related code out of <asm/tdx.h> and make them only visible to TDX host core. Since TDX host core tdx.c is already very heavy, don't put low level seamcall*() code there but to a new dedicated "seamcall_internal.h". Also, currently tdx.c has seamcall_prerr*() helpers which additionally print error message when calling seamcall*() fails. Move them to "seamcall_internal.h" as well. In such way all low level SEAMCALL helpers are in a dedicated place, which is much more readable. Copy the copyright notice from the original files and consolidate the date ranges to: Copyright (C) 2021-2023 Intel Corporation Signed-off-by: Kai Huang <kai.huang@intel.com> Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Zhenzhong Duan <zhenzhong.duan@intel.com> Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com> Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Vishal Annapurve <vannapurve@google.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-6-chao.gao@intel.com |
||
|
Chao Gao
|
77525820de |
x86/virt/tdx: Move TDX_FEATURES0 bits to asm/tdx.h
Future changes will add support for new TDX features exposed as TDX_FEATURES0 bits. The presence of these features will need to be checked outside of arch/x86/virt. The feature query helpers and the TDX_FEATURES0 defines they reference will need to live in the widely accessible asm/tdx.h header. Move the existing TDX_FEATURES0 to asm/tdx.h so that they can all be kept together. Opportunistically switch to BIT_ULL() since TDX_FEATURES0 is 64-bit. No functional change intended. [ dhansen: grammar fixups ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://lore.kernel.org/kvm/20260427152854.101171-17-chao.gao@intel.com/ # [1] Link: https://lore.kernel.org/kvm/20251121005125.417831-16-rick.p.edgecombe@intel.com/ # [2] Link: https://patch.msgid.link/20260520133909.409394-5-chao.gao@intel.com |
||
|
Chao Gao
|
451735bf90 |
x86/virt/tdx: Consolidate TDX global initialization states
The kernel uses several global flags to guard one-time TDX initialization flows and prevent them from being repeated. When the TDX module is updated, all of those states must be reset so that the module can be initialized again. Today those states are kept as separate global variables, which makes the reset path awkward and easy to miss when a new state is added. Group the states into a single structure so they can be reset together, for example with memset(), and so a newly added state won't be missed. Drop the __ro_after_init annotation from tdx_module_initialized because the other two states do not have it. And with TDX module update support, all the states need to be writable at runtime. Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-4-chao.gao@intel.com |
||
|
Chao Gao
|
9c0c68708d |
x86/virt/tdx: Move TDX global initialization states to file scope
TDX module global initialization is executed only once. The first call caches both the result and the "done" state, and later callers reuse the saved result. A lock protects that cached states. Those states and the lock are currently kept as function-local statics because they are used only by try_init_module_global(). TDX module updates need to reset the cached states so TDX global initialization can be run again after an update. That will add another access site in the same file. Move the cached states to file scope so it is accessible outside try_init_module_global(), and move the lock along with the states it protects. No functional change intended. Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-3-chao.gao@intel.com |
||
|
Chao Gao
|
1ffa6a1025 |
x86/virt/tdx: Clarify try_init_module_global() result caching
TDX module global initialization is executed only once. The first call caches both the return code and the "done" state in static function variables. Later callers read the variables. A lock protects the saved state and serializes callers. These variables will soon be moved to a global structure. Prepare for that by treating the variables as a unit. Assign them together and limit accesses to while the lock is held. [ dhansen: mostly rewrite changelog ] Signed-off-by: Chao Gao <chao.gao@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260520133909.409394-2-chao.gao@intel.com |
||
|
Yan Zhao
|
3f330fbb91 |
x86/virt/tdx: Move mk_keyed_paddr() to tdx.c due to no external users
Move mk_keyed_paddr() from tdx.h to tdx.c to avoid unnecessary header inclusion and improve encapsulation since there are no users outside of tdx.c. No functional change intended. Signed-off-by: Yan Zhao <yan.y.zhao@intel.com> Acked-by: Kiryl Shutsemau <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260430015014.24261-1-yan.y.zhao@intel.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Yan Zhao
|
4a72a6dc44 |
x86/tdx: Drop exported function tdx_quirk_reset_page()
KVM invokes tdx_quirk_reset_page() to reset TDX control pages (including S-EPT pages, TDR page, etc.), as all those pages are allocated by KVM TDX and thus always have struct page. However, it's also reasonable for KVM to reset those TDX control pages via tdx_quirk_reset_paddr() directly, eliminating the need to export two parallel APIs. Keeping tdx_quirk_reset_page() as a one-line helper in the header file is also unnecessary. No functional change intended. Suggested-by: Paolo Bonzini <pbonzini@redhat.com> Suggested-by: Xiaoyao Li <xiaoyao.li@intel.com> Signed-off-by: Yan Zhao <yan.y.zhao@intel.com> Acked-by: Kiryl Shutsemau <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Ackerley Tng <ackerleytng@google.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260430015001.24242-1-yan.y.zhao@intel.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
4c7a124764 |
x86/tdx: Use PFN directly for unmapping guest private memory
Remove struct page assumptions/constraints in APIs for unmapping guest private memory and have them take physical address directly. Having core TDX make assumptions that guest private memory must be backed by struct page (and/or folio) will create subtle dependencies on how KVM/guest_memfd allocates/manages memory (e.g., whether it uses memory allocated from core MM, if the memory is refcounted, or if the folio is split) that are easily avoided. [1]. KVM's MMUs work with PFNs. This is very much an intentional design choice. It ensures that the KVM MMUs remain flexible and are not too tightly tied to the regular CPU MMUs and the kernel code around them. Using "struct page" for TDX guest memory is not a good fit anywhere near the KVM MMU code [2]. Therefore, for unmapping guest private memory: export tdx_quirk_reset_paddr() for direct KVM invocation, and convert the SEAMCALL wrapper API tdh_phymem_page_wbinvd_hkid() to take PFN as input (thus updating mk_keyed_paddr() and tdh_phymem_page_wbinvd_tdr()). Intentionally have KVM pass PAGE_SIZE (rather than KVM_HPAGE_SIZE(level)) to tdx_quirk_reset_paddr() in tdx_sept_remove_private_spte() to avoid mixing in huge page changes. The KVM_BUG_ON() check for !PG_LEVEL_4K in tdx_sept_remove_private_spte() justifies using PAGE_SIZE. Do not convert tdx_reclaim_page() to use PFN as input since it currently does not remove guest private memory. Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate since APIs tdh_phymem_page_wbinvd_hkid() and tdx_quirk_reset_paddr() are exported to KVM only. [Yan: Use kvm_pfn_t,exclude tdx_reclaim_page(),use tdx_quirk_reset_paddr()] Signed-off-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://lore.kernel.org/all/aWgyhmTJphGQqO0Y@google.com [1] Link: https://lore.kernel.org/all/ac7V0g2q2hN3dU5u@google.com [2] Acked-by: Kiryl Shutsemau <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Ackerley Tng <ackerleytng@google.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260430014948.24226-1-yan.y.zhao@intel.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
6ad0badd76 |
x86/tdx: Use PFN directly for mapping guest private memory
Remove struct page assumptions/constraints in the SEAMCALL wrapper APIs for mapping guest private memory and have them take PFN directly. Having core TDX make assumptions that guest private memory must be backed by struct page (and/or folio) will create subtle dependencies on how KVM/guest_memfd allocates/manages memory (e.g., whether it uses memory allocated from core MM, if the memory is refcounted, or if the folio is split) that are easily avoided. [1]. KVM's MMUs work with PFNs. This is very much an intentional design choice. It ensures that the KVM MMUs remain flexible and are not too tied to the regular CPU MMUs and the kernel code around them. Using 'struct page' for TDX guest memory is not a good fit anywhere near the KVM MMU code [2]. Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate since APIs tdh_mem_page_add() and tdh_mem_page_aug() are exported to KVM only. [ Yan: Replace "u64 pfn" with "kvm_pfn_t pfn" ] Signed-off-by: Yan Zhao <yan.y.zhao@intel.com> Link: https://lore.kernel.org/all/aWgyhmTJphGQqO0Y@google.com [1] Link: https://lore.kernel.org/all/ac7V0g2q2hN3dU5u@google.com [2] Acked-by: Kiryl Shutsemau <kas@kernel.org> Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com> Reviewed-by: Ackerley Tng <ackerleytng@google.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260430014929.24210-1-yan.y.zhao@intel.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Sean Christopherson
|
4487492b92 |
x86/tdx: Use pg_level in TDX APIs, not the TDX-Module's 0-based level
Rework the TDX APIs to take the kernel's 1-based pg_level enum, not the TDX-Module's 0-based level. The APIs are _kernel_ APIs, not TDX-Module APIs, and the kernel (and KVM) uses "enum pg_level" literally everywhere. Using "enum pg_level" eliminates ambiguity when looking at the APIs (it's NOT clear that "int level" refers to the TDX-Module's level), and will allow for using existing helpers like page_level_size() when support for hugepages is added to the S-EPT APIs. No functional change intended. Cc: Kai Huang <kai.huang@intel.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Rick Edgecombe <rick.p.edgecombe@intel.com> Cc: Yan Zhao <yan.y.zhao@intel.com> Cc: Vishal Annapurve <vannapurve@google.com> Cc: Ackerley Tng <ackerleytng@google.com> Acked-by: Kiryl Shutsemau <kas@kernel.org> Reviewed-by: Kai Huang <kai.huang@intel.com> Tested-by: Kai Huang <kai.huang@intel.com> Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Tested-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Link: https://patch.msgid.link/20260129011517.3545883-2-seanjc@google.com Signed-off-by: Sean Christopherson <seanjc@google.com> |
||
|
Mikhail Gavrilov
|
fff82ea9d9 |
x86/virt: Silence RCU lockdep splat in emergency virt callback path
x86_virt_invoke_kvm_emergency_callback() reaches rcu_dereference() through machine_crash_shutdown() with IRQs disabled but with RCU not necessarily watching the crashing CPU, which triggers a suspicious RCU usage splat on debug kernels (CONFIG_PROVE_RCU=y) during panic/kdump: WARNING: suspicious RCU usage arch/x86/virt/hw.c:52 suspicious rcu_dereference_check() usage! rcu_scheduler_active = 2, debug_locks = 1 1 lock held by tee/11119: #0: ffff8881fa32c440 (sb_writers#3){.+.+}-{0:0}, at: ksys_write Call Trace: <TASK> dump_stack_lvl+0x84/0xd0 lockdep_rcu_suspicious.cold+0x37/0x8f x86_virt_invoke_kvm_emergency_callback+0x5f/0x70 x86_svm_emergency_disable_virtualization_cpu+0x2a/0x30 x86_virt_emergency_disable_virtualization_cpu+0x6b/0x90 native_machine_crash_shutdown+0x72/0x170 __crash_kexec+0x137/0x280 panic+0xce/0xd0 sysrq_handle_crash+0x1f/0x20 __handle_sysrq.cold+0x192/0x335 write_sysrq_trigger+0x8c/0xc0 proc_reg_write+0x1c3/0x3c0 vfs_write+0x1d0/0xf80 ksys_write+0x116/0x250 do_syscall_64+0x11c/0x1480 entry_SYSCALL_64_after_hwframe+0x76/0x7e </TASK> A truly correct fix is non-trivial: the RCU usage genuinely is wrong in panic context (RCU may ignore the crashing CPU during synchronization), and a concurrent KVM module unload could in principle race with the callback read; see commit |
||
|
Tycho Andersen (AMD)
|
39f1de2fff |
x86/sev: Do not initialize SNP if missing CPUs
The SEV firmware checks that the SNP enable bit is set on each CPU during SNP initialization, and will fail if not. If there are some CPUs offline, they will not run the setup functions, so SNP initialization will always fail. Skip the IPIs in this case and return an error so that the CCP driver can skip the SNP_INIT that will fail. Also print the CPU masks in order to leave breadcrumbs so people can figure out what happened. [ bp: Massage commit message. ] Suggested-by: Borislav Petkov (AMD) <bp@alien8.de> Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Reviewed-by: Nikunj A Dadhania <nikunj@amd.com> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Link: https://20260429155636.540040-1-tycho@kernel.org |
||
|
Vishal Verma
|
5b25f249be |
x86/tdx: Disable the TDX module during kexec and kdump
Use the TDH.SYS.DISABLE SEAMCALL, which disables the TDX module, reclaims all memory resources assigned to TDX, and clears any partial-write induced poison, to allow kexec and kdump on platforms with the partial write errata. On TDX-capable platforms with the partial write erratum, kexec has been disabled because the new kernel could hit a machine check reading a previously poisoned memory location. Later TDX modules support TDH.SYS.DISABLE, which disables the module and reclaims all TDX memory resources, allowing the new kernel to re-initialize TDX from scratch. This operation also clears the old memory, cleaning up any poison. Add tdx_sys_disable() to tdx_shutdown(), which is called in the syscore_shutdown path for kexec. This is done just before tdx_shutdown() disables VMX on all CPUs. For kdump, call tdx_sys_disable() in the crash path before x86_virt_emergency_disable_virtualization_cpu() does VMXOFF. Since this clears any poison on TDX-managed memory, remove the X86_BUG_TDX_PW_MCE check in machine_kexec() that blocked kexec on partial write errata platforms. Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Acked-by: Kai Huang <kai.huang@intel.com> Link: https://patch.msgid.link/20260402-fuller_tdx_kexec_support-v3-4-34438d7094bf@intel.com |
||
|
Vishal Verma
|
b7d2173946 |
x86/virt/tdx: Add SEAMCALL wrapper for TDH.SYS.DISABLE
Some early TDX-capable platforms have an erratum where a partial write to TDX private memory can cause a machine check on a subsequent read. On these platforms, kexec and kdump have been disabled in these cases, because the old kernel cannot safely hand off TDX state to the new kernel. Later TDX modules support the TDH.SYS.DISABLE SEAMCALL, which provides a way to cleanly disable TDX and allow kexec to proceed. The new SEAMCALL has an enumeration bit, but that is ignored. It is expected that users will be using the latest TDX module, and the failure mode for running the missing SEAMCALL on an older module is not fatal. This can be a long running operation, and the time needed largely depends on the amount of memory that has been allocated to TDs. If all TDs have been destroyed prior to the sys_disable call, then it is fast, with only needing to override the TDX module memory. After the SEAMCALL completes, the TDX module is disabled and all memory resources allocated to TDX are freed and reset. The next kernel can then re-initialize the TDX module from scratch via the normal TDX bring-up sequence. The SEAMCALL can return two different error codes that expect a retry. - TDX_INTERRUPTED_RESUMABLE can be returned in the case of a host interrupt. However, it will not return until it makes some forward progress, so we can expect to complete even in the case of interrupt storms. - TDX_SYS_BUSY will be returned on contention with other TDH.SYS.* SEAMCALLs, however a side effect of TDH.SYS.DISABLE is that it will block other SEAMCALLs once it gets going. So this contention will be short lived. So loop infinitely on either of these error codes, until success or other error. An error is printed if the SEAMCALL fails with anything other than the error codes that cause retries, or 'synthesized' error codes produced for #GP or #UD. e.g., an old module that has been properly initialized, that doesn't implement SYS_DISABLE, returns TDX_OPERAND_INVALID. This prints: virt/tdx: TDH.SYS.DISABLE failed: 0xc000010000000000 But a system that doesn't have any TDX support at all doesn't print anything. Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Chao Gao <chao.gao@intel.com> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Acked-by: Kai Huang <kai.huang@intel.com> Link: https://patch.msgid.link/20260402-fuller_tdx_kexec_support-v3-3-34438d7094bf@intel.com |
||
|
Rick Edgecombe
|
597bdf6e06 |
x86/virt/tdx: Pull kexec cache flush logic into arch/x86
KVM tries to take care of some required cache flushing earlier in the kexec path in order to be kind to some long standing races that can occur later in the operation. Until recently, VMXOFF was handled within KVM. Since VMX being enabled is required to make a SEAMCALL, it had the best per-cpu scoped operation to plug the flushing into. So it is kicked off from there. This early kexec cache flushing in KVM happens via a syscore shutdown callback. Now that VMX enablement control has moved to arch/x86, which has grown its own syscore shutdown callback, it no longer make sense for it to live in KVM. It fits better with the TDX enablement managing code. In addition, future changes will add a SEAMCALL that happens immediately before VMXOFF, which means the cache flush in KVM will be too late to flush the cache before the last SEAMCALL. So move it to the newly added TDX arch/x86 syscore shutdown handler. Since tdx_cpu_flush_cache_for_kexec() is no longer needed by KVM, make it static and remove the export. Since it is also not part of an operation spread across disparate components, remove the redundant comments and verbose naming. In the existing KVM based code, CPU offline also funnels through tdx_cpu_flush_cache_for_kexec(). Add an explicit WBINVD in tdx_offline_cpu() as well, even though it may be redundant with WBINVD done elsewhere during CPU offline (e.g. hlt_play_dead()). This avoids relying on fragile code ordering for cache coherency safety. [Vishal: add explicit WBINVD in tdx_offline_cpu()] Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com> Signed-off-by: Vishal Verma <vishal.l.verma@intel.com> Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com> Reviewed-by: Chao Gao <chao.gao@intel.com> Acked-by: Kai Huang <kai.huang@intel.com> Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Acked-by: Sean Christopherson <seanjc@google.com> Link: https://patch.msgid.link/20260402-fuller_tdx_kexec_support-v3-2-34438d7094bf@intel.com |
||
|
Linus Torvalds
|
01f492e181 |
Arm:
- Add support for tracing in the standalone EL2 hypervisor code, which should help both debugging and performance analysis. This uses the new infrastructure for 'remote' trace buffers that can be exposed by non-kernel entities such as firmware, and which came through the tracing tree. - Add support for GICv5 Per Processor Interrupts (PPIs), as the starting point for supporting the new GIC architecture in KVM. - Finally add support for pKVM protected guests, where pages are unmapped from the host as they are faulted into the guest and can be shared back from the guest using pKVM hypercalls. Protected guests are created using a new machine type identifier. As the elusive guestmem has not yet delivered on its promises, anonymous memory is also supported. This is only a first step towards full isolation from the host; for example, the CPU register state and DMA accesses are not yet isolated. Because this does not really yet bring fully what it promises, it is hidden behind CONFIG_ARM_PKVM_GUEST + 'kvm-arm.mode=protected', and also triggers TAINT_USER when a VM is created. Caveat emptor. - Rework the dreaded user_mem_abort() function to make it more maintainable, reducing the amount of state being exposed to the various helpers and rendering a substantial amount of state immutable. - Expand the Stage-2 page table dumper to support NV shadow page tables on a per-VM basis. - Tidy up the pKVM PSCI proxy code to be slightly less hard to follow. - Fix both SPE and TRBE in non-VHE configurations so that they do not generate spurious, out of context table walks that ultimately lead to very bad HW lockups. - A small set of patches fixing the Stage-2 MMU freeing in error cases. - Tighten-up accepted SMC immediate value to be only #0 for host SMCCC calls. - The usual cleanups and other selftest churn. LoongArch: - Use CSR_CRMD_PLV for kvm_arch_vcpu_in_kernel(). - Add DMSINTC irqchip in kernel support. RISC-V: - Fix steal time shared memory alignment checks - Fix vector context allocation leak - Fix array out-of-bounds in pmu_ctr_read() and pmu_fw_ctr_read_hi() - Fix double-free of sdata in kvm_pmu_clear_snapshot_area() - Fix integer overflow in kvm_pmu_validate_counter_mask() - Fix shift-out-of-bounds in make_xfence_request() - Fix lost write protection on huge pages during dirty logging - Split huge pages during fault handling for dirty logging - Skip CSR restore if VCPU is reloaded on the same core - Implement kvm_arch_has_default_irqchip() for KVM selftests - Factored-out ISA checks into separate sources - Added hideleg to struct kvm_vcpu_config - Factored-out VCPU config into separate sources - Support configuration of per-VM HGATP mode from KVM user space s390: - Support for ESA (31-bit) guests inside nested hypervisors. - Remove restriction on memslot alignment, which is not needed anymore with the new gmap code. - Fix LPSW/E to update the bear (which of course is the breaking event address register). x86: - Shut up various UBSAN warnings on reading module parameter before they were initialized. - Don't zero-allocate page tables that are used for splitting hugepages in the TDP MMU, as KVM is guaranteed to set all SPTEs in the page table and thus write all bytes. - As an optimization, bail early when trying to unsync 4KiB mappings if the target gfn can just be mapped with a 2MiB hugepage. x86 generic: - Copy single-chunk MMIO write values into struct kvm_vcpu (more precisely struct kvm_mmio_fragment) to fix use-after-free stack bugs where KVM would dereference stack pointer after an exit to userspace. - Clean up and comment the emulated MMIO code to try to make it easier to maintain (not necessarily "easy", but "easier"). - Move VMXON+VMXOFF and EFER.SVME toggling out of KVM (not *all* of VMX and SVM enabling) as it is needed for trusted I/O. - Advertise support for AVX512 Bit Matrix Multiply (BMM) instructions - Immediately fail the build if a required #define is missing in one of KVM's headers that is included multiple times. - Reject SET_GUEST_DEBUG with -EBUSY if there's an already injected exception, mostly to prevent syzkaller from abusing the uAPI to trigger WARNs, but also because it can help prevent userspace from unintentionally crashing the VM. - Exempt SMM from CPUID faulting on Intel, as per the spec. - Misc hardening and cleanup changes. x86 (AMD): - Fix and optimize IRQ window inhibit handling for AVIC; make it per-vCPU so that KVM doesn't prematurely re-enable AVIC if multiple vCPUs have to-be-injected IRQs. - Clean up and optimize the OSVW handling, avoiding a bug in which KVM would overwrite state when enabling virtualization on multiple CPUs in parallel. This should not be a problem because OSVW should usually be the same for all CPUs. - Drop a WARN in KVM_MEMORY_ENCRYPT_REG_REGION where KVM complains about a "too large" size based purely on user input. - Clean up and harden the pinning code for KVM_MEMORY_ENCRYPT_REG_REGION. - Disallow synchronizing a VMSA of an already-launched/encrypted vCPU, as doing so for an SNP guest will crash the host due to an RMP violation page fault. - Overhaul KVM's APIs for detecting SEV+ guests so that VM-scoped queries are required to hold kvm->lock, and enforce it by lockdep. Fix various bugs where sev_guest() was not ensured to be stable for the whole duration of a function or ioctl. - Convert a pile of kvm->lock SEV code to guard(). - Play nicer with userspace that does not enable KVM_CAP_EXCEPTION_PAYLOAD, for which KVM needs to set CR2 and DR6 as a response to ioctls such as KVM_GET_VCPU_EVENTS (even if the payload would end up in EXITINFO2 rather than CR2, for example). Only set CR2 and DR6 when consumption of the payload is imminent, but on the other hand force delivery of the payload in all paths where userspace retrieves CR2 or DR6. - Use vcpu->arch.cr2 when updating vmcb12's CR2 on nested #VMEXIT instead of vmcb02->save.cr2. The value is out of sync after a save/restore or after a #PF is injected into L2. - Fix a class of nSVM bugs where some fields written by the CPU are not synchronized from vmcb02 to cached vmcb12 after VMRUN, and so are not up-to-date when saved by KVM_GET_NESTED_STATE. - Fix a class of bugs where the ordering between KVM_SET_NESTED_STATE and KVM_SET_{S}REGS could cause vmcb02 to be incorrectly initialized after save+restore. - Add a variety of missing nSVM consistency checks. - Fix several bugs where KVM failed to correctly update VMCB fields on nested #VMEXIT. - Fix several bugs where KVM failed to correctly synthesize #UD or #GP for SVM-related instructions. - Add support for save+restore of virtualized LBRs (on SVM). - Refactor various helpers and macros to improve clarity and (hopefully) make the code easier to maintain. - Aggressively sanitize fields when copying from vmcb12, to guard against unintentionally allowing L1 to utilize yet-to-be-defined features. - Fix several bugs where KVM botched rAX legality checks when emulating SVM instructions. There are remaining issues in that KVM doesn't handle size prefix overrides for 64-bit guests. - Fail emulation of VMRUN/VMLOAD/VMSAVE if mapping vmcb12 fails instead of somewhat arbitrarily synthesizing #GP (i.e. don't double down on AMD's architectural but sketchy behavior of generating #GP for "unsupported" addresses). - Cache all used vmcb12 fields to further harden against TOCTOU bugs. x86 (Intel): - Drop obsolete branch hint prefixes from the VMX instruction macros. - Use ASM_INPUT_RM() in __vmcs_writel() to coerce clang into using a register input when appropriate. - Code cleanups. guest_memfd: - Don't mark guest_memfd folios as accessed, as guest_memfd doesn't support reclaim, the memory is unevictable, and there is no storage to write back to. LoongArch selftests: - Add KVM PMU test cases s390 selftests: - Enable more memory selftests. x86 selftests: - Add support for Hygon CPUs in KVM selftests. - Fix a bug in the MSR test where it would get false failures on AMD/Hygon CPUs with exactly one of RDPID or RDTSCP. - Add an MADV_COLLAPSE testcase for guest_memfd as a regression test for a bug where the kernel would attempt to collapse guest_memfd folios against KVM's will. -----BEGIN PGP SIGNATURE----- iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmnftRQUHHBib256aW5p QHJlZGhhdC5jb20ACgkQv/vSX3jHroPAzwf+NKO4Ktv+7A22ImN0SBl0nlUuulsz vTcw3+hxdRoIw83GdNS+hG5js0wrpMDnbv3t4+VliDNBSSxrBzcSWX2wpilW0Xtw qGo1MWhs2lKPy1NlaRVOwPS6j7uF3AR0TQ1iQLGMedQuCU9WpiKJxyhNXJdbLrt3 8EgFzsvtEsv+jKNRUNDf9+d0j4gZsFyIe+Brhianbw+u3/UCiUClLCdsKPc4+5ZX 08otYXytacGNIf/5Ev1vT4pHkHL0yqKXAtX7LEtaS3+0KrPuLjV4slemivzE9vf5 Evafm5AhA4wpaNMb1ZerhY3T94lsMaJpWxotjR//0Q7C9B59pCQnXCm8mg== =CcE0 -----END PGP SIGNATURE----- Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm Pull kvm updates from Paolo Bonzini: "Arm: - Add support for tracing in the standalone EL2 hypervisor code, which should help both debugging and performance analysis. This uses the new infrastructure for 'remote' trace buffers that can be exposed by non-kernel entities such as firmware, and which came through the tracing tree - Add support for GICv5 Per Processor Interrupts (PPIs), as the starting point for supporting the new GIC architecture in KVM - Finally add support for pKVM protected guests, where pages are unmapped from the host as they are faulted into the guest and can be shared back from the guest using pKVM hypercalls. Protected guests are created using a new machine type identifier. As the elusive guestmem has not yet delivered on its promises, anonymous memory is also supported This is only a first step towards full isolation from the host; for example, the CPU register state and DMA accesses are not yet isolated. Because this does not really yet bring fully what it promises, it is hidden behind CONFIG_ARM_PKVM_GUEST + 'kvm-arm.mode=protected', and also triggers TAINT_USER when a VM is created. Caveat emptor - Rework the dreaded user_mem_abort() function to make it more maintainable, reducing the amount of state being exposed to the various helpers and rendering a substantial amount of state immutable - Expand the Stage-2 page table dumper to support NV shadow page tables on a per-VM basis - Tidy up the pKVM PSCI proxy code to be slightly less hard to follow - Fix both SPE and TRBE in non-VHE configurations so that they do not generate spurious, out of context table walks that ultimately lead to very bad HW lockups - A small set of patches fixing the Stage-2 MMU freeing in error cases - Tighten-up accepted SMC immediate value to be only #0 for host SMCCC calls - The usual cleanups and other selftest churn LoongArch: - Use CSR_CRMD_PLV for kvm_arch_vcpu_in_kernel() - Add DMSINTC irqchip in kernel support RISC-V: - Fix steal time shared memory alignment checks - Fix vector context allocation leak - Fix array out-of-bounds in pmu_ctr_read() and pmu_fw_ctr_read_hi() - Fix double-free of sdata in kvm_pmu_clear_snapshot_area() - Fix integer overflow in kvm_pmu_validate_counter_mask() - Fix shift-out-of-bounds in make_xfence_request() - Fix lost write protection on huge pages during dirty logging - Split huge pages during fault handling for dirty logging - Skip CSR restore if VCPU is reloaded on the same core - Implement kvm_arch_has_default_irqchip() for KVM selftests - Factored-out ISA checks into separate sources - Added hideleg to struct kvm_vcpu_config - Factored-out VCPU config into separate sources - Support configuration of per-VM HGATP mode from KVM user space s390: - Support for ESA (31-bit) guests inside nested hypervisors - Remove restriction on memslot alignment, which is not needed anymore with the new gmap code - Fix LPSW/E to update the bear (which of course is the breaking event address register) x86: - Shut up various UBSAN warnings on reading module parameter before they were initialized - Don't zero-allocate page tables that are used for splitting hugepages in the TDP MMU, as KVM is guaranteed to set all SPTEs in the page table and thus write all bytes - As an optimization, bail early when trying to unsync 4KiB mappings if the target gfn can just be mapped with a 2MiB hugepage x86 generic: - Copy single-chunk MMIO write values into struct kvm_vcpu (more precisely struct kvm_mmio_fragment) to fix use-after-free stack bugs where KVM would dereference stack pointer after an exit to userspace - Clean up and comment the emulated MMIO code to try to make it easier to maintain (not necessarily "easy", but "easier") - Move VMXON+VMXOFF and EFER.SVME toggling out of KVM (not *all* of VMX and SVM enabling) as it is needed for trusted I/O - Advertise support for AVX512 Bit Matrix Multiply (BMM) instructions - Immediately fail the build if a required #define is missing in one of KVM's headers that is included multiple times - Reject SET_GUEST_DEBUG with -EBUSY if there's an already injected exception, mostly to prevent syzkaller from abusing the uAPI to trigger WARNs, but also because it can help prevent userspace from unintentionally crashing the VM - Exempt SMM from CPUID faulting on Intel, as per the spec - Misc hardening and cleanup changes x86 (AMD): - Fix and optimize IRQ window inhibit handling for AVIC; make it per-vCPU so that KVM doesn't prematurely re-enable AVIC if multiple vCPUs have to-be-injected IRQs - Clean up and optimize the OSVW handling, avoiding a bug in which KVM would overwrite state when enabling virtualization on multiple CPUs in parallel. This should not be a problem because OSVW should usually be the same for all CPUs - Drop a WARN in KVM_MEMORY_ENCRYPT_REG_REGION where KVM complains about a "too large" size based purely on user input - Clean up and harden the pinning code for KVM_MEMORY_ENCRYPT_REG_REGION - Disallow synchronizing a VMSA of an already-launched/encrypted vCPU, as doing so for an SNP guest will crash the host due to an RMP violation page fault - Overhaul KVM's APIs for detecting SEV+ guests so that VM-scoped queries are required to hold kvm->lock, and enforce it by lockdep. Fix various bugs where sev_guest() was not ensured to be stable for the whole duration of a function or ioctl - Convert a pile of kvm->lock SEV code to guard() - Play nicer with userspace that does not enable KVM_CAP_EXCEPTION_PAYLOAD, for which KVM needs to set CR2 and DR6 as a response to ioctls such as KVM_GET_VCPU_EVENTS (even if the payload would end up in EXITINFO2 rather than CR2, for example). Only set CR2 and DR6 when consumption of the payload is imminent, but on the other hand force delivery of the payload in all paths where userspace retrieves CR2 or DR6 - Use vcpu->arch.cr2 when updating vmcb12's CR2 on nested #VMEXIT instead of vmcb02->save.cr2. The value is out of sync after a save/restore or after a #PF is injected into L2 - Fix a class of nSVM bugs where some fields written by the CPU are not synchronized from vmcb02 to cached vmcb12 after VMRUN, and so are not up-to-date when saved by KVM_GET_NESTED_STATE - Fix a class of bugs where the ordering between KVM_SET_NESTED_STATE and KVM_SET_{S}REGS could cause vmcb02 to be incorrectly initialized after save+restore - Add a variety of missing nSVM consistency checks - Fix several bugs where KVM failed to correctly update VMCB fields on nested #VMEXIT - Fix several bugs where KVM failed to correctly synthesize #UD or #GP for SVM-related instructions - Add support for save+restore of virtualized LBRs (on SVM) - Refactor various helpers and macros to improve clarity and (hopefully) make the code easier to maintain - Aggressively sanitize fields when copying from vmcb12, to guard against unintentionally allowing L1 to utilize yet-to-be-defined features - Fix several bugs where KVM botched rAX legality checks when emulating SVM instructions. There are remaining issues in that KVM doesn't handle size prefix overrides for 64-bit guests - Fail emulation of VMRUN/VMLOAD/VMSAVE if mapping vmcb12 fails instead of somewhat arbitrarily synthesizing #GP (i.e. don't double down on AMD's architectural but sketchy behavior of generating #GP for "unsupported" addresses) - Cache all used vmcb12 fields to further harden against TOCTOU bugs x86 (Intel): - Drop obsolete branch hint prefixes from the VMX instruction macros - Use ASM_INPUT_RM() in __vmcs_writel() to coerce clang into using a register input when appropriate - Code cleanups guest_memfd: - Don't mark guest_memfd folios as accessed, as guest_memfd doesn't support reclaim, the memory is unevictable, and there is no storage to write back to LoongArch selftests: - Add KVM PMU test cases s390 selftests: - Enable more memory selftests x86 selftests: - Add support for Hygon CPUs in KVM selftests - Fix a bug in the MSR test where it would get false failures on AMD/Hygon CPUs with exactly one of RDPID or RDTSCP - Add an MADV_COLLAPSE testcase for guest_memfd as a regression test for a bug where the kernel would attempt to collapse guest_memfd folios against KVM's will" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (373 commits) KVM: x86: use inlines instead of macros for is_sev_*guest x86/virt: Treat SVM as unsupported when running as an SEV+ guest KVM: SEV: Goto an existing error label if charging misc_cg for an ASID fails KVM: SVM: Move lock-protected allocation of SEV ASID into a separate helper KVM: SEV: use mutex guard in snp_handle_guest_req() KVM: SEV: use mutex guard in sev_mem_enc_unregister_region() KVM: SEV: use mutex guard in sev_mem_enc_ioctl() KVM: SEV: use mutex guard in snp_launch_update() KVM: SEV: Assert that kvm->lock is held when querying SEV+ support KVM: SEV: Document that checking for SEV+ guests when reclaiming memory is "safe" KVM: SEV: Hide "struct kvm_sev_info" behind CONFIG_KVM_AMD_SEV=y KVM: SEV: WARN on unhandled VM type when initializing VM KVM: LoongArch: selftests: Add PMU overflow interrupt test KVM: LoongArch: selftests: Add basic PMU event counting test KVM: LoongArch: selftests: Add cpucfg read/write helpers LoongArch: KVM: Add DMSINTC inject msi to vCPU LoongArch: KVM: Add DMSINTC device support LoongArch: KVM: Make vcpu_is_preempted() as a macro rather than function LoongArch: KVM: Move host CSR_GSTAT save and restore in context switch LoongArch: KVM: Move host CSR_EENTRY save and restore in context switch ... |
||
|
Linus Torvalds
|
60b8d4d492 |
- Change the SEV host code handling of when SNP gets enabled in order to allow
the machine to claim SNP-related resources only when SNP guests are really going to be launched. The user requests this by loading the ccp module and thus it controls when SNP initialization is done So export an API which module code can call and do the necessary SNP setup only when really needed - Drop an unnecessary write-back and invalidate operation that was being performed too early, since the ccp driver already issues its own at the correct point in the initialization sequence — Drop the hotplug callbacks for enabling SNP on newly onlined CPUs, which were both architecturally unsound (the firmware rejects initialization if any CPU lacks the required configuration) and buggy (the MFDM SYSCFG MSR bit was not being set) - Code refactoring and cleanups to accomplish the above -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmndWHYACgkQEsHwGGHe VUrp5w/+KtaEmeIJ8w5tkZ1haY6/FaG0mPKkBVoIEbt4TR2p7JmdCDS0yVi88+Ze GnsmKbesinhMzH3SFcQWOHu17c5BYyu8HllDhKqvAalwp0BZXSsz0LruJJC9vyqz 6J89iGPhvFDlV3aE8gSAVNu8bzfQwoiAGS4C8QXVnerUCuGMqrM31dAyroqmRw+Y QNeSqm2OW3hkisSZPga8euo7e9iwGuXufhR/mxumrLY3k7mK0U9tDqMA9XaHqfLI TssalBrhdBJkf62Cj5gByVOlVVq6E4ii3xR0Pbs35BkXchdCM+ni89BcdJTrwzcr fcmsc0HXZud5ZzTi4BYPyEUuJbNshj2EuhVTaLbIGL0kPO/MNJ1xrGfAF1NrkqmD ATH5B5uCCwUxTJJc0Fwj4McVpW/TCqXP/QDEpMoN8yQX8yejQBGRcTKVIEzyGrze GziQ+Oem/MLTBz97r8p/CXR6ADDdqIxiO1WF1hDpYoDQqtdE+DeXSscLvSH+f4C+ 6x3EjrfMD8tNrf9JYE/aRdoF72q5GlWSh06RWlQrv2A+OezQCe82yMAPiUqaS1HS GsTJtoeKVDZNW2EfIl4HNzf0PemhD8JlJSRV/euaW6ipl1gDm4YasmYyhn2pL8WI Q0I9Ud2pnDmYFjxkfPmMCsnwkdSPCXlQ7/EeclYswWAKnoDrOAc= =2eQs -----END PGP SIGNATURE----- Merge tag 'x86_sev_for_v7.1_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull x86 SEV updates from Borislav Petkov: - Change the SEV host code handling of when SNP gets enabled in order to allow the machine to claim SNP-related resources only when SNP guests are really going to be launched. The user requests this by loading the ccp module and thus it controls when SNP initialization is done So export an API which module code can call and do the necessary SNP setup only when really needed - Drop an unnecessary write-back and invalidate operation that was being performed too early, since the ccp driver already issues its own at the correct point in the initialization sequence - Drop the hotplug callbacks for enabling SNP on newly onlined CPUs, which were both architecturally unsound (the firmware rejects initialization if any CPU lacks the required configuration) and buggy (the MFDM SYSCFG MSR bit was not being set) - Code refactoring and cleanups to accomplish the above * tag 'x86_sev_for_v7.1_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: crypto/ccp: Update HV_FIXED page states to allow freeing of memory crypto/ccp: Implement SNP x86 shutdown x86/sev, crypto/ccp: Move HSAVE_PA setup to arch/x86/ x86/sev, crypto/ccp: Move SNP init to ccp driver x86/sev: Create snp_shutdown() x86/sev: Create snp_prepare() x86/sev: Create a function to clear/zero the RMP x86/sev: Rename SNP_FEATURES_PRESENT to SNP_FEATURES_IMPL x86/virt/sev: Keep the RMP table bookkeeping area mapped x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN x86/virt/sev: Drop support for SNP hotplug |
||
|
Sean Christopherson
|
e30aa03d03 |
x86/virt: Treat SVM as unsupported when running as an SEV+ guest
When running as an SEV+ guest, treat SVM as unsupported even if CPUID (and
other reporting, e.g. MSRs) enumerate support for SVM, as KVM doesn't
support nested virtualization within an SEV VM (KVM would need to
explicitly share all VMCBs and other assets with the untrusted host), let
alone running nested VMs within SEV-ES+ guests (e.g. emulating VMLOAD,
VMSAVE, and VMRUN all require access to guest register state). And outside
of KVM, there is no in-tree user of SVM enabling.
Arguably, the hypervisor/VMM (e.g. QEMU) should clear SVM from guest CPUID
for SEV VMs, especially for SEV-ES+, but super duper technically, it's
feasible to run nested VMs in SEV+ guests (with many caveats). More
importantly, Linux-as-a-guest has played nice with SVM being advertised to
SEV+ guests for a long time.
Treating SVM as unsupported fixes a regression where a clean shutdown of
an SEV-ES+ guest degrades into an abrupt termination. Due to a gnarly
virtualization hole in SEV-ES (the architecture), where EFER must NOT be
intercepted by the hypervisor (because the untrusted hypervisor can't set
e.g. EFER.LME on behalf o the guest), the _host's_ EFER.SVME is visible to
the guest. Because EFER.SVME must be always '1' while in guest mode,
Linux-the-guest sees EFER.SVME=1 even when _its_ EFER.SVME is '0', thinks
it has enabled virtualization, and ultimately can cause
x86_svm_emergency_disable_virtualization_cpu() to execute STGI to ensure
GIF is enabled. Executing STGI _should_ be fine, except Linux is a also
wee bit paranoid when running as an SEV-ES guest.
Because L0 sees EFER.SVME=0 for the guest, a well-behaved L0 hypervisor
will intercept STGI (to inject #UD), and thus generate a #VC on the STGI.
Which, again, should be fine. Unfortunately, vc_check_opcode_bytes() fails
to account for STGI and other SVM instructions, throws a fatal error, and
triggers a termination request. In a perfect world, the #VC handler would
be more forgiving of unknown intercepts, especially when the #VC happened
on an instruction with exception fixup. For now, just fix the immediate
regression.
Fixes:
|
||
|
Tycho Andersen (AMD)
|
7b2bc5f0ab |
x86/sev, crypto/ccp: Move HSAVE_PA setup to arch/x86/
Now that there is snp_prepare() that indicates when the CCP driver wants to prepare the architecture for SNP_INIT(_EX), move this architecture-specific bit of code to a more sensible place. Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Link: https://patch.msgid.link/20260324161301.1353976-6-tycho@kernel.org |
||
|
Tycho Andersen (AMD)
|
299933b118 |
x86/sev, crypto/ccp: Move SNP init to ccp driver
Use the new snp_prepare() to initialize SNP from the ccp driver instead of at boot time. This means that SNP is not enabled unless it is really going to be used (i.e. kvm_amd loads the ccp driver automatically). Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Link: https://patch.msgid.link/20260324161301.1353976-5-tycho@kernel.org |
||
|
Tycho Andersen (AMD)
|
b65546b14d |
x86/sev: Create snp_shutdown()
After SNP_SHUTDOWN, two things should be done: 1. clear the RMP table 2. disable MFDM to prevent the FW_WARN in k8_check_syscfg_dram_mod_en() in the event of a kexec Create and export to the CCP driver a function that does them. Also change the MFDM helper to allow for disabling the bit, since the SNP x86 shutdown path needs to disable MFDM. The comment for k8_check_syscfg_dram_mod_en() notes, the "BIOS" is supposed clear it, or the kernel in the case of module unload and shutdown followed by kexec. Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Link: https://patch.msgid.link/20260324161301.1353976-4-tycho@kernel.org |
||
|
Tycho Andersen (AMD)
|
ca2ca373ec |
x86/sev: Create snp_prepare()
In preparation for delayed SNP initialization, create a function snp_prepare() that does the necessary architecture setup. Export this function for the ccp module to allow it to do the setup as necessary. Introduce a cpu_read_lock/unlock() wrapper around the MFDM and SNP enable. While CPU hotplug is not supported, this makes sure that the bit setting happens on the same set of CPUs in both cases. This improvement was suggested by Sashiko: https://sashiko.dev/#/patchset/20260324161301.1353976-1-tycho%40kernel.org Also move {mfd,snp}_enable() out of the __init section, since these will be called later. Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Link: https://patch.msgid.link/20260326161110.1764303-3-tycho@kernel.org |
||
|
Tom Lendacky
|
9c016c3f49 |
x86/sev: Create a function to clear/zero the RMP
In preparation for delayed SNP initialization and disablement on shutdown, create a function, clear_rmp(), that clears the RMP bookkeeping area and the RMP entries. Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Link: https://patch.msgid.link/20260324161301.1353976-2-tycho@kernel.org |
||
|
Tom Lendacky
|
cca1494299 |
x86/virt/sev: Keep the RMP table bookkeeping area mapped
In preparation for delayed SNP initialization and disablement on shutdown, the RMP will need to be cleared each time SNP is disabled. Maintain the mapping to the RMP bookkeeping area to avoid mapping and unmapping it each time and any possible errors that may arise from that. Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Link: https://patch.msgid.link/20260309180053.2389118-4-tycho@kernel.org |
||
|
Tycho Andersen (AMD)
|
99cf1fb58e |
x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN
WBINVD is required before SNP_INIT(_EX), but not before setting MSR_AMD64_SYSCFG_SNP_EN, since the ccp driver already does its own WBINVD before SNP_INIT (and this one would be too early for that anyway...). Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org> Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com> Link: https://patch.msgid.link/20260309180053.2389118-3-tycho@kernel.org |