The low 32-bit ICR data is prepared by __prepare_ICR(), which returns a 32-bit
value. However, when this value is assigned to a new variable, it's easy to
mistakenly declare that variable with a different width.
To avoid this class of mistakes, use __prepare_ICR() directly as the function
argument instead of storing its result in an intermediate variable. This also
shaves off a bunch of lines in the code.
There should be no functionality change resulting from this.
Signed-off-by: Melody Wang <huibo.wang@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260708012117.177959-1-huibo.wang@amd.com
The return values of syscall_enter_from_user_mode[_work]() are
non-intuitive. Both functions return the syscall number which should be
invoked by the architecture specific syscall entry code. The returned
number can be:
- the unmodified syscall number which was handed in by the caller
- a modified syscall number (ptrace, seccomp, trace/probe/bpf)
That has an additional twist. If the return value is -1L then the caller is
not allowed to modify the return value as that indicates that the modifying
entity requests to abort the syscall and set the return value already. That
can obviously not be differentiated from a syscall which handed in -1 as
syscall number.
The most trivial way to deal with that is:
set_return_value(regs, -ENOSYS);
nr = syscall_enter_from_user_mode(regs, nr);
if (valid(nr))
handle_syscall(regs, nr);
That's what LOONGARCH, RISCV, and X86 do. But PowerPC and S390 do not
preset the return value, so when user space hands in -1 and there is
nothing setting the return value in the entry work code, then the syscall
is skipped but the return value is whatever random data has been in the
return value register.
Change the return values of syscall_enter_from_user_mode[_work]() to
boolean and return false, when either ptrace or seccomp request to skip the
syscall. If they return true, update the syscall number as it might have
been changed.
That results in slightly different behaviour of the architectures versus
tracing.
If the syscall tracepoint has probe/BPF attached, those might set the
syscall number to -1 and also set the return value. PowerPC and S390 will
then overwrite that value with -ENOSYS. The other architectures will just
ignore it like any other invalid syscall and use the modified one.
Originally-by: Michal Suchánek <msuchanek@suse.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Michal Suchánek <msuchanek@suse.de>
Link: https://patch.msgid.link/20260712141346.772209074@kernel.org
Thomas Gleixner pointed out that cpu-type is a per-CPU property while hybrid
is a system property; conflating the two in the CPU matching infrastructure is
wrong. Currently, on a hybrid system x86_match_cpu() matches any cpu-type.
This works if the intent is to find the possibility of a cpu-type in a system.
But fails if matching for the cpu-type of a given CPU.
Borislav posted a cleanup here:
https://lore.kernel.org/all/20260703193222.GFakgORjvxwnZTPRnI@fat_crate.local
To make way for the cleanup stop matching cpu-type in cpu_vuln_blacklist.
RFDS is the only user, so drop the VULNBL_INTEL_TYPE entries and fold their
RFDS bit into the base Alder Lake (0x97) and Raptor Lake (0xB7) blacklist
entries. For now open-code cpu-type check in vulnerable_to_rfds(). In the
future, if more vulnerabilities need cpu-type matching a helper can be added.
No functional change intended.
Fixes: 722fa0dba7 ("x86/rfds: Exclude P-only parts from the RFDS affected list")
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20260708-cpu-type-vuln-v1-1-85c1d3c704db@linux.intel.com
- Reject too long acpi_rsdp= boot parameter values
(Thorsten Blum)
- Validate console=uart8250 baud rate to fix early boot hang
(Thorsten Blum)
- Remove dead Makefile rule (Ethan Nelson-Moore)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmpcjhMRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gVUw//UmbQ3jby1pvjDfAeJeqI8x2iCY2F9N5l
AqPD3371RPNzyx5uqyz4dLY94M31JW4jqzxntbIhGJrm9aXPLRTVzfMjF6sd9Lmj
lkMyHW+a4DVtA7oVSk2INebGUX2t+cpGLcF+9/yH3QavpLIXL9BNz5vDYqS0POMc
EitNNNqmO4+/f6oLf7Xxfl8Z2z2cVhcPCj1EG8b/rOWKeFmw1teuVu+dSagnV7Cn
/Hi4S8Q5HYUvvFBIFxgtuPW5S8HlTINhaE8hiat+Z0BENlAnbNThGEpz7hdogXAq
VDFwGPsjg3AF2agTWnF3e7Vs2CdDLHgcdTxUGII2lhZ9wCXzQ72+qXajf9PPynzH
YV9xtPw0a4sTDcR4QtdgBneCLf320WmbV6aH+tm9aZ1LIcPUgBO0qxYI/Ff1Frm8
NI1LxZJMiVfNNpFBk4JW/V1QKo3Padjhzfc23Rb4/8Xpu+S+Cu31pligIH3Rtnhm
pNt6Bv8eV61QJMZvjy4+5MouFfK2rL3YyEiE/WMP5cRhOCaK10hgDDEqCbuL5IRV
AgYMqGQPLTbZbdZZ1oSub+OPIRidMDfITlLoR9ggcJgNrV5J4tGysgAJWU2fCDoR
5dvRlzeQ0cjmCVMC+FVjQAahjBX5KqAQolDuDGB/vAAvxiir711ZO7g9UTCCRWQg
Bq1slSZ0WEc=
=upW0
-----END PGP SIGNATURE-----
Merge tag 'x86-urgent-2026-07-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 fixes from Ingo Molnar:
- Reject too long acpi_rsdp= boot parameter values (Thorsten Blum)
- Validate console=uart8250 baud rate to fix early boot hang (Thorsten
Blum)
- Remove dead Makefile rule (Ethan Nelson-Moore)
* tag 'x86-urgent-2026-07-19' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/boot: Validate console=uart8250 baud rate to fix early boot hang
x86/boot: Reject too long acpi_rsdp= values
x86/cpu: Remove Makefile rule for removed UMC CPU support
The recent switch to 64-bit MSR interfaces introduced a build warning.
Work it around with a __maybe_unused annotation. After the full conversion
of the APIs we'll be able to address this more robustly by making sure
the APIs consume all the values as far the compiler is concerned, so that
the compiler doesn't emit such nuisance warnings.
[ mingo: Updated the changelog. ]
Fixes: cff219368b ("x86/mce: Stop using 32-bit MSR interfaces")
Closes: https://lore.kernel.org/oe-kbuild-all/202607031726.ZOwu4snu-lkp@intel.com/
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260703105555.1758819-1-jgross@suse.com
The 32-bit MSR interfaces rdmsr() and rdmsr_safe() are planned to be
removed. Use the related 64-bit variants instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260703112426.1763049-1-jgross@suse.com
The 32-bit MSR interfaces rdmsr(), wrmsr() and rdmsr_safe() are
planned to be removed. Use the related 64-bit variants instead.
No change in functionality intended.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: kvm@vger.kernel.org
Link: https://patch.msgid.link/20260629060526.3638272-10-jgross@suse.com
The 32-bit MSR interfaces rdmsr(), wrmsr(), rdmsr_safe() and
wrmsr_safe() are planned to be removed. Use the related 64-bit variants
instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260703112344.1762985-1-jgross@suse.com
The 32-bit MSR interfaces rdmsr(), wrmsr(), rdmsr_safe() and
wrmsr_safe() are planned to be removed. Use the related 64-bit variants
instead.
In processor_throttling.c drop needless initializers.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Rafael J. Wysocki (Intel) <rafael@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-5-jgross@suse.com
If a vCPU stays scheduled out (or blocked) while the last pCPU it ran
on goes through a hotplug cycle (online->offline->online), and the vCPU
then resumes execution on the same pCPU, then it is possible for it to
run with an ASID that has now been assigned to a different vCPU,
resulting in stale TLB translations being used.
svm_enable_virtualization_cpu() resets asid_generation to 1 and sets
next_asid to max_asid + 1 on every CPU online event, including hotplug
cycles. Because next_asid starts beyond the pool boundary, the first
call to new_asid() after an online event always wraps the pool,
incrementing asid_generation to 2 and assigning ASIDs starting from
min_asid.
Consider two vCPUs from different VMs, vCPU-A pinned to CPU-X holding
asid_generation=2 and ASID=N from before the hotplug event:
1. CPU-X goes offline and back online: asid_generation resets to 1,
next_asid = max_asid + 1.
2. One or more vCPUs migrate to CPU-X and call new_asid(), wrapping
the pool and consuming ASIDs starting from min_asid. Eventually
vCPU-B from a different VM is assigned asid_generation=2, ASID=N
— the same ASID that vCPU-A held before the hotplug.
3. vCPU-A enters pre_svm_run() on CPU-X: current_vmcb->cpu is
unchanged so the migration branch is skipped. Its saved
asid_generation=2 matches sd->asid_generation=2, so the generation
check silently passes and vCPU-A continues running with ASID=N —
the same ASID just freshly assigned to vCPU-B.
Both vCPUs from different VMs now run on CPU-X with the same ASID,
causing them to share NPT TLB entries and producing stale translations.
The collision manifests as a KVM internal error (Suberror: 1, emulation
failure). The NPT page fault reports a faulting GPA far outside the
VM's physical memory range — a sign of stale TLB translations being
used. KVM falls back to instruction emulation, which fails on
FPU/XSave instructions (XRSTOR, STMXCSR) that the emulator does not
implement.
Fix this by incrementing asid_generation instead of resetting it to 1
in svm_enable_virtualization_cpu(). On module load, asid_generation
starts at 0 (memset) and the increment produces 1, identical to the
old behaviour. On subsequent hotplug cycles the generation advances
beyond any value a vCPU previously observed on this CPU, so the
generation check in pre_svm_run() reliably forces new_asid() on every
vCPU after every hotplug cycle.
Fixes: 774c47f1d7 ("[PATCH] KVM: cpu hotplug support")
Reported-by: Chandrakanth Silveru <Chandrakanth.Silveru@amd.com>
Tested-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Message-ID: <20260715063506.672432-1-nikunj@amd.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Loading and unloading modules concurrently on several CPUs on a KASAN
build, with a short delay injected at the CPA page-table lookup to
widen the window, faults within minutes:
BUG: KASAN: use-after-free in __change_page_attr+0x7cc/0x7e0
Write of size 8 at addr ffff888181139718 by task modprobe
...
The buggy address belongs to the physical page:
pfn:0x181139 ... page_type: f2(table)
cpa_collapse_large_pages() rebuilds a leaf PMD from its 4K PTEs and
frees the old PTE-table pages, while __change_page_attr() fetches a
PTE pointer from a lockless lookup_address_in_pgd_attr() and writes
it with set_pte_atomic() only later. When module text is served from
a shared large ROX mapping the two run on the same PMD:
CPU A (module load) CPU B (module finalize)
------------------- -----------------------
execmem_make_temp_rw
set_memory_nx
__change_page_attr
split 2M -> 4K table P
kpte = &P[i] (lockless)
execmem_restore_rox
set_memory_rox (CPA_COLLAPSE)
cpa_collapse_large_pages
rebuild leaf PMD
flush_tlb_all
pagetable_free(P)
set_pte_atomic(kpte, ...)
-> writes into freed P
P is a page-table page (page_type: table), reused at once, so the
write corrupts whatever got the page next: a bad-pte or bad-page
splat, or a fatal fault once P has been turned into read-only text.
The flush_tlb_all() before the free does not close this: its IPI only
serializes against page-table walkers that run with interrupts off
(e.g. GUP-fast); the walk in __change_page_attr() runs with interrupts
on, so nothing stops it from holding a stale pointer into P.
Serialize the collapse - the PMD rebuild, TLB flush and PTE-table
free - under cpa_lock, the same lock __change_page_attr() now takes
unconditionally since commit ("x86/mm/pat: stop gating cpa_lock on
debug_pagealloc_enabled()"), so a concurrent walker can no longer
hold a pointer into a table the collapse is about to free.
Fixes: 41d88484c7 ("x86/mm/pat: restore large ROX pages after fragmentation")
Signed-off-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://patch.msgid.link/20260715183453.2381141-1-den@openvz.org
The splitting and merging of kernel page table mappings between small and
large is protected by cpa_lock. The merging is relatively new but the
splitting is ancient.
The splitting has a locking optimization: since DEBUG_PAGEALLOC forces all
mappings to 4k, there are no large pages to split. So the code that *might*
cause a split can just skip the locking (and a few other things).
This is entertaining, but it adds complexity and makes for weird locking
rules. Plus it's all for a debugging feature which makes the kernel super
slow in the first place. Optimizing something which is already super slow
and not used in production is not the best way to spend our complexity
budget.
Stop gating cpa_lock on debug_pagealloc_enabled() to simplify the code
and the locking rules.
[ dhansen: flesh out changelog ]
Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260715144519.934289-1-rppt@kernel.org
Link: https://lore.kernel.org/all/aab44f08-89f8-47fe-bee4-0ab6b25968c6@intel.com/
The header define in asm/timex,h and the naming of the function to read the
delay timer are confusing at best.
Convert it to a config switch selected by the archictures which provide the
functionality and rename the function to delay_read_timer(), which makes
the purpose clear. Move the declaration to linux/delay.h where it belongs.
Remove the resulting empty asm/timex.h files as well.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260410120317.978403520@kernel.org
This has been scheduled for removal more than a decade ago and the comments
related to it have been dutifully ignored. The last dependencies are gone.
Remove it along with various now empty asm/timex.h files.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Link: https://patch.msgid.link/20260410120317.910770161@kernel.org
CLOCK_TICK_RATE is only used in x86 but defined all over the tree for no
reason with comments that it's scheduled for removal for more than a
decade.
Use PIT_TICK_RATE for registering refined jiffies to get rid of the last
dependency.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260410120317.843419270@kernel.org
Put all vmcs12 pages if KVM synthesizes a nested VM-Exit due to invalid
guest while emulating VMLAUNCH or VMRESUME. The invalid guest state path
doesn't use nested_vmx_vmexit() as that API is intended to be used if and
only if L2 is active, and the open coded equivalent neglects to put the
vmcs12 pages. Failure to put the vmcs12 pages leaks any pinned pages
(and/or mappings) if L1 retries VMLAUNCH/VMRESUME.
Note, the !from_vmenter scenario doesn't suffer the same problem, as
vmx_get_nested_state_pages() only gets/pins/maps the vmcs12 pages if L2 is
active, i.e. if a "full" VM-Exit is guaranteed before KVM will retry
getting vmcs12 pages.
Fixes: 96c66e87de ("KVM/nVMX: Use kvm_vcpu_map when mapping the virtual APIC page")
Fixes: 3278e04925 ("KVM/nVMX: Use kvm_vcpu_map when mapping the posted interrupt descriptor table")
Fixes: fe1911aa44 ("KVM: nVMX: Use kvm_vcpu_map() to get/pin vmcs12's APIC-access page")
Reported-by: Minh Nguyen <minhnguyen.080505@gmail.com>
Cc: stable@vger.kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- Fix a bug where KVM will trigger a UAF if updating IOMMU IRTEs fails when
registering an IRQ-bypass producer.
- Ignore pending PV EOI instead of BUG()ing the host if the feature was
disabled by the guest.
- Fix nVMX bugs where KVM would run L1 with an L1-controlled CR3 after a
failed "late" consistency check when KVM is NOT using EPT.
- Disallow intra-host migration/mirroring of SNP VMs as KVM doesn't yet
support moving/mirroring SNP state.
- Fix a TOCTOU bug in KVM's handling of the "trusted" CPUID for TDX guests.
- Fix a NULL pointer deref in trace_kvm_inj_exception() where a change to the
core infrastructure missed KVM's unique (ab)use of __print_symbolic().
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmpWb/kACgkQOlYIJqCj
N/1eAA//RDKkj5grTOKN7n9aY+uBrEFSH0VsEoyfg0GgiFZNOFWq4J6yiPTYb9kk
5Jxnl0LObI1sQ1Ini7LvB2sNa7YR5M3za3WNzUvynjw2gBvQGxU/7JMUqwcFlcXt
85RH/Wg7eNPCUvRUSsKEQwtdfi2NiGCxdg/AVnHs8/IPl8Ce7n3tQeKq4zm44mN9
eJ5hV5cCp6ytV7iM3N89yXo2LgLg2FWTg4vPA6MwxmTjNaouft3XH0428sNyI4zw
4NhLAnPLL8zmUs2zCxMSBYNkKiJ8Ou7jwD+1hny8MdHFbnrkNeNbk4Pbpol9AQ+Z
PhYeSsL0rk4bw+l5OEH5mxvtqjPV0qsH/ntN99W6kDh/gYDgJr7d70DsWxHWvluV
1Ewu26fjaUhKKwBoi4pDsXJVSHK2rTfeufeKegVW5IqEKLXYDzG2Kaj41UdPcFbo
HDv2FdUiqQo8PG2PUZzTduMOpcWggDjVgiqhAr4dL/Mf+IH+pDtPdCHM4XpvYHkQ
a+lFO76ZA/7fBVl8QgeZy8qRkbqSZLh00isUXXSRBamSL/rD27ODGOFrw+R9ct4s
hfUlC5GQDgrMmkqdweILfVip3/4B3Im2UI/5/QJB1bouf4r6w5A5fseTnMek445j
9zcFWquD17H/hg2DaS7+J0eZMKxhOZXegVLcE3UVqZLYw4VUBEw=
=+cUz
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-fixes-7.2-rc4' of https://github.com/kvm-x86/linux into HEAD
KVM x86 fixes for 7.2-rcN
- Fix a bug where KVM will trigger a UAF if updating IOMMU IRTEs fails when
registering an IRQ-bypass producer.
- Ignore pending PV EOI instead of BUG()ing the host if the feature was
disabled by the guest.
- Fix nVMX bugs where KVM would run L1 with an L1-controlled CR3 after a
failed "late" consistency check when KVM is NOT using EPT.
- Disallow intra-host migration/mirroring of SNP VMs as KVM doesn't yet
support moving/mirroring SNP state.
- Fix a TOCTOU bug in KVM's handling of the "trusted" CPUID for TDX guests.
- Fix a NULL pointer deref in trace_kvm_inj_exception() where a change to the
core infrastructure missed KVM's unique (ab)use of __print_symbolic().
Now that GENERIC_GETTIMEOFDAY and VDSO_GETRANDOM enable
HAVE_GENERIC_VDSO automatically, the architecture specific kconfig files
do not need to do so anymore.
Remove the corresponding lines.
Keep them for riscv, loongarch and powerpc as those have their own
explicit usage of the vDSO datastore provided by HAVE_GENERIC_VDSO.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Link: https://patch.msgid.link/20260709-vdso-have_generic_vdso-v1-5-d2e1061f268b@linutronix.de
Add missing ENCODE_FRAME_POINTER macro invocation into FRED_ENTER macro,
to prevent the unwinder from encountering a NULL stack frame pointer
when CONFIG_UNWINDER_FRAME_POINTER is enabled
Fixes: 14619d912b ("x86/fred: FRED entry/exit and dispatch code")
Signed-off-by: David Stevens <stevensd@google.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Link: https://patch.msgid.link/20260424191456.2679717-12-stevensd@google.com
Currently, only x86 genuinely implements and supports Syscall User Dispatch
(SUD).
Multiple architectures provide a stub arch_syscall_is_vdso_sigreturn()
returning 'false' simply to satisfy GENERIC_ENTRY compilation, which
creates a false impression of feature support.
Introduce ARCH_SUPPORTS_SYSCALL_USER_DISPATCH to decouple this mechanism
from GENERIC_ENTRY. Select it exclusively on x86 and remove the redundant
stub functions from other architectures.
Suggested-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20260713035422.582771-1-ruanjinjie@huawei.com
When the baud rate is empty, 0, invalid, or overflows to 0 when stored
as an int, the system will hang during early boot because of a division
by zero in early_serial_init().
Fall back to DEFAULT_BAUD when the resulting baud rate is 0 to prevent
an early system hang.
Fixes: ce0aa5dd20 ("x86, setup: Make the setup code also accept console=uart8250")
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260713194924.126472-3-thorsten.blum@linux.dev
Convert xen_mm_unpin_all() to ptdescs in preparation for the eventual
splitting of ptdescs from struct page. Continue checking PagePinned
through the underlying page as we do not have a per-memdesc api yet.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-10-vishal.moola@gmail.com
Convert xen_mm_pin_all() to ptdescs in preparation for the eventual
splitting of ptdescs from struct page. Continue checking PagePinned
through the underlying page as we do not have a per-memdesc api for
page flags yet.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Reviewed-by: Juergen Gross <jgross@suse.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-9-vishal.moola@gmail.com
Convert pgd_page_get_mm() to ptdescs. Define struct ptdesc in our
pgtable_types so that our declarations recognize ptdesc as an
appropriate page table type.
Now that all callers are using ptdescs, we can pass in that
ptdesc to get the underlying mm_struct.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-8-vishal.moola@gmail.com
Convert sync_global_pgds_l4() to ptdescs in preparation for the
eventual splitting of ptdescs from struct page.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-7-vishal.moola@gmail.com
Convert sync_global_pgds_l5() to ptdescs in preparation for the
eventual splitting of ptdescs from struct page.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-6-vishal.moola@gmail.com
Convert arch_sync_kernel_mappings() to ptdescs in preparation for
the eventual splitting of ptdescs from struct page.
Following this patch, we can successfully boot a 32-bit x86 kernel with
separately allocated ptdescs.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-5-vishal.moola@gmail.com
Convert collapse_pmd_page() to ptdescs in preparation for the
eventual splitting of ptdescs from struct page.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-4-vishal.moola@gmail.com
Convert __set_pmd_pte() to ptdescs in preparation for the eventual
splitting of ptdescs from struct page.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-3-vishal.moola@gmail.com
Use IS_ENABLED() to check if we are on 32 bit. This standardizes this
check with the other 32 bit check in the file. No functional changes.
Signed-off-by: Vishal Moola <vishal.moola@gmail.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: William Kucharski <william.kucharski@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://patch.msgid.link/20260629185742.126987-2-vishal.moola@gmail.com
According to x86 architecture rules, 32-bit operations zero-extend the
result to 64 bits. The current implementation of handle_in() only masks
the lower 32 bits, which preserves the upper 32 bits of RAX when a
32-bit port IN instruction is emulated.
Use insn_assign_reg() to write the result back into RAX with proper
partial-register-write semantics: 1- and 2-byte forms leave the upper
bits untouched, the 4-byte form zero-extends to the full register.
Fixes: 0314994883 ("x86/tdx: Port I/O: Add runtime hypercalls")
Reported-by: Borys Tsyrulnikov <tsyrulnikov.borys@gmail.com>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://lore.kernel.org/all/CAKw_Dz96rfSQc6Rn+9QBcUFHhmkK+9zu+P=bxowfZwxrATCBRg@mail.gmail.com/
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260713133753.223947-4-kirill@shutemov.name
KVM's instruction emulator has a small helper, assign_register(), that
writes a value into a register following the x86 rules for writes to
general-purpose registers: an 8- or 16-bit write leaves the rest of the
register untouched, a 32-bit write zero-extends the result to 64 bits,
and a 64-bit write replaces the whole register.
The TDX guest #VE handler needs the same logic for port I/O emulation
to get 32-bit zero-extension right. Rather than add a third copy of
the same switch, move the helper verbatim to <asm/insn-eval.h>, rename
it to insn_assign_reg(), and route KVM's callers through it.
Add <asm/insn.h> to the header's includes so it builds standalone in
callers that have not pulled it in transitively.
No functional change.
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Sean Christopherson <seanjc@google.com>
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260713133753.223947-3-kirill@shutemov.name
handle_in() and handle_out() in arch/x86/coco/tdx/tdx.c use:
u64 mask = GENMASK(BITS_PER_BYTE * size, 0);
GENMASK(h, l) includes bit h. For size=1 (INB), this produces
GENMASK(8, 0) = 0x1FF (9 bits) instead of GENMASK(7, 0) = 0xFF (8
bits). The mask is one bit too wide for all I/O sizes.
Fix the mask calculation.
Fixes: 0314994883 ("x86/tdx: Port I/O: Add runtime hypercalls")
Reported-by: Borys Tsyrulnikov <tsyrulnikov.borys@gmail.com>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://lore.kernel.org/all/CAKw_Dz96rfSQc6Rn+9QBcUFHhmkK+9zu+P=bxowfZwxrATCBRg@mail.gmail.com/
Cc:stable@vger.kernel.org
Link: https://patch.msgid.link/20260713133753.223947-2-kirill@shutemov.name
A resctrl domain consists of the domain structure self that includes
pointers to dynamically allocated filesystem as well as architecture
specific data. For example, the L3 monitoring domain structure consists
of the architecture specific struct rdt_hw_l3_mon_domain that contains
the dynamically allocated rdt_hw_l3_mon_domain::arch_mbm_states
architectural state and the embedded struct rdt_l3_mon_domain contains
the dynamically allocated rdt_l3_mon_domain::mbm_states resctrl fs state.
The domains are added to and removed from an RCU protected list while
cpus_write_lock() is held so that readers could access domains via
cpus_read_lock() or from an RCU read-side critical section. A reader
accessing a domain via the RCU list expects that the domain and all its
dynamically allocated data is accessible.
Only place the domain on the RCU list when all its dynamically allocated
data is ready, similarly unlink it from RCU list (again with cpus_write_lock()
held) before removing any of its dynamically allocated data.
Calling resctrl_online_mon_domain() before adding the domain to the RCU
list creates the kernfs files that expose the domain's monitoring data to
user space before adding the domain to the RCU list. This is safe because
rdtgroup_mondata_show() acquires cpus_read_lock() before it traverses the
RCU list and will thus block until the domain is added to the RCU list.
There are no readers accessing a domain via RCU list. Ensure safety of
access when such a reader arrives.
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tony Luck <tony.luck@intel.com>
Reviewed-by: Chen Yu <yu.c.chen@intel.com>
Link: https://patch.msgid.link/31ae67084c983e8cb8c5ef2c65e1096de5e8f9b0.1783963505.git.reinette.chatre@intel.com
rdt_resource::ctrl_domains and rdt_resource::mon_domains are RCU lists with
entries added and removed by architecture from CPU hotplug callbacks that are
run with cpus_write_lock() held. These lists can be traversed safely from
resctrl fs by either holding cpus_read_lock() or relying on an RCU read-side
critical section.
resctrl fs traversals of rdt_resource::ctrl_domains and
rdt_resource::mon_domains are done using list_for_each_entry() with
cpus_read_lock() held. Similarly, x86 architecture callbacks use
list_for_each_entry() expecting that resctrl fs makes the call with
cpus_read_lock() held. Inconsistently, a lockdep_assert_cpus_held() precedes
the list_for_each_entry() call with varying distance to document this safe RCU
list traversal.
In preparation for an upcoming traversal of rdt_resource::ctrl_domains that
needs to be done from RCU read-side critical section there is a requirement
for developers to always know exactly in which context the list is being
traversed.
Replace the list_for_each_entry() traversals of RCU list with
list_for_each_entry_rcu() to document that an RCU list is being traversed
while making use of the built-in lockdep expression that additionally
documents that it is cpus_read_lock() that enables the list to be
traversed from non-RCU protection. Only revert to documenting the
safety of traversal using a comment when lockdep does not have needed
visibility in functions called via smp_call*().
The lockdep expression within list_for_each_entry_rcu() depends on
RCU_EXPERT that is not set in a typical debug kernel so keep the existing
lockdep_assert_cpus_held() that is active with CONFIG_LOCKDEP=y found in
typical debug kernel.
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/d9373f8da8ffde667740e186ffc96ab69628ac9a.1783963505.git.reinette.chatre@intel.com
This was there to be able to patch out locking instructions when running
a SMP kernel build on a UP CPU. The times are long gone when single-CPU
x86 machines were relevant so drop that machinery and simplify the code
considerably.
LOCK_PREFIX needs to stay for when one wants to do a UP build for
whatever reason. That'll go away when CONFIG_SMP becomes unconditional.
Kill a bunch of leftover, unused prototypes while at it.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Link: https://lore.kernel.org/r/20260328081634.797552-1-rppt@kernel.org
The trace_kvm_inj_exception tracepoint takes as arguments the exception
vector, whether the exception has an error code (and subsequently, the
error code), and whether it is being reinjected. Because '0' is a valid
error code, KVM uses __print_symbolic() to format the error code as a
string to avoid printing the error code entirely if the exception doesn't
have an error code (see commit 21d4c575eb ("KVM: x86: Print error code
in exception injection tracepoint iff valid").
KVM's abuse of __print_symbolic() was all fine and dandy, until commit
754e38d2d1 ("tracing: Use explicit array size instead of sentinel
elements in symbol printing") reworked the printing to avoid terminating
the arrays with NULL/0 values, and missed KVM's clever use of not-quite
empty array of symbols.
BUG: kernel NULL pointer dereference, address: 0000000000000000
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: Oops: 0000 [#1] SMP
CPU: 20 UID: 0 PID: 791 Comm: less Not tainted 7.2.0-rc2 #401 PREEMPT
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
RIP: 0010:strlen+0x0/0x20
Call Trace:
<TASK>
trace_seq_puts+0x18/0x80
trace_print_symbols_seq+0x68/0xa0
trace_raw_output_kvm_inj_exception+0x64/0xf0 [kvm]
s_show+0x47/0x110
seq_read_iter+0x2a5/0x4c0
seq_read+0xfd/0x130
vfs_read+0xb6/0x330
? vfs_write+0x2f2/0x3f0
ksys_read+0x61/0xd0
do_syscall_64+0xb7/0x570
entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7ff283714862
</TASK>
Simply drop the dummy array entirely, so that __print_symbolic() generates
a truly empty array.
Signed-off-by: Daniel Paziyski <danielpaziyski@gmail.com>
Fixes: 754e38d2d1 ("tracing: Use explicit array size instead of sentinel elements in symbol printing")
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20260710134055.16432-1-danielpaziyski@gmail.com
[sean: massage changelog, add splat, add Fixes, cc stable]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Converting from int to long, back to int and then to unsigned int is
confusing at best.
None of this voodoo is required. Negative syscall numbers including -1
don't need any of this treatment and the low level ASM code already does
the sign extension to 64-bit on a 64-bit kernel.
The only point where signedness matters is the comparison against the
maximum syscall number, but that can be simplified by just using a unsigned
argument for the various syscall invocation functions.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Link: https://patch.msgid.link/20260707190254.545214398@kernel.org
Invoking sys_ni_syscall() from a code path, which already knows that the
syscall number is invalid just to assign -ENOSYS to regs->ax is a pointless
exercise. It's even redundant as the low level entry code already has set
regs->ax to -ENOSYS on entry.
Remove the extra conditionals and the function calls.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Link: https://patch.msgid.link/20260707190254.493733289@kernel.org
They are only used in the respective source files. No point in exposing
them.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Link: https://patch.msgid.link/20260707190254.438361565@kernel.org
The return value of __secure_computing() currently uses 0 to indicate
that a system call should be allowed, and -1 to indicate that it should
be blocked/killed. This 0/-1 pattern is non-intuitive for a security
check function and makes the control flow at the call sites less readable.
Furthermore, any potential future changes to these return values would
require a high-risk, error-prone audit of all its users across different
architectures.
Sanitize this logic by converting the return type of __secure_computing()
to a proper boolean, where 'true' explicitly means 'allow' and 'false'
means 'fail/deny'.
Update all the two dozen or so call sites across the tree to align with
this new boolean semantic. No functional changes are intended, as the
callers still return -1 to the lower-level assembly entry code upon
seccomp denial.
Rename the function to __seccomp_permit_syscall() so that the purpose is
entirely clear.
[ tglx: Rename the function ]
Suggested-by: Thomas Gleixner <tglx@kernel.org>
Suggested-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260707190254.230735780@kernel.org
These functions integrate the stack randomization.
syscall_enter_from_user_mode_randomize_stack() has the advantage that the
randomization happens early right after enter_from_user_mode().
In both cases also the overhead of get/put_cpu_var() in
add_random_kstack_offset() is avoided.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Reviewed-by: Radu Rendec <radu@rendec.net>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Link: https://patch.msgid.link/20260707190254.079478122@kernel.org
cmdline_find_option() returns the full length of the parsed acpi_rsdp=
value. get_cmdline_acpi_rsdp() then silently truncates values which do
not fit in the val[] buffer.
Prevent boot_kstrtoul() from parsing a truncated value and then the
kernel from silently using the wrong RSDP address, see discussion in
Link:.
Issue a warning so that the user is aware that s/he supplied a malformed
value and can get feedback instead of silent crashes.
[ bp: Make commit message more precise. ]
Fixes: 3c98e71b42 ("x86/boot: Add "acpi_rsdp=" early parsing")
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/20260617130417.36651-4-thorsten.blum@linux.dev
Support for UMC CPUs was removed in
7d328c5de4 ("x86/cpu: Remove CPU_SUP_UMC_32 support"),
but a Makefile rule for the support code remained. Remove it.
Fixes: 7d328c5de4 ("x86/cpu: Remove CPU_SUP_UMC_32 support")
Signed-off-by: Ethan Nelson-Moore <enelsonmoore@gmail.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Ahmed S. Darwish <darwi@linutronix.de>
Link: https://patch.msgid.link/20260610033252.164571-1-enelsonmoore@gmail.com
A user-only branch stack can contain branches that originate from
the kernel. As a result, kernel addresses are exposed to user space
even when PERF_SAMPLE_BRANCH_USER is requested. On AMD processors
supporting X86_FEATURE_BRS (Zen 3 only), perf can still report entries
such as SYSRET/interrupt returns for which the branch-from addresses
are in the kernel.
E.g.
$ perf record -j any,u -c 4000 -e branch-brs -o - -- \
perf bench syscall basic --loop 1000 | \
perf script -i - -F brstack|tr ' ' '\n'| \
grep -E '0x[89a-f][0-9a-f]{15}'
...
0xffffffff810001c4/0x72e2e32955eb/-/-/-/0//-
0xffffffff810001c4/0x72e2d94a9821/-/-/-/0//-
0xffffffff810001c4/0x72e2d94ffa1b/-/-/-/0//-
...
BRS provides no hardware branch filtering, so privilege level
filtering is performed entirely in software. However, amd_brs_match_plm()
only validates the branch-to address against the requested privilege
levels. For branches from the kernel to user space, the branch-from
address is left unchecked and is leaked. Extend the software filter to
also validate the branch-from address, so that any branch record whose
branch-from address is in the kernel is dropped when
PERF_SAMPLE_BRANCH_USER is requested.
Fixes: 8910075d61 ("perf/x86/amd: Enable branch sampling priv level filtering")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Stephane Eranian <eranian@google.com>
Link: https://patch.msgid.link/f05931c4f89a146c364bd5dc6b8170b1ac611c65.1783701239.git.sandipan.das@amd.com
Closes: https://lore.kernel.org/all/20260710110235.F3FD81F000E9@smtp.kernel.org/
Commit aebc3ca190 ("KVM: x86: Enable CMCI capability by default and
handle injected UCNA errors") introduced kvm_vcpu_x86_set_ucna(), which
accesses @vcpu->arch.mci_ctl2_banks[] using @mce->bank as the index. The
@mce struct is user-controlled, provided via the KVM_X86_SET_MCE ioctl.
The caller of this function, kvm_vcpu_ioctl_x86_set_mce(), bounds-checks
@mce->bank and applies array_index_nospec() to advance the @banks
pointer, but @mce->bank itself is passed through unclamped. On a
speculative path that bypasses the bounds check, the raw @mce->bank
value can index mci_ctl2_banks[] out-of-bounds.
In practice this is a very weak gadget, and would at most allow leaking
a single bit in a 64-bit integer, but prevent potential future issues by
clamping @mce->bank in place with array_index_nospec(), before passing
the struct to kvm_vcpu_x86_set_ucna().
Fixes: aebc3ca190 ("KVM: x86: Enable CMCI capability by default and handle injected UCNA errors")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260609131856.2562222-3-clopez@suse.de
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin big-number arithmetic engine to guests via
CPUID 0xC0000001 EDX bits 27 (RSA) and 28 (RSA_EN). The RSA feature
provides two unprivileged instructions for modular arithmetic on big
integers, documented in the Zhaoxin PadLock Instruction Reference,
chapter 4 ("Modular Multiplication and Exponentiation Engine"). Both
support operand sizes from 256 to 32768 bits (in 128-bit increments):
- REP XMODEXP (encoding F3 0F A6 F8, subsection 4.1) computes
A^B mod M
- REP MONTMUL2 (encoding F3 0F A6 F0, subsection 4.2) computes
A*B mod M
REP MONTMUL2 is the long-mode replacement of legacy REP MONTMUL, which
is restricted to compatibility and 32-bit protected modes. These
primitives accelerate RSA and related public-key operations.
Both instructions are unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The RSA and RSA_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because different
software may probe either one when checking for RSA availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-6-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin PadLock Hash Engine v2 to guests via CPUID
0xC0000001 EDX bits 25 (PHE2) and 26 (PHE2_EN). PHE2 extends the
PadLock hash family with SHA-384 and SHA-512 support per FIPS 180-3,
complementing the existing PHE feature (SHA-1 and SHA-256).
Two unprivileged instructions are exposed, documented in the Zhaoxin
PadLock Instruction Reference, chapter 3 ("Hash Engine"):
- REP XSHA384 (encoding F3 0F A6 D8, subsection 3.3)
- REP XSHA512 (encoding F3 0F A6 E0, subsection 3.4)
Both consume software-padded 128-byte blocks (RCX = block count, RSI =
input, RDI = state) and produce hash output in the state buffer.
Both instructions are unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The PHE2 and PHE2_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because different
software may probe either one when checking for PHE2 availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-5-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin second-generation hardware RNG to guests via
CPUID 0xC0000001 EDX bits 22 (RNG2) and 23 (RNG2_EN).
RNG2 is exposed by the REP XRNG2 instruction (encoding F3 0F A7 F8),
documented in the Zhaoxin PadLock Instruction Reference, subsection 1.3
("REP XRNG2"). It produces random bytes from two on-die RNG sources
selectable via RAX bits[10:9] and an output mode (raw vs post-processed)
controlled by RDX bits[1:0], providing high-quality entropy intended
for cryptographic operations.
REP XRNG2 is unprivileged (no CPL restriction) and available in all
CPU modes, with no associated MSR control. The RNG2 and RNG2_EN bits
are redundant by hardware design (set or cleared together) and both
serve purely as CPUID-level feature-presence reporting flags requiring
no KVM emulation. Both bits are advertised because different software
may probe either one when checking for RNG2 availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-4-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin CCS (Chinese Cryptography Standard) feature to
guests via CPUID 0xC0000001 EDX bits 4 (CCS) and 5 (CCS_EN). CCS groups
two unprivileged instructions for Chinese national cryptographic
primitives, documented in the Zhaoxin GMI Instruction Set Reference,
chapter 2 ("CCS instruction group"):
- SM3 (encoding F3 0F A6 E8, subsection 2.1) implements the SM3 hash
algorithm specified in GM/T 0004-2012. It supports two modes
selected by RAX: auto-padding stream mode (RAX=0) and pre-padded
block mode (RAX=-1).
- SM4 (encoding F3 0F A7 F0, subsection 2.2) implements the SM4 block
cipher specified in GM/T 0002-2012, supporting ECB / CBC / CFB /
OFB / CTR modes via a control word in RAX, and CBC-MAC / CFB-MAC
when RAX bit[11] is set.
Both instructions are unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The CCS and CCS_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because different
software may probe either one when checking for CCS availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-3-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Advertise the Zhaoxin SM2 instruction support to guests via CPUID
0xC0000001 EDX bits 0 (SM2) and 1 (SM2_EN).
The SM2 instruction (encoding F2 0F A6 C0) implements the SM2
elliptic-curve public-key cryptography algorithm specified in
GM/T 0003-2012; the hardware-level behavior is documented in the
Zhaoxin GMI Instruction Set Reference, chapter 1 ("SM2"). The
instruction multiplexes its sub-functions on the RDX[5:0] control
word: encryption (subsection 1.1), decryption (1.2), signing (1.3),
signature verification (1.4), the three key-exchange sub-operations
of section 1.5 (1.5.1 SM2 key-pair generation, which the spec also
uses for the initiator's ephemeral key; 1.5.2 responder shared-key
derivation; 1.5.3 initiator shared-key derivation), and two
preprocess steps for identity and message hashing (1.6.1 and 1.6.2).
The instruction is unprivileged (no CPL restriction) and available
in all CPU modes, with no associated MSR control. The SM2 and SM2_EN
bits are redundant by hardware design (set or cleared together) and
both serve purely as CPUID-level feature-presence reporting flags
requiring no KVM emulation. Both bits are advertised because
different software may probe either one when checking for SM2
availability.
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Signed-off-by: Ewan Hai <ewanhai-oc@zhaoxin.com>
Link: https://patch.msgid.link/20260610023512.3690734-2-ewanhai-oc@zhaoxin.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Rework KVM's handling of per-vendor nested ops to copy the vendor's ops
into a global structure owned by common x86, i.e. treat nested ops just
like x86 and PMU ops. In addition to providing consistency across all ops
implementations, making a copy of the ops prevents changes to the vendor's
ops after KVM is initialized, i.e. guards against goofs where KVM *thinks*
it is updating nested ops, but which won't take effect now that KVM uses
static calls to invoke vendor hooks.
Ignoring the side effects of tagging {svm,vmx}_nested_ops as __initdata,
no functional change intended.
Link: https://patch.msgid.link/20260630202828.440724-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Use static calls to invoke nested virtualization ops, as many of the calls
are in relatively hot paths when L2 is active, e.g. checking for events,
and because there's no reason not use static calls these days.
Opportunistically use a RET0 static call for get_evmcs_version() instead
of manually checking for a non-NULL vendor hook.
Link: https://patch.msgid.link/20260630202828.440724-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a flag to explicitly track if nested virtualization is enabled, and use
it enumerate that various nested CAPs are unsupported, and to reject
enablement of said CAPs. When the nested ops hooks were moved to their
own structure, KVM's NULL-by-default behavior was deliberately dropped,
with the changelog asserting that all was well. That wasn't quite true;
there is no danger to KVM, but now KVM is over-reporting support for
KVM_CAP_NESTED_STATE and KVM_CAP_HYPERV_ENLIGHTENED_VMCS.
Fixes: 33b2217245 ("KVM: x86: move nested-related kvm_x86_ops to a separate struct")
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Link: https://patch.msgid.link/20260630202828.440724-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reject KVM_TDX_INIT_VM if userspace changes cpuid.nent between the
initial read and the subsequent copy of the initialization data.
tdx_td_init() first reads user_data->cpuid.nent to size the flexible
kvm_tdx_init_vm copy. The copied structure also contains cpuid.nent,
and that field can differ from the value used to size the allocation if
userspace modifies the input concurrently. setup_tdparams_cpuids() later
passes init_vm->cpuid.nent to kvm_find_cpuid_entry2(), which uses it as
the array bound for the copied entries.
Require the copied count to match the value used to size the allocation
so that CPUID parsing cannot access beyond the entries actually copied.
Fixes: 0bd0a4a142 ("KVM: TDX: Replace kmalloc + copy_from_user with memdup_user in tdx_td_init()")
Reported-by: Sashiko:gemini-3.1-pro-preview
Cc: <stable@vger.kernel.org>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Thorsten Blum <thorsten.blum@linux.dev>
Link: https://patch.msgid.link/20260710035324.3170534-1-binbin.wu@linux.intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Wrap the .gmem_prepare() declarations with HAVE_KVM_GMEM_PREPARE so that
non-SEV code doesn't try to wire up a callback without doing the necessary
enabling.
No functional change intended.
Fixes: 3bb2531e20 ("KVM: guest_memfd: Add hook for initializing memory")
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-13-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Track the guest as having a guest-provided VMSA as soon as control.vmsa_pa
is invalidated, instead of waiting to see if the guest-provided VMSA is
usable, so that KVM doesn't switch back to the original VMSA instead of
exiting to userspace (due to an invalid VMSA). By the time a vCPU tries
to load a guest-provided VMSA, KVM has already communicated "success" for
AP creation, i.e. KVM has committed to using the guest-provided VMSA.
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-12-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Wire up a gmem_invalidate_range() call for SNP VMs, and use it to force
vCPUs to reload/recheck their guest-provided VMSA if the backing gmem
page is being invalidated, e.g. is being PUNCH_HOLE'd. Use the same core
logic to handle invalidations as VMX does for the APIC-access page, as the
two concepts are nearly identical: shove the physical address of a page
into the vCPU's control structure:
1. Snapshot the invalidation sequence counter
2. Grab the pfn (from guest_memfd in this case)
3. Acquire mmu_lock for read
4. Re-request reload if retry is needed, otherwise commit the change.
Note, the re-request action in #4 is necessary as KVM's retry logic is
fuzzy, i.e. can get false positives. If the guest_memfd page has been
dropped, at some point a subsequent reload will fail to get a PFN from
guest_memfd, and KVM will fail KVM_RUN. If the retry was due to a false
positive, KVM will retry until there are no relevant MMU notifier events
(and will retry in the "outer" loop, i.e. will drop locks and resched as
needed).
Note #2! Take care to invalidate the VMSA when a relevant memslot is
DELETED or MOVED, as invalidations in response to PUNCH_HOLE are predicated
on memslot bindings (KVM doesn't know what GFN range(s) to invalidate
without a binding). And more importantly, the VMSA mapping requires a
memslot, i.e. must be invalidated if its memslots disappears, regardless of
the state of the underlying guest_memfd inode.
Failure to invalidate the vCPU's control.vmsa_pa (which is checked by
pre_sev_run()) can prevent KVM from properly freeing the page as firmware
will reject the RMPUPDATE to reclaim the page with FAIL_INUSE if the vCPU
is actively running, i.e. if VMSA page is in-use. That in turn leads to an
RMP #PF on the next use, as the page will still be assigned to the SNP VM.
SEV-SNP: RMPUPDATE failed for PFN 78d198, pg_level: 1, ret: 3
SEV-SNP: PFN 0x78d198, RMP entry: [0xfff0000000144001 - 0x000000000000000f]
CPU: 3 UID: 0 PID: 31345 Comm: sev_snp_vmsa_pu Tainted: G U O
Tainted: [U]=USER, [O]=OOT_MODULE
Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026
Call Trace:
<TASK>
dump_stack_lvl+0x54/0x70
rmpupdate+0x12c/0x140
rmp_make_shared+0x3b/0x60
sev_gmem_invalidate+0xe0/0x170 [kvm_amd]
delete_from_page_cache_batch+0x1d8/0x220
truncate_inode_pages_range+0x120/0x3d0
kvm_gmem_fallocate+0x19a/0x270 [kvm]
vfs_fallocate+0x1bc/0x1f0
__x64_sys_fallocate+0x48/0x70
do_syscall_64+0x10a/0x480
entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x496c7e
</TASK>
------------[ cut here ]------------
SEV: Failed to update RMP entry for PFN 0x78d198 error -14
WARNING: arch/x86/kvm/svm/sev.c:5160 at sev_gmem_invalidate+0x126/0x170 [kvm_amd], CPU#3: sev_snp_vmsa_pu/31345
CPU: 3 UID: 0 PID: 31345 Comm: sev_snp_vmsa_pu Tainted: G U O
Tainted: [U]=USER, [O]=OOT_MODULE
Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026
RIP: 0010:sev_gmem_invalidate+0x12b/0x170 [kvm_amd]
Call Trace:
<TASK>
delete_from_page_cache_batch+0x1d8/0x220
truncate_inode_pages_range+0x120/0x3d0
kvm_gmem_fallocate+0x19a/0x270 [kvm]
vfs_fallocate+0x1bc/0x1f0
__x64_sys_fallocate+0x48/0x70
do_syscall_64+0x10a/0x480
entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x496c7e
</TASK>
irq event stamp: 20689
hardirqs last enabled at (20699): [<ffffffff8e76092c>] __console_unlock+0x5c/0x60
hardirqs last disabled at (20708): [<ffffffff8e760911>] __console_unlock+0x41/0x60
softirqs last enabled at (20722): [<ffffffff8e6cd74e>] __irq_exit_rcu+0x7e/0x140
softirqs last disabled at (20717): [<ffffffff8e6cd74e>] __irq_exit_rcu+0x7e/0x140
---[ end trace 0000000000000000 ]---
BUG: unable to handle page fault for address: ffff99a64d198000
#PF: supervisor write access in kernel mode
#PF: error_code(0x80000003) - RMP violation
PGD 13eb001067 P4D 13eb001067 PUD 78d1d1063 PMD 1184e0063 PTE 800000078d198163
SEV-SNP: PFN 0x78d198, RMP entry: [0x6030000000144001 - 0x000000000000000f]
Oops: Oops: 0003 [#1] SMP
CPU: 3 UID: 0 PID: 31407 Comm: highlanderd_hea Tainted: G U W O
Tainted: [U]=USER, [W]=WARN, [O]=OOT_MODULE
Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 34.86.0-102 01/25/2026
RIP: 0010:prep_new_page+0x67/0x220
Call Trace:
<TASK>
get_page_from_freelist+0x1c40/0x1c70
__alloc_frozen_pages_noprof+0xca/0x1f0
alloc_pages_mpol+0x10b/0x1b0
alloc_pages_noprof+0x81/0x90
pte_alloc_one+0x1b/0xd0
do_pte_missing+0xdf/0x1020
handle_mm_fault+0x7c7/0xb20
do_user_addr_fault+0x268/0x6b0
exc_page_fault+0x67/0xa0
asm_exc_page_fault+0x26/0x30
RIP: 0033:0x4a6b1e
</TASK>
gsmi: Log Shutdown Reason 0x03
CR2: ffff99a64d198000
---[ end trace 0000000000000000 ]---
RIP: 0010:prep_new_page+0x67/0x220
Drop the pseudo-TODO comment about needing to pin the page if guest_memfd
every supports migration, as integrating with invalidations events means
KVM will Just Work if/when page migration is ever supported (assuming SNP
hardware supports migrating VMSA pages).
Note #3, invalidate() and invalidate_range() have _completely_ different
semantics; the new invalidate_range() is a true invalidation, whereas the
existing invalidate() is really a "make shared" operation. Ignore the
confusing naming and poor Kconfig bundling for the moment to minimize the
delta for LTS kernels, the mess will be cleaned up shortly.
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Closes: https://lore.kernel.org/all/aimMWzAf5b3luM0b@v4bel
Fixes: e366f92ea9 ("KVM: SEV: Support SEV-SNP AP Creation NAE event")
Cc: stable@vger.kernel.org
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Michael Roth <michael.roth@amd.com>
Cc: Jörg Rödel <joro@8bytes.org>
Cc: Fuad Tabba <tabba@google.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Manually invoke the front half and back half of the "zap all fast" flow
when invalidating a memslot so that mmu_lock is acquired at function scope
in kvm_arch_flush_shadow_memslot(). This will allow putting more code
inside the critical section without having to take mmu_lock twice in quick
succession.
Opportunistically open code checking whether or not to do the fast zap, to
discourage removing the local "zap_all" in a future cleanup, i.e. to ensure
the SLOT_ZAP_ALL quirk is queried exactly once. Processing the front half
but not the back half of the fast zap (if SLOT_ZAP_ALL were disabled
concurrently) would result in KVM unnecessarily keeping invalid TDP MMU
roots until the VM is destroyed.
No functional change intended.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Split kvm_mmu_zap_all_fast() into a "front half" and a "back half", where
the front half is everything that runs with mmu_lock held for write, and
the back half is the code that runs outside of mmu_lock. This will allow
putting more code inside kvm_arch_flush_shadow_memslot()'s critical section
without having to take mmu_lock twice in quick succession.
No functional change intended.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Fold kvm_mmu_zap_memslot() into its sole caller so that its GFN range
structure can be used to trigger guest_memfd invalidations regardless of
whether KVM will do a partial or full zap of the MMU.
No functional change intended.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Wrap the sole reader of disabled_quirks with READ_ONCE(), and wrap the
post-VM-creation write to disabled_quirks with WRITE_ONCE(), to ensure
checking the status of a quirk doesn't re-read disabled_quirks *if* the
caller needs such a guarantee. This will allow splitting the "fast" MMU
zap into front and back halves, without potentially skipping the back
half if SLOT_ZAP_ALL were concurrently disabled (which would be "fine" in
the current code base, but far from ideal).
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Protect writes to disabled_quirks with kvm->lock to ensure KVM doesn't
clobber state in the unlikely scenario that userspace disables disparate
quirks from multiple tasks. More importantly, this will allow wrapping
accesses with {READ,WRITE}_ONCE without "needing" to also guard the writer
with a useless and confusing READ_ONCE (since the RMW wouldn't be atomic
anyways).
Ideally, KVM would disallow disabling quirks once quirks are "live", but
that would be a potentially breaking userspace ABI change, and while all
existing quirks are fully live only after vCPUs have been created, several
MMU-related quirks, IGNORE_GUEST_PAT and SLOT_ZAP_ALL, are partially live
at all times. Because populating MMUs requires a vCPU, the guest-visible
behavior of IGNORE_GUEST_PAT and SLOT_ZAP_ALL requires a vCPU, but for KVM
itself, processing the quirk (or not) has functional impact, i.e. for all
intents and purposes, KVM can't prevent those quirks from being disabled
after they've been consumed.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Wire up the SEV-SNP guest_memfd kvm_x86_ops hooks if and only if SEV is
actually enabled, and drop the now-unnecessary stubs. Leaving the hooks
NULL allows the static call infrastructure to elide the CALL+RET, and more
importantly, referencing the hooks if and only if SEV support is enabled
will allow conditionally definining the hooks using their corresponding
HAVE_KVM_ARCH_GMEM_XXX Kconfig.
No functional change intended.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Link: https://patch.msgid.link/20260709204948.1988414-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Always mark the vCPU as RUNNABLE after responding to AP_CREATE, even if the
guest-specified VMSA is unusable, e.g. isn't backed by a memslot or doesn't
have a backing guest_memfd page. If the VMSA is unusable, leaving the vCPU
in a non-running state will effectively hang the vCPU instead of reporting
an error to userspace. This will also allow retrying the VMSA load in the
future, to fix a bug where KVM doesn't honor guest_memfd invalidation
events, e.g. if AP_CREATION races with PUNCH_HOLE.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Extract the loading/retrieval of a guest-provided VMSA to a separate helper
so that KVM can reuse the core logic when refreshing the VMSA after an MMU
invalidation from guest_memfd.
No functional change intended.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Track the GPA of the guest-provided VMSA used after AP_CREATION events when
running SNP guests, instead of simply tracking whether or not the vCPU is
using a guest-provided VMSA. KVM needs to know the GPA of the VMSA that's
actively being used so that it can react to MMU invalidation events, i.e.
so that KVM can drop the VMSA if its backing guest_memfd page is punched
out of existence.
Opportunistically rename snp_vmsa_gpa to clarify that it tracks the pending
VMSA GPA, whereas snp_guest_vmsa_gpa now tracks the in-use VMSA GPA.
Note! Take care to track the GPA, not the GFN, as VALID_PAGE() won't
behave correctly if an invalid GFN is converted to a GPA for checking.
Note #2! Keep snp_has_guest_vmsa so that switching to a guest-provided
VMSA is sticky, even if the guest-provided VMSA becomes invalid.
No functional change intended.
Cc: stable@vger.kernel.org # 6.12.x
Reviewed-by: Michael Roth <michael.roth@amd.com>
Link: https://patch.msgid.link/20260709204948.1988414-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
The intra-host migration/mirroring feature is not fully implemented for
SEV-SNP VMs. The proper migration requires additional SNP-specific
state such as guest_req_mutex, guest_req_buf, and guest_resp_buf to be
transferred or initialized on the destination.
The SNP VM mirroring requires vmsa features to be copied as well otherwise
ASID would be bound to SNP range while VM is detected as a SEV VM.
Reject SNP source VMs in migration/mirroring until proper SNP state
transfer is implemented.
Fixes: 1dfe571c12 ("KVM: SEV: Add initial SEV-SNP support")
Reported-by: Chris Mason <clm@meta.com>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Atish Patra <atishp@meta.com>
Link: https://patch.msgid.link/20260602-sev_snp_fixes-v3-1-24bfd3ae047c@meta.com
Cc: stable@vger.kernel.org
[sean: let lines poke past 80 chars, tag for stable]
Signed-off-by: Sean Christopherson <seanjc@google.com>
In svm_set_nested_state(), the success path reaches out_free with
ret already set to 0 from nested_svm_load_cr3(). The explicit
'ret = 0' assignment before out_free is therefore redundant.
Remove it.
No functional change.
Signed-off-by: Qiang Ma <maqianga@uniontech.com>
Link: https://patch.msgid.link/20260618085217.3934985-1-maqianga@uniontech.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
A user-only branch stack can contain branches that originate from
the kernel. As a result, kernel addresses are exposed to user space
even when PERF_SAMPLE_BRANCH_USER is requested. On AMD processors
supporting X86_FEATURE_AMD_LBR_V2, perf can still report SYSRET/ERET
entries for which the branch-from addresses are in the kernel.
E.g.
$ perf record -e cycles -o - -j any,save_type,u -- \
perf bench syscall basic --loop 1000 | \
perf script -i - -F brstack|tr ' ' '\n'| \
grep -E '0x[89a-f][0-9a-f]{15}'
...
0xffffffff81001268/0x717a90a38f1a/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a39157/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a2c628/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a41b60/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a260db/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a90a260db/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a8bef1c30/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
0xffffffff81001268/0x717a8e4d3c90/M/-/-/0/ERET/NON_SPEC_CORRECT_PATH
...
The reason is that the hardware filter only considers the privilege
level applicable to the branch target. Extend software filtering to
also validate the branch-from addresses against br_sel, so that any
branch record whose branch-from address is in the kernel is dropped
when PERF_SAMPLE_BRANCH_USER is requested.
Fixes: f4f925dae7 ("perf/x86/amd/lbr: Add LbrExtV2 hardware branch filter support")
Reported-by: Ian Rogers <irogers@google.com>
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/a898a29725f6b2f30518354cdc2e432db66c43cf.1783680119.git.sandipan.das@amd.com
Add CPUID and MSR bit definitions required to support Intel Directed
Package Thermal Interrupt.
A CPU requests directed package-level thermal interrupts by setting bit 25
in IA32_THERM_INTERRUPT. Hardware acknowledges by setting bit 25 in
IA32_PACKAGE_THERM_STATUS, indicating that only CPUs that opted in will
receive the interrupt. If no CPU in the package requests it, delivery
falls back to broadcast.
Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Link: https://patch.msgid.link/20260613-rneri-directed-therm-intr-v3-2-3a26d1e47fc8@linux.intel.com
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Move the comment about setting the encryption mask above the line which
does that and, especially, inside the ifdeffery, where it belongs.
Move comments on top of the code lines they refer to and not on the
side, which impairs readability.
No functional changes.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://patch.msgid.link/20260620043757.347076-1-bp@kernel.org
PTI requires the begin and end of ENTRY_TEXT be aligned to PMD_SIZE. SRSO
requires srso_alias_untrain_ret to be 2M aligned. This costs between 2-4 MiB
of RAM (depending on the size of the preceding section). So, only align when
either of the two are enabled.
Co-developed-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Jared White <jaredwhite@microsoft.com>
Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260623152430.1473481-1-hamzamahfooz@linux.microsoft.com
The driver sets acpi_device_name() and acpi_device_class() which are
never read afterward, so make it stop doing that and drop the symbols
defined specifically for this purpose.
No intentional functional impact.
This will facilitate the removal of device_name and device_class from
struct acpi_device_pnp in the future.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://patch.msgid.link/2866696.mvXUDI8C0e@rafael.j.wysocki
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
vmx_get_msr() and vmx_set_msr() already cache to_vmx(vcpu) in a local
'vmx' pointer, but a few cases still open-code to_vmx(vcpu). Use the
cached pointer for consistency.
Likewise, cache to_vmx(vcpu) in vmx_get_segment_base() instead of
open-coding it in both the real-mode check and the VMCS read path.
No functional change intended.
Signed-off-by: Hao Zhang <zhanghao1@kylinos.cn>
Link: https://patch.msgid.link/tencent_A78DC401911634111A3391650CB00FCD0409@qq.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When a VM is being destroyed, delete it from the list used to process GA
Log interrupts before vCPUs are freed, otherwise avic_ga_log_notifier()
could theoretically hit a use-after-free if a GA Log notification arrives
for a vCPU after the last reference to the VM has been put.
Note, in practice, it's likely all but impossible to trigger UAF, as all
all irqfds and thus all IRTEs are cleaned up by:
kvm_irqfd_release()
|
|-> irqfd_deactivate()
|
|-> irqfd_shutdown()
|
|-> irq_bypass_unregister_consumer()
And kvm_irqfd_release() is guaranteed to run before the last reference to
the VM is put. KVM also configures GA Log interrupts only when a vCPU is
blocking (older versions of KVM configre GA Log interrupts at all times,
but AVIC is off by default on those kernels). Hitting UAF would require
tearing down a VM shortly after a vCPU stopped blocking, and with a very,
very delayed IRQ from hardware.
Opportunistically use guard() to avoid a local "flags" variable.
Fixes: 5881f73757 ("svm: Introduce AMD IOMMU avic_ga_log_notifier")
Cc: Naveen N Rao (AMD) <naveen@kernel.org>
Cc: Xiao Wu <xiaowu.417@qq.com>
Reviewed-by: Naveen N Rao (AMD) <naveen@kernel.org>
Link: https://patch.msgid.link/20260630210156.457151-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move all per-VM AVIC initialization from VM creation to vCPU pre-creation,
i.e. defer allocating the logical ID table and adding the VM to the GA Log
list until vCPUs are created. This will allow removing the VM from the GA
Log list before vCPUs are destroyed without needing yet another kvm_x86_ops
hook (.vm_pre_destroy() is very intentionally called if and only if VM
creation fully succeeds).
As a bonus, this re-unites physical and logic table allocation, and avoids
allocating a logical table in the unlikely scenario that userspace creates
a VM without an in-kernel local APIC.
Another bonus to hooking .vcpu_precreate() is that there is no need to
unwind on failure, as the VM has already been created, i.e. KVM will run
through all phases of VM destruction. In fact, unwinding is undesirable,
as KVM tries to keep VM-wide behavior idempotent/sticky across creaton of
multiple vCPUs.
Reviewed-by: Naveen N Rao (AMD) <naveen@kernel.org>
Link: https://patch.msgid.link/20260630210156.457151-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
In anticipation of deferring all per-VM AVIC initialization until a vCPU
is first created, move SVM's kvm_x86_ops.vcpu_precreate() hook into avic.c
as avic_vcpu_precreate() and nullify the hook if AVIC is disabled (and
WARN if the hook is somehow invoked without AVIC enabled).
Reviewed-by: Naveen N Rao (AMD) <naveen@kernel.org>
Link: https://patch.msgid.link/20260630210156.457151-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move "struct kvm_vcpu_hv" and all of its child structures to hyperv.h,
guarded by CONFIG_KVM_HYPERV=y, as "struct kvm_vcpu_arch" holds a pointer
to the structure, i.e. only needs the structure to be declared, not fully
defined.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move the definition of "struct kvm_apic_map", a.k.a. the optimized local
APIC map, to lapic.h, as it is very nearly an implementation details that's
internal to KVM's local APIC emulation (KVM also uses the map to do quick
lookups when a vCPU is yielding to a different vCPU).
No functional change intended.
Suggested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Relocate KVM's TASK_SWITCH_<reason> enums from kvm_host.h to x86.h, as the
enums are arbitrary values, i.e. not architectural, and are intended to be
used only to translate vendor specific information to a common x86 reason
when invoking kvm_task_switch().
Opportunistically name the overall enum to help document the role of the
values.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add static asserts to sanity check the I/O permission map and TSS size
macros against tss_segment_32. Alternatively, the macros could simply use
offsetof() and sizeof(), but having literal numbers makes it easier to
understand the bigger picture, and provides a good excuse for the sanity
checks.
Opportunistically add the necessary includes to make tss.h self sufficient.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move KVM_GUESTDBG_VALID_MASK into x86.c so that it's not globally visible.
As explained by commit 462474588b ("KVM: x86: Move misc "VALID MASK"
defines from kvm_host.h => x86.c"), which unintentionally missed GUESTDBG,
the set of valid flags/bits is very much a KVM-internal detail, as the
values from the hardcoded #defines are often captured and massaged by KVM's
setup code, i.e. *directly* using the macros outside of KVM x86 would be
actively dangerous.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Relocate a variety of Control/Debug Register macros that unintentionally
got left behind when the related helper function prototypes were moved to
regs.h.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Tag tdp_enabled with __read_mostly as the variable is only ever written
during vendor module load, same as all the other global MMU variables that
are handled by kvm_configure_mmu().
Opportunistically annotate the tdp_mmu_enabled and eager_page_split
declarations with __read_mostly, to match their definitions. The compiler
will warn if there are conflicting annotations, i.e. there's minimal risk
of the declaration annotation becoming stale.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move the macros that define the mostly-obsolete apic_attention bits into
lapic.c, as the gory details of PV EOIs and the pre-APICv TPR acceleration
are 100% internal to KVM's local APIC emulation.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When activating Hyper-V's Synthetic Interrupt Controller (SynIC), mark it
active with WRITE_ONCE() and query it using READ_ONCE() in synic_get(),
the only known cross-task reader, to document that the flag is accessed
without holding the vCPU's mutex.
Note, there are no data dependencies on the SynIC being marked active,
e.g. the vector read by synic_set_irq() is set (usually in response to
guest activity) long after the SynIC is initially activated, and a false
negative on the SynIC being active would be benign (ignoring that such a
race is likely to be problematic for the guest irrespective of what KVM
does).
Link: https://patch.msgid.link/20260630225619.511632-12-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Assert that either vcpu->mutex is held or the VM is otherwise unreachable
when using the normal vCPU => HyperV accessor to help detect improper
cross-task usage of the HyperV structure. When accessing the structure
without holding the vCPU's mutex, e.g. to send interrupts or to queue TLB
flushes, KVM needs to use the more paranoid to_hv_vcpu_safe() to guarantee
that it can't see a half-baked structure.
To avoid false positives, open code accesses to vcpu->arch.hyperv in the
Synthetic Timer callbacks (can be reached if and only if HyperV state is
fully initialized).
Link: https://patch.msgid.link/20260630225619.511632-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Extract nVMX's lockdep assertion that a vCPU is locked or otherwise
unreachable into a common helper, as KVM x86 is about to gain another user,
but there is nothing x86-specific about the logic, i.e. the assertion may
be useful for other architectures.
No functional change intended.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Explicitly invalidate KVM's internal Xen vCPU ID during vCPU creation
instead of *trying* to set the Xen ID to the vCPU index by default, and
forward singleshot timer hypercalls to userspace if the VMM hasn't set the
Xen ID via KVM_XEN_VCPU_ATTR_TYPE_VCPU_ID. Using the vCPU's index as its
default Xen ID is reasonable in concept, but in practice is horribly flawed
as the index is left as '0' until after vCPU initialization completes, i.e.
every vCPU gets a Xen ID of '0' by default.
Forward hypercalls to userspace instead of trying to salvage any kind of
default behavior, as all userspace implementations that support multiple
vCPUs either don't enable the timer, are guaranteed to set Xen ID, or work
only because *all* guests also screw up the singleshot timer hypercalls.
The last scenarios is extremely unlikely given that Linux-as-a-guest uses
the actual Xen vCPU ID when making timer hypercalls. In other words, for
all intents and purposes, KVM's ABI is already that userspace must set the
Xen vCPU ID, so just commit to that ABI.
Note, KVM's handling of KVM_XEN_VCPU_ATTR_TYPE_VCPU_ID restricts the ID to
KVM_MAX_VCPUS, so there's no chance of a valid ID colliding with U32_MAX.
Add a compile-time assertion to ensure this holds true in the future (KVM
doesn't care what value is used for "invalid", only that there can't be a
collision).
Link: https://lore.kernel.org/all/20260612233017.1F9771F000E9@smtp.kernel.org
Suggested-by: David Woodhouse <dwmw2@infradead.org>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260630225619.511632-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Hoist the checks on the Xen vCPU ID when handling set_singleshot_timer and
stop_singleshot_timer hypercalls out of their individual if-statements,
so that both checks on the ID are in common code. kvm_xen_hcall_vcpu_op()
is already doubly committed to handling only singleshot timer hypercalls,
and even if that were to change in the future, the function could simply
be renamed and turned into a helper specifically for timer hypercalls.
Opportunistically add a comment to explain why the check exists; the code
looks rather nonsensical without the knowledge that @vcpu_id is a common
param for all per-vCPU hypercalls.
No functional change intended.
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When handling Xen vCPU hypercalls, explicitly route non-singleshot-timer
commands to userspace, *before* checking if in-kernel emulation of the Xen
timer is enabled. Punting hypercalls that are never accelerated by KVM
because some other hypercall happens to be disabled is confusing and
actively dangerous, e.g. it's easy to miss that the only reason KVM can
bail early is because the timer-disabled case provides the same semantics
as the implicit "default" path in the switch-statement.
Opportunistically convert the switch-statement to an if-else-statement to
avoid having to carry code for an impossible "default" case.
For all intents and purposes, no functional change intended.
Link: https://patch.msgid.link/20260630225619.511632-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When initializing a vCPU's Hyper-V object, ensure the object is fully
initialized prior to exposing it through the vCPU, and ensure accesses from
other tasks (e.g. other vCPUs) see the fully initialized object if
vcpu->arch.hyperv is non-NULL.
Lack of ordering manifests as a lockdep splat due to attempting to lock a
TLB flush FIFO before the spinlock is initialized.
INFO: trying to register non-static key.
The code is fine but needs lockdep annotation, or maybe
you didn't initialize this object before use?
turning off the locking correctness validator.
CPU: 1 PID: 5005 Comm: syz-executor189 Not tainted 6.6.120-smp-DEV #1
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/18/2026
Call Trace:
<TASK>
[<ffffffff810dd10c>] dump_stack_lvl+0xcc/0x130 lib/dump_stack.c:106
[<ffffffff8192bddd>] assign_lock_key+0x1fd/0x230 kernel/locking/lockdep.c:977
[<ffffffff8191cb97>] register_lock_class+0x187/0x7a0 kernel/locking/lockdep.c:1291
[<ffffffff8191e7a9>] __lock_acquire+0x179/0x7650 kernel/locking/lockdep.c:5016
[<ffffffff8191e28f>] lock_acquire+0x13f/0x3d0 kernel/locking/lockdep.c:5756
[<ffffffff8101a65b>] __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
[<ffffffff8101a65b>] _raw_spin_lock+0x2b/0x40 kernel/locking/spinlock.c:154
[<ffffffff81319d44>] spin_lock include/linux/spinlock.h:351 [inline]
[<ffffffff81319d44>] hv_tlb_flush_enqueue+0xb4/0x270 arch/x86/kvm/hyperv.c:1946
[<ffffffff813160c6>] kvm_hv_flush_tlb+0xa96/0x1dc0 arch/x86/kvm/hyperv.c:2145
[<ffffffff8131438b>] kvm_hv_hypercall+0x103b/0x1fe0 arch/x86/kvm/hyperv.c:-1
[<ffffffff8133bff3>] __vmx_handle_exit arch/x86/kvm/vmx/vmx.c:6624 [inline]
[<ffffffff8133bff3>] vmx_handle_exit+0x12e3/0x21f0 arch/x86/kvm/vmx/vmx.c:6641
[<ffffffff81215d11>] vcpu_enter_guest arch/x86/kvm/x86.c:11649 [inline]
[<ffffffff81215d11>] vcpu_run+0x4d01/0x79c0 arch/x86/kvm/x86.c:11832
[<ffffffff8120fe39>] kvm_arch_vcpu_ioctl_run+0xb49/0x1c80 arch/x86/kvm/x86.c:12179
[<ffffffff8119cd60>] kvm_vcpu_ioctl+0xc80/0xff0 virt/kvm/kvm_main.c:6029
[<ffffffff8226fefd>] vfs_ioctl fs/ioctl.c:52 [inline]
[<ffffffff8226fefd>] __do_sys_ioctl fs/ioctl.c:872 [inline]
[<ffffffff8226fefd>] __se_sys_ioctl+0xfd/0x170 fs/ioctl.c:858
[<ffffffff85ac97d9>] do_syscall_x64 arch/x86/entry/common.c:52 [inline]
[<ffffffff85ac97d9>] do_syscall_64+0x69/0xb0 arch/x86/entry/common.c:93
[<ffffffff85c000d0>] entry_SYSCALL_64_after_hwframe+0x68/0xd2
</TASK>
Use the "safe" variant in all paths that are known to access the Hyper-V
object, as detected by an upcoming lockdep assertion, with an assist or two
from Sashiko.
Link: https://lore.kernel.org/all/20260612232258.0D9131F000E9@smtp.kernel.org
Fixes: 0823570f01 ("KVM: x86: hyper-v: Introduce TLB flush fifo")
Fixes: fc08b628d7 ("KVM: x86: hyper-v: Allocate Hyper-V context lazily")
Reported-by: syzbot+5b32c49cd8f005e65654@syzkaller.appspotmail.com
Reported-by: syzbot+5d2b94b77112148d1744@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a396a66.52ae72c2.136ac7.0002.GAE@google.com
Tested-by: syzbot+5d2b94b77112148d1744@syzkaller.appspotmail.com
Link: https://patch.msgid.link/20260630225619.511632-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Check for a NULL Hyper-V object in kvm_hv_get_tlb_flush_fifo() instead of
relying on the caller to do so. This will allow fixing a cross-vCPU race
where KVM can access a vCPU's FIFO before it's fully initialized, without
having to jump through too many cognitive hoops to reason about the
correctness of the logic.
Ignoring changes in ordering that only affect the aforementioned race, no
functional change intended.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When handling Hyper-V PV TLB flushes, retrieve the to-be-used FIFO in
hv_tlb_flush_enqueue() instead of having the caller pass in the FIFO. This
will make it easier to fix a cross-vCPU race where KVM can access a vCPU's
FIFO before it's fully initialized.
No functional change intended.
Link: https://patch.msgid.link/20260630225619.511632-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Micro-optimize emulation of MOV DR instructions by checking CR4.DE if and
only if DR4 or DR5 is being accessed.
No functional change intended.
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
WARN if ->set_dr() => kvm_set_dr() fails when emulating a MOV DR write,
as the emulator _must_ pre-check for #GPs in order to get the event
priority right when emulating MOV DR for L2 on SVM (all exceptions have
higher priority than the instruction intercept).
Opportunistically update the comment as the blurb about "#UD" being
checked is incomplete and misleading.
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Use kvm_dr{6,7}_valid() to validate the incoming DR{4,5,6,7} value in the
emulator instead of open coding an equivalent check. In the unlikely event
that the behavior of DR6/7 (and their aliases) changes in the future, using
common helpers will hopefully make it less likely the emulator logic will
be overlooked.
No functional change intended.
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When emulating a MOV DR on Intel with DR7.GD=1 at CPL>0, prioritize the #DB
due to DR7.GD over the #GP due to CPL>0, as empirical testing shows that
Intel CPUs (Skylake, Icelake and Emerald Rapids) prioritize the DR7.GD #DB
over all #GPs, whereas AMD CPUs prioritize the CPL>0 #GP (but not illegal
value #GPs) over the #DB.
Outside of the emulator, don't bother trying to provide the "correct"
priority based on the virtual CPU model, as it's simply impossible to do
so without intercepting *all* MOV DR accesses, which would result in a
massive, unacceptable performance hit. Note, getting the priority right
when advertising Intel on AMD would also require intercepting #GP, as SVM
prioritizes all exceptions over the instruction intercept.
Note, neither Intel's SDM nor AMD's APM says anything about the relative
priority, hence the empirical testing. Arguably Intel's description of
DR7.GD:
causes a debug exception to be generated prior to any MOV instruction
that accesses a debug register.
implies that DR7.GD has higher priority. But that's a fairly weak argument
as the statement would still hold true if the #GP due to CPL>0 had higher
priority, as the #GP would prevent any access to a DR.
Fixes: 3b88e41a41 ("KVM: SVM: Add intercept check for accessing dr registers")
Link: https://patch.msgid.link/20260612230113.684301-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Manually handle the CPL check for MOV DR instructions instead of using the
Priv flag, *after* checking for #UD scenarios, as #GP due to CPL>0 has
lower priority than all #UDs.
Fixes: 1e470be5a1 ("KVM: x86 emulator: fix mov dr to inject #UD when needed.")
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Manually (pre)check the values being written to DR4/5, i.e. the DR6/DR7
aliases, instead of relying on ->set_dr() => kvm_set_dr() to signal a #GP.
SVM unfortunately prioritizes all exceptions over an instruction intercept,
i.e. nSVM is relying on the emulator to perform *all* exception checks
prior to attempting to execute the instruction.
Fixes: 3b88e41a41 ("KVM: SVM: Add intercept check for accessing dr registers")
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When emulating a MOV DR, specifically a write to DR6 or DR7, treat a #DB
due to DR7.GD (General Detect) as higher priority than a #GP due to an
illegal value. While neither Intel's SDM nor AMD's APM says anything
about the relative priority, empirical testing on Intel and AMD shows that
the #DB has higher priority. And for VMX, where the instruction intercept
has priority over *all* exceptions, KVM already treats the #DB as having
higher priority.
Cc: Maciej W. Rozycki <macro@orcam.me.uk>
Fixes: 3b88e41a41 ("KVM: SVM: Add intercept check for accessing dr registers")
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When emulating a MOV to a debug register, em_dr_write() calls
@ctxt->ops->set_dr(), which is forwarded to emulator_set_dr() and
then kvm_set_dr(). The latter checks that the written value is valid,
otherwise returning an error, in which case the emulator is supposed to
inject a #GP fault into the guest.
Commit 996ff5429e ("KVM: x86: move kvm_inject_gp up from kvm_set_dr
to callers") changed the contract of kvm_set_dr() (and thus
emulator_set_dr()), returning 1 as an error instead of -1, but the
caller in em_dr_write() was never updated, checking only if the returned
value is negative. The end result is that em_dr_write() does not detect
the error, so an invalid write does not generate a #GP, but at the same
time the register value is not updated.
The practical impact is limited, as check_dr_write() already checks DR6
and DR7 manually. However, it misses DR4/DR5, which alias DR6/DR7 when
CR4.DE=0.
Fix the bug by treating any non-zero return from set_dr() as a reason to
inject #GP.
Note, the manual checks on DR6 and DR7 are flawed, as they incorrectly
prioritize the #GP over a DR7.GD=1 #DB (the General Detect #DB has
priority on both Intel and AMD).
Note #2, relying on ->set_dr() to detect #GP is also flawed as all
exceptions have higher priority than the instruction intercept on SVM,
i.e. the manual checks need to be extended to DR4 and DR5 (after the
priority bug is fixed).
Fixes: 996ff5429e ("KVM: x86: move kvm_inject_gp up from kvm_set_dr to callers")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260601133320.91479-2-clopez@suse.de
[sean: drop explicit "!= 0", massage changelog]
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Pass the CPUID index into cpuid_func_emulated() and return no emulated
features for indexed CPUID leaves with a non-zero index.
KVM currently emulates CPUID features only for index 0, but
kvm_vcpu_after_set_cpuid() looks up emulated features by function alone.
As a result, reverse_cpuid[] entries that share a function but use a
non-zero index, e.g. CPUID.7.1:ECX, can inherit emulated features that
belong to index 0. For example, RDPID, which is CPUID.7.0:ECX[22], can
be incorrectly OR'd into CPUID.7.1:ECX.
This is benign today because the affected bits do not correspond to
features KVM cares about, but it can become a real bug as new CPUID
features are defined. Make the helper index-aware so emulated features
are applied only to the CPUID entry they actually describe.
Fixes: e592ec657d ("KVM: x86: Initialize guest cpu_caps based on KVM support")
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Link: https://patch.msgid.link/20260609075748.612704-1-binbin.wu@linux.intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Return EINVAL instead of EOPNOTSUPP if userspace attempts to pass a NULL
pointer for the source page of INIT_MEM_REGION, so that KVM's ABI is
consistent between TDX and SNP (for LAUNCH_UPDATE). EOPNOTSUPP was chosen
to be a forward-looking error code for when guest_memfd supports in-place
conversion, but even when in-place conversion comes along, it's an awkward
error code as KVM is deliberately choosing to disallow virtual address '0',
which is technically a legal userspace address. I.e. it's not so much a
lack of support as it is that KVM reserves address '0' to simplify KVM's
internal implementation.
Opportunistically move the check so that it's co-located with the other
checks on the userspace address, and so that it's more obvious that a NULL
source address is explicitly disallowed.
Fixes: 2a62345b30 ("KVM: guest_memfd: GUP source pages prior to populating guest memory")
Cc: Yan Zhao <yan.y.zhao@intel.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Binbin Wu <binbin.wu@linxu.intel.com>
Reviewed-by: Yan Zhao <yan.y.zhao@intel.com>
Tested-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Link: https://patch.msgid.link/20260630213711.479692-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Explicitly reject a NULL userspace virtual address for the source page of
SNP_LAUNCH_UPDATE instead of relying on the post-populate callback to do
the check, and don't WARN on failure, as the scenario is blatantly user-
triggerable, as reported by Sashiko. Waiting until post-populate to check
the address "works", but makes it unnecessarily difficult to see that KVM's
ABI is to disallow a NULL source page for non-ZERO pages.
Note, several existing VMMs pass a valid userspace address for the ZERO
case, i.e. KVM can't *require* the userspace address to be NULL for ZERO
pages, at least not without breaking userspace.
Fixes: dee5a47cc7 ("KVM: SEV: Add KVM_SEV_SNP_LAUNCH_UPDATE command")
Reported-by: Sashiko Bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260611125849.9ED631F00893@smtp.kernel.org
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Link: https://patch.msgid.link/20260630213711.479692-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When emulating INVVPID, KVM executes INVVPID on the physical CPU using
vpid02 (instead of the L1 assigned VPID), after doing some validations
on the operands. However, it is possible that the physical CPU KVM
executes INVVPID on is different from the CPU L2 is running on.
For example, in the following scenario:
- L2 runs on CPU #1 and exits to L1 (vmx->nested.vmcs02.cpu=1)
- L1 migrates to CPU #2 and executes INVVPID
- KVM executes INVVPID on CPU #2
- L1 migrates back to CPU #1 and runs L2 (vmx->nested.vmcs02.cpu=1)
The TLB entries on CPU #1 are never invalidated, because INVVPID was
executed on CPU #2, and vmcs02 never ran on a different pCPU (i.e.
vmx_vcpu_load_vmcs() will *not* request KVM_REQ_TLB_FLUSH).
Ensure that INVVPID is being executed on the same pCPU that L2 last ran
on, and if not, fallback to clearing last_vpid=0 to trigger a full VPID
flush on the next nested VM-Enter (as KVM will detect L1 using a
different VPID for L2). If L2 ends up running on a different pCPU, KVM
will flush the TLB anyway through vmx_vcpu_load_vmcs().
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260616214652.2157032-4-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Separate the INVVPID operand checks from the actual flushing of vpid02 so
the flushing can be adjusted to do the right thing when vmcs02 was last
loaded on a different pCPU, without having to duplicate the logic across
multiple case-statements.
Opportunistically let the VM-Fail paths poke out past 80 chars.
No functional change intended.
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260616214652.2157032-3-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Make sure vpid02 is always flushed on first use by setting last_vpid=0
when allocating vpid02. nested_vmx_transition_tlb_flush() will always
detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
vmcs12 is not allowed if L1 enables VPID.
This avoids using stale TLB entries from a previous lifetime of the
VPID, that might have been associated with a different vCPU (or a
completely different VM).
Note that last_vpid is already being initialized as 0 when the vCPU is
created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
happens to reuse a VPID that has TLB entries on the physical CPU.
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260616214652.2157032-2-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a dedicated field in "struct nested_vmx" to track L1's pre-VM-Enter CR3
instead of using vmcs01.GUEST_CR3, which isn't anywhere near as safe as the
comment purports it to be. E.g. in addition to the warn_on_missed_cc bug
(that was fixed by relocating the consistency check), if getting vmcs12
pages (during actual nested VM-Entry) fails and EPT is disabled (in KVM),
KVM will return control to userspace with vmcs01.GUEST_CR3 holding a guest-
controlled value.
Alternatively, KVM could force a reload of vmcs01.GUEST_CR3 by resetting
the MMU context in the error path, but as above, the safety of the vmcs01
approach is extremely questionable, e.g. it took all of ~4 months for the
code to break.
Fixes: 671ddc700f ("KVM: nVMX: Don't leak L1 MMIO regions to L2")
Cc: stable@vger.kernel.org
Cc: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612145642.452392-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Move the off-by-default consistency check for vmcs12.tpr_threshold vs.
the virtual APIC vTPR into the "normal" controls checks, as waiting until
KVM has loaded some amount of state is unnecessary and actively dangerous.
Specifically, failure to unwind vmcs01.GUEST_CR3 to KVM's value when EPT
is disabled results in KVM running L1 with an L1-controlled CR3, not with
KVM's CR3!
Alternatively, KVM could simply reset the MMU to force a reload of
vmcs01.GUEST_CR3, but the _only_ reason the check was shoved into a "late"
flow was to wait until the vmcs12 pages were retrieved. Rather than build
up more crusty code, simply access vTPR using a regular guest memory access
(performance isn't a concern). To circumvent the restrictions that led to
KVM deferring nested_get_vmcs12_pages(), (a) use a VM-scoped API to read
guest memory so that it always hits non-SMM memslots (for RSM), and (b)
skip the check (since its off-by-default anyways) when the vCPU doesn't
want to run, i.e. when userspace is restoring/stuffing state.
If reading guest memory fails, simply skip the consistency check, as KVM's
de facto ABI is that VMX instruction accesses to non-existent memory get
PCI Bus Error semantics, where reads return 0xFFs. And if vTPR=0xFF, then
the vTPR is guaranteed to be greater than or equal to TPR_THRESHOLD.
Fixes: 1100e4910a ("KVM: nVMX: Add an off-by-default module param to WARN on missed consistency checks")
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260612145642.452392-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Nullify irqfd->producer if updating the IRTE for bypass fails, as leaving a
dangling pointer will result in a use-after-free if the irqfd is reachable
through KVM's routing, but the producer is freed separately. E.g. for VFIO
PCI, the producer is embedded in struct "vfio_pci_irq_ctx" and freed when
the vector is disabled, which can happen independent of routing updates.
Fixes: 77e1b8332d ("KVM: x86: Decouple device assignment from IRQ bypass")
Cc: stable@vger.kernel.org
Signed-off-by: leixiang <leixiang@kylinos.cn>
Link: https://patch.msgid.link/1782119051448443.14545.seg@mailgw.kylinos.cn
[sean: drop PPC change, massage changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
Branch Sampling (BRS) and Last Branch Record (LBR) are mutually
exclusive hardware features, and users of both are tracked via
cpuc->lbr_users.
When SVM is toggled on a CPU, the host perf events are reprogrammed to
update the HostOnly filter bit (set when virtualization is enabled,
cleared when it is disabled). On PerfMonV2-capable processors, this
reprogramming is performed by calling amd_pmu_enable_all() to rewrite
the event selectors. However, amd_pmu_enable_all() also calls
amd_brs_enable_all(), which enables BRS whenever cpuc->lbr_users > 0.
Having active LBR events satisfies this gating on processors that have
LBR but not BRS. The kernel then tries to set the BRS enable bit in
DebugExtnCfg (MSR 0xc000010f). Since that bit is deprecated on such
hardware, the write results in a #GP:
Call Trace:
<IRQ>
amd_pmu_enable_all+0x1d/0x90
amd_pmu_disable_virt+0x62/0xb0
kvm_arch_disable_virtualization_cpu+0xa/0x40 [kvm]
hardware_disable_nolock+0x1a/0x30 [kvm]
__flush_smp_call_function_queue+0x9b/0x410
__sysvec_call_function+0x18/0xc0
sysvec_call_function+0x69/0x90
</IRQ>
<TASK>
asm_sysvec_call_function+0x16/0x20
RIP: 0010:cpuidle_enter_state+0xc4/0x450
? cpuidle_enter_state+0xb7/0x450
cpuidle_enter+0x29/0x40
cpuidle_idle_call+0xf5/0x160
do_idle+0x7b/0xe0
cpu_startup_entry+0x26/0x30
start_secondary+0x115/0x140
secondary_startup_64_no_verify+0x194/0x19b
</TASK>
Fix this by ensuring that BRS is not enabled from the event selector
reprogramming path even when cpuc->lbr_users > 0.
Fixes: bae19fdd7e ("perf/x86/amd/core: Fix reloading events for SVM")
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/702fa204d574b03d14e3664c7d4b201db048bbfd.1783506528.git.sandipan.das@amd.com
If CONFIG_COMPAT_32BIT_TIME is disabled then the vDSO should not
provide any 32-bit time related functionality. This is the intended
effect of the kconfig option and also the fallback system calls would
also not be implemented.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Link: https://patch.msgid.link/20260702-vdso-compat_32bit_time-v3-3-db9f36d8d432@linutronix.de
The usage of 'bool' requires linux/types.h, which is currently only
included through a transitive dependency chain.
Include linux/types.h as that chain is going to go away.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260702-hrtimer-header-dependencies-v1-7-c50b19bda473@linutronix.de
Revert
99cf1fb58e ("x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN").
Section 8.8 of the SNP spec says:
Before invoking SNP_INIT_EX with INIT_RMP set to 1, software must ensure
that no CPUs contain dirty cache lines for the memory containing the RMP.
Cachelines can be moved from cache to cache in a dirty state. The
wbinvd_on_all_cpus() before SNP_INIT_EX flushes the caches for each CPU, but
if the IPIs for WBINVD race with this dirty cacheline movement, it is possible
that they may not get flushed, violating the firmware requirement.
Doing wbinvd_on_all_cpus() before setting SNPEn is safer since the RMP
table is not yet in use.
[ Heroically bisected by Srikanth. ]
[ bp: Massage commit message. ]
Fixes: 99cf1fb58e ("x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN")
Reported-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Tested-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: <stable@kernel.org>
Link: https://patch.msgid.link/20260707150033.2364758-1-tycho@kernel.org
Some multi GPU systems may have a VGA compatible device, but that might not be
used for display. If due to enumeration order this device is found before the
one actually used for display then multiple devices may show the boot_display
attribute, confusing userspace.
When screen info is valid, use it exclusively to find the primary device so
that only the device backing the framebuffer is reported. Only when no
framebuffer has been set up does it make sense to fall back to the default VGA
device. This ensures at most one primary graphics device, preferably the one
with the framebuffer.
Fixes: ad90860bd1 ("fbcon: Use screen info to find primary device")
Closes: https://lore.kernel.org/linux-pci/20260618081803.2790848-1-aaron.ma@canonical.com/#t
Reported-by: Aaron Ma <aaron.ma@canonical.com>
Suggested-by: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Thomas Zimmermann <tzimmermann@suse.de>
Tested-by: Aaron Ma <aaron.ma@canonical.com>
Cc: <stable@kernel.org>
Link: https://patch.msgid.link/20260623141505.1816786-1-mario.limonciello@amd.com
x86 asm/setup.h includes linux/kexec_handover.h. This is because it is
used by setup.c and kaslr.c. But this inclusion is problematic. The
header is included in many places, so it results in the KHO header being
propagated there. Also, the setup header is used by realmode code. If
KHO header includes things like mm.h, it causes a big dump of
compilation failures.
Nothing in setup.h uses anything from KHO. Remove the header from
setup.h, and directly include it in setup.c. which does use things from
KHO. Since kaslr.c is a part of the decompressor, avoid including linux
headers there directly. Instead, split out struct kho_scratch, which is
the only thing the kaslr.c uses, and move it to
include/asm-generic/kexec_handover.h.
This should also help reduce files recompiled when kexec_handover.h
changes.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20260706153751.1166003-1-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
cpu_feature_enabled() is the one to use to test feature flags so hide
the static thing which doesn't pay attention to disabled mask bits
anyway.
Use the following command to do the replacement:
$ git grep --files-with-matches -w static_cpu_has -- ':(exclude)*cpufeature.h' \
| xargs sed -i 's/static_cpu_has(/cpu_feature_enabled\(/g'
There should be no functional changes resulting from this.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Link: https://patch.msgid.link/20260620015041.336288-1-bp@kernel.org
CPUs when Intel SNC (Sub-NUMA Clustering) is enabled
(Reinette Chatre)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmpKEsURHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gcWhAApp+w5vUx0FXlou1Mzvn63bXey9/CMXyX
rrUeWA+ic7PpNvImuwixdyPyS1CnF2l7lMcmOtiNE4wnRvCf+0rsrwgYy4m4qd/2
aLAftFDp9WhKKH3Bb6gnVfMXiXmYb4eCQO0pRhJY26QtVJ2RGkbenW/TdTnAAAwS
KO8bJGsllufiF0+k4G5YMgiVcgZFByn/nuqmzvIc+oKQt5LSMvkKijoM7ozX9q5g
c1ABWvag3KWU2gFI00GfvzuQ7n7ckGhBVRwtO9DMox68liOlKOHedEFF0A0+KiDt
fk6b2LFbqm6hB7TpWRQqQ+rPusbDdAVxKG8ehIkjR8BD49VwzZUcr5kiWS5/xplm
a9F7cJpAJiuSiTi2HYtQVOlrCujonmjt11qqj3cCCjkton+IC9twhDkrj6+Kxygq
W3oc8Gia2JWoFBNt/6l0iKRJjluqgPJ8crPHpnDr4C5KEnsEIqCQsSPsVjoYM5mY
+xwS9jMJM7Dhrw3OZ6b5D8jY9oonBM/BB+jMomQ9ncDijUwKybgqp0JXB/3ydIw3
ci69tewFdVD+9EyNAz+dqANuhNyF1JTFza/5QxAU88E1wp2O2WjahSPvkNU/ekNK
jDGLf8VyiI08bH2pyYIhEhcvmTZ5Ocyuj9zA4Ufi2V3gaDMkvMXRipxm50dAhaWG
CMoZ+wTnI/8=
=mae/
-----END PGP SIGNATURE-----
Merge tag 'x86-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 fix from Ingo Molnar:
- Prevent OOB access in the resctrl code while offlining
CPUs when Intel SNC (Sub-NUMA Clustering) is enabled
(Reinette Chatre)
* tag 'x86-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86,fs/resctrl: Prevent out-of-bounds access while offlining CPU when SNC enabled
- Fix a perf_event_attr::remove_on_exec bug for group
events (Taeyang Lee)
- Fix uprobes CALL emulation interaction with shadow stacks,
and add a testcase for this (David Windsor)
- Fix uprobes unregister bug (Jiri Olsa)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmpKCX8RHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hwJRAAne1HniwuV9PV2GQHNKipeKV320tod2iR
WY9yy9ez5WJbPttS7Fy28NobQKxuTanNWnAJqqM3TJF0tPPSrrYqkhAGrpEb2ab0
G7gl4KlrO//5kKTZnnI86t/quSV5BDt00UUdMBFp+hbNYXNul5/AeUxqMhNoGqB6
DpJHrV+7kNQZ4I7tjatYWL11hZHpEhrx4QWLUsnd+nDwddpmdsRNXVZwpGh9+Dh4
+XXcgLD7M8FqNFN3GCVfhJKO8x8HRaBWv3FHeGqbCUL9k5viWcn+N91IbrcPVat1
HBk0JDtpLK286WHLFy7uROafCA59AlYp5DX7mobXi0VF1FdMqPtjaXEsZM7Ng/P5
/tpbV5P4irrUnMSCEDSDzqZJWXbcBSqCJ9p6z5/Tjzo3VegHyrXe29wjVoqvXjLx
og/9OnPZv/2QEsE37rBRwNC889ihFMUDZh3T+uUc1YKUEwYWFXwtUECTg+0Oi+lL
mLTdK05j/6NmVkcY77mFjQfTMFAZeD78g6cPY3yDHiHRFtxPOIhknCKBeLD5BjXD
tz08x3MN0ItVEBXfuKAlkb1KPhHy8x02IrpdfVojoQ7lUBRH/JqLvnSGgvVsRM+0
v9D3N9BAuB3s8DmxEdh0wyRzISl6jzBHJDJ83+uWSfZWLlEW+RVn9NbH2ubNU/jt
FO3U81r1MvE=
=Mb4z
-----END PGP SIGNATURE-----
Merge tag 'perf-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar:
- Fix a perf_event_attr::remove_on_exec bug for group events
(Taeyang Lee)
- Fix uprobes CALL emulation interaction with shadow stacks, and
add a testcase for this (David Windsor)
- Fix uprobes unregister bug (Jiri Olsa)
* tag 'perf-urgent-2026-07-05' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
uprobes/x86: Use proper mm_struct in __in_uprobe_trampoline
selftests/x86: Add shadow stack uprobe CALL test
x86/uprobes: Keep shadow stack in sync for emulated CALLs
perf/core: Detach event groups during remove_on_exec
The amd_uncore driver currently does not validate event groups and
allows creation of groups with more events than the number of available
hardware counters. Because of this, pmu->event_init() succeeds but
counter assignment fails later in pmu->add() which returns -EBUSY once
all counters are exhausted.
Address this by introducing group validation in the pmu->event_init()
path. Since the uncore PMUs have no per-event constraints and all
counters of a PMU are interchangeable, validation is reduced to just
counting the group members that target a PMU and ensuring that they fit
within the available set of counters.
Fixes: c43ca5091a ("perf/x86/amd: Add support for AMD NB and L2I "uncore" counters")
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/750877d66e208603c3047f13eed6399625d43969.1782884387.git.sandipan.das@amd.com
Fix all kernel-doc warnings in imr.c:
imr.c:300: warning: Function parameter or struct member 'rmask' not described in 'imr_add_range'
imr.c:300: warning: Function parameter or struct member 'wmask' not described in 'imr_add_range'
imr.c:300: warning: Excess function parameter 'read_mask' description in 'imr_add_range'
imr.c:300: warning: Excess function parameter 'write_mask' description in 'imr_add_range'
Fixes: 28a375df16 ("x86/intel/quark: Add Isolated Memory Regions for Quark X1000")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20250620065639.3348609-1-rdunlap@infradead.org
-----BEGIN PGP SIGNATURE-----
iJEEABYKADkWIQRTLbB6QfY48x44uB6AXGG7T9hjvgUCakeC1xsUgAAAAAAEAA5t
YW51MiwyLjUrMS4xMiwyLDIACgkQgFxhu0/YY74ZBQD/chB7mZrRK5FWdhtEYZuT
5is9FKdEe7P3rRChGuDxGZoBAISClrfxdh46C90D8jhjE3G3xWsN5DIajo1YaDbH
TroI
=wiih
-----END PGP SIGNATURE-----
Merge tag 'for-linus-7.2a-rc2-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip
Pull xen fixes from Juergen Gross:
- rename function parameters and a comment related to
xen_exchange_memory() (Jan Beulich)
- replace __ASSEMBLY__ with __ASSEMBLER__ (Thomas Huth)
- add some sanity checking to the Xen pvcalls frontend driver (Michael
Bommarito)
- fix error handling in the Xen gntdev driver (Wentao Liang)
- fix several minor bugs in Xen related drivers (Yousef Alhouseen)
* tag 'for-linus-7.2a-rc2-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip:
x86/Xen: correct commentary and parameter naming of xen_exchange_memory()
xenbus: reject unterminated directory replies
xen/gntalloc: validate grant count before allocation
xen/gntalloc: make grant counters unsigned
xen/front-pgdir-shbuf: free grant reference head on errors
xen/gntdev: fix error handling in ioctl
xen: Replace __ASSEMBLY__ with __ASSEMBLER__ in header files
xen/pvcalls: bound backend response req_id before indexing rsp[]
Andrii reported an issue with optimized uprobes [1] that can clobber
redzone area with call instruction storing return address on stack
where user code may keep temporary data without adjusting rsp.
Fixing this by moving the optimized uprobes on top of 10-bytes nop
instruction, so we can squeeze another instruction to escape the
redzone area before doing the call, like:
lea -0x80(%rsp), %rsp
call tramp
Note the lea instruction is used to adjust the rsp register without
changing the flags.
We use nop10 and following transformation to optimized instructions
above and back as suggested by Peterz [2].
Optimize path (int3_update_optimize):
1) Initial state after set_swbp() installed the uprobe:
cc 2e 0f 1f 84 00 00 00 00 00
From offset 0 this is INT3 followed by the tail of the original
10-byte NOP.
After a previous unoptimization bytes 5..9 may still contain the
old call instruction, which remains valid for threads already there.
2) Rewrite the LEA tail and call displacement:
cc [8d 64 24 80 e8 d0 d1 d2 d3]
From offset 0 this traps on the uprobe INT3. Bytes 1..9 are not
executable entry points while byte 0 is trapped.
3) Publish the first LEA byte:
[48] 8d 64 24 80 e8 d0 d1 d2 d3
From offset 0 this is:
lea -0x80(%rsp), %rsp
call <uprobe-trampoline>
Unoptimize path (int3_update_unoptimize):
1) Initial optimized state:
48 8d 64 24 80 e8 d0 d1 d2 d3
Same as 3) above.
2) Trap new entries before restoring the NOP bytes:
[cc] 8d 64 24 80 e8 d0 d1 d2 d3
From offset 0 this traps. A thread that had already executed the
LEA can still reach the intact CALL at offset 5.
3) Restore bytes 1..4 of the original NOP while keeping byte 0 trapped
and byte 5 as CALL.
cc [2e 0f 1f 84] e8 d0 d1 d2 d3
From offset 0 this still traps. Offset 5 is still the CALL for any
thread that was already past the first LEA byte.
4) Publish the first byte of the original NOP:
[66] 2e 0f 1f 84 e8 d0 d1 d2 d3
From offset 0 this is the restored 10-byte NOP; the CALL opcode and
displacement are now only NOP operands. Offset 5 still decodes as
CALL for a thread that was already there.
Tthere is only a single target uprobe-trampoline for the given nop10
instruction address, so the CALL instruction will not be changed across
unoptimization/optimization cycles.
Therefore, any task that is preempted at the CALL instruction is guaranteed
to observe that CALL and not anything else.
Note as explained in [2] we need to use following nop10:
PF1 PF2 ESC NOPL MOD SIB DISP32
NOP10: 0x66, 0x2e, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00 -- cs nopw 0x00000000(%rax,%rax,1)
which means we need to allow 0x2e prefix which maps to INAT_PFX_CS
attribute in is_prefix_bad function.
Also changing the uprobe syscall error when called out of uprobe
trampoline to -EPROTO, so we are able to detect the fixed kernel.
The optimized uprobe performance stays the same:
uprobe-nop : 3.129 ± 0.013M/s
uprobe-push : 3.045 ± 0.006M/s
uprobe-ret : 1.095 ± 0.004M/s
--> uprobe-nop10 : 7.170 ± 0.020M/s
uretprobe-nop : 2.143 ± 0.021M/s
uretprobe-push : 2.090 ± 0.000M/s
uretprobe-ret : 0.942 ± 0.000M/s
--> uretprobe-nop10: 3.381 ± 0.003M/s
usdt-nop : 3.245 ± 0.004M/s
--> usdt-nop10 : 7.256 ± 0.023M/s
[1] https://lore.kernel.org/bpf/20260509003146.976844-1-andrii@kernel.org/
[2] https://lore.kernel.org/bpf/20260518104306.GU3102624@noisy.programming.kicks-ass.net/#t
Closes: https://lore.kernel.org/bpf/20260509003146.976844-1-andrii@kernel.org/
Fixes: ba2bfc97b4 ("uprobes/x86: Add support to optimize uprobes")
Reported-by: Andrii Nakryiko <andrii@kernel.org>
Assisted-by: Codex:GPT-5.5
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260703114917.238144-6-jolsa@kernel.org
When we do fork or clone without CLONE_VM the new process won't
have uprobe trampoline vma objects and at the same time it will
have optimized code calling that trampoline and crash.
Fixing this by allowing vma uprobe trampoline objects to be copied
on fork to the new process.
Fixes: ba2bfc97b4 ("uprobes/x86: Add support to optimize uprobes")
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260703114917.238144-5-jolsa@kernel.org
In case the optimization fails, we leak new-ly created trampoline
vma mapping (in case we just created it), let's unmap it.
Fixes: ba2bfc97b4 ("uprobes/x86: Add support to optimize uprobes")
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260703114917.238144-4-jolsa@kernel.org
Removing struct uprobe_trampoline object and it's tracking code,
because it's not needed. We can do same thing directly on top of
struct vm_area_struct objects.
This makes the code simpler and allows easy propagation of the
trampoline vma object into child process in following change.
Note the original code called destroy_uprobe_trampoline if the
optimiation failed, but it only freed the struct uprobe_trampoline
object, not the vma. The new vma leak is fixed in following change.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://patch.msgid.link/20260703114917.238144-3-jolsa@kernel.org
The 32-bit MSR interface rdmsr() is planned to be removed. Use the
related 64-bit variant instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-29-jgross@suse.com
The 32-bit MSR interface rdmsr() is planned to be removed. Use the
related 64-bit variant instead.
This conversion also simplifies the code a bit.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: K. Y. Srinivasan <kys@microsoft.com>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Long Li <longli@microsoft.com>
Cc: Wei Liu <wei.liu@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-28-jgross@suse.com
numa_add_memblk() now sets the node in numa_nodes_parsed itself, so the
caller's own node_set() is redundant. Remove it.
No functional change.
Signed-off-by: Sang-Heon Jeon <ekffu200098@gmail.com>
Link: https://patch.msgid.link/20260703041329.2797584-5-ekffu200098@gmail.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
<linux/mod_devicetable.h> is included transitively in nearly every
driver in an x86_64 allmodconfig build of v7.1:
$ find drivers -name \*.o -not -name \*.mod.o | wc -l
21330
$ find drivers -name \*.o.cmd -not -name \*.mod.o.cmd | xargs grep -l mod_devicetable.h | wc -l
17038
The result of this mixture of different and unrelated subsystem details
is that even when touching an obscure device id struct most of the
kernel needs to be recompiled. Given that each driver typically only
needs one or two of these structures, splitting into per subsystem
headers and only including what is really needed reduces the amount of
needed recompilation.
This split is implemented in the first commit and then after some
preparatory work in the following commits, the last two replace
includes of <linux/mod_devicetable.h> by the actually needed more
specific headers. There are still a few instances left, but the ones
with high impact (that is in headers that are used a lot) and the easy
ones (.c files) are handled. These remaining includes will be addressed
during the next merge window.
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEP4GsaTp6HlmJrf7Tj4D7WH0S/k4FAmpHSvIACgkQj4D7WH0S
/k6Rdgf/QqxMIt7DEAN6KvyCnwVnNnvEWQhac/DtYA3NF1JaynieJIoFcimrRr/j
cntxjz2zlMhLH4CYZ87IunZ7nGZgkMlsnl/OVMm5NY0wq0vVYX6VmpiSvhMAgvjV
QxKHOAyuVqPslwVE8DF2Vh9mgJILviH5t/yIy8DHF85QBrDSjcd/n6csx09uTjIy
+D6NuxQ7ZeOi+O2wZuiDhYpD1kwgO1sXAg56ogN1OSTQoo1kkKqj66b5fgbPVm4/
MX3EBOjKIytX8csfWBJjNSB7aRyWVMkMKmmDO9+7MIq6tPRbc3yYwhu2kZBAw5Pt
mQfy1oyUi+0jwew4yKHeoVLyWAc4bg==
=YhFg
-----END PGP SIGNATURE-----
Merge tag 'device-id-rework' of git://git.kernel.org/pub/scm/linux/kernel/git/ukleinek/linux
Pull mod_devicetable.h header split from Uwe Kleine-König:
"Split <linux/mod_devicetable.h> in per subsystem headers
<linux/mod_devicetable.h> is included transitively in nearly every
driver in an x86_64 allmodconfig build of v7.1:
$ find drivers -name \*.o -not -name \*.mod.o | wc -l
21330
$ find drivers -name \*.o.cmd -not -name \*.mod.o.cmd | xargs grep -l mod_devicetable.h | wc -l
17038
The result of this mixture of different and unrelated subsystem
details is that even when touching an obscure device id struct most of
the kernel needs to be recompiled. Given that each driver typically
only needs one or two of these structures, splitting into per
subsystem headers and only including what is really needed reduces the
amount of needed recompilation.
This split is implemented in the first commit and then after some
preparatory work in the following commits, the last two replace
includes of <linux/mod_devicetable.h> by the actually needed more
specific headers.
There are still a few instances left, but the ones with high impact
(that is in headers that are used a lot) and the easy ones (.c files)
are handled. These remaining includes will be addressed during the
next merge window"
* tag 'device-id-rework' of git://git.kernel.org/pub/scm/linux/kernel/git/ukleinek/linux:
Replace <linux/mod_devicetable.h> by more specific <linux/device-id/*.h> (c files)
Replace <linux/mod_devicetable.h> by more specific <linux/device-id/*.h> (headers)
parisc: #include <linux/compiler.h> for unlikely() in <asm/ptrace.h>
media: em28xx: Add include for struct usb_device_id
LoongArch: KVM: Add include defining struct cpu_feature
ALSA: hda/core: Add include defining struct hda_device_id
usb: dwc2: Add include defining struct pci_device_id
platform/x86: int3472: Add include defining struct dmi_system_id
platform/x86: x86-android-tablets: Add include defining struct dmi_system_id
i2c: Let i2c-core.h include <linux/i2c.h>
of: Explicitly include <linux/types.h> and <linux/err.h>
platform/x86: msi-ec: Ensure dmi_system_id is defined
usb: serial: Include <linux/usb.h> in <linux/usb/serial.h>
driver core: platform: Include header for struct platform_device_id
driver: core: Include headers for acpi_device_id and of_device_id for struct device_driver
media: ti: vpe: #include <linux/platform_device.h> explicitly
mod_devicetable.h: Split into per subsystem headers
Replace the #include of <linux/mod_devicetable.h> by the more specific
<linux/device-id/*.h> where applicable. For most cases the include
can be dropped completely, only a few drivers need one or two headers
added.
Acked-by: Danilo Krummrich <dakr@kernel.org>
Acked-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://patch.msgid.link/1a3f2007c5c5dcf555c09a4035ce3ae8ef1b6c49.1782808461.git.u.kleine-koenig@baylibre.com
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
<linux/mod_devicetable.h> is included in a many files:
$ git grep '<linux/mod_devicetable.h>' ef0c9f75a1 | wc -l
1598
; some of them are widely used headers. To stop mixing up different and
unrelated driver( type)s let the subsystem headers only use the subset
of the recently split <linux/mod_devicetable.h> that are relevant for
them.
The fallout (I hope) is addressed in the previous commits that handle
sources relying on e.g. <linux/i2c.h> pulling in the full legacy header
and thus providing pci_device_id.
Acked-by: Danilo Krummrich <dakr@kernel.org>
Acked-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://patch.msgid.link/199fe46b624ba07fb9bd3e0cd6ff13757932cb5f.1782808461.git.u.kleine-koenig@baylibre.com
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
Call xbc_prepend_embedded_cmdline() in setup_arch() right after the
CONFIG_CMDLINE merge and before strscpy(command_line, ...) so the
build-time-rendered embedded bootconfig "kernel" subtree is part of
boot_command_line by the time parse_early_param() runs. early_param()
handlers (mem=, earlycon=, loglevel=, ...) now see values supplied via
CONFIG_BOOT_CONFIG_EMBED_FILE without parsing bootconfig at runtime.
Gate the prepend on the same opt-in the runtime parser uses: prepend
when "bootconfig" is present on the command line, or when
CONFIG_BOOT_CONFIG_FORCE is set. Detect it with parse_args(), exactly
as setup_boot_config() does, so both agree on what counts as opt-in:
any "bootconfig" key regardless of value (bare, =0, =1, ...), and only
before the "--" that separates init arguments. Sharing the parser keeps
the early and late paths from diverging -- e.g. "bootconfig=0" or a
"-- bootconfig" meant for init must not apply the embedded keys early
while the runtime parser skips them.
The prepend necessarily runs before setup_boot_config() detects an
initrd bootconfig, so an initrd cannot override the embedded "kernel"
keys for early_param(). This is intentional: the embedded cmdline acts
like a build-time CONFIG_CMDLINE. An initrd bootconfig's "kernel" keys
never reached early_param() anyway (they apply late via
extra_command_line), so nothing is lost -- the initrd keys still apply
late, with last-wins keeping the embedded values in effect.
Link: https://lore.kernel.org/all/20260626-bootconfig_using_tools-v7-7-24ab72139c29@debian.org/
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
In the unregister path we use __in_uprobe_trampoline check with
current->mm for the VMA lookup, which is wrong, because we are
in the tracer context, not the traced process.
Add mm_struct pointer argument to __in_uprobe_trampoline and
changing related callers to pass proper mm_struct pointer.
Fixes: ba2bfc97b4 ("uprobes/x86: Add support to optimize uprobes")
Reported-by: syzbot+61ce80689253f42e6d80@syzkaller.appspotmail.com
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: syzbot+61ce80689253f42e6d80@syzkaller.appspotmail.com
Link: https://patch.msgid.link/20260701111337.53943-2-jolsa@kernel.org
Uprobe CALL emulation updates the normal user stack, but not the CET user
shadow stack. The subsequent RET then sees a stale shadow stack entry and
raises #CP.
Update the relative CALL emulation and XOL CALL fixup paths to keep the
shadow stack in sync.
Fixes: 488af8ea71 ("x86/shstk: Wire in shadow stack interface")
Signed-off-by: David Windsor <dwindsor@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Jiri Olsa <jolsa@kernel.org>
Link: https://patch.msgid.link/8b5b1c7407b98f31664ad7b6a6faf20d2d4a6cad.1782777969.git.dwindsor@gmail.com
The 32-bit MSR interfaces rdmsr() and wrmsr() are planned to be
removed. Use the related 64-bit variants instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-20-jgross@suse.com
The 32-bit MSR interfaces rdmsr() and wrmsr() are planned to be
removed. Use the related 64-bit variants instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-19-jgross@suse.com
The 32-bit MSR interfaces rdmsr() and wrmsr() are planned to be
removed. Use the related 64-bit variants instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Acked-by: Reinette Chatre <reinette.chatre@intel.com>
Link: https://patch.msgid.link/20260629060526.3638272-18-jgross@suse.com
The 32-bit MSR interfaces rdmsr() and rdmsr_safe() are planned to be
removed. Use the related 64-bit variants instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-15-jgross@suse.com
The 32-bit MSR interfaces rdmsr(), wrmsr() and rdmsr_safe() are
planned to be removed. Use the related 64-bit variants instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-13-jgross@suse.com
The 32-bit MSR interface rdmsr_safe()is planned to be removed. Use the
related 64-bit variant instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Pu Wen <puwen@hygon.cn>
Link: https://patch.msgid.link/20260629060526.3638272-11-jgross@suse.com
The 32-bit MSR interfaces rdmsr(), wrmsr() and rdmsr_safe() are
planned to be removed. Use the related 64-bit variants instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260629060526.3638272-9-jgross@suse.com
The 32-bit MSR interfaces rdmsr(), wrmsr(), rdmsr_safe() and
wrmsr_safe() are planned to be removed. Use the related 64-bit variants
instead.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260629060526.3638272-7-jgross@suse.com
As documented in comments in struct xen_memory_exchange, the input to the
hypercall is a set of MFNs which are to be removed from the domain, plus a
set of PFNs where the newly allocated MFNs are to appear. Present comment
and parameter naming don't correctly reflect that.
Signed-off-by: Jan Beulich <jbeulich@suse.com>
Reviewed-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <7e0c8795-cc60-4b78-8601-6a999739467a@suse.com>
Commit
bc8e80d56c ("x86/mce: Merge mce_amd_inj into mce-inject")
removed the last .c file from arch/x86/ras/, so that there is now only
a lonely Kconfig file in this folder. Its config switches CONFIG_RAS_CEC and
CONFIG_RAS_CEC_DEBUG are only used in code that resides in drivers/ras/, so
those Kconfig switches should better reside in drivers/ras/, too. Thus move
the contents of arch/x86/ras/ into drivers/ras/Kconfig now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20260410122748.29978-1-thuth@redhat.com
It is not only used in the microcode loader anymore and the platform ID is
cached in the cpuinfo_x86 structure so move the getter to Intel CPU-specific
code.
No functional changes.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Link: https://lore.kernel.org/all/20260430020953.1405535-1-binbin.wu@linux.intel.com
The architecture updates the cpu_mask in a domain's header to track which
online CPUs are associated with the domain. When this mask becomes empty
the architecture initiates offline of the domain that includes calling
on resctrl fs to offline the domain. If it is a monitoring domain in
which LLC occupancy is tracked resctrl fs forces the limbo handler to
clear all busy RMID state associated with the domain.
The limbo handler always reads the current event value associated with a
busy RMID irrespective of it being checked as part of regular "is it still
busy" check or whether it will be forced released anyway. When reading an
RMID on a system with SNC enabled the "logical RMID" is converted to the
"physical RMID" and this conversion requires the NUMA node ID of the
resctrl monitoring domain that is in turn determined by querying the NUMA
node ID of any CPU belonging to the monitoring domain.
When the monitoring domain is going offline its cpu_mask is empty causing
the NUMA node ID query via cpu_to_node() to be done with "nr_cpu_ids" as
argument resulting in an out-of-bounds access.
Refactor the limbo handler to skip reading the RMID when the RMID will
just be forced to no longer be dirty in the domain anyway. Add a safety
check to the architecture's RMID reader to protect against this scenario.
Fixes: e13db55b5a ("x86/resctrl: Introduce snc_nodes_per_l3_cache")
Closes: https://sashiko.dev/#/patchset/cover.1780456704.git.reinette.chatre%40intel.com?part=9
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Cc: <stable@kernel.org>
Link: https://patch.msgid.link/16137433df42f85013b2f7a53626795cbd6637b9.1781029125.git.reinette.chatre@intel.com
Currently predictor flush on memory reuse is done for all BPF JIT
allocations, but only cBPF programs can be loaded by an unprivileged user.
eBPF is privileged by default, and flushing predictors for all CPUs on
every eBPF reuse penalizes the common case for no security benefit.
eBPF allocations can be frequent on busy systems, only flush predictors
for cBPF programs. Trampoline and dispatcher allocations also skip the
flush as they are eBPF-only.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Enable hardening against JIT spraying when Spectre-v2 mitigations are in
use. Specifically, issue an IBPB flush on BPF JIT memory reuse. Skip
enabling the IBPB flush if the BPF dispatcher is already using a retpoline
sequence.
This hardening applies only when BPF-JIT is in use. Guard the enabling
under CONFIG_BPF_JIT so that bugs.c still builds with CONFIG_BPF_JIT=n.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Adjust the indentation and use tabs between function parameters and
their descriptions to align with the convention used in FPU code.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20250620052152.490414-1-chao.gao@intel.com
Use the newly introduced crash_prepare_headers() function to replace
the existing prepare_elf_headers(), allocate cmem and exclude crash kernel
memory in the crash core, which reduce code duplication.
Only the following three architecture functions need to be implemented:
- arch_get_system_nr_ranges(). Call get_nr_ram_ranges_callback()
to pre-count the max number of memory ranges.
- arch_crash_populate_cmem(). Use prepare_elf64_ram_headers_callback()
to collect the memory ranges and fills them into cmem.
- arch_crash_exclude_ranges(). Exclude the low 1M for x86.
While at it, remove the unused "nr_mem_ranges" in
arch_crash_handle_hotplug_event().
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Vivek Goyal <vgoyal@redhat.com>
Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Sourabh Jain <sourabhjain@linux.ibm.com>
Acked-by: Baoquan He <bhe@redhat.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Link: https://patch.msgid.link/20260629094746.191843-6-ruanjinjie@huawei.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
On Sapphire Rapids (SPR) guests where pebs_baseline is not advertised,
running command
$ ./perf record -e cpu/event=0x00,umask=0x01,i\
name=INST_RETIRED.PREC_DIST/p -c 10000 sleep 1
can trigger:
unchecked MSR access error: WRMSR to 0x3f1 ... in\
intel_pmu_pebs_enable_all()
This occurs because SPR-specific PEBS constraints allow fixed-counter
scheduling (for example, INST_RETIRED.PREC_DIST on fixed counter 0).
In guests lacking pebs_baseline, KVM does not support PEBS sampling on
fixed counters, so enabling such events reaches an invalid MSR
programming path.
Starting with Icelake, regardless of whether Extended PEBS or
Architectural PEBS is in use, all counters (including fixed counters)
support PEBS sampling, and the PMU_FL_PEBS_ALL flag is set by default.
As long as PMU_FL_PEBS_ALL is set, constraint lookup automatically falls
back to the corresponding non-PEBS constraints if no matching entry is
found in the PEBS constraints table. Since non-PEBS event constraints
already contain the same fixed-counter constraints, it is safe to remove
these fixed-counter entries from the PEBS constraints table. The fallback
mechanism will ensure that fixed PEBS events are scheduled onto the
correct fixed counters.
So, directly drop the fixed-counter entries from the PEBS constraint
table. Without pebs_baseline, these fixed-counter PEBS events will now
resolve to empty constraints and will not be scheduled or enabled,
thereby avoiding the warning and bypassing the broken guest PEBS path.
Reported-by: Yi Lai <yi1.lai@intel.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-7-dapeng1.mi@linux.intel.com
The intel_pmu_init_hybrid() function allocates memory for the
x86_pmu.hybrid_pmu[] array. If this allocation fails under memory
pressure, hybrid PMU initialization will fail.
Currently, the caller does not check the return value of
intel_pmu_init_hybrid(). This can lead to a null-pointer dereference or
invalid memory access when attempting to use the uninitialized array,
potentially triggering a system panic.
Fix this by validating the return value of intel_pmu_init_hybrid().
Additionally, reset x86_pmu.num_hybrid_pmus to 0 on failure, and defer
intel_pmu_arch_lbr_init() until after hybrid PMU initialization
succeeds. This reordering avoids the need to explicitly destroy the
kmem cache if the memory allocation fails.
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-6-dapeng1.mi@linux.intel.com
Before Arch LBR gained CPL filtering support, a user-only branch stack
could still contain kernel addresses. As a result, kernel branch records
may be exposed to user space even when PERF_SAMPLE_BRANCH_USER is
requested.
For example, on Intel Tiger Lake, the following command can still report
SYSRET/ERET entries with kernel-space from addresses:
$ ./perf record -e cycles:p -o - --branch-filter any,save_type,u -- \
./perf bench syscall basic --loop 1000 | \
./perf script -i - --fields brstack|tr ' ' '\n'| \
grep -E '0x[89a-f][0-9a-f]{15}'
Total time: 0.000 [sec]
0.219000 usecs/op
4,566,210 ops/sec
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.551 MB - ]
0xffffffff93c001c8/0x7f12a2b1d647/P/-/-/16959/SYSRET/-
0xffffffff93c001c8/0x7f12a2b1d5c2/P/-/-/17535/SYSRET/-
0xffffffff93c01928/0x7f12a2861000/P/-/-/6719/ERET/-
0xffffffff93c01928/0x7f12a297a000/P/-/-/8575/ERET/-
The problem is that intel_pmu_lbr_filter() does not fully validate the
privilege level of sampled entries. It filters some mismatches based on
the branch type and the to address, but it does not reject entries whose
from address violates the requested branch privilege filter.
Fix this by extending software filtering to validate both from and to
addresses against br_sel. Any LBR entry contains kernel address does not
match the requested user filter is dropped. This prevents kernel
addresses from appearing in user-only branch stacks.
Fixes: 47125db27e ("perf/x86/intel/lbr: Support Architectural LBR")
Reported-by: Ian Rogers <irogers@google.com>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260616044654.3468742-5-dapeng1.mi@linux.intel.com
intel_pmu_lbr_filter() currently assumes arch-LBR provides hardware
branch-type decoding and skips software decoding on that path.
However, arch-LBR may not always expose branch-type information. In
that case, treating entries as hardware-decoded can misclassify sampled
branches (for example, defaulting to JCC), which breaks branch-type
filtering results.
Fix this by using software branch-type decoding when hardware
branch-type decoding is unavailable (that is, when x86_lbr_type is not
enabled). This keeps branch classification and filtering behavior
correct across arch-LBR configurations.
Since x86_lbr_type is only set to true when arch-LBR is supported,
it's unnecessary to check if the CPU has the arch-LBR feature.
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-4-dapeng1.mi@linux.intel.com
After introducing the RDPMC user disable feature, user-space RDPMC may
return 0 instead of the actual event count. This creates an inconsistency
with cap_user_rdpmc, where cap_user_rdpmc is set, but user-space RDPMC
only returns 0.
To accurately represent the user-space RDPMC capability, update
cap_user_rdpmc (depending on PERF_EVENT_FLAG_USER_READ_CNT) according to
the RDPMC user disable state. If RDPMC user disable is enabled,
cap_user_rdpmc is updated to false eventually, allowing user-space
programs to fall back to the read() syscall to obtain the real event
count.
Because PERF_EVENT_FLAG_USER_READ_CNT is evaluated in
x86_pmu_event_init(), move intel_pmu_update_rdpmc_user_disable()
earlier into intel_pmu_hw_config(). This ensures that the user-disable
state is updated before updating PERF_EVENT_FLAG_USER_READ_CNT. Note that
since event->ctx is not yet assigned at this stage, use the
PERF_ATTACH_TASK flag to detect whether the event is task-attached.
While at it, fix the indentation of x86_pmu_has_rdpmc_user_disable()
to adhere to the kernel coding style.
Fixes: 59af95e028 ("perf/x86/intel: Add support for rdpmc user disable feature")
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260616044654.3468742-3-dapeng1.mi@linux.intel.com
AnyThread mode deprecation is enumerated by CPUID.0AH:EDX[15] instead of
PERF_CAPABILITIES MSR. It's not a good practice to define a bit to
represent "anythread deprecation" in perf_capabilities. It leads to the
anythread_deprecated bit could be overwritten by the real value of
PERF_CAPABILITIES MSR, just like the below code in update_pmu_cap() does.
if (!intel_pmu_broken_perf_cap()) {
/* Perf Metric (Bit 15) and PEBS via PT (Bit 16) are hybrid enumeration */
rdmsrq(MSR_IA32_PERF_CAPABILITIES, hybrid(pmu, intel_cap).capabilities);
}
It leads to the anythread_deprecated bit is cleared to 0 and the "any"
attribute is incorrectly shown in the /sys/devices/cpu/format/ folder on
these support Perfmon v6 platforms, like Clearwater Forest.
$ grep . /sys/devices/cpu/format/*
/sys/devices/cpu/format/acr_mask:config2:0-63
/sys/devices/cpu/format/any:config:21
/sys/devices/cpu/format/cmask:config:24-31
So remove the anythread_deprecated bit from perf_capabilities structure
and directly depends on CPUID.0AH:EDX[15] to judge if anythread is
deprecated.
Fixes: cadbaa039b ("perf/x86/intel: Make anythread filter support conditional")
Reported-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Zide Chen <zide.chen@intel.com>
Reviewed-by: Thomas Falcon <thomas.falcon@intel.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260616044654.3468742-2-dapeng1.mi@linux.intel.com
MSR and MMIO uncore PMUs are currently registered at module init time
and appear in sysfs even when no PMU boxes are functional.
Apply the same lazy registration model used by PCI uncore PMUs: the
PMU is registered when the first box is successfully initialized, and
unregistered when the last box exits. If a box fails to initialize on
a subsequent die, the PMU is marked broken but remains registered to
avoid disrupting any in-flight perf events.
Box allocation and freeing remain at module init/exit time to avoid
repeated kfree/alloc cycles across CPU offline/online events.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-9-zide.chen@intel.com
In uncore_event_cpu_online(), uncore_box_ref() was called before
uncore_change_context(). uncore_box_ref() gates on box->cpu >= 0,
but box->cpu is still -1 at that point because uncore_change_context()
has not run yet. As a result, the box is never initialized on the
first CPU to come online in a die, leaving it permanently
uninitialized in the single-CPU-per-die case.
Thus, box->refcnt is one count below the true value, and in the CPU
offline path, the box will be torn down on the second-to-last CPU.
In uncore_event_cpu_offline(), uncore_box_unref() was called after
uncore_change_context(), so box->cpu is already -1 when the collector
CPU goes offline, which prevents it from tearing down the box.
Fix by swapping the call order in both paths so that
uncore_box_{ref,unref}() runs at the point where box->cpu reflects
the correct context.
Move allocate_boxes() out of uncore_box_ref() to enable this
reordering.
Fixes: c74443d92f ("perf/x86/uncore: Support per PMU cpumask")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-8-zide.chen@intel.com
Replace the boolean 'registered' field in intel_uncore_pmu with an
unsigned long 'flags' field, and add a PMU_BROKEN flag to track box
setup failures. The broken flag is sticky, meaning it is cleared only
by a module reload or system reboot.
Broken PMUs are skipped in the CPU hotplug and box allocation paths.
When any box fails to initialize, the PMU is marked broken. Broken
PMUs reject new event assignments and skip future box setup attempts.
If the PMU was already registered, it remains so to avoid disrupting
in-flight events on other boxes.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-7-zide.chen@intel.com
The PCI uncore PMU path already implements a lazy registration model:
the PMU is registered when the first active box appears and
unregistered when the last active box is removed.
Factor this registration management into a shared helper, so the same
code can be reused by the MSR and MMIO paths in later changes.
No functional change intended.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-6-zide.chen@intel.com
uncore_event_cpu_online() returns -ENOMEM early when both the MSR and
MMIO box allocations fail. This also aborts PCI uncore setup, even
though PCI PMUs are independent of the MSR/MMIO paths.
Remove the early return so PCI uncore setup always runs regardless
of whether MSR or MMIO box allocation succeeds.
Fixes: 3da04b8a00 ("perf/x86/intel/uncore: Support MMIO type uncore blocks")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-5-zide.chen@intel.com
The init_box() callback currently returns void, so initialization
failures are silently ignored and the box is still marked initialized.
Change the callback to return int so platform code can report errors
back to the common uncore layer.
Update uncore_box_init() to set the initialized flag only when
init_box() succeeds. Because box->refcnt guarantees that at most
one CPU calls uncore_box_init() for a given box at a time, plain
__set_bit() is safe for the initialized flag without atomic overhead.
Convert all init_box() implementations to return 0 on success or a
negative error code on failure. This is a prerequisite for propagating
initialization errors to the caller so they can be handled properly.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-4-zide.chen@intel.com
Fix typo UNCORE_BOX_FLAG_INITIATED to UNCORE_BOX_FLAG_INITIALIZED.
Rename the 'id' parameter in uncore_box_{ref,unref}() to 'die' to
reflect its actual meaning and be consistent with other functions.
box->refcnt is incremented in the PCI PMU register path but has never
been checked or decremented. Although for PCI PMUs box->refcnt
effectively tracks only a single user, add atomic_dec_return() in the
PCI PMU unregister path to make the reference counting complete and
consistent.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260611160033.66760-3-zide.chen@intel.com
When uncore_pci_pmu_register() fails, pmu->boxes[die] is set to NULL
before returning. In the uncore_pci_remove() path, this causes
uncore_pci_pmu_unregister() to be skipped entirely, leaking
pmu->activeboxes. In the uncore_bus_notify() path,
uncore_pci_pmu_unregister() may still be called and must exit early
when pmu->boxes[die] is NULL to avoid a NULL pointer dereference, and
to ensure activeboxes is only decremented for a previously active box.
Additionally, since pci_get_drvdata() returns NULL on registration
failure, uncore_pci_remove() can no longer treat NULL drvdata as an
indicator of an auxiliary PCI device. Remove the associated
WARN_ON_ONCE().
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://sashiko.dev/#/patchset/20260512233048.9577-1-zide.chen@intel.com?part=1
Link: https://patch.msgid.link/20260611160033.66760-2-zide.chen@intel.com
Commit
ccace936ee ("x86: Add types to indirectly called assembly functions")
introduced a x86-specific implementation of __CFI_TYPE() using an asymmetric
combination of SYM_START() and SYM_FUNC_END() to add a symbol to the KCFI type
identifier that precedes a function.
This asymmetric combination is an issue if SYM_FUNC_END() ever gets extended
in a way that requires it to be used symmetrically with SYM_FUNC_START*().
For instance to emit DWARF CFI directives that denote the start/end of
a function. [1]
Use SYM_END() with SYM_T_FUNC instead. No functional change, as the generic
implementation of SYM_FUNC_END(name) expands into SYM_END(name, SYM_T_FUNC).
Fixes: ccace936ee ("x86: Add types to indirectly called assembly functions")
Closes: https://sashiko.dev/#/patchset/20260522110427.2816637-1-jremus@linux.ibm.com?part=3 [1]
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260611155716.830563-1-jremus@linux.ibm.com
* Fix S390_USER_OPEREXEC so it can now be enabled regardless of other
unrelated capabilities
* Fix handling of the _PAGE_UNUSED pte bit that could lead to guest
memory corruption in some scenarios
* A bunch of misc gmap fixes (locking, behaviour under memory pressure)
* Fix CMMA dirty tracking
x86:
* Tidy up some WARN_ON() and BUG_ON(), replacing them with WARN_ON_ONCE()
or KVM_BUG_ON(). All of these have obviously never triggered, or
somebody would have been annoyed earlier, but still.
* Fix missing interrupt due to stale CR8 intercept
* Add a statistic that can come in handy to debug leaks as well as the
vulnerability to a class of recently-discovered issues.
* Do not ask arch/x86/kernel to export default_cpu_present_to_apicid()
just for KVM.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmo7xaAUHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroN7bwf/Ua6PrRrgdzfUGiIruBDnFGtXnaKf
UPGpgl35Kl4ntPlmH6wRG6N0jP9sloa17Qj/Y33O1Um3Mi/eDV0UESW8o8tfFE2J
CWUPc3/adw2urhnkXsoleSFy21a89TgLd0p4tot832p+IHi3iNJ45fx7XxftBeGt
fa/liscncEX5tGwie0ZVkIEE/ob3+eX4ZYsTLXMi6FxAlyGPYNrp4jwJeYH9AFSd
ZYGqVbDjDq14vhnppKXM8DMfHTXN4OVYw2RJZ9Y7u2h7ku3sN0XakzVTAUz5gtoM
h+iu8dn5yslODBZiNGlls8UYm2bweOBCumH1ZITe/eTNuEdUOJGILorKRw==
=QzfV
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm fixes from Paolo Bonzini:
"s390:
- Fix S390_USER_OPEREXEC so it can now be enabled regardless of other
unrelated capabilities
- Fix handling of the _PAGE_UNUSED pte bit that could lead to guest
memory corruption in some scenarios
- A bunch of misc gmap fixes (locking, behaviour under memory
pressure)
- Fix CMMA dirty tracking
x86:
- Tidy up some WARN_ON() and BUG_ON(), replacing them with
WARN_ON_ONCE() or KVM_BUG_ON(). All of these have obviously never
triggered, or somebody would have been annoyed earlier, but still...
- Fix missing interrupt due to stale CR8 intercept
- Add a statistic that can come in handy to debug leaks as well as
the vulnerability to a class of recently-discovered issues
- Do not ask arch/x86/kernel to export
default_cpu_present_to_apicid() just for KVM"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (22 commits)
x86/apic: KVM: Use cpu_physical_id() to get APIC ID of running vCPU for AVIC
KVM: x86/mmu: Expose number of shadow MMU shadow pages as a stat
KVM: x86: Unconditionally recompute CR8 intercept on PPR update
KVM: VMX: Grab vmcs12 on CR8 interception update iff vCPU is in guest mode
KVM: x86: WARN (once) if RTC pending EOI tracking goes off the rails
KVM: x86: WARN and fail kvm_set_irq() if a PIC or I/O APIC vector is invalid
KVM: x86: Bug the VM, not the kernel, if the ISR count {under,over}flows
KVM: x86/mmu: Bug the VM, not the host kernel, if KVM write-protects upper SPTEs
KVM: x86: Replace BUG_ON() with WARN_ON_ONCE() on "bad" nested GPA translation
KVM: Replace guest-triggerable BUG_ON() in ioeventfd datamatch with get_unaligned()
KVM: s390: Return failure in case of failure in kvm_s390_set_cmma_bits()
KVM: s390: selftests: Fix cmma selftest
KVM: s390: Fix cmma dirty tracking
KVM: s390: Fix locking in kvm_s390_set_mem_control()
KVM: s390: Fix handle_{sske,pfmf} under memory pressure
KVM: s390: Fix code typo in gmap_protect_asce_top_level()
KVM: s390: Do not set special large pages dirty
KVM: s390: Fix dat_peek_cmma() overflow
s390/mm: Fix handling of _PAGE_UNUSED pte bit
KVM: s390: Fix typo in UCONTROL documentation
...
- fix locking context with shared GPIOs in gpio-tegra
- fix IRQ domain leak in error path in gpio-davinci
- fix returning a potentially uninitialized integer in
gpiochip_set_multiple()
- use raw spinlock in gpio-eic-sprd and gpio-sch to address locking
context issues
- bail out of probe() if registering the GPIO chip fails in gpio-mlxbf3
- fix varible type for storing the "ngpios" property in gpio-pisosr
- fix out-of-bounds pin access in GPIO ACPI
- make GPIO ACPI core only trigger interrupts on boot that are marked as
ActiveBoth
- fix kerneldoc in gpio-tb10x
- reference the real software node of the cs5535 GPIO controller in
Geode board file
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEkeUTLeW1Rh17omX8BZ0uy/82hMMFAmo86vUACgkQBZ0uy/82
hMNNWRAAmZmRGrQtGsl50IOsZBgdONYapPftkk36rAJ7JKfxIAnO1ebcU+GjHve5
0F1D12Oz8mhrPnpGXm9im5msrwzVJtO6BGiQUqz8YvoajTDEqzCcOkDPNMbkseob
3L/XEzsz6MWtc4JvEyx6sFLCsIr2z/uzitX8nq+9KGkeC1PCEQ7zROLuoTxGcMQe
33nftgrJ2Ch+NP7QYDayI7imB+y7j01jQgfNuhsdwcqPam9CAJYX/hBkke8WyaIm
c84u9fXEto+u3CIgK3zxLDkzodSDWw+8zPtwEYV7QgyAZyp7rXEEkbOJLxQCMiGK
puJUDLCt+3jCAp6GjORewrPysReR27zn0KGaL70OKObKvIJ4tDEHnPI/GdTsUEOh
n9/bZkF8eXOfT3RQV5ezdQgLrt0+/J3lM44nVkx3bno2OXumXXIW7+8frvhFy9TB
C5Pn638XUPAPdlHiZlazLbJzQnchNPqH3wu0Hfcwrid+43byrb/lidtT6e6cmezl
l+AdqV3ppZcVJlW6ORLvIk1UT7JNsE7+DHL8/8XWLsLZrtjG8aotVMBpxAqgsl1s
adbJ3q96C4P27Zz93D7qyR/zopqRBa2BA+F3j8PDQOzuSO0Ps3y3kyIJH9OCEnLY
ztyyuMH3tXFkkq9zStbe66VlMpEMMW8YZyHx6jIyc7/gVYnK4dk=
=53+g
-----END PGP SIGNATURE-----
Merge tag 'gpio-fixes-for-v7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/brgl/linux
Pull gpio fixes from Bartosz Golaszewski:
- fix locking context with shared GPIOs in gpio-tegra
- fix IRQ domain leak in error path in gpio-davinci
- fix returning a potentially uninitialized integer in
gpiochip_set_multiple()
- use raw spinlock in gpio-eic-sprd and gpio-sch to address locking
context issues
- bail out of probe() if registering the GPIO chip fails in gpio-mlxbf3
- fix varible type for storing the "ngpios" property in gpio-pisosr
- fix out-of-bounds pin access in GPIO ACPI
- make GPIO ACPI core only trigger interrupts on boot that are marked
as ActiveBoth
- fix kerneldoc in gpio-tb10x
- reference the real software node of the cs5535 GPIO controller in
Geode board file
* tag 'gpio-fixes-for-v7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/brgl/linux:
gpio: davinci: fix IRQ domain leak on devm_kzalloc failure
gpio: tegra: do not call pinctrl for GPIO direction
gpio: tb10x: fix struct tb10x_gpio kernel-doc
gpiolib: initialize return value in gpiochip_set_multiple()
x86/platform/geode: reference the real node of the cs5535 GPIO controller
gpio: eic-sprd: use raw_spinlock_t in the irq startup path
gpio: sch: use raw_spinlock_t in the irq startup path
gpiolib: acpi: Prevent out-of-bounds pin access in OperationRegion handler
gpiolib: acpi: Add robust bounds-checking for GPIO pin resources
gpio: mlxbf3: fail probe if gpiochip registration fails
gpio: pisosr: Read "ngpios" as u32
gpiolib: acpi: Only trigger ActiveBoth interrupts on boot
is_access_allowed(), and is_executable_pte() within it, are effectively a
special version of permission_fault() that only supports a subset of roles.
In particular it does not allow SMEP, SMAP and PKE; while SMAP and PKE are
not a problem (they are not supported by EPT/NPT and is_access_allowed() is
only used for either EPT/NPT or CR0.PG=0), lack of ring-0 execution checks
support means that KVM can only use MBEC and GMET with shadow paging.
Replace is_access_allowed() with a modified version of permission_fault(),
looking at the same bitmasks but using the struct kvm_mmu's fmt member;
the new version supports MBEC/GMET for free, just by virtue of filling
in the ACC_* masks from the SPTE.
This prepares for a possible future where TDP entries could have XS!=XU,
for example as part of implementing Hyper-V VSM natively inside KVM.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Make it possible to apply the computation loop to both guest
and shadow PTEs formats; the latter do not have an extended role, so
pass the four parameters to the function one by one.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This remove one level of indirection, and adds the data for the permission
bitmask machinery to struct kvm_mmu. This way, it will be possible to
reuse the permission bitmasks for SPTEs as well.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Structs kvm_mmu ("build page tables") and kvm_pagewalk ("walk page
tables") share a piece of common functionality, namely "looking at PTEs".
Both of them have code to check is a specific access (described by
PFERR_* constants) is allowed by a PTE, and both of them also validate
that reserved bits are zero on the respective page tables page tables;
for SPTEs the code is only there to check internal consistency, but in
this case it is indeed shared via struct rsvd_bits_validate.
In preparation for sharing more PTE parsing code between struct
kvm_pagewalk and struct kvm_mmu, create a new struct that contains all
precalculated tables, including the data that is extracted from the
CPU role. For now only struct kvm_pagewalk uses it.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that the GVA->GPA page walker is initialized independently,
init_kvm_softmmu() does not do anything more than calling
kvm_init_shadow_mmu() so eliminate it from the call chain.
At the same time, rename kvm_init_shadow_mmu() to
init_kvm_shadow_mmu() for consistency with init_kvm_tdp_mmu().
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
At this point, vcpu->arch.ngva_walk and vcpu->arch.root_gva_walk contain
the same information; compare init_kvm_page_walk() on one side with
init_kvm_softmmu() + shadow_mmu_init_context() on the other. They only
differ in when each is active, and root_gva_walk is also used by shadow
paging, via FNAME(walk_addr) and its callers.
Always use the same instance of kvm_pagewalk to do GVA->GPA translations,
for both guest emulation and shadow paging, instead of flipping the
gva_walk pointer back and forth. After all the page walking does behave
the same no matter if you are in guest mode or not; the difference lies
in the behavior of kvm_translate_gpa and thus in vcpu->arch.mmu, not in
the page walker itself.
This completes the transition from walk_mmu/nested_mmu as the page
walking entry points to gva_walk/ngpa_walk, and removes duplicated
code between the initialization of root_mmu.w and ngva_walk (The
Struct Formerly Known As nested_mmu).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Replace kvm_mmu's w field with a pointer to an external instance of
struct kvm_pagewalk. This is the first step towards using a single
kvm_pagewalk struct for all GVA walks, whether nested or not.
With this patch, non-MMU code basically does not use kvm_mmu anymore:
it does care about page walks, but it funnels (almost) all interactions
with the TLB to mmu.c.
kvm_mmu_invalidate_addr() still needs to go from kvm_pagewalk to kvm_mmu,
and it cannot anymore use container_of() to do it; the mapping is hardcoded
based on the provided kvm_pagewalk, since there are just three of them.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
nested_mmu is now only used for its w member. While there is still a
single container_of() going from (possibly) gva_walk to its containing
struct kvm_mmu, it is never reached for nested_mmu and therefore it is
safe to strip nested_mmu.w out of its containing struct kvm_mmu.
So do it, and rename it following the model of gva_walk itself.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Now that walk_mmu is only accessed for its "w" member, store
directly the pointer to it. Since it is only used to convert
guest GVAs or nGVAs, call it gva_walk.
Note that there is still one container_of() going from (possibly)
walk_mmu.w to its containing struct kvm_mmu, but for now all instances
of struct kvm_pagewalk do live within a kvm_mmu.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
kvm_mmu_invalidate_addr()'s callers only want to tell it whether to
invalidate a GVA or GPA. This will ultimately be represented by two
different kvm_pagewalk structs, so adjust the type of the parameter.
As of this patch, the GVA case is represented by both root_mmu.w and
nested_mmu.w. Since nested_mmu never has a sync_spte callback, it would
exit at its check, but really nested_mmu should not be a kvm_mmu in the
first place: it is only used as the walk_mmu, and walk_mmu is only used
for its struct kvm_pagewalk member.
Since calling container_of() on the nested_mmu would be bogus after it is
turned into a struct kvm_pagewalk, introduce a separate check to check
if no work is needed beyond kvm_x86_call(flush_tlb_gva). Implement it
so that it is as similar as possible to what was happening until now; that
is, do nothing if the invalidation is happening for a nested GVA.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
As promised, this removes the remaining instances of
container_of(w, struct kvm_mmu, w), meaning that struct
kvm_pagewalk's definition is pretty much complete.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
With this change, walk_addr_generic and its callees do not need to use
container_of() anymore. There are only two remaining occurrences of
container_of, in permission_fault() and kvm_mmu_refresh_passthrough_bits().
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
struct kvm_pagewalk's behavior depends on the CPU state and its
page format. Move related fields so that walk_mmu remains
self contained.
Note that for now, some of the accessors still use kvm_mmu
to split the churn.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Injection of page faults is also part of accesses to guest
page tables; in particular, __kvm_inject_emulated_page_fault()
calls it on walk_mmu. Move it to struct kvm_pagewalk as
part of converting walk_mmu to a struct kvm_pagewalk.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Continue with yet another callback used in FNAME(walk_addr_generic),
as another step towards removing container_of() from there.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
gva_to_gpa is the main entry point into walk_mmu, which
is only used for guest page table walking (as opposed to building
the page tables). Moving gva_to_gpa to struct kvm_pagewalk
is a step towards making walk_mmu a struct kvm_pagewalk, and
removes several uses of struct kvm_mmu in x86.c.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Start moving page walking functionality out of kvm_mmu; the easiest
target is the callbacks.
Change the kvm_mmu_get_guest_pgd() wrapper to take a struct kvm_pagewalk
too, avoiding the MMU indirection (and associated container_of) whenever
the caller already has one. All container_of uses need to go before
nested_mmu can be changed to a struct kvm_pagewalk, signifying that it
is not used to build page tables.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
In preparation for separating walking and building of page tables,
introduce a dummy struct kvm_pagewalk and pass it around instead of
its containing kvm_mmu to functions that do not build the page tables.
Outermost functions retrieve the mmu via container_of, while internal
functions can pass around the struct kvm_pagewalk pointer.
x86.c is still mostly oblivious to the existence of struct kvm_pagewalk,
with are only a couple exceptions for now, but the plan is for it to
use struct kvm_pagewalk whenever dealing with guest page tables and have
only limited knowledge of struct kvm_mmu.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Just always go through kvm_translate_gpa(), which will either invoke
the vendor check or just return hc->ingpa back.
This is a better way to fix the issue of commit 464af6fc2b ("KVM:
x86: check for nEPT/nNPT in slow flush hypercalls", 2026-05-03).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the majority of remaining KVM-internal declarations and defines in
kvm_host.h to x86.h, so that kvm_host.h only holds structure and function
definitions that need to be visible to arch-neutral KVM.
Land the emulator interfaces in x86.h, even though kvm_emulate.h *seems*
like a good home, as the interfaces and defines being moved are provided by
x86.c. I.e. keep kvm_emulate.h as an interface to the emulator proper.
Note, any "misses" are likely unintentional.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260613000329.732085-31-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move kvm_mmu_do_page_fault() into mmu.c, as there are no users outside of
mmu.c, and the function typically isn't inlined by the compiler anyways.
This will allow moving the EMULTYPE_xxx definitions into x86.h without
having to include x86.h in mmu_internal.h, i.e. will help preserve the
goal of making x86.h KVM x86's "top-level" include.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-30-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the implementation of kvm_arch_async_page_ready() "down" in mmu.c so
that it lives below kvm_tdp_page_fault(). This will allow moving
kvm_mmu_do_page_fault() into mmu.c without needing a forward declaration.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-29-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Rename kvm_arch_interrupt_allowed() to kvm_is_interrupt_allowed() and
change its return type to a boolean, as the purpose of the helper is purely
to check if interrupts are architecturally allowed. I.e. whether or not
interrupts are temporarily disallowed due a pending nested VM-Enter is
irrelevant (and callers are most definitely not supposed to care).
Opportunistically bury the helper in x86.c, as it hasn't been referenced by
arch-neutral code since commit a1b37100d9 ("KVM: Reduce runnability
interface with arch support code"), and has long since gained _very_
x86-specific semantics (see above).
Opportunistically add a comment to call out that treating -EBUSY as
"allowed" is intentional.
For all intents and purposes, no functional change intended (KVM treats
-EBUSY as "allowed" before and after).
Cc: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260613000329.732085-28-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When querying whether or not interrupts (IRQs) are allowed, check for a
pending nested run _after_ checking whether or not interrupts are blocked.
If L1 is running L2 _without_ nested_exit_on_intr(), i.e. if L1 IRQs can
be blocked while running L2, and interrupts will indeed be blocked once the
nested VM-Enter to L2 is completed, then KVM should treat interrupts as not
being allowed.
For injection, this avoids an unnecessary (forced) VM-Exit, as KVM can
immediately request an IRQ window, instead of forcing an exit and _then_
requesting an IRQ window (because after the forced exit, KVM will see that
interrupts are blocked).
For non-injection usage, only kvm_vcpu_ready_for_interrupt_injection() is
affected in practice. Barring KVM bugs or misbehaving userspace (at which
point all architectural guarantees are off), kvm_vcpu_has_events() is
unreachable when a nested run is pending. To reach kvm_vcpu_has_events(),
kvm_vcpu_running() needs to return false, i.e. vcpu->arch.mp_state needs
to be something other than RUNNABLE. If nested_run_pending is true, then
mp_state *must* be RUNNABLE (again barring bugs or stupid userspace),
because KVM shouldn't emulate VMRUN/VMLAUNCH/VMRESUME while the vCPU is
!RUNNABLE.
The one "near miss" is VMX's GUEST_ACTIVITY_STATE field, which allows L1 to
put the vCPU into HLT or WFS as part of nested VMLAUNCH/VMRESUME. However,
KVM clears nested_run_pending prior to calling kvm_emulate_halt_noskip()
when putting L2 into HLT via GUEST_ACTIVITY_HLT, and also clears the flag
before setting mp_state to INIT_RECEIVED. SVM has no equivalent to
GUEST_ACTIVITY_STATE.
I.e. the vCPU will always be runnable if a nested run is pending, and thus
kvm_arch_vcpu_runnable() => kvm_vcpu_has_events() is effectively dead code,
as is __kvm_emulate_halt() => kvm_vcpu_has_events(). Oh, and TDX doesn't
support nested VMX. Similarly, kvm_can_do_async_pf() is unreachable as
KVM shouldn't be faulting in memory with a pending nested VM-Enter.
As for kvm_vcpu_ready_for_interrupt_injection(), KVM's current behavior of
incorrectly treating interrupts as being allowed could result in KVM
prematurely exiting to userspace to accept an ExtINT. But, KVM will still
hold/block the ExtINT and request its own IRQ window. I.e. the net effect
is more or less the same as the for-injection case, the unnecessary exit
just happens at a different boundary.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260613000329.732085-27-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the declaration of kvm_pv_send_ipi() into lapic.h, as its
implementation is provided by lapic.c (sending PV IPIs relies on the
optimized APIC map provided by the in-kernel local APIC), and it's only
used by KVM x86 code.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-26-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the function declaration for APIs to get/query pending IRQs from
kvm_host.h to irq.h, as the APIs are only used by KVM x86 code.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-25-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Bury __kvm_irq_line_state() in CONFIG_KVM_IOAPIC=y code, as it's only used
by PIC and I/O APIC code.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-24-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move a variety of "VALID MASK" defines, e.g. that capture which flags in
a given ioctl are supported by KVM, from kvm_host.h to x86.c. The set of
valid flags/bits is very much a KVM-internal detail, as the values from the
hardcoded #defines are often captured and massaged by KVM's setup code, i.e.
*directly* using the macros outside of KVM x86 would be actively dangerous.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-23-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move kvm_{load,read}_ldt() into vmx.c, as vmx_{load,store}_ldt(), as they
are exclusively used by VMX to save/restore host state, and have no
business being globally visible.
Ideally, KVM-specific helpers wouldn't exist at all, as they are nothing
more than assembly wrappers for SLDT and LLDT, i.e. should be provided by
the kernel, not by KVM. Punt that cleanup to the future, as
arch/x86/include/asm/desc.h _does_ provide helpers, but load_ldt() is only
available for CONFIG_PARAVIRT_XXL=n builds, and both {load,store}_ldt()
unnecessarily constrain the operands to memory.
No functional change intended.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-22-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move a pile of MMU helper declarations and macros into mmu.h, as they are
very much KVM x86 internal APIs and details, and not intended to be exposed
to arch-neutral KVM, and certainly not to the broader kernel.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-21-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the definition of "struct kvm_x86_pmu_event_filter" to pmu.c, as the
as the details of the filters are very much implementation details that can
and should be buried in pmu.c. While the _existence_ of filters is public
knowledge, almost by definition, the contents don't need to be exposed
outside of the PMU code as the filter data is provided by userspace, i.e.
it pretty much has to be dynamically allocated, and thus never should be
fully embedded in a globally visible structure.
No functional change intended
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-20-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the definition of "struct kvm_x86_msr_filter" and its associate,
"struct msr_bitmap_range", to msrs.c, as the details of the filters are
very much implementation details that can and should be buried in msrs.c.
While the _existence_ of filters is public knowledge, almost by definition,
the contents don't need to be exposed outside of the MSR code as the filter
data is provided by userspace, i.e. it pretty much has to be dynamically
allocated, and thus never should be fully embedded in a globally visible
structure.
Note, this creates a discrepancy with the PMU event filter structure; that
will be remedied shortly.
No functional change intended.
Suggested-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-19-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Relocate declarations of MSR helpers (and kvm_nr_uret_msrs) from x86's
kvm_host.h to msrs, to continue trimming down kvm_host.h.
Deliberately leave the funky read_msr() where it is, as it will hopefully
be removed entirely as part of a broader kernel-API cleanup.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-18-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Define kvm_{g,s}et_segment() as inline functions in regs.h, as they are
literally one-line wrappers to invoke vendor code.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-17-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Relocate declarations of Control/Debug Register, EFLAGS and RIP helpers
from x86's kvm_host.h to regs.h, to continue trimming down kvm_host.h.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-16-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Introduce msrs.{c,h}, and move the vast majority of MSR specific code out
of x86.{c,h}. Use a plural "msrs" instead of just "msr" to be consistent
with regs.{c,h}, and to make it easier to differentiate KVM's code from the
other 5+ msr.c files in the kernel.
Opportunistically drop the "x86.h" include from mtrr.c, mostly as proof
that the bulk of the MSR code is indeed being relocated to msrs.c.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-15-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Begrudgingly move adjust_tsc_offset_{guest,host}() to x86.h as inlines,
and expose several other TSC helpers in anticipation of moving KVM's MSR
code to a dedicated msrs.c. Unfortunately for KVM, several MSRs that KVM
emulates can affect TSC state.
Opportunistically drop a superfluous local "tsc_offset" variable, whose
existence causes checkpatch to complain about lack of a blank line.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260613000329.732085-14-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Extract the code for getting/setting MSRs and MSR lists to dedicated
helpers in anticipation of moving the MSR code to a new msrs.c.
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-13-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move the kvm_{load,put}_guest_fpu() helpers to fpu.h in anticipation of
moving the bulk of KVM's MSR handling code out of x86.c. KVM needs to
load and put the FPU when accessing MSRs that are managed via XSTATE,
a.k.a. the so called "FPU".
No functional change intended.
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-12-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Drop an mostly unused include of x86.h from hyperv.h, and instead pull in
regs.h, which is need for at least is_guest_mode(). This eliminates the
last include of x86.h from a common x86 header, i.e. solidifies that x86.h
is the top of the pyramid.
Add a missing x86.h include in cpuid.c to avoid build breakage.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-11-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move KVM's eager_page_split module param to the MMU, as it is very much an
MMU knob.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-10-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Relocated the declaration of tdp_enabled into mmu.h, and opportunistically
hoist tdp_mmu_enabled up to the top so that the two are co-located.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-9-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Invert the include ordering between x86.h and mmu.h, and move
mmu_is_nested() to mmu.h where it belongs (mmu_is_nested()'s placement in
x86.h was solely responsible for the existing ordering), so that x86.h is
not included by most KVM internal headers (but includes them).
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-8-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Relocate the kvm_caps and kvm_host_values struct definitions and their
associated global variable declarations to asm/kvm_host.h to allow for a
variety of cleanups in x86.h and mmu.h, and to establish a (hopefully)
maintainable rule that asm/kvm_host.h's role is to define common
structures (and declare any associated globals), and anything needed by
arch-neutral KVM.
While it would be lovely to trim kvm_host.h down to the point where it
*only* holds things needed by arch-neutral and/or non-KVM code, multiple
attempts to do just that have failed miserably. Trying to "hide" code
from arch-neutral KVM is too restrictive (and ultimately pointless), and
KVM x86 itself also needs a place to define common structures and their
globals, e.g. to avoid inconsistent header include chains and/or misplaced
helpers.
E.g. as pointed out by Kai, it's weird that x86.h, which is a kitchen sink
of sorts, includes regs.h, but not mmu.h. Literally the only reason that
x86.h doesn't include mmu.h is that mmu.h references kvm_host, which is
currently defined in x86.h. As a result of odd include ordering, the
very clearly MMU-specific helper mmu_is_nested() lives in x86.h, not mmu.h
"Fix" the kvm_host dependency so that x86.h can be the "central" include
everyone expects it to be, and set KVM x86 on the path to having somewhat
sensible "rules" for what goes where:
- asm/kvm_host.h holds "common" structure definitions and associated key
global variables, and things that are referenced by arch-neutral KVM.
- <thing>.{c,h} holds relevant declarations and definitions.
- x86.{c,h} is the kitchen sink for everything else.
Cc: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260613000329.732085-7-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move local APIC IRQ helpers out of asm/kvm_host.h so that they
are co-located with the structs they use, and exposed to a smaller
portion of the broader world.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-6-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Introduce regs.c, and move the vast majority of register specific code out
of x86.c and into regs.c. Deliberately leave behind MSR code, as KVM's MSR
support is complex enough to warrant its own compilation unit, and doesn't
have much in common with the other register code.
Note, "struct kvm_sregs" has fields for EFER and MSR_IA32_APICBASE, and so
the {G,S}ET_REGS flows technically contain a tiny amount of MSR code.
MSR_IA32_APICBASE is already managed by lapic.c, and so doesn't require a
"placement decision". As for EFER, leave all other EFER handling in x86.c
(later to be moved to msrs.c). The primary interface to EFER, set_efer(),
is very much MSR specific, even though EFER is arguably more of a Control
Register than an MSR.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-5-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Rename the KVM_{G,S}ET_SREGS2 helpers in anticipation of moving them out of
x86.c (while leaving the ioctl dispatch behind). Having globally visible
APIs named __{g,s}et_sregs2() would be "fine", but ugly, given that
__{g,s}et_sregs() will NOT be globally visible. As a bonus, this makes it
a bit more obvious that the helpers implement newer versions of
kvm_arch_vcpu_ioctl_set_sregs().
No functional change intended.
Cc: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20260613000329.732085-4-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Move get_segment_base() to regs.h, as kvm_get_segment_base(), so that the
bulk of the register code can be moved from x86.c to a new regs.c, without
simultaneously needing to rename "public" helpers to explicitly scope them
to KVM.
No functional change intended.
Cc: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Extract the REGS and SREGS portions of {store,sync}_regs() into separate
helpers in anticipation of moving the register specific code out of x86.c
and into regs.c.
No functional change intended.
Cc: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Message-ID: <20260613000329.732085-2-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Use cpu_physical_id() instead of default_cpu_present_to_apicid() when
getting the APIC ID of the pCPU on which a vCPU is running/loaded, as the
kernel has gone way off the rails if a vCPU is loaded on a pCPU that has
been physically removed from the system. Even if the impossible were to
happen, the absolutely worst case scenario is that hardware will ring the
AIVC doorbell on the wrong pCPU, i.e. a severely broken system will
experience mild performance issues.
Kill off KVM's superfluous kvm_cpu_get_apicid() wrapper along with the
for-KVM export of default_cpu_present_to_apicid(), as they existed purely
for the wonky AVIC usage.
Cc: Kai Huang <kai.huang@intel.com>
Cc: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Acked-by: Naveen N Rao (AMD) <naveen@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Message-ID: <20260612185459.591892-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Turn arch.n_used_mmu_pages into a stat, mmu_shadow_pages, as the number of
live shadow pages is arguably _the_ most critical datapoint when it comes
to analyzing the shadow MMU. Before the TDP MMU came along, i.e. when the
shadow MMU was the only MMU, explicitly tracking the number of shadow pages
wasn't as interesting, because the same information could more or less be
gleaned from the pages_{1g,2m,4k} stats. But with the TDP MMU, where the
shadow MMU is only used for nested TDP, it becomes extremely difficult, if
not impossible, to determine which SPTEs are coming from the TDP MMU, and
which are coming from the shadow MMU.
E.g. when triaging/debugging shadow MMU performance issues due to "too many
shadow pages", being able to observe that 99%+ of all shadow pages are
unsync is critical to being able to deduce that KVM is effectively leaking
shadow pages.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260612133727.411902-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The TPR_THRESHOLD field in the VMCS is used by VMX to induce VM exits
when the guest's virtual TPR falls under the specified threshold,
allowing KVM to inject previously masked interrupts.
KVM handles these VM exits in handle_tpr_below_threshold().
Commit eb90f3417a ("KVM: vmx: speed up TPR below threshold vmexits")
optimized this function by calling apic_update_ppr() instead of raising
KVM_REQ_EVENT. apic_update_ppr() then raises KVM_REQ_EVENT if there is
a pending, deliverable interrupt.
However, if there are no new interrupts pending, apic_update_ppr() does
not issue the request. Thus, kvm_lapic_update_cr8_intercept() and
vmx_update_cr8_intercept() are not called before VM entry, which results
in a high, stale TPR_THRESHOLD. This is problematic due to the following
sentence in 28.2.1.1 "VM-Execution Control Fields" in the SDM:
The following check is performed if the “use TPR shadow” VM-execution
control is 1 and the “virtualize APIC accesses” and “virtual-interrupt
delivery” VM-execution controls are both 0: the value of bits 3:0 of
the TPR threshold VM-execution control field should not be greater
than the value of bits 7:4 of VTPR.
This error condition is typically not observed when KVM runs on a bare
metal system because modern processors support APICv, which enables
virtual-interrupt delivery, and which KVM uses when possible. This
causes the processor to no longer generate TPR-below-threshold exits
and to no longer check TPR_THRESHOLD on entry. However, when running
on older platforms, or under nested virtualization on a hypervisor that
does not support virtual-interrupt delivery and enforces this check
(like Hyper-V) this can cause a VM entry failure with hardware error
0x7, as seen in [1].
Call kvm_lapic_update_cr8_intercept() if apic_update_ppr() does not
find a deliverable interrupt (and thus does not raise KVM_REQ_EVENT).
Remove calls to kvm_lapic_update_cr8_intercept() on paths that end up in
apic_update_ppr(), as they now become redundant. This ensures that any
path that updates the guest's PPR also figures out if KVM needs to wait
for a TPR change (using TPR_THRESHOLD on VMX or CR8 intercepts on SVM).
Link: https://github.com/coconut-svsm/svsm/issues/1081 [1]
Tested-by: Stefano Garzarella <sgarzare@redhat.com>
Cc: stable@vger.kernel.org
Fixes: eb90f3417a ("KVM: vmx: speed up TPR below threshold vmexits")
Signed-off-by: Carlos López <clopez@suse.de>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Message-ID: <20260618174347.1981064-3-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
WARN once if KVM's tracking for pending EOIs for Real-Time Clock IRQs goes
off the rails, as there's no reason to bug the host or risk a DoS due to
spamming dmesg with endless WARNs. Absolute worst case scenario, guest
time will go awry.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20260618174527.1982333-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
WARN and return an error up the stack if the PIC or I/O APIC encounters an
invalid vector when injecting an IRQ, as there is no danger to the host and
thus no justification for potentially panicking the kernel. Don't bug the
VM either, as the risk of corrupting the guest is minuscule, and the guest
might even be completely tolerant of a lost interrupt.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20260618185213.2019937-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Bug the VM, not the host kernel, if KVM's ISR count {under,over}flows when
tracking in-flight ISRs. There is zero danger to the host if KVM messes up
its IRQ tracking.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20260618185350.2020845-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Instead of bugging the host kernel, WARN and terminate the VM if KVM
attempts to write-protect at a level that cannot use leaf SPTEs.
There is no reason to bring down the entire host; even termininating
the VM is likely overkill, but in theory a missed write could corrupt
guest memory, so play it safe.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20260618185641.2022368-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
If KVM attempts to translate what it thinks is an L2 GPA with a non-nested
MMU, simply WARN and return the GPA, i.e. trust the MMU more than the
caller, as there is zero reason to potentially panic the host kernel just
because KVM misused an API.
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Message-ID: <20260618185746.2023283-1-seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Highlights:
- amd/hfi: Add support for dynamic ranking tables (version 3)
- amd/pmc:
- Add PMC driver support for AMD 1Ah M80H SoC
- Delay suspend for some Lenovo Laptops to avoid keyboard and lid
switch problems after s2idle
- arm64: qcom-hamoa-ec: Add Hamoa/Purwa/Glymur EC driver
- asus-armoury: add support for G614PR, GA402NJ, GA403UM, and FX608JPR
- asus-wmi: add keystone dongle support
- dell-dw5826e: Add reset driver for DW5826e
- dell-laptop: Fix rollback path
- hp-wmi:
- Add support for Omen 16-ap0xxx (board ID 8D26) and board ID 8B2F
- intel-hid:
- Add HP ProBook x360 440 G1 5 button array support
- Prevent racing ACPI notify handlers
- intel/pmc:
- Add Nova Lake support
- Rate-limit LTR scale-factor warning
- intel-uncore-freq:
- Expose instance ID in the sysfs
- Fix current_freq_khz after CPU hotplug
- intel/vsec: Restore BAR fallback for header walk
- ISST: Restore SST-PP control to all domains
- lenovo-wmi-*:
- Add more CPU tunable attributes
- Add GPU tunable attributes
- Add WMI battery charge limiting
- oxpec: add support for OneXPlayer Super X
- sel3350-platform: Retain LED state on load and unload
- surface: SAM: Add support for Surface Pro 12in
- uniwill-laptop: Add support for battery charge modes
- tools/power/x86/intel-speed-select: Harden daemon pidfile open
- Major refactoring efforts:
- ACPI driver to platform driver conversion
- Converting drivers to use the improved WMI API
- Miscellaneous cleanups / refactoring / improvements
The following is an automated shortlog grouped by driver:
ACPI:
- provide acpi_bus_find_device_by_name()
alienware-wmi-base:
- Transition to new WMI API
amd: hfi:
- Support for ranking table versions
amd/hsmp:
- Clamp ioctl/send_message indices (Spectre v1)
amd/pmc:
- Add delay_suspend module parameter
- Add PMC driver support for AMD 1Ah M80H SoC
- Check for intermediate wakeup in function
- Delay suspend for some Lenovo Laptops
- Don't log during intermediate wakeups
- Use per-SoC cpu_info struct for SMU mailbox and IP info
apple-gmux:
- Drop unused assignment of pnp_device_id driver data
arm64:
- Add driver for EC found on Qualcomm reference devices
arm64: qcom-hamoa-ec:
- Fix indentation in comment tables
arm64 Use named initializers for struct i2c_device_id:
- arm64 Use named initializers for struct i2c_device_id
asus-armoury:
- add support for FX608JPR
- add support for G614PR
- add support for GA402NJ
- add support for GA403UM
asus-wmi:
- add keystone dongle support
barco-p50-gpio:
- attach software node to its target GPIO device
classmate-laptop:
- Address memory leaks on driver removal
- Convert accel driver to a platform one
- Convert ipml driver to a platform one
- Convert keys driver to a platform one
- Convert tablet driver to a platform one
- Convert v4 accel driver to a platform one
- Pass struct device pointer to helpers
- Register ACPI notify handlers directly
- Rename two helper functions
- Unify probe rollback and remove code
dell-ddv:
- Use new buffer-based WMI API
dell-descriptor:
- Use new buffer-based WMI API
dell-dw5826e:
- Add reset driver for DW5826e
dell-laptop:
- add Inspiron N5110 to touchpad LED quirk table
- fix missing cleanups in init error path
dell-privacy:
- correct CONFIG_DELL_WMI_PRIVACY macro name in comments
- Use new buffer-based WMI API
dell_rbu:
- use strscpy in image_type_write
dell-smbios-wmi:
- Use new buffer-based WMI API
dell-wmi-base:
- Use new buffer-based WMI API
docs:
- fix typo in uniwill-laptop.rst
hp-wmi:
- Add support for Omen 16-ap0xxx (8D26)
- Add support for Omen 16-ap0xxx (8E35)
- Add thermal support for board 8B2F
hwmon:
- (dell-smm) Use new buffer-based WMI API
intel-hid:
- Add HP ProBook x360 440 G1 to button_array_table
- Protect ACPI notify handler against recursion
intel/pmc:
- Add ACPI PWRM telemetry driver for Nova Lake S
- Add Nova Lake support to intel_pmc_core driver
- Add NVL PCI IDs for SSRAM telemetry discovery
- Add PMC SSRAM Kconfig description
- Add support for variable DMU offsets
- Enable Pkgc blocking residency counter
- Enable PkgC LTR blocking counter
- rate-limit LTR scale-factor warning
- Retrieve PMC info only for available PMCs
intel/pmc/ssram:
- Add ACPI discovery scaffolding
- Add PCI platform data
- Make PMT registration optional
- Refactor DEVID/PWRMBASE extraction into helper
- Rename probe and PCI ID table for consistency
- Switch to static array with per-index probe state
intel/pmc:
- Use __free() in pmc_core_punit_pmt_init()
- Use PCI DID for PMC SSRAM device discovery
intel/pmt:
- Add pre/post decode hooks around header parsing
- Cache the telemetry discovery header
intel/pmt/crashlog:
- Split init into pre-decode
intel/pmt:
- Pass discovery index instead of resource
intel/pmt/telemetry:
- Move overlap check to post-decode hook
intel/pmt:
- Unify header fetch and add ACPI source
intel/tpmi:
- convert mutex in mem_write() to guard
- use cleanup helpers in mem_write()
intel-uncore-freq:
- Expose instance ID in the sysfs
- Fix current_freq_khz after CPU hotplug
- Rename instance_id
intel/vsec:
- allocate res with intel_vsec_dev
- Restore BAR fallback for header walk
ISST:
- Restore SST-PP control to all domains
lenovo-wmi-capdata:
- Add debugfs file for dumping capdata
lenovo-wmi-helpers:
- Add helper for creating per-device debugfs dir
lenovo-wmi-other:
- Add force_load_psy_ext module parameter
- Add GPU tunable attributes
- Add missing CPU tunable attributes
- Add WMI battery charge limiting
- Rename LWMI_OM_FW_ATTR_BASE_PATH
meraki-mx100:
- use real software node references
modpost:
- Handle malformed WMI GUID strings
Move delayed work on system_dfl_wq:
- Move delayed work on system_dfl_wq
msi-ec:
- Add support for MSI Pulse GL66 12th Gen
oxpec:
- add support for OneXPlayer Super X
panasonic-laptop:
- simplify allocation of sinf
pcengines-apuv2:
- reduce indiraction in swnode assignment
pmc_atom:
- Use named initializer for pci_device_id array
sel3350-platform:
- Retain LED state on load and unload
sonypi:
- use strscpy() in sonypi_acpi_probe
surface: SAM:
- Add support for Surface Pro 12in
thinkpad_acpi:
- Add debugfs entry to display HWDD raw
- Remove unneeded goto
tools/power/x86/intel-speed-select:
- Harden daemon pidfile open
uniwill-laptop:
- Add support for battery charge modes
- Enable battery charge modes on supported devices
- Mark EC_ADDR_OEM_4 as volatile
- Rework FN lock/super key suspend handling
use u8 * for raw byte buffers:
- use u8 * for raw byte buffers
wmi:
- Make sysfs attributes const
- Make wmi_bus_class const
x86-android-tablets:
- enable fwnode matching of GPIO chips
- Use named initializers for struct i2c_device_id
x86/platform/olpc: xo15:
- Convert ACPI driver to a platform one
- Drop wakeup source on driver removal
xo15-ebook:
- Convert ACPI driver to a platform one
- Fix formatting of labels
- Fix wakeup source and GPE handling
- Register ACPI notify handler directly
- Use devres-based resource management
Merges:
- Merge branch 'fixes' into for-next
- Merge branch 'fixes' of into for-next
- Merge branch 'intel-sst' of https://github.com/spandruvada/linux-kernel into review-ilpo-next
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQSCSUwRdwTNL2MhaBlZrE9hU+XOMQUCajl8LgAKCRBZrE9hU+XO
MZ47AP0Z0IoiT8WEcXL4wuTQyR6ruXWClYk82wiroAFEko8Z5AD/WYNWCBXqjZk+
FlJbtleRLA/ADmgT3Mj+biHNTIqyBA0=
=Q3d+
-----END PGP SIGNATURE-----
Merge tag 'platform-drivers-x86-v7.2-1' of git://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86
Pull x86 platform driver updates from Ilpo Järvinen:
- amd/hfi: Add support for dynamic ranking tables (version 3)
- amd/pmc:
- Add PMC driver support for AMD 1Ah M80H SoC
- Delay suspend for some Lenovo Laptops to avoid keyboard and lid
switch problems after s2idle
- arm64: qcom-hamoa-ec: Add Hamoa/Purwa/Glymur EC driver
- asus-armoury: add support for G614PR, GA402NJ, GA403UM, and FX608JPR
- asus-wmi: add keystone dongle support
- dell-dw5826e: Add reset driver for DW5826e
- dell-laptop: Fix rollback path
- hp-wmi:
- Add support for Omen 16-ap0xxx (board ID 8D26) and board ID 8B2F
- intel-hid:
- Add HP ProBook x360 440 G1 5 button array support
- Prevent racing ACPI notify handlers
- intel/pmc:
- Add Nova Lake support
- Rate-limit LTR scale-factor warning
- intel-uncore-freq:
- Expose instance ID in the sysfs
- Fix current_freq_khz after CPU hotplug
- intel/vsec: Restore BAR fallback for header walk
- ISST: Restore SST-PP control to all domains
- lenovo-wmi-*:
- Add more CPU tunable attributes
- Add GPU tunable attributes
- Add WMI battery charge limiting
- oxpec: add support for OneXPlayer Super X
- sel3350-platform: Retain LED state on load and unload
- surface: SAM: Add support for Surface Pro 12in
- uniwill-laptop: Add support for battery charge modes
- tools/power/x86/intel-speed-select: Harden daemon pidfile open
- Major refactoring efforts:
- ACPI driver to platform driver conversion
- Converting drivers to use the improved WMI API
- Miscellaneous cleanups / refactoring / improvements
* tag 'platform-drivers-x86-v7.2-1' of git://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86: (115 commits)
platform/x86/intel/pmc: Add NVL PCI IDs for SSRAM telemetry discovery
platform/x86/intel/pmc/ssram: Make PMT registration optional
platform/x86/intel/pmc/ssram: Add ACPI discovery scaffolding
platform/x86/intel/pmc/ssram: Switch to static array with per-index probe state
platform/x86/intel/pmc/ssram: Refactor DEVID/PWRMBASE extraction into helper
platform/x86/intel/pmc/ssram: Add PCI platform data
platform/x86/intel/pmc/ssram: Rename probe and PCI ID table for consistency
platform/x86/intel/pmc: Add ACPI PWRM telemetry driver for Nova Lake S
platform/x86/intel/pmc: Add PMC SSRAM Kconfig description
platform/x86/intel/pmt: Unify header fetch and add ACPI source
platform/x86/intel/pmt: Cache the telemetry discovery header
platform/x86/intel/pmt: Pass discovery index instead of resource
platform/x86/intel/pmt/telemetry: Move overlap check to post-decode hook
platform/x86/intel/pmt/crashlog: Split init into pre-decode
platform/x86/intel/pmt: Add pre/post decode hooks around header parsing
modpost: Handle malformed WMI GUID strings
platform/wmi: Make sysfs attributes const
platform/wmi: Make wmi_bus_class const
hwmon: (dell-smm) Use new buffer-based WMI API
platform/x86: dell-ddv: Use new buffer-based WMI API
...
-----BEGIN PGP SIGNATURE-----
iQFHBAABCgAxFiEEIbPD0id6easf0xsudhRwX5BBoF4FAmo41jgTHHdlaS5saXVA
a2VybmVsLm9yZwAKCRB2FHBfkEGgXjo1CACiN+Ifxj67WZAy1c1tqlHTJbD/OcAh
KlbZQCVG3QoV9RC8yc9U2DyzSv488YujrqaJhsRg35Cjqs91dFlaXOemArF9dk8f
ICy3SU48sCGvm6v95ndClUyvncAkqcPR/tyfNs7l5rPh6fZnXEov2SdtOlMMayOA
HD/s+urd4AiIT3ffj/Ikl/+VrxC7rkYw5oXRF83y4oG79BDtXxgsSlHCudCAM15O
FvBeUyTT7/7Phz7jk+oNiNBEtJTq6CQo/2Xp5TBpSBl2F/MjfCudAa/XqJakZO27
UMi5lH8DALYNek4dckyfvt7cVHfeCma0ho77V9AscChFtnAnv74sJyEK
=ADO8
-----END PGP SIGNATURE-----
Merge tag 'hyperv-next-signed-20260621' of git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux
Pull hyperv updates from Wei Liu:
- Use wakeup mailbox to boot APs in Hyper-V VTL2 TDX guests (Yunhong
Jiang, Ricardo Neri)
- Move the Hyper-V IOMMU to its own subdirectory (Mukesh Rathor)
- Cosmetic changes to mshv and balloon driver (Junrui Luo, Markus
Elfring)
* tag 'hyperv-next-signed-20260621' of git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux:
mshv: add bounds check on vp_index in mshv_intercept_isr()
hv_balloon: Simplify data output in hv_balloon_debug_show()
x86/hyperv: Cosmetic changes in irqdomain.c for readability
iommu/hyperv: Create hyperv subdirectory under drivers/iommu
x86/hyperv/vtl: Use the wakeup mailbox to boot secondary CPUs
x86/hyperv/vtl: Mark the wakeup mailbox page as private
x86/acpi: Add a helper to get the address of the wakeup mailbox
x86/hyperv/vtl: Setup the 64-bit trampoline for TDX guests
x86/realmode: Make the location of the trampoline configurable
x86/hyperv/vtl: Set real_mode_header in hv_vtl_init_platform()
x86/dt: Parse the Wakeup Mailbox for Intel processors
dt-bindings: reserved-memory: Wakeup Mailbox for Intel processors
x86/acpi: Add functions to setup and access the wakeup mailbox
x86/topology: Add missing struct declaration and attribute dependency
- Remove the per-arch strncpy implementations in alpha, m68k, powerpc,
x86, and xtensa
- Remove strncpy API
Over the last 6 years working on strncpy removal there were 362 commits
by 70 contributors. Folks with more than 1 commit were:
211 Justin Stitt <justinstitt@google.com>
22 Xu Panda <xu.panda@zte.com.cn>
21 Kees Cook <kees@kernel.org>
17 Thorsten Blum <thorsten.blum@linux.dev>
12 Arnd Bergmann <arnd@arndb.de>
4 Pranav Tyagi <pranav.tyagi03@gmail.com>
4 Lee Jones <lee@kernel.org>
2 Steven Rostedt <rostedt@goodmis.org>
2 Sam Ravnborg <sam@ravnborg.org>
2 Marcelo Moreira <marcelomoreira1905@gmail.com>
2 Krzysztof Kozlowski <krzk@kernel.org>
2 Kalle Valo <kvalo@kernel.org>
2 Jaroslav Kysela <perex@perex.cz>
2 Daniel Thompson <danielt@kernel.org>
2 Andrew Lunn <andrew@lunn.ch>
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRSPkdeREjth1dHnSE2KwveOeQkuwUCajSC5QAKCRA2KwveOeQk
u+L0AQCqbRKDuNHKWwbdwv5Sy6nbsHLLd4//m+dJDesqbdMYfgD/UQyygfeicqep
tXcP2krB9Mcl6qsMqNgkKAPpNTCPAwE=
=FZ+4
-----END PGP SIGNATURE-----
Merge tag 'strncpy-removal-v7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux
Pull strncpy removal from Kees Cook:
- Remove the per-arch strncpy implementations in alpha, m68k, powerpc,
x86, and xtensa
- Remove strncpy API
Over the last 6 years working on strncpy removal there were 362
commits by 70 contributors. Folks with more than 1 commit were:
211 Justin Stitt <justinstitt@google.com>
22 Xu Panda <xu.panda@zte.com.cn>
21 Kees Cook <kees@kernel.org>
17 Thorsten Blum <thorsten.blum@linux.dev>
12 Arnd Bergmann <arnd@arndb.de>
4 Pranav Tyagi <pranav.tyagi03@gmail.com>
4 Lee Jones <lee@kernel.org>
2 Steven Rostedt <rostedt@goodmis.org>
2 Sam Ravnborg <sam@ravnborg.org>
2 Marcelo Moreira <marcelomoreira1905@gmail.com>
2 Krzysztof Kozlowski <krzk@kernel.org>
2 Kalle Valo <kvalo@kernel.org>
2 Jaroslav Kysela <perex@perex.cz>
2 Daniel Thompson <danielt@kernel.org>
2 Andrew Lunn <andrew@lunn.ch>
* tag 'strncpy-removal-v7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux:
string: Remove strncpy() from the kernel
xtensa: Remove arch-specific strncpy() implementation
x86: Remove arch-specific strncpy() implementation
powerpc: Remove arch-specific strncpy() implementation
m68k: Remove arch-specific strncpy() implementation
alpha: Remove arch-specific strncpy() implementation
Everything:
Total patches: 321
Reviews/patch: 1.52
Reviewed rate: 68%
Excluding DAMON:
Total patches: 227
Reviews/patch: 2.07
Reviewed rate: 90%
Excluding DAMON and selftests:
Total patches: 203
Reviews/patch: 2.14
Reviewed rate: 91%
Patchsets in this merge:
- The 2 patch series "selftests/mm: clean up build output and verbosity"
from Li Wang removes some noise from the MM selftests build out.
- The 3 patch series "mm: Free contiguous order-0 pages efficiently"
from Ryan Roberts speeds up the freeing of a batch of 0-order pages by
first scanning them for coalescing opportunities. This is applicable to
vfree() and to the releasing of frozen pages.
- The 11 patch series "mm/damon: introduce DAMOS failed region quota
charge ratio" from SeongJae Park addresses a DAMOS usability issue: The
DAMOS quota often exhausts prematurely because it charges for all memory
attempted, causing slow and inconsistent performance when actions fail
on unreclaimable memory.
To fix this, a new feature lets users set a smaller, flexible quota
charge ratio (via a numerator and denominator) for failed regions.
Since failed actions cause less overhead, reducing their quota cost
ensures more predictable and efficient DAMOS processing
- The 8 patch series "selftests/cgroup: improve zswap tests robustness
and support large page sizes" from Li Wang fixes various spurious
failures and improves the overall robustness of the cgroup zswap
selftests.
- The 3 patch series "fix MAP_DROPPABLE not supported errno" from
Anthony Yznaga fixes an issue in the mlock selftests on arm32.
- The 2 patch series "mm: huge_memory: clean up defrag sysfs with
shared" from Breno Leitao does some maintenance work in the huge_memory
code.
- The 3 patch series "treewide: fixup gfp_t printks" from Brendan
Jackman uses the special vprintf() gfp_t conversion in various places.
- The 6 patch series "mm: Fix vmemmap optimization accounting and
initialization" from Muchun Song fixes several bugs in the vmemmap
optimization, mainly around incorrect page accounting and memmap
initialization in the DAX and memory hotplug paths. It also fixes
pageblock migratetype initialization and struct page initialization for
ZONE_DEVICE compound pages.
- The 4 patch series "mm/damon: repost non-hotfix reviewed patches in
damon/next tree" from is a sprinkle of unrelated minor bugfixes for
DAMON.
- The 3 patch series "mm: remove page_mapped()" from David Hildenbrand
remove this function from the tree, replacing it with folio_mapped().
- The 10 patch series "mm/damon: let DAMON be paused and resumed" from
SeongJae Park permits DAMON to be paused and resumed without losing its
current state.
- The 3 patch series "kasan: hw_tags: Disable tagging for stack and
page-tables" from Muhammad Usama Anjum simplifies and speeds up kasan by
removing its ineffective tagging of stacks and page tables.
- The 7 patch series "mm/damon/reclaim,lru_sort: monitor all system rams
by default" from SeongJae Park simplifies deployment on diverse hardware
like NUMA systems by updating DAMON_RECLAIM and DAMON_LRU_SORT to
automatically monitor the physical address range covering all System RAM
areas by default, replacing the overly restrictive behavior that only
targeted the single largest memory block to save on negligible overhead.
- The 2 patch series "mm/damon/sysfs: document filters/ directory as
deprecated" from SeongJae Park updates some DAMON docs.
- The 8 patch series "mm: use spinlock guards for zone lock" from Dmitry
Ilvokhin switches zone->lock handling over to using the guard()
mechanisms.
- The 2 patch series "mm/filemap: tighten mmap_miss hit accounting" from
fujunjie fixes a flaw where the mmap_miss counter over-credited page
cache hits during fault-arounds and page-fault retries. This results in
significant reduction of redundant synchronous mmap readahead I/O,
drastically cutting down execution time and gigabytes read for sparse
random or strided memory access workloads.
- The 2 patch series "selftests/cgroup: Fix false positive failures in
test_percpu_basic" from Li Wang fixes a couple of false-positives in the
cgroup kmem selftests.
- The 2 patch series "mm/damon/reclaim: support monitoring intervals
auto-tuning" from SeongJae Park adds a new parameter to DAMON permitting
DAMON_RECLAIM to automatically tune DAMON's sampling and aggregation
intervals.
- The 2 patch series "mm/damon/stat: add kdamond_pid parameter" from
SeongJae Park chnges DAMON_STAT to provide the pid of its kdamond.
- The 2 patch series "mm/kmemleak: dedupe verbose scan output" from
Breno Leitao removes large amounts of duplicated backtraces from the
verbose-mode kmemleak output.
- The 8 patch series "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 1)"
from David Hildenbrand reduces our use of CONFIG_HAVE_BOOTMEM_INFO_NODE,
with a view to removing it entirely in a later series.
- The 2 patch series "mm/damon: validate min_region_size to be power of
2" from Liew Rui Yan prevents users from passing a non-power-of-2 value
of `addr_unit', as this later results in undesirable behavior.
- The 2 patch series "mm: document read_pages and simplify usage" from
Frederick Mayle does as claimed.
- The 3 patch series "tools/mm/page-types: Fix misc bugs" from Ye Liu
fixes three issues in tools/mm/page-types.c.
- The 4 patch series "mm: misc cleanups from __GFP_UNMAPPED series" from
Brendan Jackman implements several cleanups in the page allocator and
related code.
- The 12 patch series "mm, swap: swap table phase IV: unify allocation"
from Kairui Song unifies the allocation and charging of anon and shmem
swap in folios, provides better synchronization, consolidates the
metadata management, hence dropping the static array and map, and
improves performance.
- The 28 patch series "mm/damon: introduce data attributes monitoring"
from SeongJae Park extends DAMON for monitoring general data attributes
other than accesses.
- The 5 patch series "mm/vmalloc: free unused pages on vrealloc()
shrink" from Shivam Kalra implements the TODO in vrealloc() to unmap and
free unused pages when shrinking across a page boundary.
- The 3 patch series "mm/damon: documentation and comment fixes" from
niecheng does as advertised.
- The 3 patch series "remove mmap_action success, error hooks" from
Lorenzo Stoakes eliminates custom hooks from mmap_action by removing the
problematic success_hook which allowed drivers to improperly access
uninitialized VMAs. It replaces the error_hook with a simple error-code
field and updates the memory char driver accordingly.
- The 14 patch series "mm/damon: minor improvements for code readability
and tests" from SeongJae Park implements minor improvements in code
readability and tests for DAMON.
- The 2 patch series "mm/damon: fix macro arguments and clarify quota
goals doc" from Maksym Shcherba does those things.
- The 2 patch series "userfaultfd: merge fs/userfaultfd.c into
mm/userfaultfd.c" from Mike Rapoport performs that code movement.
- The 15 patch series "mm/mglru: improve reclaim loop and dirty folio"
from Kairui Song and others cleans up and slightly improves MGLRU's
reclaim loop and dirty writeback handling. Large performance
improvements are measured.
- The 3 patch series "use vma locks for proc/pid/{smaps|numa_maps}
reads" from Suren Baghdasaryan uses per-vma locks when reading
/proc/pid/smaps and /proc/pid/numa_maps similar to reduce contention on
central mmap_lock.
- The 2 patch series "refactors thpsize_shmem_enabled_store() and
thpsize_shmem_enabled_show()" from Ran Xiaokai provides some cleanup
work in the THP code.
- The 2 patch series "selftests/memfd: fix compilation warnings" from
Konstantin Khorenko fixes a few build glitches in the memfd selftest
code.
- The 4 patch series "memcg: shrink obj_stock_pcp and cache multiple
objcgs" from Shakeel Butt resolves a 68% performance regression caused
by NUMA-node cache thrashing around struct obj_stock_pcp by shrinking
its existing fields and expanding it into a multi-slot array that caches
up to five obj_cgroup pointers per CPU, allowing per-node variants of
the same memcg to coexist within a single 64-byte cache line.
- The 2 patch series "zram: writeback fixes" from Sergey Senozhatsky
addresses a couple of unrelated zram writeback issues.
- The 9 patch series "mm: switch THP shrinker to list_lru" from Johannes
Weiner resolves NUMA-awareness issues and streamlines callsite
interaction by refactoring and extending the list_lru API to completely
replace the complex, open-coded deferred split queue for Transparent
Huge Pages (THPs).
- The 2 patch series "mm: improve large folio readahead for exec memory"
from Usama Arif improves large-folio readahead on systems like 64K-page
arm64 by preventing the mmap_miss check from permanently disabling
target-oriented VM_EXEC readahead, and by generalizing the
force_thp_readahead gate to support mappings with any usefully large
maximum folio order under the cache cap.
- The 6 patch series "userfaultfd/pagemap: pre-existing fixes" from
Kiryl Shutsemau fixes a bunch of minor issues in the
userfaultfd/pagemap, all of which were flagged by Sashiko review of
proposed new material.
- The 5 patch series "mm/sparse-vmemmap: Provide generic
vmemmap_set_pmd() and vmemmap_check_pmd()" from Muchun Song provides
generic versions of these two functions so the four arch-specific
implementations can be removed.
- The 2 patch series "mm/swap, PM: hibernate: fix swapoff race in
uswsusp by pinning swap device" from Youngjun Park addresses a
uswsusp-vs-swapoff race and reduces the swap device reference
taking/releasing frequency.
- The 2 patch series "mm/hmm: A fix and a selftest" from Dev Jain does
as claimed.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCajQcWQAKCRDdBJ7gKXxA
jkjFAQD14ZCAS7c/lsNHPRpOARR2zrCJiIp+vhZDEx5V+d8u0QD+I4ca11CnQ+OA
dKCeabWWYuTR1GDtTHbLVCRfpGexBwM=
=cWPO
-----END PGP SIGNATURE-----
Merge tag 'mm-stable-2026-06-18-09-26' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM updates from Andrew Morton:
- "selftests/mm: clean up build output and verbosity" (Li Wang)
Remove some noise from the MM selftests build
- "mm: Free contiguous order-0 pages efficiently" (Ryan Roberts)
Speed up the freeing of a batch of 0-order pages by first scanning
them for coalescing opportunities. This is applicable to vfree() and
to the releasing of frozen pages
- "mm/damon: introduce DAMOS failed region quota charge ratio"
(SeongJae Park)
Address a DAMOS usability issue: The DAMOS quota often exhausts
prematurely because it charges for all memory attempted, causing slow
and inconsistent performance when actions fail on unreclaimable
memory.
To fix this, a new feature lets users set a smaller, flexible quota
charge ratio (via a numerator and denominator) for failed regions.
Since failed actions cause less overhead, reducing their quota cost
ensures more predictable and efficient DAMOS processing
- "selftests/cgroup: improve zswap tests robustness and support large
page sizes" (Li Wang)
Fix various spurious failures and improves the overall robustness of
the cgroup zswap selftests
- "fix MAP_DROPPABLE not supported errno" (Anthony Yznaga)
Fix an issue in the mlock selftests on arm32
- "mm: huge_memory: clean up defrag sysfs with shared" (Breno Leitao)
Some maintenance work in the huge_memory code
- "treewide: fixup gfp_t printks" (Brendan Jackman)
Use the special vprintf() gfp_t conversion in various places
- "mm: Fix vmemmap optimization accounting and initialization" (Muchun
Song)
Fix several bugs in the vmemmap optimization, mainly around incorrect
page accounting and memmap initialization in the DAX and memory
hotplug paths. It also fixes pageblock migratetype initialization and
struct page initialization for ZONE_DEVICE compound pages
- "mm/damon: repost non-hotfix reviewed patches in damon/next tree"
A sprinkle of unrelated minor bugfixes for DAMON
- "mm: remove page_mapped()" (David Hildenbrand)
Remove this function from the tree, replacing it with folio_mapped()
- "mm/damon: let DAMON be paused and resumed" (SeongJae Park)
Allow DAMON to be paused and resumed without losing its current state
- "kasan: hw_tags: Disable tagging for stack and page-tables" (Muhammad
Usama Anjum)
Simplify and speed up kasan by removing its ineffective tagging of
stacks and page tables
- "mm/damon/reclaim,lru_sort: monitor all system rams by default"
(SeongJae Park)
Simplify deployment on diverse hardware like NUMA systems by updating
DAMON_RECLAIM and DAMON_LRU_SORT to automatically monitor the
physical address range covering all System RAM areas by default,
replacing the overly restrictive behavior that only targeted the
single largest memory block to save on negligible overhead
- "mm/damon/sysfs: document filters/ directory as deprecated" (SeongJae
Park)
Update some DAMON docs
- "mm: use spinlock guards for zone lock" (Dmitry Ilvokhin)
Switch zone->lock handling over to using the guard() mechanisms
- "mm/filemap: tighten mmap_miss hit accounting" (fujunjie)
Fix a flaw where the mmap_miss counter over-credited page cache hits
during fault-arounds and page-fault retries. This results in
significant reduction of redundant synchronous mmap readahead I/O,
drastically cutting down execution time and gigabytes read for sparse
random or strided memory access workloads
- "selftests/cgroup: Fix false positive failures in test_percpu_basic"
(Li Wang)
Fix a couple of false-positives in the cgroup kmem selftests
- "mm/damon/reclaim: support monitoring intervals auto-tuning"
(SeongJae Park)
Add a new parameter to DAMON permitting DAMON_RECLAIM to
automatically tune DAMON's sampling and aggregation intervals
- "mm/damon/stat: add kdamond_pid parameter" (SeongJae Park)
Change DAMON_STAT to provide the pid of its kdamond
- "mm/kmemleak: dedupe verbose scan output" (Breno Leitao)
Remove large amounts of duplicated backtraces from the verbose-mode
kmemleak output
- "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 1)" (David
Hildenbrand)
Reduce our use of CONFIG_HAVE_BOOTMEM_INFO_NODE, with a view to
removing it entirely in a later series
- "mm/damon: validate min_region_size to be power of 2" (Liew Rui Yan)
Prevent users from passing a non-power-of-2 value of `addr_unit', as
this later results in undesirable behavior
- "mm: document read_pages and simplify usage" (Frederick Mayle)
- "tools/mm/page-types: Fix misc bugs" (Ye Liu)
Fix three issues in tools/mm/page-types.c
- "mm: misc cleanups from __GFP_UNMAPPED series" (Brendan Jackman)
Implement several cleanups in the page allocator and related code
- "mm, swap: swap table phase IV: unify allocation" (Kairui Song)
Unify the allocation and charging of anon and shmem swap in folios,
provides better synchronization, consolidates the metadata
management, hence dropping the static array and map, and improves
performance
- "mm/damon: introduce data attributes monitoring" (SeongJae Park(
Extend DAMON to monitor general data attributes other than accesses
- "mm/vmalloc: free unused pages on vrealloc() shrink" (Shivam Kalra)
Implement the TODO in vrealloc() to unmap and free unused pages when
shrinking across a page boundary
- "mm/damon: documentation and comment fixes" (niecheng)
- "remove mmap_action success, error hooks" (Lorenzo Stoakes)
Eliminate custom hooks from mmap_action by removing the problematic
success_hook which allowed drivers to improperly access uninitialized
VMAs. It replaces the error_hook with a simple error-code field and
updates the memory char driver accordingly
- "mm/damon: minor improvements for code readability and tests"
(SeongJae Park)
- "mm/damon: fix macro arguments and clarify quota goals doc" (Maksym
Shcherba)
- "userfaultfd: merge fs/userfaultfd.c into mm/userfaultfd.c" (Mike
Rapoport)
- "mm/mglru: improve reclaim loop and dirty folio" (Kairui Song and
others)
Clean up and slightly improves MGLRU's reclaim loop and dirty
writeback handling. Large performance improvements are measured
- "use vma locks for proc/pid/{smaps|numa_maps} reads" (Suren
Baghdasaryan)
Use per-vma locks when reading /proc/pid/smaps and numa_maps similar
to reduce contention on central mmap_lock
- "refactors thpsize_shmem_enabled_store() and thpsize_shmem_enabled_show()"
(Ran Xiaokai)
Some cleanup work in the THP code
- "selftests/memfd: fix compilation warnings" (Konstantin Khorenko)
Fix a few build glitches in the memfd selftest code.
- "memcg: shrink obj_stock_pcp and cache multiple objcgs" (Shakeel
Butt)
Resolve a 68% performance regression caused by NUMA-node cache
thrashing around struct obj_stock_pcp by shrinking its existing
fields and expanding it into a multi-slot array that caches up to
five obj_cgroup pointers per CPU, allowing per-node variants of the
same memcg to coexist within a single 64-byte cache line.
- "zram: writeback fixes" (Sergey Senozhatsky)
address a couple of unrelated zram writeback issues
- "mm: switch THP shrinker to list_lru" (Johannes Weiner)
Resolve NUMA-awareness issues and streamlines callsite interaction by
refactoring and extending the list_lru API to completely replace the
complex, open-coded deferred split queue for Transparent Huge Pages
- "mm: improve large folio readahead for exec memory" (Usama Arif)
Improve large-folio readahead on systems like 64K-page arm64 by
preventing the mmap_miss check from permanently disabling
target-oriented VM_EXEC readahead, and by generalizing the
force_thp_readahead gate to support mappings with any usefully large
maximum folio order under the cache cap.
- "userfaultfd/pagemap: pre-existing fixes" (Kiryl Shutsemau)
Fix a bunch of minor issues in the userfaultfd/pagemap, all of which
were flagged by Sashiko review of proposed new material
- "mm/sparse-vmemmap: Provide generic vmemmap_set_pmd() and
vmemmap_check_pmd()" (Muchun Song)
Provide generic versions of these two functions so the four
arch-specific implementations can be removed.
- "mm/swap, PM: hibernate: fix swapoff race in uswsusp by pinning swap
device" (Youngjun Park)
Address a uswsusp-vs-swapoff race and reduces the swap device
reference taking/releasing frequency.
- "mm/hmm: A fix and a selftest" (Dev Jain)
* tag 'mm-stable-2026-06-18-09-26' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (321 commits)
selftests/mm/hmm-tests: test pagemap reads of PMD device-private entries
fs/proc/task_mmu: do not warn on seeing non-migration pmd entry
lib/test_hmm: check alloc_page_vma() return value and handle OOM
mm/compaction: cap compact_gap() at COMPACT_CLUSTER_MAX
mm/swap: remove redundant swap device reference in alloc/free
mm/swap, PM: hibernate: fix swapoff race in uswsusp by pinning swap device
mm/filemap: use folio_next_index() for start
vmalloc: fix NULL pointer dereference in is_vm_area_hugepages()
sparc/mm: drop vmemmap_check_pmd helper and use generic code
loongarch/mm: drop vmemmap_check_pmd helper and use generic code
riscv/mm: drop vmemmap_pmd helpers and use generic code
arm64/mm: drop vmemmap_pmd helpers and use generic code
mm/sparse-vmemmap: provide generic vmemmap_set_pmd() and vmemmap_check_pmd()
rust: page: mark Page::nid as inline
userfaultfd: build __VMA_UFFD_FLAGS from config-gated masks
userfaultfd: gate must_wait writability check on pte_present()
mm/huge_memory: preserve pmd_swp_uffd_wp on device-private PMD downgrade
fs/proc/task_mmu: fix hugetlb self-deadlock in pagemap_scan_pte_hole()
fs/proc/task_mmu: use huge_page_size() in pagemap_scan_hugetlb_entry()
fs/proc/task_mmu: fix make_uffd_wp_huge_pte() prot-update race
...
"This is a bit of an odd merge window on the KVM/arm64 front. There is
absolutely no new feature in the pull request. It is purely fixes,
because it is simply becoming too hard to review new stuff when so
many AI-fuelled fixes hit the list".
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX)
that caused the FPU load request.
- Some enhancements about interrupt injection.
- Some bug fixes and other small changes.
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic
PTE updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on
a memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes.
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace.
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root.
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both
of these are preparatory changes for upcoming APX support.
- Clean up KVM's register tracking and storage, primarily to prepare for
APX support, which expands the maximum number of GPRs from 16 to 32.
- Keep a single copy of the PDPTRs rather than two, since architecturally
there is just one.
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in shadow page table handling, where KVM fails to recursively
zap nested EPT/NPT shadow page tables when the nested hypervisor tears
down its own EPT/NPT page tables from the bottom up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in the
generic section; also run with MBEC enabled even for non-nested mode.
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Fix a largely benign bug where KVM TDX would incorrectly state it could
emulate several x2APIC MSRs.
- Use the "safe" WRMSR API when proxying LBR MSR writes as the to-be-written
value is guest controlled and completely unvalidated.
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in the
generic section; also run with GMET enabled even for non-nested mode.
- Fixes and minor cleanups to GHCB handling, on top of the earlier work
already merged into 7.1-rc.
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers.
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
- Don't advertise support for unusable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to follow
the kernel's nomenclature, e.g. for mmu_notifiers.
- Use guard() to cleanup up various KVM+VFIO flows.
- Minor cleanups.
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a gmem
range to multiple memslots, and fix the test that was supposed to ensure
KVM returns -EEXIST.
- Treat memslot binding offsets and sizes as unsigned values to fix a bug
where KVM interprets a large "offset + size" as a negative value and allows
a nonsensical offset.
- Use the inode number instead of the page offset for the NUMA interleaving
index to fix a bug where the effective index would jump by two for
consecutive pages (the caller also adds in the page offset).
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the first
pass, as always waiting only 1ms hid a KVM RISC-V bug as the test reaped the
bitmap before KVM could build up enough state to hit the bug.
- A pile of one-off fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmoypI0UHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroNCSAf8CyuvicojnZGFsNcoY4DZ0whBj5l8
IwVkHaptuyGLSwPAebC/DwtU2NBY4CQdbtLjGmxenf+fSQOtcz1YWKbdY4Nr0hJG
euAb/wO47RqKp1C4WPi7lL2P4XHgmUgD5XZ1qDnRV9gnGddZi3QLLmKgAtNq9Z4m
+pAdGD2/RyZeI0Dx61+5ufE/z8YBlH6GrzgBTxfvvRNKqNDFykBenxYHI7seEfHC
BPz0LxjHDD8BMDUo86qaylZnaeQSmFTGO/eTgvCvabxG6WbM1s0w3K1owmUmP7o3
p9o8274Xh53iK2JYD/qGOtAJXejIwbYKT5bnvUQX0O1EH2+ouvSFapLQ9Q==
=sSqM
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
GPIO software node lookup should rely exclusively on matching the
addresses of the referenced firmware nodes. Commit e5d527be7e ("gpio:
swnode: don't use the swnode's name as the key for GPIO lookup") tried to
enforce this but had to be reverted: it broke existing users who abuse the
software node mechanism by creating "dummy" software nodes named after the
device they want to get GPIOs from, without ever attaching them to the
actual GPIO devices. Those users rely on GPIOLIB matching the label of the
GPIO controller against the name of the software node rather than on a real
firmware node link.
Un-reverting e5d527be7e therefore requires converting all such users to
real firmware node lookup. The geode board setup is one of them: it
references the cs5535 GPIO controller through a locally-defined dummy node
named "cs5535-gpio".
The cs5535 MFD driver now exports the software node associated with its
GPIO controller cell as cs5535_gpio_swnode. Use it as the target of the
GPIO software node references in geode-common.c instead of the dummy node,
so the lookup resolves by firmware node address. As the referenced node
must exist at lookup time, make the cs5535 driver built-in for all boards
selecting GEODE_COMMON (depend on GPIO_CS5535=y). The node is exported in
the "CS5535" namespace, so import it in this module.
Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20260611-cs5535-swnode-v3-1-2b0c517c0c03@oss.qualcomm.com
Signed-off-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
strncpy() has no remaining callers in the kernel[1]. Remove the
x86-32-specific inline assembly implementation and __HAVE_ARCH_STRNCPY
define, falling back to the generic version in lib/string.c.
Link: https://github.com/KSPP/linux/issues/90 [1]
Signed-off-by: Kees Cook <kees@kernel.org>
- new handy helper FIELD_GET_SIGNED() (Yury);
- arch test_and_set_bit_lock() and clear_bit_unlock() cleanup (Randy);
- __bf_shf() simplification (Yury);
- bitmap_print_to_pagebuf() removal (Yury);
- RISCV/bitrev conditional support (Jindie, Yury);
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEEi8GdvG6xMhdgpu/4sUSA/TofvsgFAmowS0sACgkQsUSA/Tof
vshqLgwAw2RkV8ZHAniyPgz4noGztto+fcD70UbMgkh48n46PSKIUPCUvOj4RGHc
Nzcjg6VkHLlypb7U3fHoXOHhkt+nTt+OeVDtZCmr8OzlRYmWXAo9TptfalDAlH1A
OucRocjYSkwl/nbuYk0EQ/0+AcTFlfVTeyyus6wUYt7Nj1ncCNjKF2K5EjtF4aoQ
JiA4d9LKWCIN4qFodP/9aFDzLX5MvqHgvOmBPSSFZKkMAer9HHCa/V3jHH3JMh21
bMPjZvaPUCI9oywLL+pDe0RRhP1CYFevxJ/Tnpcx0elV+CekrzuSsM3zpRbPt6hR
0P+8VVzUuoI8yth5mSH1gvUn9ZDvDeognTpKpXR574lB83A8Ub/Gu/NGMXO3TY0q
mtuVbI5ZG0dCqahjEZvJQJmZ/qqbmwDhQDBtPS0x3S9XyzhNAga+JFpr6FXhULx8
b8ZrDPc5BGLtTt5aBOAgpN10CeFM8JHxku1B489sqRXQWaeKW++GkVfVTZ9UOK1E
8c3l/1Bl
=z8f4
-----END PGP SIGNATURE-----
Merge tag 'bitmap-for-7.2' of https://github.com/norov/linux
Pull bitmap updates from Yury Norov:
"This includes the new FIELD_GET_SIGNED() helper,
bitmap_print_to_pagebuf() removal, RISCV/bitrev support, and a couple
cleanups.
- new handy helper FIELD_GET_SIGNED() (Yury)
- arch test_and_set_bit_lock() and clear_bit_unlock() cleanup (Randy)
- __bf_shf() simplification (Yury)
- bitmap_print_to_pagebuf() removal (Yury)
- RISCV/bitrev conditional support (Jindie, Yury)"
* tag 'bitmap-for-7.2' of https://github.com/norov/linux:
MAINTAINERS: BITOPS: include bitrev.[ch]
arch/riscv: Add bitrev.h file to support rev8 and brev8
bitops: Define generic___bitrev8/16/32 for reuse
lib/bitrev: Introduce GENERIC_BITREVERSE
arch: select HAVE_ARCH_BITREVERSE conditionally on BITREVERSE
bitmap: fix find helper documentation
bitmap: drop bitmap_print_to_pagebuf()
cpumask: switch cpumap_print_to_pagebuf() to using scnprintf()
bitfield: wire __bf_shf to __builtin_ctzll
bitops: use common function parameter names
ptp: switch to using FIELD_GET_SIGNED()
rtc: rv3032: switch to using FIELD_GET_SIGNED()
wifi: rtw89: switch to using FIELD_GET_SIGNED()
iio: mcp9600: switch to using FIELD_GET_SIGNED()
iio: pressure: bmp280: switch to using FIELD_GET_SIGNED()
iio: magnetometer: yas530: switch to using FIELD_GET_SIGNED()
iio: intel_dc_ti_adc: switch to using FIELD_GET_SIGNED()
x86/extable: switch to using FIELD_GET_SIGNED()
bitfield: add FIELD_GET_SIGNED()
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE+soXsSLHKoYyzcli6rmadz2vbToFAmowIHwACgkQ6rmadz2v
bTqRog/+KIO6wpvuizt8N+vzgKeEYR/KlEPEyV5dGAopIk1Ct5Ahzti1/V6zg7hk
oddWhPewF7QncVLdnL6zRCRahXCHOi30148acG8pj3Fwj/iYw/zVihVkppVt9qRU
REAvFj0/hYJOGzttSt9UfQ78ZftrZNhXFCRIK/zUWj72IVMJD04Vbawo8iIixVtu
I+fCB3A1kv+RMlZ/3TWk11z3Fko4lK/jKpujegIzv6s+n+0lAraF0aw0cWMwNpUj
mtJx2PktPHYbjA5CwE1Fyxf6EhV4OhV5gwyET7gzqITG5zjUo45pMNcA7EAxDyiC
NvTGIZMsAC+9iSSLNSfdjBWExXFYHHuCkeoxkw9ei05Qrye/7lYzwwtgImuyhBZG
xNXrnSNCG5BPl+DCLy09LzzwNdRoP7zYZV3QH/ZU0uMTS6foegYoSM8jSa2skmM+
mR8a4qrMeZTv+XWvScduOqtFMBOv3mSPkZMkoyqReLaMIPse5NuUOTxOGn5REu4X
tQLW0GLa6ad/7udv7DYrSfmwK1dihyOPmZl/GMuGroXm4rW4jNojljNBdTFEWjCQ
Rd9qArq1Z1z/g8mWjTdiRRGa/QzF5rmJRLgjjUKIzUlQ5/77t96ls0jLexsRZQx5
hCdu/HoK+pIRIUy20NtdqA8ykowsfUkyEBjYR28GV4RHvtmgW5k=
=rtQD
-----END PGP SIGNATURE-----
Merge tag 'bpf-next-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Alexei Starovoitov:
"Major changes:
- Recover from BPF arena page faults using a scratch page and add
ptep_try_set() for lockless empty-slot installs on x86 and arm64.
This allows BPF kfuncs to access arena pointers directly.
The 'arena_direct_access' stable branch was created for this work
and was pulled into sched-ext and bpf-next trees (Tejun Heo, Kumar
Kartikeya Dwivedi)
- Lift old restriction and support 6+ arguments in BPF programs and
kfuncs on x86 and arm64 (Yonghong Song, Puranjay Mohan)
Other features and fixes:
- Add 24-bit BTF vlen and reclaim unused bits in the BTF UAPI to ease
addition of new BTF kinds (Alan Maguire)
- Raise the maximum BPF call chain depth from 8 to 16 frames (Alexei
Starovoitov)
- Refactor object relationship tracking in the verifier and fix a
dynptr use-after-free bug (Amery Hung)
- Harden the signed program loader and reject exclusive maps as inner
maps (Daniel Borkmann)
- Replace the verifier min/max bounds fields with a circular number
(cnum) representation and improve 32->64 bit range refinements
(Eduard Zingerman)
- Introduce the arena library and runtime (libarena) with a buddy
allocator, rbtree and SPMC queue data structures, ASAN support and
a parallel test harness. Allow subprograms to return arena pointers
and switch to a BTF type-tag based __arena annotation (Emil
Tsalapatis)
- Cache build IDs in the sleepable stackmap path and avoid faultable
build ID reads under mm locks (Ihor Solodrai)
- Introduce the tracing_multi link to attach a single BPF program to
many kernel functions at once. Allow specifying the uprobe_multi
target via FD (Jiri Olsa)
- Extend the bpf_list family of kfuncs with bpf_list_add/del(), and
bpf_list_is_first/is_last/empty() (Kaitao Cheng)
- Extend the BPF syscall with common attributes support for
prog_load, btf_load and map_create (Leon Hwang)
- Wrap rhashtable as BPF map (Mykyta Yatsenko, Herbert Xu)
- Add sleepable support for tracepoint programs and fix deadlocks in
LRU map due to NMI reentry (Mykyta Yatsenko)
- Fix OOB access in bpf_flow_keys, fix nullness analysis of inner
arrays, enforce write checks for global subprograms (Nuoqi Gui)
- Report the maximum combined stack depth and print a breakdown of
instructions processed per subprogram (Paul Chaignon)
- Add an XDP load-balancer benchmark and arm64 JIT support for stack
arguments (Puranjay Mohan)
- Add kfuncs to traverse over wakeup_sources (Samuel Wu)
- Allow sleepable BPF programs to use LPM trie maps directly (Vlad
Poenaru)
- Many more fixes and cleanups across the verifier, BTF, sockmap,
devmap, bpffs, security hooks, s390/riscv/loongarch JITs,
rqspinlock, libbpf, bpftool, selftests"
* tag 'bpf-next-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (336 commits)
selftests/bpf: Work around llvm stack overflow in crypto progs
selftests/bpf: add test for bpf_msg_pop_data() overflow
bpf, sockmap: fix integer overflow in bpf_msg_pop_data() bounds check
sockmap: Fix use-after-free in udp_bpf_recvmsg()
bpf, sockmap: keep sk_msg copy state in sync
bpf, sockmap: Fix wrong rsge offset in bpf_msg_push_data()
bpf, sockmap: reject overflowing copy + len in bpf_msg_push_data()
selftsets/bpf: Retry map update on helper_fill_hashmap()
selftests/bpf: Add test for sleepable lsm_cgroup rejection
selftests/bpf: Add test to verify the fix for bpf_setsockopt() helper
bpf: Fix bpf_get/setsockopt to tos for ipv4-mapped ipv6 socket
selftests/bpf: Avoid static LLVM linking for cross builds
selftests/bpf: Use common CFLAGS for urandom_read
selftests/bpf: Initialize operation name before use
tools/bpf: build: Append extra cflags
libbpf: Initialize CFLAGS before including Makefile.include
bpftool: Append extra host flags
bpftool: Avoid adding EXTRA_CFLAGS to HOST_CFLAGS
bpftool: Pass host flags to bootstrap libbpf
selftests/bpf: correct CONFIG_PPC64 macro name in comment
...
* Make kexec and TDX finally place nice together
* Put TDX error codes into a single header
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEV76QKkVc4xCGURexaDWVMHDJkrAFAmowVgUACgkQaDWVMHDJ
krDwsw/9FKXndYdeSc7P8sJwjSlc9U4PFy/6rdUfnYCP1c2BTeVqNxZvCz+T9E4V
Ld4UjjTLCbEq1uQ2UdYugpLnSKSNzf6MEp06Tsq2cRlEcPQBLGaACo+7dnBrUf6g
zVUY0lPh1Jlxa+FBzMLf/H1qyUGl/Lv8msmgrC9fHHwQgsPYMgKUU6t72ScRG6FA
kUk0crzI1DlSpFFVbXuRIpZSmLFPgyx5TNkteGJrP6q3iR6zYuc30MQsV2GalFrv
qrn4cz3EmW0vousqpg7EA6J7okTUem/iswbxbQnpR4AMdJ7NwkKfa/OvHa6UB9UL
Nb5gE7lxplKOlJKfeGjrL6ABazpwsk0J4oS9r2GoWDfs30ji3TX5ixVvJ1KCFAFZ
dEkS1SazNjAWuDWuptVbu3+4DV34XSUg+kvcCXNTlOHz5KVg5ASzjF0QFvXexjCD
BhcIqJ3OI8VtL4AI+HXg+GTpHOUYtGct/Y+5gB16VvoSuX9vNOPBYx2M9Q4Nhb/C
U9eM7wqaS6lxX4UGId0ZgrNwHROreTEeGBScn1q1Rm7B6qJgeYo7wrzuKXCyRgcb
/evbcZ1DWJTXEORsJobzNiXHQWBtAEs4rLycOKAGx8//kD7vEmy2itslPlEaSgFq
jvoAHRNpiE3VW/hItB8WFTgWxiKaIz624gNrMjI2RYI/JSNhKVo=
=Y3yy
-----END PGP SIGNATURE-----
Merge tag 'x86_tdx_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 TDX updates from Dave Hansen:
"There are a few cleanups, and some changes that should allow TDX and
kexec to coexist nicely.
The biggest change, however, is support for updating the TDX module
after boot, just like CPU microcode. TDX users really want this
because it lets them do security updates without tearing things down
and rebooting.
- Add TDX module update support
- Make kexec and TDX finally place nice together
- Put TDX error codes into a single header"
* tag 'x86_tdx_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (30 commits)
x86/virt/tdx: Document TDX module update
x86/virt/tdx: Enable TDX module runtime updates
x86/virt/tdx: Refresh TDX module version after update
coco/tdx-host: Lock out module updates when reading version
x86/virt/seamldr: Add module update locking
x86/virt/tdx: Restore TDX module state
x86/virt/seamldr: Initialize the newly-installed TDX module
x86/virt/seamldr: Install a new TDX module
x86/virt/tdx: Reset software states during TDX module shutdown
x86/virt/seamldr: Shut down the current TDX module
x86/virt/seamldr: Abort updates after a failed step
x86/virt/seamldr: Introduce skeleton for TDX module updates
x86/virt/seamldr: Allocate and populate a module update request
coco/tdx-host: Implement firmware upload sysfs ABI for TDX module updates
coco/tdx-host: Don't expose P-SEAMLDR information on CPUs with erratum
coco/tdx-host: Expose P-SEAMLDR information via sysfs
x86/virt/seamldr: Add a helper to retrieve P-SEAMLDR information
x86/virt/seamldr: Introduce a wrapper for P-SEAMLDR SEAMCALLs
coco/tdx-host: Expose TDX module version
coco/tdx-host: Introduce a "tdx_host" device
...
* Update comment about pgd_list
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEV76QKkVc4xCGURexaDWVMHDJkrAFAmowVdwACgkQaDWVMHDJ
krARXA//c+FCH86tcGuZndWmebCP5ZuPFDkZMQuK53cg2hZq6OHUSxDL989iq/AO
YMfKR+CFJtpMd3fiLiASbks+mctWLJYjX1eWFvjtGaZPsH4u0CdlbPkYsfY5OKcA
NYeZ9Pbag/DYDu/SvSylDaYR0P1VgncaF4fhP5gH3YbhEU4rzNyrf6T+HBPz0B1h
iXWre0QAkXLrx2pkKRx8h55uy+hNLjLfExD6tbMMxnOHtOVaBL+JHG8czBgv6P7F
VrQ2vu8BKBqq8+rPGFVuolvGri2iM+hpddemKzc/wFHxP9oW7bLj9bieuoQk3R3p
1ILhDF0rfwoTzgzw2wD3yJpzP5BNB+tqlOqnN+KUFK8w1l01I1dbzfz/Ib/ozH3O
fAOLCAC9nlZ0VFozzPvpe3YHGI2feK/sW0Wh3X7YcfibmofpCHn8BkSgCo0cEjXs
JkcMqyCPEyYQUwWvbV39yeMKUnfi447UQdr95q4QNxi8stozQewLN6r/Y4DnGuEi
qZWzOSBGr3wCK25Tn0MF4vv0oVWb+4UFhF/79FkJBPZq78390XcmyYBC/tiWx8KJ
qQPhpfk3TIga2mmQsq1qwr4am7XXwjmB6sZHaprV7LDq/YLFnJTsmhIsq0nicrYZ
88S+XF94z/woCEUMaYjcp7E+bBUu5n4i2yWocu5yHwFd0pmuDGo=
=tM/K
-----END PGP SIGNATURE-----
Merge tag 'x86_mm_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 mm updates from Dave Hansen:
"There's a small comment fixup, followed by an actual bug fix. The bug
fix is slightly new territory for us. It fixes a real regression, but
it is from ~6.18 not a _new_ regression. It was mostly only biting
folks doing hotplug memory testing. So, it was queued in here and held
for the merge window instead of going via x86/urgent.
- Fix freeing of PMD-sized vmemmap pages
- Update comment about pgd_list"
* tag 'x86_mm_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/mm: Fix freeing of PMD-sized vmemmap pages
x86: Update comment about pgd_list
* Remove unused header
* Ensure printf() validation in all configs
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEV76QKkVc4xCGURexaDWVMHDJkrAFAmowVYsACgkQaDWVMHDJ
krCtFhAAjNOBz4+N7RHyG45U8AX5zLFSvKLia0SIEvgnprYNT7WJqFHDYLajRV4C
62m6OBinY4NkG5tbwABOj4XsdYGC1S2tVOTqTxtvC/6FvsUE8gKkjYDL58fnFeST
m1RCryPiDyrauc5mrN41pMvYIHE8Dgw+8Yor35RzZP1Nr4wwN5LtOPHmIZbz/QVj
XVcheDNBQU4jRa5OnTnmx97spywTlT9JnWyO1F9gKkTwBA6+sl1tAd9FXsfxGw9t
JWLJqPxpz/1G/gI13KOau6WkfwdPUxQioJ4ZIbO+C2UemUFYu+ZGwbdHnSVkdzdJ
4xU6w3lS9MdprS01V7EQrwaZmsdH7z6lvoHsnwN2Wal+Sagt3Ukr/dEBWDlJmzTB
+HfFFEmZL9UTPDoHGUoLOVEy4NafAwZZBDOciJe8K0qLGqg4S8xCw2W9LhfJGIGp
W6fSYssmOJX40qFvDpLX6GOK/MEkCerqfUfcFFCaKd2ZGnM4qsKtfHxYLbHL1Soq
TkWsC5hMWImi1tiuEdqDtPUODWiZ7k0HTN2C/9pI4IIUMUxdB3Jm3c43GuGsZNl0
y9PGVq8uIZiRV8FAu8NBsPb65UzRNVFPH8WATxTK2mVsMqzGxwMpM5ROQ3pkiPu+
DL1izUOADIYkxkvt00K5Lc5tOP29vCE6lKH5neXeZjhvfLEHb0U=
=8iBm
-----END PGP SIGNATURE-----
Merge tag 'x86_misc_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull misc x86 updates from Dave Hansen:
"These are the usual random pile, with the one exception of moving Rick
over to be a TDX maintainer. Rick has been doing a great job with TDX
contributions, especially on the host side of things. It's time to
promote him to "M".
- Move Rick Edgecombe to TDX maintainer
- Remove unused header
- Ensure printf() validation in all configs"
* tag 'x86_misc_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
MAINTAINERS: Move Rick Edgecombe to TDX maintainer
x86: Remove unnecessary architecture-specific <asm/device.h>
x86/bug: Put HAVE_ARCH_BUG_FORMAT_ARGS WARN definitions inside __ASSEMBLER__
x86/bug: Add printf() validation to HAVE_ARCH_BUG_FORMAT_ARGS WARNs
call paths now that the GHCB helpers handle early-boot fallback internally
- Skip SNP initialization in the CCP driver immediately when the preparation
step fails rather than proceeding to an operation that will certainly fail
- Abort SNP preparation and return an error when not all CPUs are online,
since the firmware enforces that every CPU enables SNP and will fail init if
not
- Simplify the VMM communication exception entry path by replacing separate
kernel and user mode macros with a single handler that dispatches based on
the current privilege level
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmowORwACgkQEsHwGGHe
VUqphg//VVXH3Svs2K5+++WQvavXDVwB5B+W4KA5WYYz3I77nmXgb2++aGSptGfy
ckC8ESFwyRWRh6aaOMtWGdbTQryS9e72sqiZEphbAZCe2XwR/A0N60uUktUCGm1W
3ce/VRkeTJI4LTkx3g+VSPTGzDpqfbwwTAJQZt8Ny8kooQCUH+VA7JvyLOYFvyTg
4NtjP2MVTzjE1io9SpZdSAbG/wdn7l7cFI+xC1QXYKZBzHSK5e4vUwnSGd86lrCa
2Q+zVcGcA7w2J6Obx4dNde5Y7AhGmYnhDxbFtNp9iUmG/o75LpVvP+8v7wshSGiL
HjTsRy5tkjrF9MUNXZd6cwFQ9gs6wWAwxXUsPtfs2N8R6Ve0W9aR49MUp1sEFuKm
hNgRLA7idSQ0RUSrL4f3c7Zx8ML6nXM2yRgE+4UTbr5/untHCqYunHelEIy1uay5
2vbqJ5VzT+pfY9qZn+q41AM2E1+ZzLmXFuGPBSOU4G+dU0CoVo+OUrsKZh8Q/4Ff
DajeiTW1+UidZyk2nb+HoX3DIa7jsUBeA3IGAQv8wK31jj3XhKwJTFPwPGkdjcpw
iR35vPzZFh0JLm+AHGrjgLI2+DkWEKl3j6lxI94UxsugaqkywtPZ5XJhJC54YHS9
iOwlpZK37eYy6zITx1+bwfzrW4oSsEZyYHkNgYJhU/RW4MrNJiU=
=Wq1K
-----END PGP SIGNATURE-----
Merge tag 'x86_sev_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 SEV updates from Borislav Petkov:
- Remove redundant GHCB initialization guards in the SEV page state and
SVSM call paths now that the GHCB helpers handle early-boot fallback
internally
- Skip SNP initialization in the CCP driver immediately when the
preparation step fails rather than proceeding to an operation that
will certainly fail
- Abort SNP preparation and return an error when not all CPUs are
online, since the firmware enforces that every CPU enables SNP and
will fail init if not
- Simplify the VMM communication exception entry path by replacing
separate kernel and user mode macros with a single handler that
dispatches based on the current privilege level
* tag 'x86_sev_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/sev: Remove redundant ghcbs_initialized checks around __sev_{get,put}_ghcb()
crypto/ccp: Skip SNP_INIT if preparation fails
x86/sev: Do not initialize SNP if missing CPUs
x86/entry: Zap the #VC entry user and kernel macros
retrieval function and restrict it to Zen family processors, ensuring
patch level arithmetic always operates on a valid revision
- Fix an incorrect comment about which CPUID bit is checked when
determining whether the microcode loader should be disabled
- Add the latest Intel microcode revision data for a broad range of
processor models and steppings and add the script which generates the
header of minimum expected Intel microcode revisions
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmowMfYACgkQEsHwGGHe
VUq4jRAAiUoRmAPSgxfw6fNzn+VcH2pxRa987HtZM/teJO9dec2fswhnOsOdx1Y+
hbQ1uxPdgZzowg2CJL/Il+QNGjpVkA0SaVbnlamX5bRMuEKdWXXawf0eeV3EMmmh
f+LaN/jQ3+vuE5boSOh1tVGQOTPyuyOg977leXSPlXxyTMyrVuLAN6+Zs/qsuhSY
vWTKY40E+1ZeDHCXcNtOhxKXR7tGjgGrjqLMx0bbs8z1DOP2OJpILHdgPB9igghb
hg5uUSj2LTd+28/H0HIu5RqfI7/ulQPBmLFYmw6ENLSR4U3GEtg0gItZEXL6E8Dz
UZVMqorpAMV41cPyBvStfK6R8cvGEe6m+iidmsTQDVvQWX9xnSNX1N+HV6Xr+fjN
kuPZLEGhkmDRe8mup+n5t/wqw4iDFrfqhgPGgFN2fwDFcs9oJfAHTgMvpdvWGG/2
hbME8PGWFv+N3Piy4GqROuVjcm0cbTTzdzWmWEZ8qO3OakSDC9vex5061/O5DmFB
vI7QHanc3Xe3vuf20Jnbc8LFkw54zHBSITrguah0yJFsVqE4tktSsy3NIHLxdP87
7e4oCzmP7oelCgxzvKyZiHzNtzrngGT9L0Nowcg9y/iibWzOkW8F9Ek5NqMaUyo4
JEKXizudArM6oZUtk+E6CAhGotL1en/FhfBmungtaExA2BnCJtc=
=KarZ
-----END PGP SIGNATURE-----
Merge tag 'x86_microcode_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 microcode loader updates from Borislav Petkov:
- Move the zero-revision fixup for AMD microcode to the patch level
retrieval function and restrict it to Zen family processors, ensuring
patch level arithmetic always operates on a valid revision
- Fix an incorrect comment about which CPUID bit is checked when
determining whether the microcode loader should be disabled
- Add the latest Intel microcode revision data for a broad range of
processor models and steppings and add the script which generates the
header of minimum expected Intel microcode revisions
* tag 'x86_microcode_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/microcode/AMD: Move the no-revision fixup to get_patch_level()
x86/microcode: Fix comment in microcode_loader_disabled()
scripts/x86/intel: Add a script to update the old microcode list
x86/microcode/intel: Refresh old_microcode defines with Nov 2025 release
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmowF/UACgkQEsHwGGHe
VUqPFQ//WJ62+aEsmuuZZ/HWuU/Warawn3R5Hm0DxgXAssUviuyOKBJhoP9ApaCK
SfyNxSqcu2QJed0tZXfJsF+qH8OrNf/FjkenWpgrBJDtl+qiRRT/WRekVtzUR5WO
HFemO+vnR5lgKKPwPlDFGs3/rARHPWs8HEl984PrjZJajwWnQujhkdZA88Hj8ehH
hglS780Uitdp/8aqYB8mlsDdb1JPL2m3Ajoagij7nX9FjLz4fayMjiQW+w/XfYTw
VP9vJtwXsVHP8inFLJPctKx2XRNYKU4g6WOGd2j3tIIeE9pvOpRbLJopeFgWAzbU
zhxCMMlW30KmuBIRAUQAG6B2xlJxAqsZbH7om7QPXRNLYJ8wMlqPqZ5Q3WW52cmo
YLbFDtcrHSn79Gukn0RZIN66xc6h1zKakhByZ5IPAB4GK2aJcS1f6OJCMIVknZUy
FlkCH+YiRSWn3yJVUgeVK8QbG0+n4r+a2QhnT/ems2nVzmBvLYHbV9QtKEdfxTj8
aD8Nwjh40mYvzOkbzCVyPO7QR/7SxIumaT/LsDvxMMMKBzuzS6BEy8WBtztxhhsU
yTABDf8WInRwTPe8P8jCArpFRlRbLeXkkBqzvQlWMJEty1Md1Id+LdAF+hCEOTip
jEYPWnmsaEnIFcJUQ/Am9f+ST8sq91kR92fLadxiWIuLjyQA1bo=
=sSrt
-----END PGP SIGNATURE-----
Merge tag 'x86_cleanups_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 cleanups from Borislav Petkov:
- The usual pile of cleanups and fixlets the cat dragged in
* tag 'x86_cleanups_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/cpu: Remove obsolete aperfmperf_get_khz() declaration
x86/pmem: Check for platform_device_alloc() retval
x86/platform/uv: Use str_enabled_disabled() in uv_nmi_setup_hubless_intr()
x86/cpu: Keep the PROCESSOR_SELECT menu together
x86/tlb: Convert copy_from_user() + kstrtouint() to kstrtouint_from_user()
x86/purgatory: Fix #endif comment
x86/boot: Get rid of kstrtoull()
x86/boot/compressed: Use boot_kstrtoul() for hugepages= parsing
- Simplify the error handling path when creating monitor group event
configuration directories
- Make the MBM event filter configurable only on architectures that
support it and expose this with the respective file modes in the event
config
- Disallow the MBA software controller on systems where MBM counters are
assignable, as it requires continuous bandwidth measurement that
assignable counters do not guarantee
- Replace a compile-time Kconfig option for fixed counter assignment
with a per-architecture runtime property, and expose whether the
counter assignment mode is changeable to userspace
- Continue counter allocation across all domains instead of aborting at
the first failure
- Document that automatic MBM counter assignment is best effort and may
not assign counters to all domains
- Document the behavior of task ID 0 and idle tasks in the resctrl tasks
file
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEzv7L6UO9uDPlPSfHEsHwGGHeVUoFAmowEfYACgkQEsHwGGHe
VUoqpA//SQnC9ehAKVg+tf/V0H5s4sAQimTI3fl9pMYWfduoHCQkdPs/lSWF4Su+
YNhjr/+nffC1oqSpklG2XEDHYD3HoKnBqSrALX0lmEVPr4MHtAnOAQs/G57Jqp8G
p1nniOgPqwDltndHYUl2vLX/dFpjIprqPG3lWfjPCFUzrVaOlTPd7Kkv8kSdmZOL
IYMpMP8IPjA8QfLdGtcqKbtjsNu6gxNs9TS2pXSyy/NuPqaS9CAzsOwj0KNlYPOk
Hi7KVXc87GAHAUS1LK8+ZjEao/BHtUE8XJlsnTNtPlvwfS7uuOCIrrQyooJS8uT4
CKv5KbCxZKlRlWNnZLW6IsfYd85N+7gse9n0U2BNlqMor535AEp/X5bka6QO4mLa
CuN72A8Tkw2bNEcUfOc1akeKNj7bQ9lgz0AESJefmaFNkLmaWJi5mNA/27JFAXBQ
KfGvPHlKcwt7e8Mj3OaDnhEstf9kVppD2NAL+tl5eGA0mHixyK0WNrDqX1ZjiPkL
Lf2w5bHA0tTUPtFR/71RqWyHL5N7jjctC6aeVmGDgji7dIkodD8bUFEj/ORY02iS
oAX7n0E8ccC/E8+jAb8n+zukpbHE4V2ASr+tncLuGVNRuq03YfK5RXIwK0wN6OnM
dDN1DjaNNMI5pkB48zytDhl1RK6QKVtOqBRBO2TMbQHypnjUMDI=
=6iVq
-----END PGP SIGNATURE-----
Merge tag 'x86_cache_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 resource control updates from Borislav Petkov:
"Preparatory work for MPAM counter assignment:
- Simplify the error handling path when creating monitor group event
configuration directories
- Make the MBM event filter configurable only on architectures that
support it and expose this with the respective file modes in the
event config
- Disallow the MBA software controller on systems where MBM counters
are assignable, as it requires continuous bandwidth measurement
that assignable counters do not guarantee
- Replace a compile-time Kconfig option for fixed counter assignment
with a per-architecture runtime property, and expose whether the
counter assignment mode is changeable to userspace
- Continue counter allocation across all domains instead of aborting
at the first failure
- Document that automatic MBM counter assignment is best effort and
may not assign counters to all domains
- Document the behavior of task ID 0 and idle tasks in the resctrl
tasks file"
* tag 'x86_cache_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
fs/resctrl: Document tasks file behaviour for task id 0 and idle tasks
fs/resctrl: Document that automatic counter assignment is best effort
fs/resctrl: Continue counter allocation after failure
fs/resctrl: Add monitor property 'mbm_cntr_assign_fixed'
fs/resctrl: Disallow the software controller when MBM counters are assignable
x86,fs/resctrl: Create 'event_filter' files read only if they're not configurable
fs/resctrl: Tidy up the error path in resctrl_mkdir_event_configs()
When recovering hugepages in the shadow MMU, verify that the base gfn of
the shadow page is actually contained within the target memslot, *before*
querying the max mapping level given the shadow page's gfn. Failure to
pre-check the validity of the gfn can lead to an out-of-bounds access to
the slot's lpage_info (which typically manifests as a host #PF because the
lpage_info is vmalloc'd) if the guest creates a hugepage mapping (in its
PTEs) that extends "below" the bounds of a memslot.
When faulting in memory for a guest, and the size of the guest mapping is
greater than KVM's (current) max mapping, then KVM will create a "direct"
shadow page (direct in that there are no gPTEs to shadow, and so the target
gfn is a direct calculation given the base gfn of the shadow page). The
hugepage recovery flow looks for such direct shadow pages, as forcing 4KiB
mappings when dirty logging generates the guest > host mapping size case.
When the 4KiB restriction is lifted, then KVM can replace the shadow page
with a hugepage.
But if KVM originally used a smaller mapping than the guest because the
range of memory covered by the guest hugepage exceeds the bounds of a
memslot, then KVM will link a direct shadow page with a gfn that is outside
the bounds of the memslot being used to fault in memory. The rmap entry
added for the leaf mapping is correct and within bounds, but the gfn of the
leaf SPTE's parent shadow page will be out of bounds.
BUG: unable to handle page fault for address: ffffc90000806ffc
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 100000067 P4D 100000067 PUD 1002a7067 PMD 10612f067 PTE 0
Oops: Oops: 0000 [#1] SMP
CPU: 13 UID: 1000 PID: 757 Comm: mmu_stress_test Not tainted 7.1.0-rc1-48ce1e26eace-x86_pir_to_irr_comments-vm #341 PREEMPT
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
RIP: 0010:kvm_mmu_max_mapping_level+0x79/0x2b0 [kvm]
Call Trace:
<TASK>
kvm_mmu_recover_huge_pages+0x21b/0x320 [kvm]
kvm_set_memslot+0x1ee/0x590 [kvm]
kvm_set_memory_region.part.0+0x3a1/0x4d0 [kvm]
kvm_vm_ioctl+0x9bf/0x15d0 [kvm]
__x64_sys_ioctl+0x8a/0xd0
do_syscall_64+0xb7/0xbb0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7f21c0f1a9bf
</TASK>
Don't bother pre-checking the bounds of the potential hugepage, i.e. don't
check that e.g. sp->gfn + KVM_PAGES_PER_HPAGE(sp->role.level + 1) is also
within the memslot, as the checks performed by kvm_mmu_max_mapping_level()
are a superset of the basic bounds checks. I.e. pre-checking the full
range would be a dubious micro-optimization.
Fixes: 9eba50f8d7 ("KVM: x86/mmu: Consult max mapping level when zapping collapsible SPTEs")
Cc: stable@vger.kernel.org
Cc: David Matlack <dmatlack@google.com>
Cc: James Houghton <jthoughton@google.com>
Cc: Alexander Bulekov <bkov@amazon.com>
Cc: Fred Griffoul <fgriffo@amazon.co.uk>
Cc: Alexander Graf <graf@amazon.de>
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Filippo Sironi <sironi@amazon.de>
Cc: Ivan Orlov <iorlov@amazon.co.uk>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Commit 0cb2af2ea6 ("KVM: x86: Fix shadow paging use-after-free due
to unexpected GFN") fixed a shadow paging mismatch between stored and
computed GFNs; the bug could be triggered by changing a PDE mapping from
outside the guest, and then deleting a memslot. The rmap_remove()
call would miss entries created after the PDE change because the GFN
of the leaf SPTE does not match the GFN of the struct kvm_mmu_page.
A similar hole however remains if the modified PDE points to a non-leaf
page. In this case the gfn can be made to match, but the role does not
match: the original large 2MB page creates a kvm_mmu_page with direct=1,
while the new 4KB needs a kvm_mmu_page with direct=0. However,
kvm_mmu_get_child_sp() does not compare the role, and therefore reuses
the page.
The next step is installing a leaf (4KB) SPTE on the new path which
records an rmap entry under the gfn resolved by the walk. But when
that child is zapped its parent kvm_mmu_page has direct=1 and
kvm_mmu_page_get_gfn() computes the gfn for the 4KB page as
sp->gfn + index instead of using sp->shadowed_translation[] (or sp->gfns[]
in older kernels). It therefore fails to remove the recorded entry.
When the memslot is dropped the shadow page is freed but the rmap
entry survives, as in the scenario that was already fixed. Code that
later walks that gfn (dirty logging, MMU notifier invalidation, and
so on) dereferences an sptep that lies in the freed page, causing the
use-after-free.
Fixes: 2032a93d66 ("KVM: MMU: Don't allocate gfns page for direct mmu pages")
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- CPUID API updates (Ahmed S. Darwish):
- Introduce a centralized CPUID parser
- Introduce a centralized CPUID data model
- Introduce <asm/cpuid/leaf_types.h>
- Rename cpuid_leaf()/cpuid_subleaf() APIs
- treewide: Explicitly include the x86 CPUID headers
- Update to x86-cpuid-db v3.1 (Maciej Wieczor-Retman)
- Continued removal of pre-i586 support and related
simplifications (Ingo Molnar)
- Add Intel CPU model number for rugged Panther Lake
(Tony Luck)
- Misc fixes, updates and cleanups by Arnd Bergmann, Chao Gao,
Lukas Bulwahn, Sohil Mehta, Maciej Wieczor-Retman.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmou1LcRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1h16Q/9GvxrQDX13qd7/XrjOKNYtoLIGh1GhvHM
2ZdbYh80LX36bDOg7Mhoy0bvElwmzaz2x1Zrb5SAIqpQXYqjfuRCe7a1SPQxYtCK
A6j1YeD/CJMm36jaQITYkuCYxwaw5LQ62u0ShpEvyzZzarEJt8c8COWJjbE57FN2
qusr+6K1sBwpEUl5mLabbJWXqhpPRYCz+nl2GF9BCHe6z7Nw7Q8VZf4w1weONJOy
lMpC2X08w5TE2f4OwvnRbZLelyROz6pr1c3osUoQSVtBkprO9TsYHgd9yBva1E2E
KxdWm+uSSUXM7cFBzY3RLBzKpG6iLtqircoDFBVdxOlT8I8KFggzbYy4JPlDEHQS
FqUwWL+2duoRciOncnZn9hydxlg5So4w7OscvTVNp94/Pb2pMdpOB+bsN05WTpXc
VMQVxrfsmwT1rV1oppzRizTMnb6yGg8BQvZPE+wrMDdH+/wwhzok4GdZzKVCzUoD
nDFhJL7t8ikRwwuq5RAl1EcT7qLFJx3ba77B5REf2Uos2liTExO/XzcmQnzSV1M1
RXpajQ1ikEBtJPAD3Zy0ASm/QPjB27o+6Kj6hW3NLlPoi5l0Lu6xQzH3wNrpQpst
02NCWEn2X00YUQt2ukNcDFJiMHJzXAnBQDBSrWwwd97wol3sKnT9RR2NZVFzn7ls
GOF8yJ0Wuec=
=tpai
-----END PGP SIGNATURE-----
Merge tag 'x86-cpu-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 cpuid updates from Ingo Molnar:
- CPUID API updates (Ahmed S. Darwish):
- Introduce a centralized CPUID parser
- Introduce a centralized CPUID data model
- Introduce <asm/cpuid/leaf_types.h>
- Rename cpuid_leaf()/cpuid_subleaf() APIs
- treewide: Explicitly include the x86 CPUID headers
- Update to x86-cpuid-db v3.1 (Maciej Wieczor-Retman)
- Continued removal of pre-i586 support and related simplifications
(Ingo Molnar)
- Add Intel CPU model number for rugged Panther Lake (Tony Luck)
- Misc fixes, updates and cleanups by Arnd Bergmann, Chao Gao, Lukas
Bulwahn, Sohil Mehta, Maciej Wieczor-Retman.
* tag 'x86-cpu-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (25 commits)
x86/cpu: Make CONFIG_X86_CX8 unconditional
x86/cpu: Remove unused !CONFIG_X86_TSC code
x86/cpuid: Update bitfields to x86-cpuid-db v3.1
tools/x86/kcpuid: Update bitfields to x86-cpuid-db v3.1
x86/cpu: Make CONFIG_X86_TSC unconditional
MAINTAINERS: Drop obsolete FPU EMULATOR section
x86/cpu: Fix a F00F bug warning and clean up surrounding code
x86/cpu: Add Intel CPU model number for rugged Panther Lake
x86/cpuid: Introduce a centralized CPUID parser
x86/cpu: Introduce a centralized CPUID data model
x86/cpuid: Introduce <asm/cpuid/leaf_types.h>
x86/cpuid: Rename cpuid_leaf()/cpuid_subleaf() APIs
x86/cpu: Do not include the CPUID API header in asm/processor.h
Documentation: core-api/cpu_hotplug: Remove stale cpu0_hotplug docs
x86/cpu, cpufreq: Remove AMD ELAN support
x86/fpu: Remove the math-emu/ FPU emulation library
x86/fpu: Remove the 'no387' boot option
x86/fpu: Remove MATH_EMULATION and related glue code
treewide: Explicitly include the x86 CPUID headers
x86/cpu: Remove the CONFIG_X86_INVD_BUG quirk
...
SMP load-balancing updates:
- A large series to introduce infrastructure for cache-aware load
balancing, with the goal of co-locating tasks that share data
within the same Last Level Cache (LLC) domain. By improving cache
locality, the scheduler can reduce cache bouncing and cache misses,
ultimately improving data access efficiency.
Implemented by Chen Yu and Tim Chen, based on early prototype work
by Peter Zijlstra, with fixes by Jianyong Wu, Peter Zijlstra and
Shrikanth Hegde.
- A series to simplify CONFIG_SCHED_SMT ifdef usage (Shrikanth Hegde)
Fair scheduler updates:
- A series to improve SD_ASYM_CPUCAPACITY scheduling by introducing
SMT awareness (Andrea Righi, K Prateek Nayak)
- A series to optimize cfs_rq and sched_entity allocation for
better data locality (Zecheng Li)
- A preparatory series to change fair/cgroup scheduling to a single
runqueue, without the final change (Peter Zijlstra)
- Auto-manage ext/fair dl_server bandwidth (Andrea Righi)
- Fix cpu_util runnable_avg arithmetic (Hongyan Xia)
- Optimize update_tg_load_avg()'s rate-limiting code
(Rik van Riel)
- Allow account_cfs_rq_runtime() to throttle current hierarchy
(K Prateek Nayak)
- Update util_est after updating util_avg during dequeue, to fix
the util signal update logic, which reduces signal noise
(Vincent Guittot)
Scheduler topology updates:
- Allow multiple domains to claim sched_domain_shared (K Prateek Nayak)
- Add parameter to split LLC (Peter Zijlstra)
Core scheduler updates:
- Use trace_call__<tp>() to save a static branch (Gabriele Monaco)
Scheduler statistics updates:
- Drop now-stale mul_u64_u64_div_u64() cputime over-approximation guard
(Nicolas Pitre)
Deadline scheduler updates:
- Reject debugfs dl_server writes for offline CPUs (Andrea Righi)
- Fix replenishment logic for non-deferred servers
(Yuri Andriaccio)
RT scheduling updates:
- Turn RT_PUSH_IPI default off for non PREEMPT_RT (Steven Rostedt)
- Update default bandwidth for real-time tasks to 1.0
(Yuri Andriaccio)
Proxy scheduling updates:
- A series to implement Optimized Donor Migration for Proxy Execution
(John Stultz, Peter Zijlstra)
- Various proxy scheduling cleanups and fixes
(Peter Zijlstra, K Prateek Nayak)
Misc fixes, improvements and cleanups by Aaron Lu, Andrea Righi,
Zenghui Yu, Chen Yu, Guanyou.Chen, John Stultz, Shrikanth Hegde,
Peter Zijlstra, Liang Luo and Yiyang Chen.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmouy9ERHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1iLrxAApGz9fhzT8k8slnaSe9kzX5K3OtBLivi1
ZHPeyNHggonASdZ7+apwJq+sdtmAgEw/phMN69fHhteIIa24WCHEpFtLndDHAkAm
eUKRgBI/eAMrOBECldZ69QO1pfYmMV/2aZH/IyTDtgFvXYZS2+UtsE9fPCvx84V2
Uxa5r7a8NTrpbhsZ2YhncpLkrJx0SfaBQBtvckDVWQBbh5stvIZncCatYjX0M33x
yddLVkM7e5f6nenb5+rwW3TMayfzhkeaR+r9vUuZE1mt9ItwIqKfsi9PBFRFXNyr
zzTxGpN6iQCgtFtySJQpIYamjxJhMZYpWcirYXRkXbmjPgc3PIgA9fRcPcj9bZ0A
Z1sM9yBDFjvZ7Eds9TW7iEsmxSpbXDAXDs6SjQa34QU8bh7qJEOG9AQYDcULXZQM
5NK7+b6GEMvcS7P4Y66CKwvxYFFGVGzX3CrFEgngkOWNJeoWhz5EpgmYIocgl/uS
1TPIWASdFLyvvtnGxMx9iWcdb4mGbXvMLbDa3luSBDWAtYxQe0w9iLi87eqypSvU
irZBSbKX737OlrOzNy/d4DKk9sbNsk/In1IK3jfHCt+7iRpVB5aWBfekrg0DIREw
Jo83ln1nApdusK11DLQpbZcUerLHsvmGXRlGJa3IRGAXUS7MAnJ+XpYkKgmjW8Vm
rw4QLfkMbP4=
=V3Wc
-----END PGP SIGNATURE-----
Merge tag 'sched-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull scheduler updates from Ingo Molnar:
"SMP load-balancing updates:
- A large series to introduce infrastructure for cache-aware load
balancing, with the goal of co-locating tasks that share data
within the same Last Level Cache (LLC) domain. By improving cache
locality, the scheduler can reduce cache bouncing and cache misses,
ultimately improving data access efficiency.
Implemented by Chen Yu and Tim Chen, based on early prototype work
by Peter Zijlstra, with fixes by Jianyong Wu, Peter Zijlstra and
Shrikanth Hegde.
- A series to simplify CONFIG_SCHED_SMT ifdef usage (Shrikanth Hegde)
Fair scheduler updates:
- A series to improve SD_ASYM_CPUCAPACITY scheduling by introducing
SMT awareness (Andrea Righi, K Prateek Nayak)
- A series to optimize cfs_rq and sched_entity allocation for better
data locality (Zecheng Li)
- A preparatory series to change fair/cgroup scheduling to a single
runqueue, without the final change (Peter Zijlstra)
- Auto-manage ext/fair dl_server bandwidth (Andrea Righi)
- Fix cpu_util runnable_avg arithmetic (Hongyan Xia)
- Optimize update_tg_load_avg()'s rate-limiting code (Rik van Riel)
- Allow account_cfs_rq_runtime() to throttle current hierarchy
(K Prateek Nayak)
- Update util_est after updating util_avg during dequeue, to fix the
util signal update logic, which reduces signal noise (Vincent
Guittot)
Scheduler topology updates:
- Allow multiple domains to claim sched_domain_shared (K Prateek
Nayak)
- Add parameter to split LLC (Peter Zijlstra)
Core scheduler updates:
- Use trace_call__<tp>() to save a static branch (Gabriele Monaco)
Scheduler statistics updates:
- Drop now-stale mul_u64_u64_div_u64() cputime over-approximation
guard (Nicolas Pitre)
Deadline scheduler updates:
- Reject debugfs dl_server writes for offline CPUs (Andrea Righi)
- Fix replenishment logic for non-deferred servers (Yuri Andriaccio)
RT scheduling updates:
- Turn RT_PUSH_IPI default off for non PREEMPT_RT (Steven Rostedt)
- Update default bandwidth for real-time tasks to 1.0 (Yuri
Andriaccio)
Proxy scheduling updates:
- A series to implement Optimized Donor Migration for Proxy Execution
(John Stultz, Peter Zijlstra)
- Various proxy scheduling cleanups and fixes (Peter Zijlstra,
K Prateek Nayak)
Misc fixes, improvements and cleanups by Aaron Lu, Andrea Righi,
Zenghui Yu, Chen Yu, Guanyou.Chen, John Stultz, Shrikanth Hegde,
Peter Zijlstra, Liang Luo and Yiyang Chen"
* tag 'sched-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (91 commits)
sched/fair: Fix newidle vs core-sched
sched/deadline: Use task_on_rq_migrating() helper
sched/core: Combine separate 'else' and 'if' statements
sched/fair: Fix cpu_util runnable_avg arithmetic
sched/fair: Unify cfs_rq throttling via account_cfs_rq_runtime()
sched/fair: Move the throttled tasks to a local list in tg_unthrottle_up()
sched/fair: Call update_curr() before unthrottling the hierarchy
sched/fair: Use throttled_csd_list for local unthrottle
sched/fair: Convert cfs bandwidth throttling to use guards
sched/fair: Allocate cfs_tg_state with percpu allocator
sched/fair: Remove task_group->se pointer array
sched/fair: Co-locate cfs_rq and sched_entity in cfs_tg_state
sched: restore timer_slack_ns when resetting RT policy on fork
MAINTAINERS: Fix spelling mistake in Peter's name
sched: Simplify ttwu_runnable()
sched/proxy: Remove superfluous clear_task_blocked_in()
sched/proxy: Remove PROXY_WAKING
sched/proxy: Switch proxy to use p->is_blocked
sched/proxy: Only return migrate when needed
sched: Be more strict about p->is_blocked
...
Core perf code updates:
- Reveal PMU type in fdinfo (Chun-Tse Shao)
Intel CPU PMU driver updates:
- Fix various inaccurate hard-coded event configurations
(Dapeng Mi)
Intel uncore PMU driver updates (Zide Chen):
- Fix discovery unit lookup bug for multi-die systems
- Guard against invalid box control address
- Fix PCI device refcount leak in UPI discovery
- Defer ADL global PMON enable to enable_box() to save power
- Fix uncore_die_to_cpu() for offline dies
- Implement global init callback for GNR uncore
AMD CPU PMU driver updates:
- Always use the NMI latency mitigation (Sandipan Das)
AMD uncore PMU driver updates:
- Use Node ID to identify DF and UMC domains (Sandipan Das)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmouvyARHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hVSA//RmBeZ9R7QiCJn770b+9ZPvk90PKN8TYQ
TXcPMTB1gZRtw7q4CF8/CFxUvxGySD+/RaekXkQzfATPurwgHkwKv3xpOZuSGcfW
dTsuAV3k+Azj3on3dGjmGZoZeWsdG12c0uF7YNylW/hfPQ/KAW1THVKv7ufX6Mcx
VrEh7oh/5N9paAqtZz5TmesB2wfC05vToj27jE0HL6tnDovF4Q8IsZkfC3P139Kd
iCwvOnc/jpfke26UP5csPnQIF5zTIy2TZrvDQVlqryPZ4NrmMnA6XC117u8VAIm2
29JBP72wbMvRT4hXSs8thfvPj1zF/y3YMCN0D+wkxpYD6gqlsjlHhoY9fJ+Ann7t
gMi0ccSGNC6COVMq1RhzrGdf6JscGcIxXzS1abNJw6cLmfv45UmlbF1ePq9nPV3n
8ZbotrBovS9TonLPmmXNpR9dKKLSH2URCF/xD92KS5Wbvod7sum8N/rpV3Ncmxxf
6g8qTsOxpJjEq9pnTndycqvtzZnyf0/68db5Y+5ag1zKNVPhbsV7AU/tVytZt1Hk
1bTNcc7lCp2GRBPAmDCk5KI235C0UpLS/llG2jEEwXmTzLxk7pzwoJMcl5sABHNA
VL6AhLP8rmHY+r0e8otdI10xg8Pt/WWvKNuOY4i4PiIf34YG1ARoY0EwzmlwcSmF
WsKdagUmIqQ=
=Di4Y
-----END PGP SIGNATURE-----
Merge tag 'perf-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull performance events updates from Ingo Molnar:
"Core perf code updates:
- Reveal PMU type in fdinfo (Chun-Tse Shao)
Intel CPU PMU driver updates:
- Fix various inaccurate hard-coded event configurations (Dapeng Mi)
Intel uncore PMU driver updates (Zide Chen):
- Fix discovery unit lookup bug for multi-die systems
- Guard against invalid box control address
- Fix PCI device refcount leak in UPI discovery
- Defer ADL global PMON enable to enable_box() to save power
- Fix uncore_die_to_cpu() for offline dies
- Implement global init callback for GNR uncore
AMD CPU PMU driver updates:
- Always use the NMI latency mitigation (Sandipan Das)
AMD uncore PMU driver updates:
- Use Node ID to identify DF and UMC domains (Sandipan Das)"
* tag 'perf-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (22 commits)
perf/x86/amd/uncore: Use Node ID to identify DF and UMC domains
perf: Reveal PMU type in fdinfo
perf/x86/intel/uncore: Implement global init callback for GNR uncore
perf/x86/intel/uncore: Fix uncore_die_to_cpu() for offline dies
perf/x86/intel/uncore: Move die_to_cpu() to uncore.c
perf/x86/intel/uncore: Defer ADL global PMON enable to enable_box()
perf/x86/intel/uncore: Fix PCI device refcount leak in UPI discovery
perf/x86/intel/uncore: Guard against invalid box control address
perf/x86/intel/uncore: Fix discovery unit lookup for multi-die systems
perf/x86/amd/core: Always use the NMI latency mitigation
perf/x86/intel: Update event constraints and cache_extra_regsfor CWF
perf/x86/intel: Update event constraints and cache_extra_regsfor SRF
perf/x86/intel: Update event constraints and cache_extra_regsfor NVL
perf/x86/intel: Update event constraints for PTL
perf/x86/intel: Update event constraints and cache_extra_regsfor ARL
perf/x86/intel: Update event constraints and cache_extra_regsfor LNL
perf/x86/intel: Update event constraints and cache_extra_regsfor MTL
perf/x86/intel: Update event constraints and cache_extra_regsfor ADL
perf/x86/intel: Update event constraints for DMR
perf/x86/intel: Update event constraints and cache_extra_regsfor SPR
...
- A large series of KLP fixes and improvements, in preparation of
the arm64 port (Josh Poimboeuf)
- Fix a number of bugs and issues on specific distro, LTO,
FineIBT and kCFI configs (Josh Poimboeuf)
- Misc other fixes by Josh Poimboeuf and Joe Lawrence.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmouvAgRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1g1rBAAuJwhyKKa5KxdgoGC3uSCyILZ77n5en+I
e5kCpeON2zbiGzg5N1D6Rgr1D1m1Fj1puiPy6FtqDkIO4a5b86Pt/Nv91CZG3A7n
lYNRJtqVyMNpBr7O9aaxalEnhrEY1kDK5DRebSAidy97hL6HelMFBy3EkV2nzsUI
eekUg3frCNq3k2oYWdtC5DZ6S7oK8rdAi/Ay3A3XTL6CLjYDPpMHd+pjDdtQsBYg
ybMoVQthfKb1SDazy2od9bbTJDngJkMVgIqHPrQcnvqMqp1knI46YvEwMJzQ9DWB
AaS8Xnm1jyk0/cFjIp5xNWo+VVuKiY0upewtFAodH35tvIuqptbt7btIkV5ZX7nu
IaMfqwh88yHvf3PJMrbAy+5lIE9h6dKEDLTJVBT8Swb7Y6rUG0qMyJ3Ah/RrlAgu
jcLUN89u8HXVaffWYAfpEj2ehSRsuy9BX2zCUeQoMH/5MCbDY2q4j6i07ylTJCkr
flYjTosIXma/NwimXIyPzMl+vUtrznMwoubH6Ns2yu7DKAK1Ntb0W9H7oy/vZIPi
ZAiV2OdmRet6iXdcIwBlS+7Du/eUMIkKlJsf4twWMgU7R4ApxTWoylr1z8yKqQam
109MNt7d98wFu+iYEWB0S+Uz7I/ZAGwY1Rj7tZ7mHUlSBfcVsYp3b3NEOG4J9YLy
K6L9ZECJ/Ow=
=D5SB
-----END PGP SIGNATURE-----
Merge tag 'objtool-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull objtool updates from Ingo Molnar:
- A large series of KLP fixes and improvements, in preparation of the
arm64 port (Josh Poimboeuf)
- Fix a number of bugs and issues on specific distro, LTO, FineIBT and
kCFI configs (Josh Poimboeuf)
- Misc other fixes by Josh Poimboeuf and Joe Lawrence
* tag 'objtool-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (53 commits)
objtool/klp: Cache dont_correlate() result
objtool: Improve and simplify prefix symbol detection
objtool/klp: Fix kCFI prefix finding/cloning
objtool: Grow __cfi_* prefix symbols for all CFI+CALL_PADDING
objtool/klp: Fix position-dependent checksums for non-relocated jumps/calls
objtool: Add insn_sym() helper
objtool/klp: Add correlation debugging output
objtool/klp: Rewrite symbol correlation algorithm
objtool/klp: Calculate object checksums
klp-build: Validate short-circuit prerequisites
objtool/klp: Remove "objtool --checksum"
klp-build: Use "objtool klp checksum" subcommand
objtool/klp: Add "objtool klp checksum" subcommand
objtool: Consolidate file decoding into decode_file()
objtool/klp: Extricate checksum calculation from validate_branch()
objtool: Add is_cold_func() helper
objtool: Add is_alias_sym() helper
objtool/klp: Handle Clang .data..Lanon anonymous data sections
objtool/klp: Create empty checksum sections for function-less object files
objtool: Include libsubcmd headers directly from source tree
...
Futex updates:
- Optimize futex hash bucket access patterns (Peter Zijlstra)
- Large series to address the robust futex unlock race for real,
by Thomas Gleixner:
"The robust futex unlock mechanism is racy in respect
to the clearing of the robust_list_head::list_op_pending
pointer because unlock and clearing the pointer are not
atomic. The race window is between the unlock and clearing the
pending op pointer. If the task is forced to exit in this
window, exit will access a potentially invalid pending op
pointer when cleaning up the robust list. That happens if
another task manages to unmap the object containing the lock
before the cleanup, which results in an UAF. In the worst case
this UAF can lead to memory corruption when unrelated content
has been mapped to the same address by the time the access
happens.
User space can't solve this problem without help from the kernel.
This series provides the kernel side infrastructure to help it
along:
1) Combined unlock, pointer clearing, wake-up for the
contended case
2) VDSO based unlock and pointer clearing helpers with a
fix-up function in the kernel when user space was interrupted
within the critical section.
... with help by André Almeida:
- Add a note about robust list race condition (André Almeida)
- Add self-tests for robust release operations (André Almeida)
Context analysis updates:
- Implement context analysis for 'struct rt_mutex'.
(Bart Van Assche)
- Bump required Clang version to 23 (Marco Elver)
Guard infrastructure updates:
- Series to remove NULL check from unconditional guards
(Dmitry Ilvokhin)
Lockdep updates:
- Restore self-test migrate_disable() and sched_rt_mutex
state on PREEMPT_RT (Karl Mehltretter)
Membarriers updates:
- Use per-CPU mutexes for targeted commands (Aniket Gattani)
- Modernize membarrier_global_expedited with cleanup guards
(Aniket Gattani)
- Add rseq stress test for CFS throttle interactions (Aniket Gattani)
percpu-rwsems updates:
- Extract __percpu_up_read() to optimize inlining overhead
(Dmitry Ilvokhin)
Seqlocks updates:
- Allow UBSAN_ALIGNMENT to fail optimizing (Heiko Carstens)
Lock tracing:
- Add contended_release tracepoint to sleepable locks
such as mutexes, percpu-rwsems, rtmutexes, rwsems
and semaphores. (Dmitry Ilvokhin)
MAINTAINERS updates:
- MAINTAINERS: Add RUST [SYNC] entry (Boqun Feng)
Misc updates and fixes by Randy Dunlap, YE WEI-HONG,
Fabricio Parra, Dmitry Ilvokhin and Peter Zijlstra.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmoujYURHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1jQFw//SNo1zxqGJuVudIWTpk5hUbWO6La9dc2C
2RW4lMNM551AaGFK6V01u1hD7dBZkWBqER+f3+CMtkqi734pAevTfkYJOitnGOOX
YZicGqTufV+Q+hcEiHAY3ADF6384SiyvAGNHL5wk3eLqC+Raug0f0cNt4opzFvr7
I6vRcW02JQMRKBe50+Aoa5zg7ymmbqaH0KkiPvfY5UFl38MKABhkY1izA9o+/Q/S
1/joNo5iQ3Ciqf6tn76vTg4Xiz7napikHB5LfWxorgmQo7FEdUY54GGMaa8aBOPi
EfgIV9p+qOJhmWn4H1HgVSYlnUn3tJICYnVVCruXWXD+pzITS+F5qSQ3p25F3VIt
c3jh3NaVW71pR/Sbj4zGvf8QguJL340RGJs/e5qcj91pjOSCH5B75ruNgFgYK9Lv
iRi6dnHBBHxaN57YHTOrhphvjyfc0L7i6cGyJgHJWwyi0y3XBYlNuP/VLPuwquKl
anK86hlkl9W4kO0OOukD1xhfuRv7066EdJc3iWWlp9yz+yI5xDGuV4FhvEHRoy38
R9VY2bwzgZrW/NkodHIb95+W30qEmeq2z1W4wiBu+mf1JnyW2g/n8jKkc6AINMpI
VGcom3MdinxjUtmybZT9Fx0SzBYCc+aRPkh7lPV7eXdZAWufxUO/6eVKiHBQ2HUO
RFDkOZU0r+c=
=gpHw
-----END PGP SIGNATURE-----
Merge tag 'locking-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull locking updates from Ingo Molnar:
"Futex updates:
- Optimize futex hash bucket access patterns (Peter Zijlstra)
- Large series to address the robust futex unlock race for real, by
Thomas Gleixner:
"The robust futex unlock mechanism is racy in respect to the
clearing of the robust_list_head::list_op_pending pointer because
unlock and clearing the pointer are not atomic.
The race window is between the unlock and clearing the pending op
pointer. If the task is forced to exit in this window, exit will
access a potentially invalid pending op pointer when cleaning up
the robust list.
That happens if another task manages to unmap the object
containing the lock before the cleanup, which results in an UAF.
In the worst case this UAF can lead to memory corruption when
unrelated content has been mapped to the same address by the time
the access happens.
User space can't solve this problem without help from the kernel.
This series provides the kernel side infrastructure to help it
along:
1) Combined unlock, pointer clearing, wake-up for the
contended case
2) VDSO based unlock and pointer clearing helpers with a
fix-up function in the kernel when user space was interrupted
within the critical section.
... with help by André Almeida:
- Add a note about robust list race condition (André Almeida)
- Add self-tests for robust release operations (André Almeida)
Context analysis updates:
- Implement context analysis for 'struct rt_mutex'. (Bart Van Assche)
- Bump required Clang version to 23 (Marco Elver)
Guard infrastructure updates:
- Series to remove NULL check from unconditional guards (Dmitry
Ilvokhin)
Lockdep updates:
- Restore self-test migrate_disable() and sched_rt_mutex state on
PREEMPT_RT (Karl Mehltretter)
Membarriers updates:
- Use per-CPU mutexes for targeted commands (Aniket Gattani)
- Modernize membarrier_global_expedited with cleanup guards (Aniket
Gattani)
- Add rseq stress test for CFS throttle interactions (Aniket Gattani)
percpu-rwsems updates:
- Extract __percpu_up_read() to optimize inlining overhead (Dmitry
Ilvokhin)
Seqlocks updates:
- Allow UBSAN_ALIGNMENT to fail optimizing (Heiko Carstens)
Lock tracing:
- Add contended_release tracepoint to sleepable locks such as
mutexes, percpu-rwsems, rtmutexes, rwsems and semaphores (Dmitry
Ilvokhin)
MAINTAINERS updates:
- MAINTAINERS: Add RUST [SYNC] entry (Boqun Feng)
Misc updates and fixes by Randy Dunlap, YE WEI-HONG, Fabricio Parra,
Dmitry Ilvokhin and Peter Zijlstra"
* tag 'locking-core-2026-06-14' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (36 commits)
locking: Add contended_release tracepoint to sleepable locks
locking/percpu-rwsem: Extract __percpu_up_read()
tracing/lock: Remove unnecessary linux/sched.h include
futex: Optimize futex hash bucket access patterns
rust: sync: completion: Mark inline complete_all and wait_for_completion
MAINTAINERS: Add RUST [SYNC] entry
cleanup: Specify nonnull argument index
selftests: futex: Add tests for robust release operations
Documentation: futex: Add a note about robust list race condition
x86/vdso: Implement __vdso_futex_robust_try_unlock()
x86/vdso: Prepare for robust futex unlock support
futex: Provide infrastructure to plug the non contended robust futex unlock race
futex: Add robust futex unlock IP range
futex: Add support for unlocking robust futexes
futex: Cleanup UAPI defines
x86: Select ARCH_MEMORY_ORDER_TSO
uaccess: Provide unsafe_atomic_store_release_user()
futex: Provide UABI defines for robust list entry modifiers
futex: Move futex related mm_struct data into a struct
futex: Make futex_mm_init() void
...
- Remove the redundant CONFIG_GENERIC_TIME_VSYSCALL after converting the
remaining users over.
- Rework and sanitize the MIPS VDSO handling, so it does not handle the
time related VDSO if there is no VDSO capable clocksource available.
Also stop mapping VDSO data pages unconditionally even if there is no
usage possible.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotog4QHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoYD5D/41UOOboDxkv3Ei0gS12RfFL/pHWPotWMvA
eUS4Ohko2pYubZB+iv87QQD5HC6TBN9yXbUjpvS0wN9jVSlXgQOdJ0JkNYYbPfjH
nsBt7g/ksCA0Tv0KxnUUtECCTu6frtXkSi5iDU03DVNvxLMDmRaXg+D10TKGiCfb
q1g2IoenA6YWz8pUkwOSI5wInepN8Dt9s5VXRsBQS9J9Uin/ZWAIwghIEhfFPNZu
braNUSyJ+A7xISipOjWD1kEfBjcUzr24AuMZ9XHpxnykdWoa57qNEoTnjm+bVWpU
hK0ED4nXSM8qxioV1Kye0h+hUi0T7amNtq6OSfAL+oXWHINvUe2PybqK+sZ1pPFb
0MSP/J4mV9CIvriPA33fTXw/VxigI/puCMtGuSLLXHwh2xW8hadz7wGU9lqF4E7E
f+X9NYqZSmx/82z0KKPFtIQyfrozquRBzy2ZiUloh7ek7GKF5tvuN8lds3Ghuf/6
9Hqr/YquG1Zuu+j95ozVIm7fhDjmMktxAwyAbg5WkNGqOEw7MyhoUGXjKKG29uLa
PRitT9DklL5vndzjizkLc1C2DuC++OhpnJCqRWOjJ9/TbvBC6Rn6aKwYyVTFX9xM
4VauNLQg0q9uG0TyW6o3aakaFk9OIKT+vl7bWO0G4g2YZaKDOs7s+b9sdNNNQq5R
lTL5F0KUCQ==
=Z90m
-----END PGP SIGNATURE-----
Merge tag 'timers-vdso-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull vdso updates from Thomas Gleixner:
- Remove the redundant CONFIG_GENERIC_TIME_VSYSCALL after converting
the remaining users over.
- Rework and sanitize the MIPS VDSO handling, so it does not handle the
time related VDSO if there is no VDSO capable clocksource available.
Also stop mapping VDSO data pages unconditionally even if there is no
usage possible.
* tag 'timers-vdso-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
MIPS: VDSO: Fold MIPS_CLOCK_VSYSCALL into MIPS_GENERIC_GETTIMEOFDAY
MIPS: VDSO: Gate microMIPS restriction on GCC version
MIPS: VDSO: Fold MIPS_DISABLE_VDSO into MIPS_GENERIC_GETTIMEOFDAY
clocksource/drivers/mips-gic-timer: Only use VDSO_CLOCKMODE_GIC when it is a available
MIPS: csrc-r4k: Only use VDSO_CLOCKMODE_R4K when it is a available
MIPS: VDSO: Only map the data pages when the vDSO is used
MIPS: Introduce Kconfig MIPS_GENERIC_GETTIMEOFDAY
vdso/datastore: Always provide symbol declarations
MAINTAINERS: Add include/linux/vdso_datastore.h to vDSO block
vdso/gettimeofday: Rename __arch_get_vdso_u_timens_data()
vdso/treewide: Drop GENERIC_TIME_VSYSCALL
vdso/vsyscall: Gate update_vsyscall() behind CONFIG_GENERIC_GETTIMEOFDAY
riscv: vdso: Drop CONFIG_GENERIC_TIME_VSYSCALL guard around syscall fallbacks
- Expand timekeeping snapshot mechanisms
The various snapshot functions are mostly used for PTP to collect
"atomic" snapshots of various involved clocks.
They lack support for the recently introduced AUX clocks and do not
provide the underlying counter value (e.g. TSC) to user space. Exposing
the counter value snapshot allows for better control and steering.
Convert the hard wired ktime_get_snapshot() to take a clock ID, which
allows the caller to select the clock ID to be captured along with
CLOCK_MONONOTONIC_RAW. Additionally capture the underlying hardware
counter value and the clock source ID of the counter.
Expand the hardware based snapshot capture where devices provide a
mechanism to snapshot the hardware PTP clock and the system counter
(usually via PCI/PTM) to support AUX clocks and also provide the
captured counter value back to the caller and not only the clock
timestamps derived from it.
- Add a new optional read_snapshot() callback to clocksources
That is required to capture atomic snapshots from clocksources which
are derived from TSC with a scaling mechanism (e.g. Hyper-V, KVMclock).
The value pair is handed back in the snapshot structure to the callers,
so they can do the necessary correlations in a more precise way.
This touches usage sites of the affected functions and data structure all
over the tree, but stays fully backwards compatible for the existing user
space exposed interfaces. New PTP IOCTLs will provide access to the
extended functionality in later kernel versions.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotoSAQHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoYWbEACv/g3pGDxWxfzOI2h6vQgGxDvD3LwmdhPE
bzXRRaxp3/J0rZTQmCghknVGDPVjepNQgKkUMXfaFG2UZmiPHG5qVTXO6DddguS4
cQc0SUO3e422lUPCoBmTULZ+vlctb4LJsWXPQHYNKC73KqMJtWte7T2HBiFDK5RB
O0S34DZtkvOW4tHIu0RwlwCXZ0gcO+zsjxKA8K/P6sMtKBQU1/rRkZZCx2KCvq0F
Rx3NTGoY4if/C83YBq1cEn8BvXrcQQH4ZOOWuySsLGJGRPZ1dXGP+JtfRWutk/f9
HZztlaXcEz71dJXlhBxc0Eb/86uC3POEq7ZYvQdzLbsSZ/3AbalksL9CLyxgdHtc
U964SuwOVPcYfEytd4TWb1nu7JgOR0olYK+l4AbCt4EdKst5TADCJ7rtlZV3Idp+
Yg1GN3TwJcKItUNX9Szk+7MbvB8EWOEl7Obahfm48qDK1pqFe08qhOzSCeRXu+Bb
QiupC3ndzUB1Yjf3DPV6wQl4Fl/TscrAVrPlnGCOJEKXtUKFxvcKquy/W29UD//w
NuxKO2zK05UDsbBEwnZiCrdSGGNiLBYUbHfx2UvA7M0rfrrbjmG4rCFPotxhNb54
UuqgdM8G45MkyBV3qSSh3VC0XeD7UqzQtMYgUjjhvtapLlsri69vzL2DnQUcajSG
dgjzIg9O3g==
=kBrz
-----END PGP SIGNATURE-----
Merge tag 'timers-ptp-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull timekeeping updates from Thomas Gleixner:
"Updates for NTP/timekeeping and PTP:
- Expand timekeeping snapshot mechanisms
The various snapshot functions are mostly used for PTP to collect
"atomic" snapshots of various involved clocks.
They lack support for the recently introduced AUX clocks and do not
provide the underlying counter value (e.g. TSC) to user space.
Exposing the counter value snapshot allows for better control and
steering.
Convert the hard wired ktime_get_snapshot() to take a clock ID,
which allows the caller to select the clock ID to be captured along
with CLOCK_MONONOTONIC_RAW. Additionally capture the underlying
hardware counter value and the clock source ID of the counter.
Expand the hardware based snapshot capture where devices provide a
mechanism to snapshot the hardware PTP clock and the system counter
(usually via PCI/PTM) to support AUX clocks and also provide the
captured counter value back to the caller and not only the clock
timestamps derived from it.
- Add a new optional read_snapshot() callback to clocksources
That is required to capture atomic snapshots from clocksources
which are derived from TSC with a scaling mechanism (e.g. Hyper-V,
KVMclock).
The value pair is handed back in the snapshot structure to the
callers, so they can do the necessary correlations in a more
precise way.
This touches usage sites of the affected functions and data structure
all over the tree, but stays fully backwards compatible for the
existing user space exposed interfaces. New PTP IOCTLs will provide
access to the extended functionality in later kernel versions"
* tag 'timers-ptp-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (28 commits)
ptp: vmclock: Use hw_cycles from snapshot for precise TSC pairing
x86/kvmclock: Implement read_snapshot() for kvmclock clocksource
clocksource/hyperv: Implement read_snapshot() for TSC page clocksource
timekeeping: Add clocksource read_snapshot() method and hw_cycles to snapshot
ptp: Switch to ktime_get_snapshot_id() for pre/post timestamps
timekeeping: Add support for AUX clock cross timestamping
timekeeping: Remove system_device_crosststamp::sys_realtime
ALSA: hda/common: Use system_device_crosststamp::sys_systime
wifi: iwlwifi: Use system_device_crosststamp::sys_systime
ptp: Use system_device_crosststamp::sys_systime
timekeeping: Prepare for cross timestamps on arbitrary clock IDs
timekeeping: Remove ktime_get_snapshot()
virtio_rtc: Use provided clock ID for history snapshot
net/mlx5: Use provided clock ID for history snapshot
igc: Use provided clock ID for history snapshot
ice/ptp: Use provided clock ID for history snapshot
wifi: iwlwifi: Adopt PTP cross timestamps to core changes
timekeeping: Add CLOCK ID to system_device_crosststamp
timekeeping: Add system_counterval_t to struct system_device_crosststamp
timekeeping: Add CLOCK_AUX support for ktime_get_snapshot_id()
...
- Rework of /proc/interrupt handling:
/proc/interrupts was subject to micro optimizations for a long time,
but most of the low hanging fruit was left on the table. This rework
addresses the major time consuming issues:
- Printing a long series of zeros one by one via a format string
instead of counting subsequent zeros and emitting a string
constant.
- Simplify and cache the conditions whether interrupts should be printed
- Use a proper iteration over the interrupt descriptor xarray
instead of walking and testing one by one.
- Provide helper functions for the architecture code to emit the
architecture specific counters
- Convert the counter structure in x86 to an array, which
simplifies the output and add mechanisms to suppress unused
architecture interrupts, which just occupy space for
nothing. Adopt the new core mechanisms.
This adjusts the gdb scripts related to interrupt counter statitics
to work with the new mechanisms.
- Prevent a string overflow in the /proc/irq/$N/ directory name
creation code.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmoths8QHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoSoMEACRODwHjNfULjgD2heHbiPKsmPMRZvwO1Ud
xu5XAoNT1gwxnLo4D+KrGCZeyxka+byRpby6eNg7HdRJuu3DUf8umwt/Q472I9a9
ck8OGFp8ntbxnueISKfzxY/O2eXHYxSKmmfZMv3wdOKbvn5OUlFT6eHPjb8PzVUM
7DiXsBL8s3MNHwdJ3grG5lBh60pt5fujzURwYAqvh/i8jlDHxsFRTMGuhR710knr
YZrgZ4/7ffnEbDsn98xezPewRomIbhhEijgfjkkbnYYUub6Y2RHJqOzZhlp6zNgi
vTsU/suW3ryVuzG34rL2uHvsxOcJY1HNA+ING7fkRmPuKxRGKOMBQfPmLQcWqP69
GxwGIlBvNbAEYievgTCS7GNHTy3t0JbxTGhHcBvX3oMtnnOSTttqH9XzvrTwGxjj
fMUykfvB+40Fp47D+t0JDhgyNNEkixSBjW8/gogZFQ0OdMFX6BQZNT/DLhMMC0LR
JbqMpfsffp5+gYam/wixv3sPlxajMpQ2w8ocgyUHVAeFMo1LOY1spUuO3+Tq7nSj
xt95xVg6HQDr+L+8QmZmnRq27uG276CxPpLotbPMsrn0Ax5PL+fymfmVsFmJFjAR
ZHKK3tSD6M94GtklfKlB/yBJGNRafH4MVZbMa0iUxGI6UyAFr/Yror3mfDK9NsIA
WTwwaqI8qw==
=z6vj
-----END PGP SIGNATURE-----
Merge tag 'irq-core-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull interrupt core updates from Thomas Gleixner:
- Rework of /proc/interrupt handling:
/proc/interrupts was subject to micro optimizations for a long time,
but most of the low hanging fruit was left on the table. This rework
addresses the major time consuming issues:
- Printing a long series of zeros one by one via a format string
instead of counting subsequent zeros and emitting a string
constant.
- Simplify and cache the conditions whether interrupts should be
printed
- Use a proper iteration over the interrupt descriptor xarray
instead of walking and testing one by one.
- Provide helper functions for the architecture code to emit the
architecture specific counters
- Convert the counter structure in x86 to an array, which
simplifies the output and add mechanisms to suppress unused
architecture interrupts, which just occupy space for nothing.
Adopt the new core mechanisms.
This adjusts the gdb scripts related to interrupt counter statistics
to work with the new mechanisms.
- Prevent a string overflow in the /proc/irq/$N/ directory name
creation code.
* tag 'irq-core-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/irq: Add missing 's' back to thermal event printout
genirq/proc: Speed up /proc/interrupts iteration
genirq/proc: Runtime size the chip name
genirq: Expose irq_find_desc_at_or_after() in core code
genirq: Add rcuref count to struct irq_desc
genirq/proc: Increase default interrupt number precision to four
genirq: Calculate precision only when required
genirq: Cache the condition for /proc/interrupts exposure
genirq/manage: Make NMI cleanup RT safe
genirq: Expose nr_irqs in core code
scripts/gdb: Update x86 interrupts to the array based storage
x86/irq: Move IOAPIC misrouted and PIC/APIC error counts into irq_stats
x86/irq: Suppress unlikely interrupt stats by default
x86/irq: Make irqstats array based
genirq/proc: Utilize irq_desc::tot_count to avoid evaluation
genirq/proc: Avoid formatting zero counts in /proc/interrupts
x86/irq: Optimize interrupts decimals printing
genirq/proc: Size interrupt directory names for 10-digit interrupt numbers
Outside the IOPL emulation path, the IO bitmap is always expected
to be allocated when TIF_IO_BITMAP is set. The paranoid WARN_ON_ONCE()
handles the case where the flag and the pointer got out of sync.
In this theoretical scenario, which presumes some other bug in the
code that triggers the WARN_ON_ONCe(), return early, instead of
continuing and dereferencing a NULL pointer.
[ mingo: Clarified the changelog. ]
Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Sohil Mehta <sohil.mehta@intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260615070115.4720-1-lirongqing@baidu.com
-----BEGIN PGP SIGNATURE-----
iJEEABYKADkWIQRTLbB6QfY48x44uB6AXGG7T9hjvgUCaivrshsUgAAAAAAEAA5t
YW51MiwyLjUrMS4xMiwyLDIACgkQgFxhu0/YY75HWAD/UWqFaTfWhpS3mJbcOE8G
NTfZMunls/XyAoPxL4T6ThMA/jrdrFMerb27S1xHwKIr84YH71P8naXkiv+71UZz
JCkE
=Hh59
-----END PGP SIGNATURE-----
Merge tag 'for-linus-7.2-rc1-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip
Pull xen updates from Juergen Gross:
- Several small cleanups of various Xen related drivers
(xen/platform-pci, xen-balloon, xenbus, xen/mcelog)
- Cleanup for Xen PV-mode related code (includes dropping the Xen
debugfs code)
- Drop the additional lazy mmu mode tracking done by Xen specific code
* tag 'for-linus-7.2-rc1-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip:
xen/xenbus: Replace strcpy() with memcpy()
x86/xen: Replace generic lazy tracking with cpu specific one
x86/xen: Get rid of last XEN_LAZY_MMU uses
mm: Refactor lazy_mmu_mode_pause() and lazy_mmu_mode_resume()
x86/xen: Change interface of xen_mc_issue()
x86/xen: Drop lazy mode from trace entries
x86/xen: Remove Xen debugfs support
x86/xen: Cleanup Xen related trace points
x86/xen: Guard PV-only stuff in xen-ops.h with CONFIG_XEN_PV
xen: balloon: Replace sprintf() with sysfs_emit()
xen/mcelog: mark g_physinfo, ncpus and xen_mce_chrdev_device as __ro_after_init
xen: constify xsd_errors array
xen/platform-pci: Simplify initialization of pci_device_id array
Kbuild:
- Remove broken module linking exclusion for BTF
- Add documentation around how offset header files work
- Include unstripped vDSO libraries in pacman packages
- Bump minimum version of LLVM for building the kernel to 17.0.1 and
clean up unnecessary workarounds
- Use a context manager in run-clang-tools
- Add dist macro value if present to release tag for RPM packages
- Detect and report truncated buf_printf() output in modpost
- Add __llvm_covfun and __llvm_covmap to section whitelist in modpost
- Support Clang's distributed ThinLTO mode
- Remove architecture specific configurations for AutoFDO and Propeller
to ease individual architecture maintenance
Kconfig:
- Add kconfig-sym-check target to look for dangling Kconfig symbol
references and invalid tristate literal values
- Harden against potential NULL pointer dereference
- Fix typo in Kconfig test comment
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQR74yXHMTGczQHYypIdayaRccAalgUCaijIpwAKCRAdayaRccAa
lszAAQD0PuP+a0IejIyubuvEeB0ecG5nvKZIV99veIaivp9J4QD+PwYuPf+Y9A0r
PqiV0IBrnhbmjNrSj8Clt2eHXqa4jg4=
=J1xQ
-----END PGP SIGNATURE-----
Merge tag 'kbuild-7.2-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux
Pull Kbuild / Kconfig updates from Nathan Chancellor:
"Kbuild:
- Remove broken module linking exclusion for BTF
- Add documentation around how offset header files work
- Include unstripped vDSO libraries in pacman packages
- Bump minimum version of LLVM for building the kernel to 17.0.1 and
clean up unnecessary workarounds
- Use a context manager in run-clang-tools
- Add dist macro value if present to release tag for RPM packages
- Detect and report truncated buf_printf() output in modpost
- Add __llvm_covfun and __llvm_covmap to section whitelist in modpost
- Support Clang's distributed ThinLTO mode
- Remove architecture specific configurations for AutoFDO and
Propeller to ease individual architecture maintenance
Kconfig:
- Add kconfig-sym-check target to look for dangling Kconfig symbol
references and invalid tristate literal values
- Harden against potential NULL pointer dereference
- Fix typo in Kconfig test comment"
* tag 'kbuild-7.2-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux: (31 commits)
kconfig: tests: fix typo in comment
kconfig: Remove the architecture specific config for Propeller
kconfig: Remove the architecture specific config for AutoFDO
modpost: Add __llvm_covfun and __llvm_covmap to section_white_list
kconfig: add kconfig-sym-check static checker
kbuild: Remove unnecessary 'T' modifier in cmd_ar_builtin_fixup
kbuild: distributed build support for Clang ThinLTO
kbuild: move vmlinux.a build rule to scripts/Makefile.vmlinux_a
scripts: modpost: detect and report truncated buf_printf() output
kbuild: rpm-pkg: append %{?dist} macro to Release tag
run-clang-tools: run multiprocessing.Pool as context manager
compiler-clang.h: Drop explicit version number from "all" diagnostic macro
compiler-clang.h: Remove __cleanup -Wunused-variable workaround
kbuild: Remove check for broken scoping with clang < 17 in CC_HAS_ASM_GOTO_OUTPUT
x86/entry/vdso32: Remove conditional omission of '.cfi_offset eflags'
x86/module: Revert "Deal with GOT based stack cookie load on Clang < 17"
x86/build: Drop unnecessary '-ffreestanding' addition to KBUILD_CFLAGS
scripts/Makefile.warn: Drop -Wformat handling for clang < 16
riscv: Drop tautological condition from TOOLCHAIN_NEEDS_OLD_ISA_SPEC
riscv: Remove tautological condition from selection of ARCH_SUPPORTS_CFI
...
The /proc/interrupt handling rework dropped a 's' in the thermal event
printout, which breaks the thermal test in the Intel LKVS suite.
Bring the important letter back.
Fixes: 2b57c69917 ("x86/irq: Make irqstats array based")
Reported-by: kernel test robot <oliver.sang@intel.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Closes: https://lore.kernel.org/oe-lkp/202606121325.97b29701-lkp@intel.com
PDPTRs are part of the CPU state. A bit unconventionally, they are
reached via vcpu->arch.walk_mmu instead of being stored in vcpu->arch
directly. That is nice in principle---it would allow TDP shadow paging
to have its own PDPTRs---but it is not necessary, because EPT has no
PDPTRs and NPT does not cache them.
Since kvm_pdptr_read does not otherwise need the MMU, drop the pdptrs
from the MMU altogether. There is however something to be careful
about, in that PDPTRs are now not stored separately in root_mmu and
nested_mmu for L1 and L2 guests. In practice this was already not
an issue:
- for EPT the VMCS0x has to keep them up to date; and for the purpose
of emulation they are always loaded from the VMCS on vmentry/vmexit,
thanks to the clearing of dirty and available register bitmaps in
vmx_switch_vmcs()
- for NPT, VCPU_EXREG_PDPTR is similarly cleared for nNPT, which does
not cache the PDPTRs; while for non-nNPT the PDPTRs are loaded
together with the load of CR3.
Note that page table PDPTRs are not affected, since they are stored
in pae_root.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260530165545.25599-6-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This is true for both Intel and AMD. On Intel, "enable INVPCID" is
set unconditionally if supported, but the vmexit is triggered by the
"INVLPG exiting" control which is disabled by enable_ept. On AMD, KVM
can intercept INVPCID if NPT is enabled but only in order to inject #UD
in the guest.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260530165545.25599-5-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When L2 runs under nested NPT and uses PAE paging, KVM's cached PDPTRs
in mmu->pdptrs[] can hold stale or wrong values after nested
transitions and across migration restore, because both
nested_svm_load_cr3() and svm_get_nested_state_pages() only refresh
PDPTRs on the !nested_npt path.
The user-visible bug is on migration restore of an L2 running with nested
NPT and 32-bit PAE paging, if userspace uses KVM_SET_SREGS rather than
KVM_SET_SREGS2. In that case, load_pdptrs() leaves VCPU_EXREG_PDPTR
marked as available, and kvm_pdptr_read() will use a stale translation
that used L1 GPAs instead of L2 nGPAs. svm_get_nested_state_pages()
runs on first KVM_RUN but skips the refresh because nested_npt_enabled()
is true. The CPU itself reads L2's PDPTRs correctly from memory via
L1's NPT, but KVM-side walking of guest PAE page tables uses the bogus
cached values.
Unlike Intel's GUEST_PDPTR0..3 fields in the VMCS, SVM has no
VMCB-cached PDPTR state: the in-memory PDPTEs at the current CR3 are
the only source of truth, and svm_cache_reg(VCPU_EXREG_PDPTR) simply
reloads them from memory via load_pdptrs(). Clearing the avail
bit (and the dirty bit because !avail/dirty is invalid) to force
a reload when PDPTRs as needed fixes the bug.
Do the same for nested_svm_load_cr3()'s nested_npt branch, so that
the invariant "PDPTRs need reloading" is handled similarly for both
immediate and deferred loading.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260530165545.25599-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The early vmwrite of the PDPTRs in prepare_vmcs02_rare() is redundant, because
every write it does will be performed by prepare_vmcs02() if it is actually
needed.
In any case where the emulator or the processor need the PDPTR, either
is_pae_paging() is true on vmentry, or a write of CR0, CR4 or EFER will
cause a vmexit to L0. The next vmentry will refresh the PDPTRs in the
vmcs02 from vmcs12.
In fact, the original version[1] of what ended up being commit
c7554efc83 ("KVM: nVMX: Copy PDPTRs to/from vmcs12 only when
necessary"), the writes in what is now prepare_vmcs02_rare() were removed.
When the mega-collection of optimizations was posted[2], the removal of
that code got dropped as a rebase good, so reinstate it.
[1] https://lore.kernel.org/all/20190507160640.4812-16-sean.j.christopherson@intel.com
[2] https://lore.kernel.org/all/1560445409-17363-31-git-send-email-pbonzini@redhat.com
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260530165545.25599-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
nested_mmu is always stored into vcpu->arch.walk_mmu at the same time as
guest_mmu is stored into vcpu->arch.mmu. But nested_mmu is not even
a proper MMU, it is only used for page walking; plus the fact that
walk_mmu has to be switched at all is just an implementation detail.
In the end what matters here is whether the guest is using nested
page tables; vmx/nested.c and svm/nested.c check it to see if they
are in nEPT or nNPT context respectively. So switch to checking
root_mmu vs. guest_mmu, which is a more cogent test.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260511150648.685374-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20260530165545.25599-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorQ78ACgkQOlYIJqCj
N/3bCxAAoXU8uJ37PIm58P7ok5hF2OFXeDLCYE1BpOfYOaT4b0vWW+Rb5I3y+2K6
511IxlBE2RcGBg6LHb1d1a/IEvs0NpzXOZFp7NseJC8sIn01Lndgn2SvoLoMVRy8
Yda4WyCNV1rSeq+duZKtMRhXiNY5Zdgsz6S5MdnslAK9OCbjS2XsQ6F7afMwM1g0
D3/kVUdBwUrKrllgmJ+XprLvYGEVji8xVCosdJm1ZAxb0xIr395/Oy7tnx3O2XNz
NjpEqabFRjcL8JMiOPnYdTfdqsNagr7qSqunJkCMUtuq+ZBaAA332ZKRbR/Rh83c
XprJMMw3+t77AjMSOHGmQY9ykVugT29bG7mjN/5bqhqqSzS1zjpGKd7ToEqvpKBn
lCnbUoSUQkvfBMZr6WUQppZvNfZaNi2Nuq4EKkZzqOiKUMO6ggfIh/1+M7s93mmT
rCa/35JEoBL3uwJcc75c25Y3QZWQyJjuNmFOuDt4s+yzDJ70F6II8FMykC5LGlFn
m9oZ/KIFEY1H9r8GyjoCIksQI/GBPxAKULA8wfY4WsMr+NzzKIhB39FfR6FnyZTN
XmPRlAKvWCGAqnrbOwgWvdu8Gsj18SpU38+AhngZRS7cVqB4+tQEZ4tcp9oSz6dy
UMYsZD0N8TFgGNqRf34DmgB+NSIamttAAYhMPWgAoqdvseHSvp0=
=4LNF
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-svm-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM SVM changes for 7.2
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
- Fix a largely benign bug where KVM TDX would incorrectly state it could
emulate several x2APIC MSRs.
- Use the "safe" WRMSR API when proxying LBR MSR writes as the to-be-written
value is guest controlled and completely unvalidated.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorRMoACgkQOlYIJqCj
N/2G7hAApCZFvkg2WXBZUxzl4iD17IF0CYk0CTNCY11xLghkSprvPviVSrHfj+lH
dLA5cCmngKkoK/6MgvPwDIfEg2Owhv3vBEIVmJTCkDZ4I5312LE6QYwtBvEY/iX3
BthUMhmWpnsmZUFRkQFHZJt3bcCo9oVjD50txSKHzvstZzXiZfOeR9gXE5h5rX6h
4yu5dd4c0/vdmCKfbzB4hNW6XbBIYd9q5xJWrycmpXKhx4hPp4V5DmmUwHUTRgXx
7xy4TBrG+R7pfkvVpfwLj9AErS5YUpM99dqA1a9H3J/yEOxHTR5oDixTxtm+9DNR
XSQ4Fh2ZPk+IZ4lPVdkoEhqKPUkz3ho+UcrctS6JPs+lcQpUDBnuNtdLsKI7MznH
rjFv19AJHv8/Ayy6OigYAMirRC19RxgAU9EeuA3wbG3iKpo/3SZV3ZtuJJrOi5pF
S5zOH2oOnNPlgP/la24vI9RiahtXwXDoMrOWEqICr5ESYEMrhfItWXPJK2j+WpFC
tk4AAF6faDWMLAA1rhiM9NJsg9RdGEfAgaA/+vE9iFmyFxR3K+UBHDUc80764SX3
hqgfenM5wJKlxzYiSzG+erFn3F7oHHFWwktS2RD8ynz3t6kW9k7KGvqI/3vBwWL6
oj+9rHVT3w+RLDP+OCZ1iPR/PuvHOYR2j5rtw2V6WgaZU7cqMNM=
=de2r
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-vmx-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM VMX changes for 7.2
- Fix a largely benign bug where KVM TDX would incorrectly state it could
emulate several x2APIC MSRs.
- Use the "safe" WRMSR API when proxying LBR MSR writes as the to-be-written
value is guest controlled and completely unvalidated.
- Don't advertise support for unusuable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorQawACgkQOlYIJqCj
N/1e8RAAhQWGpKAs6kcyd+WhJCPm5bQPtDMRVNM8piCdXdZ1jy/qO2phuxWWXtxu
7a5Kvmz5WODuUBJxdyzMLJJv+o8+8j/ReNA5XkdDBZrnvxZeQpm6yfjq7ZTfmgBx
qc6gFETNjhu8gvb2tIqAfOe87TtBb2Ruw3fMiJR8+ATKIN4bgLrqOcnczBxHo80X
RQZqM3CNjac+6q5Gc4YEK2RbKvdzl/QNEuMrpk25O9fPZ5rnFO6K6ojZQQ2tTdCB
G6tnwg5zlPGSi41l4fjvbWfaj09UQ1/zupXaXfq6aoj4og773G8GmXGEvh9Kd6Dc
GurSJMhjnOPY9kJJ1YLHcxX4HtSc0M9WLqV/uZB8PtmbKfotUMJsHz5bv7rJa98N
XQnt+9U08JhURJaKF12p+s0bmO/K7Dmp7VrUhCP1jeNuHsSdaoXlrfC/qZp1oLip
t0jSxwZELkCSNsgbNNzpTmED1+xfj9DhqY+FvzZN9HPBedZb2ZHCuWFXJ4DMhn5p
TiS2cL6eu8gG7vFXCoOYIHJbZoLNm823nqEneFbSJdy9ZSn55/UTYyFuG0I6VY1W
rfv52Vz7AuhR4WGDAAqz6fPX5otJsHKkS6Kqy9dqTcV2jDsLvu97bef3qgGj/pe+
5q+J7eeOGTtyJeQZwxljJ7Z+SOSB8vEkKPX6d6bqM7YXWKzn6oo=
=SlF7
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-sev-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM SEV changes for 7.2
- Don't advertise support for unusuable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in the shadow MMU where KVM fails to recursively zap nested TDP
shadow when L1 is tearing its TDP page tables from the bottom up, as KVM's
TDP MMU now does.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorQCsACgkQOlYIJqCj
N/3aAA//Y3Hsd5F4/p2HgI+qYxpskugi4BAInXjCzs92ShOaxxSTEPnzRNCAjDDb
OTpvfJm3ZI2if6A9yYngP7R8tzueMAhHMcCAnkL/+VBsN+ILm4yflwGgs2Tc7pUS
jpd5pioljG16b4B2LC4QM0RsAsfxqjcbLcY1XIFQVObuiSoquv6hjaebbY2IW4n3
IS6QSKRSLZb5qh9IBbWihn7Qyr4E3r5fmoaTptExSb4zKYVMPqpqMnngqYQWfZxg
6lfoh1eMrIU5LdFRgJeifwRzBvrtu2YI2pyih7bzXUe/XTE9AYpIvA1CB+OVy15g
AjDllcYX6B7B5i7uc3trk4eqD75xDnQYRQfnkWj9OuxVKGp7Fcydk1VQHtPLS3K7
HpWB4/KvXbAqGXZltE4qnqycqY64hp5k0B59I/Q+n9hC9hHXjW5etp9e+zZsNSEy
IswGEONdNHszOF0XZE0WE1zsi1D1HW3rg4OANClE+X+z55xqC1evEt7BbvbsDGmX
41sJCyRjc2yClVh5kZhruP9G5w5OT+M8En5fdou5QlgeAWcaGiUXj/e2mF5fV+8p
PtEtih3Q82MsB78Y0jiA8dkRDRkqDqh69I1BCUJJTIdhYO1fTq8t3iV9MGhJz6C1
5TXyA895FsGrp85+IN1Chi09tkCqktlM866mTg0Ph6kPHCWNQpE=
=PUlE
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-mmu-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM x86 MMU changes for 7.2
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in the shadow MMU where KVM fails to recursively zap nested TDP
shadow when L1 is tearing its TDP page tables from the bottom up, as KVM's
TDP MMU now does.
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Ensure KVM's copy of CR0 and CR3 are up-to-date on SVM prior to invoking
fastpath handlers.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Misc fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorPacACgkQOlYIJqCj
N/2f4BAAiWMEfbazgtGLYQyiSd2QtIw41Dpcz9cnSBxvuFztulewTpd5z2B4/8kX
dSeZ3vyofgOVUVIj4AlD6Ap8ZO7XL3FizLcjWv9c8l4j/sTREL1ix0oejPJpBASg
5EmSueBjmj+8VbBNxHGCYMuU4okR0UBmU86sC0wBuho+CvqkNegSrmCpntHUh4Kn
34/HZ6QTik7iZNXV7KVT7/2De/bZV/9RJ7T8EAOkOGRfWft7cVUmkCXBP184pqee
Z9X1gcymXb17Z73GbOFbM3SEewcYLiy1LdDyk1Tid3Wk6nJlMj0Tc0djsI6Vz7+8
UvaKNoAXyWydPzJmYuCuTP+OnrZrha3RuEZvpP9UBJqGQgNjzs1jis92jiPL3yf4
38zcNesukLU8RWKMHw4PYMQtDIVwqPloSPN9tPZ6U+9IWZx88H/IRMd080Jxsbem
UNCuB88PawtlYV7gNk2XvQ0gcLib3LTjVjrWyTloUDWTjsMEXYxQ7Ehn6Jk+dfya
BMjivWY246SqsMBmkj4Lj0S6uD25z9Yv3uPcwshZ3nuXCjdmBKlWNSs7cFYopI1t
h4urP9xX/eURs63f++nB+6ZsqMR2ESQXmbybvsevR/+GFb0Iuy3obJk4ab2kKvP/
OhFpo8AmJ3jOK2RrIbZ/wUb5rfZbvnCxjAqSQ1YZHIMkrD9vlTY=
=IHTg
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-misc-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM misc x86 changes for 7.2
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Ensure KVM's copy of CR0 and CR3 are up-to-date on SVM prior to invoking
fastpath handlers.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Misc fixes and cleanups.
- Rename invalidate_begin() to invalidate_start() throughout KVM to follow
the kernel's nomenclature, e.g. for mmu_notifiers.
- Minor cleanups.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmorNxkACgkQOlYIJqCj
N/0qxRAAtRQCb/6nmZFYR7Dna8ap0fgfXbdVrVGmI8/zG8zclQbIzTMMQvfgCP6H
D62WTbqZsVnPQz3Bwf7O8CRgbKXeDlRN+lf0agE7T+Gl1jIGyKlnUCiB2/9XyByP
oWmZM8ogZIqhi4I90OrRI8QxFEMR8z/6rErRyzj4D9lvzgys5ggSz911aPkUe9A0
wgbG0wX6GDS1kDP34lQHADM+WD0KACn5S4OwIdEx2YgYGvLKos2hbJMv7fbDV8LM
//bS/Y+uMMgcw9iM12EUQ/Fhu50PgNfCtqg16x/ZnKhyHsLVfgRLBfBz++2S27dC
/FbH50SIonJYY6r1y/MlbW7Qa4h0joKuvkDJn4Up72/5n31wBNnnK4J4yReVnXjw
MlJaeX7dptMDch5rMCJg/5/7CKD1Qo5xFaVeeuHcSzOczqFtnvUfGlRlkj99ukFT
lNHuphhKfl0MrK32qyFwK3WRFXG9pCV2dyhCEn537Sx7pxgpA04nrX8Q6mN9vBNE
jqMtM2zpmQeQ0zvazr/fH75HXqYPBFTghxhNHYyyufYJpn/OxUvr83LDZXOlzozz
V4LumtvZyKoLZn4gEQG0f8HV6c3yVhWyTWMiHO6ifR4xCZhXDkBaytLfK3VzVXfZ
x7p9gr2arrZgTxkH/VbQv+U4BaoQbM7tlKSfpqY6LXllSBoPaBQ=
=QHYY
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-generic-7.2' of https://github.com/kvm-x86/linux into HEAD
KVM generic changes for 7.2
- Rename invalidate_begin() to invalidate_start() throughout KVM to follow
the kernel's nomenclature, e.g. for mmu_notifiers.
- Minor cleanups.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEoWuZBM6M3lCBSfTnuARItAMU6BMFAmoqvdUACgkQuARItAMU
6BNHOA/8CSEH30EeXobbblnAiywthizP8z3gQ5LPnTjDNb4tS/UlPCwwapqFOrDs
iPS48rv9SowmVFqXfUX1C47d5V4zDApD2699eRa5MOoXLTY9JtpafC8zsSKgKevY
LW0OwiC7lDGXop9WmHwcy6vc6zgns3DMvUxyrxzVykVArA5ghZ2eMl/ElNDsRAJS
1v1Z8V/vvBLU4XCIuNFeGSx+y8qaymhIeD7LkLFqUsfTb261dzIIHeD+WSMJeYG0
tuc1UK9cSBu8zh5BLcNZ5EVh4LKJInOyju0bgeLtysQXGUBx/j9SM/kP9O6WMYiD
krOeQMHPLpyh5N00lxVg3h2xyuvD3/95YbmZEKB1sOQZjp6zdP1lNZZ/cuzJKmMo
l8LTQ4oHKFJW3GgzrAU5jKyg5DlJxvzatL8oMsHeEDMI8W5RP7wcb3jTF78meKsT
uCkeYCdGiISF/l8vJeNjPFBXJZkP6M6bdIEFrJMRahlDFkOL9K3PRa5g3UnKZQqh
ZczANXRBJoazSwM9i7x4J75NmV3hUeC6VFqEfg1oiiNqDIhmQDz9rYoRp+Kj9AZd
/MjuaOozFOKK+Zkhg6hm5aVgYMkdfkdZPByvOzNOypFckNyV0yITCOqhGISTXKvb
9tmLNdphYA3SAlu/rhqFCrJ0K4aF+qR/5rn5PU2y/CZg15qymDM=
=y8Ea
-----END PGP SIGNATURE-----
Merge tag 'kvm-s390-master-7.1-4' of https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD
KVM: s390: A few more misc gmap fixes.
I.e. from now on it's a boot time requirement for the CPU to support it.
( This was preceded by the removal of all non-CX8 platform build
options, so it should have no functional effects in theory. )
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Ahmed S . Darwish <darwi@linutronix.de>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: H . Peter Anvin <hpa@zytor.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20250425084216.3913608-14-mingo@kernel.org
Now that the Kconfig space always enables CONFIG_X86_TSC (on x86),
remove !CONFIG_X86_TSC code from the x86 arch code.
We still keep the Kconfig option to catch any eventual code still
pending in maintainer or non-mainline trees, plus some drivers
have raw TSC timestamping hacks that use CONFIG_X86_TSC.
It's also still possible to disable TSC support runtime.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Ahmed S . Darwish <darwi@linutronix.de>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: H . Peter Anvin <hpa@zytor.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://lore.kernel.org/r/20250425084216.3913608-13-mingo@kernel.org
Recursively zap orphaned nested TDP shadow pages when emulating a guest
write to a shadowed page table, regardless of whether or not the associated
(parent) shadow page will be zapped, e.g. due to detected write-flooding.
This plugs a hole where KVM fails to reclaim defunct, unsync shadow pages
for select L1 hypervisor patterns. Commit 2de4085ccc ("KVM: x86/MMU:
Recursively zap nested TDP SPs when zapping last/only parent") modified KVM
to recursively zap synchronized shadow pages (KVM already recursively zaps
unsync children) when a child is orphaned. But the fix effectively only
applied the logic to kvm_mmu_page_unlink_children(), i.e. only performs the
recursive zap when KVM is already zapping a parent SP and processing its
children.
If L1 zaps SPTEs bottom-up (4KiB => 2MiB => ...), as KVM's TDP MMU does
with CONFIG_KVM_PROVE_MMU=n since commit 8ca983631f ("KVM: x86/mmu: Zap
invalidated TDP MMU roots at 4KiB granularity"), then KVM (as L0) will leak
upwards of 4 shadow pages per GiB of L2 guest memory. Over hundreds or
thousands of L2 boots, if the VM is "lucky" enough to escape write-flooding
detection, i.e. not trigger reclaim of the orphaned shadow pages by dumb
luck, then it's possible to end up with tens or even hundreds of thousands
of unsync shadow pages and associated rmap entries.
Polluting the hash table and rmap entries with a horde of stale entries
can eventually degrade L2 guest boot time by an order of magnitude,
especially if there is any antagonistic activity in the host, i.e. anything
that will contend for mmu_lock and/or needs to walk rmaps.
With "top"-down zapping, where "top" is 1GiB or above, then L0 KVM is
effectively limited to leaking 4 shadow pages per 256 GiB of memory, as
KVM's write flooding detection will kick in on the third write to an L1
TDP PUD, and thus recursively zap the entire 256 GiB range of the parent
PGD. I.e. even though L1 KVM still recursively zaps 2MiB => 4KiB SPTEs
when zapping each 1GiB SPTE, KVM only gets through two of the 1GiB SPTEs
before dropping everything. E.g. hacking tracing into L0 KVM's
kvm_mmu_track_write(), the top-down zapping of L1's TDP MMU for an L2 with
16GiB of memory leads to:
gpa = 107407000, old = 800000010741bd07, new = 8000000000000000, level = 3, flood = 0
gpa = 10741b000, old = 8000000112fb2d07, new = 80000000000001a0, level = 2, flood = 0
gpa = 10741b008, old = 800000012509cd07, new = 80000000000001a0, level = 2, flood = 1
gpa = 10741b010, old = 80000001114b9d07, new = 80000000000001a0, level = 2, flood = 2
gpa = 107407008, old = 8000000112fb5d07, new = 8000000000000000, level = 3, flood = 1
gpa = 112fb5298, old = 8000000106f43d07, new = 80000000000001a0, level = 2, flood = 0
gpa = 112fb52a0, old = 8000000106f4dd07, new = 80000000000001a0, level = 2, flood = 1
gpa = 112fb5ea0, old = 8000000120490d07, new = 80000000000001a0, level = 2, flood = 2
gpa = 107407010, old = 8000000106df2d07, new = 8000000000000000, level = 3, flood = 2
gpa = 107410000, old = 8000000107408d07, new = 8000000000000000, level = 5, flood = 0
gpa = 107408000, old = 8000000107407d07, new = 80000000000001a0, level = 4, flood = 0
Contrast that with a bottom-up zap, which effectively allows all 2MiB SPTEs
in L1 to leak their children.
gpa = 167939000, old = 800000011c8f4d07, new = 8000000000000000, level = 2, flood = 0
gpa = 167939020, old = 8000000104407d07, new = 8000000000000000, level = 2, flood = 1
gpa = 167939028, old = 800000011ed20d07, new = 8000000000000000, level = 2, flood = 2
gpa = 118c70bb0, old = 8000000167ab9d07, new = 8000000000000000, level = 2, flood = 0
gpa = 118c70bb8, old = 8000000163913d07, new = 8000000000000000, level = 2, flood = 1
gpa = 118c70de8, old = 800000011cc9dd07, new = 8000000000000000, level = 2, flood = 2
gpa = 160be7fb0, old = 800000011d322d07, new = 8000000000000000, level = 2, flood = 1
gpa = 160be7fb8, old = 8000000126b1bd07, new = 8000000000000000, level = 2, flood = 2
gpa = 1634ab000, old = 800000010e984d07, new = 8000000000000000, level = 2, flood = 0
gpa = 1634ab008, old = 800000016879fd07, new = 8000000000000000, level = 2, flood = 1
gpa = 1634ab010, old = 800000016879ed07, new = 8000000000000000, level = 2, flood = 2
gpa = 11e3f1e48, old = 8000000168a33d07, new = 8000000000000000, level = 2, flood = 0
gpa = 11e3f1e50, old = 80000001664dcd07, new = 8000000000000000, level = 2, flood = 1
gpa = 1167eacb8, old = 8000000166544d07, new = 8000000000000000, level = 2, flood = 0
gpa = 1167eacc0, old = 800000015c16bd07, new = 8000000000000000, level = 2, flood = 1
gpa = 1689e89b8, old = 800000015f296d07, new = 8000000000000000, level = 2, flood = 0
gpa = 1689e89c0, old = 8000000167ca8d07, new = 8000000000000000, level = 2, flood = 1
gpa = 107b35eb8, old = 8000000161e71d07, new = 8000000000000000, level = 2, flood = 0
gpa = 107b35ec0, old = 8000000118cf3d07, new = 8000000000000000, level = 2, flood = 1
gpa = 118cf2d48, old = 8000000118cf1d07, new = 8000000000000000, level = 2, flood = 0
gpa = 118cf2d50, old = 8000000118cf0d07, new = 8000000000000000, level = 2, flood = 1
gpa = 118dcb770, old = 8000000118dcad07, new = 8000000000000000, level = 2, flood = 0
gpa = 118dcb778, old = 8000000118dc9d07, new = 8000000000000000, level = 2, flood = 1
gpa = 118dc87e8, old = 8000000126997d07, new = 8000000000000000, level = 2, flood = 0
gpa = 118dc87f0, old = 8000000126996d07, new = 8000000000000000, level = 2, flood = 1
gpa = 126995148, old = 8000000126994d07, new = 8000000000000000, level = 2, flood = 0
gpa = 126995150, old = 8000000103477d07, new = 8000000000000000, level = 2, flood = 1
gpa = 1034764c8, old = 8000000103475d07, new = 8000000000000000, level = 2, flood = 0
gpa = 1034764d0, old = 8000000103474d07, new = 8000000000000000, level = 2, flood = 1
gpa = 10ea4b788, old = 800000010ea4ad07, new = 8000000000000000, level = 2, flood = 0
gpa = 10ea4b790, old = 800000010ea49d07, new = 8000000000000000, level = 2, flood = 1
gpa = 10ea48928, old = 800000011a5bfd07, new = 8000000000000000, level = 2, flood = 0
gpa = 10ea48930, old = 800000011a5bed07, new = 8000000000000000, level = 2, flood = 1
gpa = 11a5bd0d8, old = 800000011a5bcd07, new = 8000000000000000, level = 2, flood = 0
gpa = 11a5bd0e0, old = 800000011d323d07, new = 8000000000000000, level = 2, flood = 1
gpa = 122ce2b40, old = 800000011fe0bd07, new = 8000000000000000, level = 2, flood = 0
gpa = 122ce2b48, old = 800000010e985d07, new = 8000000000000000, level = 2, flood = 1
gpa = 122ce2b50, old = 8000000161c9dd07, new = 8000000000000000, level = 2, flood = 2
gpa = 16864c000, old = 8000000167939d07, new = 8000000000000000, level = 3, flood = 0
gpa = 16864c008, old = 8000000118c70d07, new = 8000000000000000, level = 3, flood = 1
gpa = 16864c010, old = 80000001688a6d07, new = 8000000000000000, level = 3, flood = 2
gpa = 11c8f7000, old = 80000001608a7d07, new = 8000000000000000, level = 5, flood = 0
gpa = 1608a7000, old = 800000016864cd07, new = 80000000000001a0, level = 4, flood = 0
Note, in the shadow MMU, "level" describes the level a shadow page "points"
at, not the level of its associated SPTE. I.e. when write-flooding of 1GiB
PUD entries is detected, KVM recursively zaps shadow pages covering 256GiB
worth of memory. And as shown above, KVM's write-flooding detection
operates at all levels, so a single PMD (in L1) can effectively only leak
two unsync children (4KiB shadow pages) before it gets recursively zapped.
As a result, for the top-down zap, L0 KVM will leak at most 4 unsync shadow
pages per 256GiB of L2 memory.
The top-down zap also makes it more likely that L1 will self-heal (to some
extent), as any shadow pages that are "rediscovered" by future runs of L2
can get reclaimed by a recursive zap, whereas bottom-up zapping orphans
shadow pages over and over.
Note, in theory, there is some risk of over-zapping, e.g. due to zapping a
a large branch of the paging tree that L1 is only temporarily removing. In
practice, the usage patterns of hypervisors are highly unlikely to trigger
false positives. E.g. temporarily changing paging protections is typically
done at the leaf, not on a non-leaf entry. And if the L1 hypervisor is
updating large swaths of PTEs, e.g. to (temporarily?) remove chunks of
memory from L2, then L0 KVM's write-flooding detection will kick in, and
the children would be zapped anyways.
Fixes: 2de4085ccc ("KVM: x86/MMU: Recursively zap nested TDP SPs when zapping last/only parent")
Cc: Yosry Ahmed <yosry@kernel.org>
Cc: Jim Mattson <jmattson@google.com>
Cc: James Houghton <jthoughton@google.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260605174611.2222504-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When (conditionally) reprogramming fixed counters, use the hardware value
of FIXED_CTR_CTRL to detect changes, not the guest's original value. For
guests with a mediated PMU, overwriting fixed_ctr_ctrl_hw at the start of
reprogramming without actually reacting to changes in fixed_ctr_ctrl_hw can
lead to KVM ignoring PMU event filters.
E.g. if the guest attempts to enable a fixed PMC that is disallowed, and
then toggles a different PMC in a subsequent WRMSR, KVM will update
pmu->fixed_ctr_ctrl_hw and reprogram the PMC that is changing, but not the
others that are now effectively enabled in pmu->fixed_ctr_ctrl_hw.
Note, the perf-based PMU is unaffected, as it doesn't use fixed_ctr_ctrl_hw
(which is also why keying off fixed_ctr_ctrl_hw works for both PMUs.
Note #2, fixed_ctr_ctrl_hw won't mess up pmc_in_use either, because the
latter isn't used by the mediated PMU. Its purpose is solely to release
perf events that are no longer being actively used, and the meadiated PMU
obviously doesn't create perf events.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260528005419.0228F1F00A3A@smtp.kernel.org
Link: https://patch.msgid.link/20260603231905.1738487-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When checking if a VP ID is included in a sparse bank set, explicitly check
that the ID can actually be contained in a sparse bank (the TLFS allows for
a maximum of 64 banks of 64 vCPUs each). When handling a paravirtual TLB
flush for L2, the VP ID is copied verbatim from the enlightened VMCS,
without any bounds check, i.e. isn't guaranteed to be under the limit of
4096.
Failure to check the bounds of the VP ID leads to an out-of-bounds read
when testing the sparse bank, and super strictly speaking could lead to KVM
performing an unnecessary TLB flush for an L2 vCPU.
==================================================================
BUG: KASAN: use-after-free in hv_is_vp_in_sparse_set+0x85/0x100 [kvm]
Read of size 8 at addr ffff88811ba5f598 by task hyperv_evmcs/2802
CPU: 12 UID: 1000 PID: 2802 Comm: hyperv_evmcs Not tainted 7.1.0-rc2 #7 PREEMPT
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 0.0.0 02/06/2015
Call Trace:
<TASK>
dump_stack_lvl+0x51/0x60
print_report+0xcb/0x5d0
kasan_report+0xb4/0xe0
kasan_check_range+0x35/0x1b0
hv_is_vp_in_sparse_set+0x85/0x100 [kvm]
kvm_hv_flush_tlb+0xe9e/0x16c0 [kvm]
kvm_hv_hypercall+0xe6b/0x1e60 [kvm]
vmx_handle_exit+0x485/0x1b60 [kvm_intel]
kvm_arch_vcpu_ioctl_run+0x22e3/0x5070 [kvm]
kvm_vcpu_ioctl+0x5d0/0x10c0 [kvm]
__x64_sys_ioctl+0x129/0x1a0
do_syscall_64+0xb9/0xcf0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7f0e62d1a9bf
</TASK>
The buggy address belongs to the physical page:
page: refcount:0 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x11ba5f
flags: 0x4000000000000000(zone=1)
raw: 4000000000000000 0000000000000000 00000000ffffffff 0000000000000000
raw: ffffffffffffffff 0000000000000000 00000000ffffffff 0000000000000000
page dumped because: kasan: bad access detected
Memory state around the buggy address:
ffff88811ba5f480: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ffff88811ba5f500: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
>ffff88811ba5f580: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
^
ffff88811ba5f600: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
ffff88811ba5f680: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
==================================================================
Disabling lock debugging due to kernel taint
Opportunistically add a compile time assertion to ensure the maximum number
of sparse banks exactly matches the number of possible bits in the passed
in mask.
Cc: stable@vger.kernel.org
Fixes: c58a318f60 ("KVM: x86: hyper-v: L2 TLB flush")
Signed-off-by: Hyunwoo Kim <imv4bel@gmail.com>
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Link: https://patch.msgid.link/aiQyZIJtO-2Aj_xN@v4bel
[sean: add KASAN splat, drop comment, add assert, massage changelog]
Signed-off-by: Sean Christopherson <seanjc@google.com>
wrmsrl() is a deprecated synonym for wrmsrq(). Switch its users to
wrmsrq().
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Wei Liu <wei.liu@kernel.org>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Long Li <longli@microsoft.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Link: https://patch.msgid.link/20260608082809.3492719-4-jgross@suse.com
rdmsrl() is a deprecated synonym for rdmsrq(). Switch its users to
rdmsrq().
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Wei Liu <wei.liu@kernel.org>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Long Li <longli@microsoft.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Link: https://patch.msgid.link/20260608082809.3492719-2-jgross@suse.com
wrmsr_safe_on_cpu() has no users left. Delete it.
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260608051741.3207435-12-jgross@suse.com
In order to prepare retiring wrmsr_safe_on_cpu() switch
wrmsr_safe_on_cpu() users to wrmsrq_safe_on_cpu().
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Daniel Lezcano <daniel.lezcano@kernel.org>
Link: https://patch.msgid.link/20260608051741.3207435-11-jgross@suse.com
rdmsr_safe_on_cpu() has no users left. Delete it.
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260608051741.3207435-10-jgross@suse.com
In order to prepare retiring rdmsr_safe_on_cpu() switch
rdmsr_safe_on_cpu() users to rdmsrq_safe_on_cpu().
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Guenter Roeck <linux@roeck-us.net>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Daniel Lezcano <daniel.lezcano@kernel.org>
Link: https://patch.msgid.link/20260608051741.3207435-9-jgross@suse.com
In order to prepare removal of rdmsr_safe_on_cpu(), don't use it in
rdmsrq_safe_on_cpu(), but replace it with open coding it.
This will create a nearly verbatim copy of the same code, but this is
only temporary until rdmsr_safe_on_cpu() is removed.
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260608051741.3207435-8-jgross@suse.com
wrmsr_on_cpu() has no users left. Delete it.
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260608051741.3207435-7-jgross@suse.com
In order to prepare retiring wrmsr_on_cpu() switch wrmsr_on_cpu() users
to wrmsrq_on_cpu().
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Viresh Kumar <viresh.kumar@linaro.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Daniel Lezcano <daniel.lezcano@kernel.org>
Link: https://patch.msgid.link/20260608051741.3207435-6-jgross@suse.com
rdmsr_on_cpu() has no users left. Delete it.
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260608051741.3207435-5-jgross@suse.com
rdmsrl_on_cpu() has no users left. Delete it.
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Link: https://patch.msgid.link/20260608051741.3207435-3-jgross@suse.com
Now that lazy mmu state tracking no longer relies on the Xen specific
one, cpu lazy mode is the only user of the Xen generic lazy tracking.
Replace the Xen generic lazy tracking with a cpu lazy specific one.
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260526150514.129330-6-jgross@suse.com>
There are only very few use cases of XEN_LAZY_MMU left. Get rid of
them in order to avoid having to call enter_lazy(XEN_LAZY_MMU) and
leave_lazy(XEN_LAZY_MMU).
The query in xen_batched_set_pte() can be replaced by using
is_lazy_mmu_mode_active() instead.
As xen_flush_lazy_mmu() will be called only with lazy MMU mode being
active, the test for the lazy mode can just be dropped.
In xen_start_context_switch() and xen_end_context_switch() use
__task_lazy_mmu_mode_pause() and __task_lazy_mmu_mode_resume(),
allowing to drop xen_enter_lazy_mmu() and xen_leave_lazy_mmu()
completely.
Call arch_flush_lazy_mmu_mode() from arch_leave_lazy_mmu_mode(), as
this is the only required action now.
Drop the lazy mmu enter and leave paravirt hooks, leaving the flush
hook as the only needed one.
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260526150514.129330-5-jgross@suse.com>
Today xen_mc_issue() is deciding whether to call xen_mc_flush() or not
based on the lazy mode input parameter passed.
Modify this interface to pass a boolean indicating whether the flush
should be done.
For querying lazy mmu mode use the is_lazy_mmu_mode_active() helper.
This is done in preparation for dropping the xen_lazy_mode percpu
variable.
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260526150514.129330-3-jgross@suse.com>
Drop the lazy mode (cpu or mmu) from the xen_mc_batch and xen_mc_issue
trace entries.
This is done in preparation of removing the xen_lazy_mode percpu
variable.
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260526150514.129330-2-jgross@suse.com>
The only Xen file in debugfs is for dumping the p2m table when running
as a Xen PV guest. This might have been useful when the PV code was
young, but there haven't been any p2m related bugs requiring the p2m
dump since ages.
Remove the code and the related config option.
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260522152114.77319-4-jgross@suse.com>
A lot of arch/x86/xen/xen-ops.h is meant to be for PV only. Guard all
of it with CONFIG_XEN_PV in order to avoid someone misusing it in
non-PV builds. Additionally any 64-bit tests for now guarded items can
be dropped.
Move the enum pt_level definition to mmu_pv.c, as it is used only there.
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260522152114.77319-2-jgross@suse.com>
Make cosmetic changes:
o Rename struct pci_dev *dev to *pdev since there are cases of
struct device *dev in the file and all over the kernel
o Rename hv_build_pci_dev_id to hv_build_devid_type_pci in anticipation
of building different types of device ids
o Fix checkpatch.pl issues with return and extraneous printk
o Replace spaces with tabs
o Rename struct hv_devid *xxx to struct hv_devid *hv_devid given code
paths involve many types of device ids
o Fix indentation in a large if block by using goto.
There are no functional changes.
Reviewed-by: Souradeep Chakrabarti <schakrabarti@linux.microsoft.com>
Reviewed-by: Anirudh Rayabharam (Microsoft) <anirudh@anirudhrb.com>
Signed-off-by: Mukesh R <mrathor@linux.microsoft.com>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
- Add more AMD Zen6 models (Pratik Vishwakarma)
- Avoid confusing bootup message by the Intel resctl enumeration
code when running on certain AMD systems (Tony Luck)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmok/S0RHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1iOHw//Q2lG+XKJAG5wPgvkyp7EcQDgerj22LQS
ybapgg4u0scz4+24yt1vbzSRdvp+wTowSpviw7IH3rbemhWTkB7WZisjaKwPiPzo
gSYC3EtdcJvhaC2jqyhx1JSeR8I+d2U+tzE3OeOOxTvgOK4m3cvZhCsQIttPaO7I
DPW3ctqDYQc8/T4UQWMEyOhsKEGSplxrk8AdvzSG2QP+OOZ0gDBMh2s5ywREjSMq
OFf/tW6QdZm0CuxIYMV1PsdsYFZPI+v8MEtfGyjNw+NmkUEO/VYqQ80J0UC0rbbt
6MUT32vyLkDz0JN+IWW9YQn4OccuFfI+IZStmuZJr9jOFICpMogmIW77oobwN9ai
QeEXEvqvazmVlqMukzaDSlr42yV5nknr6bF+mlTbGIMUkCV2w40OkciyeI30LzNm
cCs/KsJeGb384ArsqH7+5coTHILvacAN9LWjf5E96APCp/2mzXwWT1bMvr7VM7Q/
65C1scpoQ5KgIewf+nijWOn+5FEN6cKVPgW1y06GpI55KDUUGIMgSyb+V5pwwVNa
fRa3qPbASopfyCni/mxPnNqSDVodFks9pDqSrg0WvmQLDdlaAzfwPf5l4F47YNGr
CmnZ7GJb00SZC/M9Pkecy0zXBZv+dfiwjiVnXBDjqYwIC1wOnlj4JTeVuaC7cFuu
688Dr2YMEEk=
=PMk2
-----END PGP SIGNATURE-----
Merge tag 'x86-urgent-2026-06-07' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 fixes from Ingo Molnar:
- Add more AMD Zen6 models (Pratik Vishwakarma)
- Avoid confusing bootup message by the Intel resctl enumeration
code when running on certain AMD systems (Tony Luck)
* tag 'x86-urgent-2026-06-07' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/resctrl: Only check Intel systems for SNC
x86/CPU/AMD: Add more Zen6 models
Adding struct bpf_tramp_node to decouple the link out of the trampoline
attachment info.
At the moment the object for attaching bpf program to the trampoline is
'struct bpf_tramp_link':
struct bpf_tramp_link {
struct bpf_link link;
struct hlist_node tramp_hlist;
u64 cookie;
}
The link holds the bpf_prog pointer and forces one link - one program
binding logic. In following changes we want to attach program to multiple
trampolines but we want to keep just one bpf_link object.
Splitting struct bpf_tramp_link into:
struct bpf_tramp_link {
struct bpf_link link;
struct bpf_tramp_node node;
};
struct bpf_tramp_node {
struct bpf_link *link;
struct hlist_node tramp_hlist;
u64 cookie;
};
The 'struct bpf_tramp_link' defines standard single trampoline link
and 'struct bpf_tramp_node' is the attachment trampoline object with
pointer to the bpf_link object.
This will allow us to define link for multiple trampolines, like:
struct bpf_tracing_multi_link {
struct bpf_link link;
...
int nodes_cnt;
struct bpf_tracing_multi_node nodes[] __counted_by(nodes_cnt);
};
Cc: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260606123955.345967-9-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Toolchain and infrastructure:
- Fix 'rustc-option' (the Makefile one) when cross-compiling that leads
to build or boot failures in certain configs.
- Work around a Rust compiler bug (already fixed for Rust 1.98.0) thats
lead to boot failures in certain configs due to missing 'uwtable'
LLVM module flags.
- Support a Rust compiler change (starting with Rust 1.98.0) in the
unstable target specification JSON files.
- Forbid Rust + arm + KASAN configs, which do not build.
'kernel' crate:
- Fix NOMMU build by adding a missing helper.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEPjU5OPd5QIZ9jqqOGXyLc2htIW0FAmokBWQACgkQGXyLc2ht
IW02uw//a6zLSwQh3c6wevbyfogCtFXA9zfEmS0VIgaFQ7rtBIMPGCIujXrg7mtM
uc0ZKg27dUyZQfrsKufjkP7ziQbwNaVowuR1/EVeFkF3bWH6BsaJic0/ewFAqbET
lua0IORlqIjxgCCYW4y93NdzzUAwM1ClENzxWsNBd1HdY0t3ZZqpySie6q2sbjws
MVr49Ghne5Q+HOeAgL3vK0ZblPx0Ujd+NcCBDVTzfliwNziGnVW5cS4QKdcYpEdc
nHIHUCpkV2oBivalLSXHPiZBVKRRRq8cNL0iS+6fZdukDER87HJ81EPESfndQFen
xdU5Qj7VHbr/PRR56P6dUpFdqcKCY3/HyoTBbr3GVGnH6pz0hs2605YoMsJI3Js8
IBkVNIAm2P3ud+oTDdA9+2vW/mBFCunfIdogqVPqb5DrPzjSl5uCgAOgX4J2yLLF
FfytH8kSqplLDdCuFq6bFNdZMgEK9qzZR99OqChh2N4xBxFeTuF1Ng+OPGJRSOxz
vlLXCav/j92ckoocqPzMG0V5AjxnrdRfql6lYZqrw7D6lrmMzwJuO7SaXiX+OkL4
hn6GB9FgWX+D6kKHotoMa51UjaDvCHOm3c2K/ROLQ5uuidFsK/585cQDg72/sq+3
nvAAjleYK9VESnlJ8h5BJWdgdWY5CRW/wwh3rprcc1gfXz1jtkA=
=7GFy
-----END PGP SIGNATURE-----
Merge tag 'rust-fixes-7.1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux
Pull Rust fixes from Miguel Ojeda:
"Toolchain and infrastructure:
- Fix 'rustc-option' (the Makefile one) when cross-compiling that
leads to build or boot failures in certain configs
- Work around a Rust compiler bug (already fixed for Rust 1.98.0)
thats lead to boot failures in certain configs due to missing
'uwtable' LLVM module flags
- Support a Rust compiler change (starting with Rust 1.98.0) in the
unstable target specification JSON files
- Forbid Rust + arm + KASAN configs, which do not build
'kernel' crate:
- Fix NOMMU build by adding a missing helper"
* tag 'rust-fixes-7.1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux:
rust: x86: support Rust >= 1.98.0 target spec
rust: arm64: set uwtable llvm module flag for CONFIG_UNWIND_TABLES
rust: helpers: add is_vmalloc_addr wrapper for NOMMU builds
rust: kasan/kbuild: fix rustc-option when cross-compiling
ARM: Do not select HAVE_RUST when KASAN is enabled
The CONFIG_PROPELLER_CLANG option currently depends on
ARCH_SUPPORTS_PROPELLER_CLANG, but this dependency seems unnecessary.
Remove ARCH_SUPPORTS_PROPELLER_CLANG and allow users to control
Propeller builds solely through CONFIG_PROPELLER_CLANG. This simplifies
the kconfig and avoids potential confusion.
Move the .llvm_bb_addr_map sections grouping to
include/asm-generic/vmlinux.lds.h.
The Propeller documentation has been updated to reflect the most
recent tool location and now includes instructions for arm64.
Contributor Acknowledgments:
* SPE instructions: Daniel Hoekwater <hoekwater@google.com>
Signed-off-by: Rong Xu <xur@google.com>
Suggested-by: Will Deacon <will@kernel.org>
Suggested-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Yabin Cui <yabinc@google.com>
Reviewed-by: Kees Cook <kees@kernel.org>
Link: https://patch.msgid.link/20260604195612.3757860-3-xur@google.com
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
The CONFIG_AUTOFDO_CLANG option currently depends on
ARCH_SUPPORTS_AUTOFDO_CLANG, but this dependency seems unnecessary.
Remove ARCH_SUPPORTS_AUTOFDO_CLANG and allow users to control AutoFDO
builds solely through CONFIG_AUTOFDO_CLANG. This simplifies the kconfig
and avoids potential confusion.
Expand the AutoFDO documentation to include instructions for arm64.
Contributor acknowledgments:
* SPE instructions: Daniel Hoekwater <hoekwater@google.com>
* ETM instructions: Yabin Cui <yabinc@google.com>
Signed-off-by: Rong Xu <xur@google.com>
Suggested-by: Will Deacon <will@kernel.org>
Tested-by: Yabin Cui <yabinc@google.com>
Reviewed-by: Kees Cook <kees@kernel.org>
Link: https://patch.msgid.link/20260604195612.3757860-2-xur@google.com
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
aperfmperf_get_khz() was replaced by arch_freq_get_on_cpu().
The remaining declaration in the header file is no longer used
and should be removed.
Fixes: f3eca381bd ("x86/aperfmperf: Replace arch_freq_get_on_cpu()")
Signed-off-by: Junxiao Chang <junxiao.chang@intel.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Link: https://patch.msgid.link/20260606021514.1433619-1-junxiao.chang@intel.com
Implement the read_snapshot() callback for the kvmclock clocksource. This
returns the kvmclock nanosecond value (for timekeeping) while also
providing the raw TSC value that was used to compute it.
The TSC is read inside the pvclock seqlock-protected region, ensuring the
raw TSC and derived kvmclock value are atomically paired.
This enables ktime_get_snapshot_id() to provide the raw TSC to consumers
like the vmclock PTP driver, which currently has to do a separate call to
get_cycles() to obtain a value at *approximately* the same time, to feed
through the vmclock calculation.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Kiro:claude-opus-4.6-1m
Link: https://patch.msgid.link/20260604095755.64849-3-dwmw2@infradead.org
arch_post_acpi_subsys_init() reads MSR_K8_INT_PENDING_MSG with rdmsr()
into a lo/hi pair but only uses the low 32 bits: K8_INTP_C1E_ACTIVE_MASK
(0x18000000) lies entirely within them. The 'hi' half is never consumed,
which triggers a -Wunused-but-set-variable warning under W=1:
arch/x86/kernel/process.c: In function 'arch_post_acpi_subsys_init':
arch/x86/kernel/process.c:972:17: warning: variable 'hi' set but not used
Read the full MSR into a single u64 with rdmsrq() and test the mask
against it, dropping the now-unnecessary lo/hi variables.
No functional change intended.
Signed-off-by: HyeongJun An <sammiee5311@gmail.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Jürgen Groß <jgross@suse.com>
Link: https://patch.msgid.link/20260604150052.3337246-1-sammiee5311@gmail.com
topology_num_nodes_per_package() reports values greater than one on certain
AMD systems resulting in resctrl's Intel model specific SNC detection
printing the confusing message:
"CoD enabled system? Resctrl not supported"
Add a check for Intel systems before looking at the topology.
[ reinette: Add Closes tag, fix tag typos, rework changelog ]
Fixes: 59674fc9d0 ("x86/resctrl: Fix SNC detection")
Reported-by: Babu Moger <babu.moger@amd.com>
Signed-off-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Babu Moger <babu.moger@amd.com>
Link: https://patch.msgid.link/9849330f45ac86344cc5ac54df2d313906d70bc4.1780634584.git.reinette.chatre@intel.com
Closes: https://lore.kernel.org/lkml/37ac0376-43a3-4283-a3d5-4d57b3bec578@amd.com/
On machines which don't have microcode applied yet, the revision is 0.
However, this doesn't work with the Zen family/model/stepping patch
arithmetic. So move the fixup to the patch level getter function and
this way make sure the patch level is always proper and thus the
arithmetic always works.
And now that it can be called on any family, make this Zen-only.
Assisted-by: claude/claude-opus-4-6
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20260530024213.86137-1-bp@kernel.org
Add proper error handling for the case when platform_device_alloc() returns
NULL due to memory allocation failure. This prevents a potential NULL pointer
dereference when trying to use the pdev pointer without checking if allocation
succeeded.
[ bp: Massage commit message. ]
Signed-off-by: Li Jun <lijun01@kylinos.cn>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/20260602100711.2542568-1-lijun01@kylinos.cn
When KVM added a lockdep assertion to catch unexpected cases where guest
CPUID lookups are performed in IRQ disabled context, it used
"linux/lockdep.h" for header inclusion even though lockdep.h is a
kernel wide header. Switch to using <linux/lockdep.h>.
Fixes: 9717efbe5b ("KVM: x86: Disallow guest CPUID lookups when IRQs are disabled")
Signed-off-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260604011106.315176-1-kai.huang@intel.com
[sean: add Fixes, Kai is too polite :-)]
Signed-off-by: Sean Christopherson <seanjc@google.com>
For DF and UMC PMUs, a single context is shared across all CPUs that
are connected to the same Data Fabric (DF) instance. Currently, the
Package ID, which also happens to be the Socket ID, is used to identify
DF instances. This approach works for configurations having a single IO
Die (IOD) but fails in the following cases.
* Older Zen 1 processors, where each chiplet has its own DF instance.
* Any configurations with multiple DF instances or multiple IODs in
the same package.
The correct way to identify DF instances is through the Node ID (not to
be confused with NUMA Node ID). This is available in ECX[7:0] of CPUID
leaf 0x8000001e and returned via topology_amd_node_id(). Hence, replace
usage of topology_logical_package_id() with topology_amd_node_id().
Fixes: 07888daa05 ("perf/x86/amd/uncore: Move discovery and registration")
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/e7a71a727c6a7b118c23d3e469929c538c4665aa.1780315832.git.sandipan.das@amd.com
On Sierra Forest and Clearwater Forest, the FRZ_ALL bit in the global
control register defaults to 0 at boot, but UBOX PMON units do not
work until the global control register is explicitly written with 0
to trigger hardware initialization properly.
Implement the generic uncore_msr_global_init() callback and add it to
gnr_uncore_init[], which is shared by GNR, GRR, SRF, and CWF.
Fixes: 632c4bf6d0 ("perf/x86/intel/uncore: Support Granite Rapids")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260602144908.263680-8-zide.chen@intel.com
If the die is offline when uncore_die_to_cpu() is called, it silently
returns 0, which is misleading. Return -1 in this case to indicate
that all CPUs on the die are offline and the caller can take care of
it accordingly.
Opportunistically, replace -EPERM with -ENODEV, as -ENODEV is
the appropriate error when no CPUs are online across all dies.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260602144908.263680-7-zide.chen@intel.com
Move die_to_cpu() into uncore.c so it can be reused by the MSR
initialization path, preparing for the introduction of an MSR global
initialization callback.
Move the cpus_read_{lock,unlock}() out of the API, in order to make
it possible to be called when the lock is being held.
Add the uncore_ prefix for consistency with other uncore APIs.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260602144908.263680-6-zide.chen@intel.com
On some Raptor Cove CPUs, enabling uncore PMON globally at driver init
may increase power consumption even when no perf events are in use.
Drop adl_uncore_msr_init_box() and defer programming the global control
register to enable_box(), so it is only set when a box is actually used.
IMC and IMC freerunning counters use a separate control path and are
unaffected.
Fixes: 772ed05f3c ("perf/x86/intel/uncore: Add Alder Lake support")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260602144908.263680-5-zide.chen@intel.com
pci_get_domain_bus_and_slot() increments the reference count of the
returned PCI device and therefore requires a matching pci_dev_put().
In skx_upi_topology_cb() and discover_upi_topology(), the lookup is
performed inside a loop, but pci_dev_put() is only called once after
the loop. As a result, references from all previous iterations are
leaked.
Move pci_dev_put(dev) into the if (dev) block immediately after
upi_fill_topology() returns.
Opportunistically, fix uninitialized variable in skx_upi_topology_cb().
Fixes: 4cfce57fa4 ("perf/x86/intel/uncore: Enable UPI topology discovery for Skylake Server")
Fixes: f680b6e606 ("perf/x86/intel/uncore: Enable UPI topology discovery for Icelake Server")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260602144908.263680-4-zide.chen@intel.com
Theoretically, intel_uncore_find_discovery_unit() could return NULL,
e.g., when a CPU die is offline during uncore enumeration and its PMU
units are not added to the discovery RB-tree.
Guard against a NULL return value and the resulting invalid box control
address (0) before accessing hardware.
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260602144908.263680-3-zide.chen@intel.com
In uncore_find_add_unit(), PMON units with the same unit ID may be
added to the uncore discovery RB-tree for different dies. These units
are distinguished by node->die.
However, intel_generic_uncore_box_ctl() uses a fixed die ID of -1 when
looking up the discovery unit, which may retrieve the wrong node on
multi-die systems.
Use box->dieid instead so the correct discovery unit is selected.
No functional issue has been observed so far because currently supported
platforms happen to use the same unit control register for such units.
Remove WARN_ON_ONCE() because with the above change a NULL unit can be
expected, e.g. when a CPU die is offline during uncore enumeration and
the unit is not added to the RB-tree. In this case,
intel_uncore_find_discovery_unit() returns NULL once the die becomes
online, and it is expected that the PMU box is not functional for that
die.
Fixes: b1d9ea2e1c ("perf/x86/uncore: Apply the unit control RB tree to MSR uncore units")
Signed-off-by: Zide Chen <zide.chen@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260602144908.263680-2-zide.chen@intel.com
Commit df4d29732f ("perf/x86/amd: Change/fix NMI latency mitigation
to use a timestamp") fixed handling of late-arriving NMIs but limited
the mitigation to processors having X86_FEATURE_PERFCTR_CORE. However,
it is unclear if processors without this feature are also affected.
When Mediated vPMU is enabled on affected hardware, it is also possible
to bypass the fix inside KVM guests if X86_FEATURE_PERFCTR_CORE is
removed from the guest CPUID (e.g. using "-cpu host,-perfctr-core" with
QEMU). Hence, use the mitigation at all times.
Fixes: df4d29732f ("perf/x86/amd: Change/fix NMI latency mitigation to use a timestamp")
Signed-off-by: Sandipan Das <sandipan.das@amd.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/29a3c970da289ab8f24282933bdb36545c0403e8.1780325517.git.sandipan.das@amd.com
All pieces of TDX module runtime updates are in place. Enable it if it
is supported.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-24-chao.gao@intel.com
The kernel exposes the TDX module version through sysfs so userspace
can check update compatibility. That information needs to remain
accurate across runtime updates.
A runtime update may change the module's update_version, so refresh
the cached version right after a successful update.
Drop __ro_after_init from tdx_sysinfo because it is now updated at
runtime.
Do not refresh the rest of tdx_sysinfo, even if some values change
across updates. TDX module updates are backward compatible, so
existing tdx_sysinfo consumers, such as KVM, can continue to operate
without seeing the new values.
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-22-chao.gao@intel.com
TDX metadata like the version number changes during a module update.
Add functions to lock out module updates.
The current stop_machine() implementation uses worker threads. The
scheduler actually does a full, normal context switch over to that
thread. preempt_disable() obviously inhibits that context switch and
thus, locks out stop_machine() users like the module update.
Thanks to Chao for the idea of using preempt_disable().
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
After per-CPU initialization, the module is nearly functional. It is
in a similar state to TDX initialization before TDH.SYS.CONFIG.
At this point, the kernel _could_ just repeat the boot-time sequence,
but that would land the new module in a slightly different state than
the old module. This would leave old TDs unrunnable, which is not a
good outcome.
Thankfully, the "handoff" data saved during module shutdown should
contain all the information needed to restore the TDX module state to
exactly what it was before the update.
Restore TDX module state. The TDX module only needs a single copy so
only do this on the lead CPU.
Restoration errors can theoretically be handled in a few ways. For
instance, userspace could try to load a different TDX module version.
Or, the kernel could give up on the handoff process and just
reinitialize the new module from scratch, which would lose all
existing TDs.
Simply propagate errors to userspace. Ignore the idea of a
TD-destroying reinitialization. It would destroy data like a reboot
and if things have gone that wrong a reboot is probably the best
option anyway.
Note: the location and the format of handoff data is defined by the
TDX module. The new module knows where to get handoff data and how to
parse it. The kernel does not touch it at all.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-21-chao.gao@intel.com
Continue fleshing out the update process. At this point the new module
is sitting in memory but has never been called and is not usable. It
is in a similar state to the when the system first boots.
Leave the P-SEAMLDR behind. Stop making calls to it. Transition to
calling the new TDX module itself to set up both global and per-cpu
state.
Share tdx_cpu_enable() with the fresh-boot module initialization code.
Export it and invoke it on all CPUs.
Note: "TDX global initialization" needs to be done once before "TDX
per-CPU initialization". It would be a great fit for the new runtime
update "is_lead_cpu" logic. But tdx_cpu_enable() already has some
logic to do the global initialization properly. Just use it directly
to maximize fresh-boot and runtime update code sharing.
== Background ==
The boot-time and post-update initialization flows share the same first
steps:
- TDX global initialization
- TDX per-CPU initialization
After that, they diverge:
- Fresh boot:
Prepare TDMRs/PAMTs
Configure the TDX module
Configure the global KeyID
Initialize TDMRs
- Runtime update:
Restore TDX module state from handoff data
Future changes will consume the handoff data.
[ dhansen: major changelog munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-20-chao.gao@intel.com
Continue fleshing out the update proces. The old module is shut down
and the system is ready for the new module image. Run the
SEAMLDR.INSTALL SEAMCALL on all CPUs.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-19-chao.gao@intel.com
The TDX module requires a one-time global initialization (TDH.SYS.INIT) and
per-CPU initialization (TDH.SYS.LP.INIT) before use. These initializations
are guarded by software flags to prevent repetition.
Reset all software flags guarding the initialization flows to allow the
global and per-CPU initializations to be triggered again after updates.
[ dhansen: trim down changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-18-chao.gao@intel.com
The first step of TDX module updates is shutting down the current TDX
module. This step also packs state information that needs to be
preserved across updates, called "handoff data". This handoff data is
consumed by the updated module and stored internally in the SEAM range and
hidden from the kernel.
Since the handoff data layout may change between modules, the handoff
data is versioned. Each module has a native handoff version and
provides backward support for several older versions.
The complete handoff versioning protocol is complex as it supports both
module upgrades and downgrades. See details in "Intel Trust Domain
Extensions (Intel TDX) Module Base Architecture Specification", Chapter
"Handoff Versioning".
Ideally, the kernel needs to retrieve the handoff versions supported by
the current module and the new module and select a version supported by
both. But since this implementation only supports module upgrades, simply
request handoff data from the current module using its highest supported
version. That is sufficient for this upgrade-only implementation.
Retrieve the module's handoff version from TDX global metadata and add an
update step to shut down the module. Module shutdown only needs to run on
one CPU.
Don't cache the handoff information in tdx_sysinfo. It is used only for
module shutdown, and is present only when the TDX module supports updates.
Caching it in get_tdx_sys_info() would require extra update-support guards
and refreshing the cached value across module updates.
[ dhansen: fix up function variables, remove 'cpu'.
Return from tdx_module_shutdown() early if handoff call fails. ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://patch.msgid.link/20260520133909.409394-17-chao.gao@intel.com
A TDX module update is a multi-step process, and any step can fail.
The current update flow continues to later steps after an error.
Continuing after a failure can cause the TDX module to enter an
unrecoverable state.
But certain failures during the initial module shutdown step should
simply return an error to userspace, so the update can be retried
cleanly.
To preserve that recoverability, one option would be to abort the
update only for those failures, since they occur before any TDX module
state is changed. But special-casing specific failures in specific
steps would complicate the do-while() update loop for no benefit.
Simply abort update on any failure, at any step.
Track failures for each step, stop the update loop once a failure is
observed, and do not advance the state machine to the next step.
[ dhansen: style nits ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://lore.kernel.org/linux-coco/aQFmOZCdw64z14cJ@google.com/ # [1]
Link: https://patch.msgid.link/20260520133909.409394-16-chao.gao@intel.com
tl;dr: Use stop_machine() and a state machine based on the
"MULTI_STOP" pattern to implement core TDX module update logic.
Long version:
TDX module updates require careful synchronization with other TDX
operations. The requirements are (#1/#2 reflect current behavior that
must be preserved):
1. SEAMCALLs need to be callable from both process and IRQ contexts.
2. SEAMCALLs need to be able to run concurrently across CPUs
3. During updates, only update-related SEAMCALLs are permitted; all
other SEAMCALLs shouldn't be called.
4. During updates, all online CPUs must participate in the update work.
No single lock primitive satisfies all requirements. For instance,
rwlock_t handles #1/#2 but fails #4: CPUs spinning with IRQs disabled
cannot be directed to perform update work.
Use stop_machine() as it is the only well-understood mechanism that can
meet all requirements.
And TDX module updates consist of several steps (See Intel Trust Domain
Extensions (Intel TDX) Module Base Architecture Specification, Chapter
"TD-Preserving TDX module Update"). Ordering requirements between steps
mandate lockstep synchronization across all CPUs.
multi_cpu_stop() provides a good example of executing a multi-step task
in lockstep across CPUs, but it does not synchronize the individual
steps inside the callback itself.
Implement a similar state machine as the skeleton for TDX module
updates. Each state represents one step in the update flow, and the
state advances only after all CPUs acknowledge completion of the current
step. This acknowledgment mechanism provides the required lockstep
execution.
The update flow is intentionally simpler than multi_cpu_stop() in two ways:
a) use a spinlock to protect the control data instead of atomic_t and
explicit memory barriers.
b) omit touch_nmi_watchdog() and rcu_momentary_eqs(), which exist
there for debugging and are not strictly needed for this update flow
Potential alternative to stop_machine()
=======================================
An alternative approach is to lock all KVM entry points and kick all
vCPUs. Here, KVM entry points refer to KVM VM/vCPU ioctl entry points,
implemented in KVM common code (virt/kvm). Adding a locking mechanism
there would affect all architectures KVM supports. And to lock only TDX
vCPUs, new logic would be needed to identify TDX vCPUs, which the KVM
common code currently lacks. This would add significant complexity and
maintenance overhead to KVM for this TDX-specific use case, so don't take
this approach.
[ dhansen: normal changelog/style munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-15-chao.gao@intel.com
There are two important ABIs here:
'struct tdx_image' - The on-disk and in-memory format for a TDX
module update image.
'struct seamldr_params' - The in-memory ABI passed to the TDX module
loader. Points to a single 'struct tdx_image'
broken up into 4k pages.
Userspace supplies the update image in 'struct tdx_image' format. The
image consists of a header followed by a sigstruct and the module
binary. P-SEAMLDR, however, consumes 'struct seamldr_params' rather
than the image directly.
Parse the 'struct tdx_image' provided by userspace and populate a
matching 'struct seamldr_params'.
The 'tdx_image' ABI is versioned. Two public versions exist today:
0x100 and 0x200. This kernel only accepts 0x200. The older 0x100
format is being deprecated and is intentionally not supported here.
Future versions of the module might be able to use the same ABIs
(user/kernel and kernel/SEAMLDR) but they will not be able to use this
kernel code.
Reject module images without that specific version. This ensures that
the kernel is able to understand the passed-in format.
Validate the 'struct tdx_image' header before using it, because the
header is consumed solely by the kernel to locate the sigstruct and
module within the image. Do not validate the payload itself. The
sigstruct and module pages are passed through to P-SEAMLDR, which
validates them as part of the update.
sigstruct_pages_pa_list currently has only one entry, but it will grow
to four pages in the future. Keep it as an array for symmetry with
module_pages_pa_list and for extensibility.
[ dhansen: normal changelog clarification/munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-14-chao.gao@intel.com
tl;dr: Select fw_upload for doing TDX module updates. The process of
selecting among available update images is complicated and nuanced. Punt
the selection process out to userspace. One existing userspace
implementation today is the script in the Intel TDX Module Binaries
repository[1].
Long Version:
The kernel supports two primary firmware update mechanisms:
1. request_firmware() - used by microcode, SEV firmware, hundreds of
other drivers
2. 'struct fw_upload' - used by CXL, FPGA updates, dozens of others
The key difference between is that request_firmware() loads a named file
from the filesystem where the filename is kernel-controlled, while
fw_upload accepts firmware data directly from userspace.
TDX module firmware update selection policy is too complex for the kernel.
Leave it to userspace and use fw_upload.
Add a skeleton fw_upload implementation to be fleshed out in subsequent
patches.
Refactor the sysfs visiblity attribute function so it can be used as a
more generic flag for the presence of viable runtime update support.
Why fw_upload instead of request_firmware()?
============================================
Selecting a TDX module update image is not a simple "load the latest"
decision. Userspace needs to choose an image that is compatible with both
the platform and the currently running module.
Some constraints are hard requirements:
a. Module version series are platform-specific. For example, the 1.5.x
series runs on Sapphire Rapids but not Granite Rapids, which needs
2.0.x.
b. Updates are also constrained by version distance. A 1.5.6 module
might permit updates to 1.5.7 but not to 1.5.50.
There may also be userspace policy choices:
c. Decide the update direction: upgrade or downgrade
d. Choose whether to optimize for fewer updates or smaller version
steps, for example, 1.2.3=>1.2.5 versus 1.2.3=>1.2.4=>1.2.5.
Given that complexity, leave module selection to userspace and use
fw_upload.
1. https://github.com/intel/confidential-computing.tdx.tdx-module.binaries/blob/main/version_select_and_load.py
[ dhansen: add version script link, add more explanation of code moves,
fix some minor whitespace issues ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://lore.kernel.org/kvm/01fc8946-eb84-46fa-9458-f345dd3f6033@intel.com/
Link: https://patch.msgid.link/20260520133909.409394-13-chao.gao@intel.com
TDX-capable CPUs clobber the current VMCS on P-SEAMLDR calls. Clearing
the current VMCS behind KVM's back breaks KVM.
Future CPUs will fix this by preserving the current VMCS across
P-SEAMLDR calls. A future specification update will describe the
VMCS-clearing behavior as an erratum and to state that it does not
occur when IA32_VMX_BASIC[60] is set.
Add a CPU bug bit and refuse to expose P-SEAMLDR information on
affected CPUs.
Use a CPU bug bit to stay consistent with X86_BUG_TDX_PW_MCE. As a
bonus, the bug bit is visible to userspace, which allows userspace to
determine why these sysfs files are not exposed, and it can also be
checked by other kernel components in the future if needed.
== Alternatives ==
Two workarounds were considered but both were rejected:
1. Save/restore the current VMCS around P-SEAMLDR calls. This produces ugly
assembly code [1] and doesn't play well with #MCE or #NMI if they
need to use the current VMCS.
2. Move KVM's VMCS tracking logic to the TDX core code, which would break
the boundary between KVM and the TDX core code [2].
[ dhansen: comment and changelog munging. Add seamldr_call() bug check. ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lore.kernel.org/kvm/fedb3192-e68c-423c-93b2-a4dc2f964148@intel.com/ # [1]
Link: https://lore.kernel.org/kvm/aYIXFmT-676oN6j0@google.com/ # [2]
Link: https://patch.msgid.link/20260520133909.409394-12-chao.gao@intel.com
TDX module updates require userspace to select the appropriate module
to load. Expose necessary information to facilitate this decision. Two
values are needed:
- P-SEAMLDR version: for compatibility checks between TDX module and
P-SEAMLDR
- num_remaining_updates: indicates how many updates can be performed
Expose them as tdx-host device attributes visible only when updates
are supported.
Note that the underlying P-SEAMLDR attributes are available regardless
of update support; this only restricts their visibility to userspace.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-11-chao.gao@intel.com
P-SEAMLDR reports its state via SEAMLDR.INFO, including its version and
the number of remaining runtime updates.
This information is useful for userspace. For example, userspace can
use the P-SEAMLDR version to determine whether a candidate TDX module
is compatible with the running loader, and can use the remaining
update count to determine whether another runtime update is still
possible.
Add a helper to retrieve P-SEAMLDR information in preparation for
exposing P-SEAMLDR version and other necessary information to userspace.
Export the new kAPI for use by the "tdx_host" device.
Note that there are two distinct P-SEAMLDR APIs with similar names:
"SEAMLDR.INFO" is metadata about the loader. It's metadata for the
update process.
"SEAMLDR.SEAMINFO" is metadata about SEAM mode. It is for the module
init process, not for the update process.
Use SEAMLDR.INFO here.
For details, see "Intel Trust Domain Extensions - SEAM Loader (SEAMLDR)
Interface Specification".
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-10-chao.gao@intel.com
The TDX architecture uses the "SEAMCALL" instruction to communicate with
SEAM mode software. Right now, the only SEAM mode software that the kernel
communicates with is the TDX module. But, there is actually another
component that runs in SEAM mode but it is separate from the TDX module:
the persistent SEAM loader or "P-SEAMLDR". Right now, the only component
that communicates with it is the BIOS which loads the TDX module itself at
boot. But, to support updating the TDX module, the kernel now needs to be
able to talk to it.
P-SEAMLDR SEAMCALLs differ from TDX module SEAMCALLs in areas such as
concurrency requirements.
Add a P-SEAMLDR wrapper to handle these differences and prepare for
implementing concrete functions.
Use seamcall_prerr() (not '_ret') because current P-SEAMLDR calls do not
use any output registers other than RAX.
Note: Despite the similar name, the NP-SEAMLDR ("Non-Persistent")
(ACM) invoked exclusively by the BIOS at boot rather than a component
running in SEAM mode. The kernel cannot call it at runtime. It exposes
no SEAMCALL interface.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://cdrdv2.intel.com/v1/dl/getContent/733582 # [1]
Link: https://patch.msgid.link/20260520133909.409394-9-chao.gao@intel.com
For TDX module updates, userspace needs to select compatible update
versions based on the current module version.
For example, the 1.5.x series runs on Sapphire Rapids but not Granite
Rapids, which needs 2.0.x. Updates are also constrained by version
distance, so a 1.5.6 module might permit updates to 1.5.7 but not to
1.5.20.
Start the process of punting the version selection logic to userspace.
Expose the TDX module version in the new faux device.
Define TDX_VERSION_FMT macro for the TDX version format since it will be
used multiple times. Also convert an existing print statement to use it.
== Background ==
For posterity, here's what other firmware mechanisms do:
1. AMD SEV leverages an existing PCI device for the PSP to expose
metadata. TDX uses a faux device as it doesn't have PCI device
in its architecture.
2. Microcode uses per-CPU virtual devices to report microcode revisions
because CPUs can have different revisions. But, there is only a
single TDX module, so exposing the TDX module version through a global
TDX faux device is appropriate
3. ARM's CCA implementation isn't in-tree yet, but will likely follow a
similar faux device approach, though it's unclear whether they need
to expose firmware version information
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ # [1]
Link: https://patch.msgid.link/20260520133909.409394-8-chao.gao@intel.com
TDX depends on a platform firmware module that runs on the CPU.
Unlike other CoCo architectures, TDX has no hardware "device"
running the show, just a blob on the CPU.
Create a virtual device to anchor interactions with this platform
firmware. This lets later code:
- expose metadata: TDX module version, seamldr version, to userspace
as device attributes
- implement firmware uploader APIs (which are tied to a device) to
support TDX module runtime updates
Use a faux device because the TDX module is singular within the system
and has no platform resources. Using a faux device eliminates the need
to create a stub bus.
The call to tdx_get_sysinfo() ensures that the TDX module is ready to
provide services.
Note that AMD has a PCI device for the PSP for SEV and ARM CCA will
likely have a faux device [1].
Thanks to Dan and Yilun for all the help on this one.
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ # [1]
Link: https://patch.msgid.link/20260520133909.409394-7-chao.gao@intel.com
TDX host core code implements three seamcall*() helpers to make SEAMCALLs
to the TDX module. Currently, they are implemented in <asm/tdx.h> and
are exposed to other kernel code which includes <asm/tdx.h>.
However, other than the TDX host core, seamcall*() are not expected to
be used by other kernel code directly. For instance, for all SEAMCALLs
that are used by KVM, the TDX host core exports a wrapper function for
each of them.
Move seamcall*() and related code out of <asm/tdx.h> and make them only
visible to TDX host core.
Since TDX host core tdx.c is already very heavy, don't put low level
seamcall*() code there but to a new dedicated "seamcall_internal.h". Also,
currently tdx.c has seamcall_prerr*() helpers which additionally print
error message when calling seamcall*() fails. Move them to
"seamcall_internal.h" as well. In such way all low level SEAMCALL helpers
are in a dedicated place, which is much more readable.
Copy the copyright notice from the original files and consolidate the
date ranges to:
Copyright (C) 2021-2023 Intel Corporation
Signed-off-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-6-chao.gao@intel.com
The kernel uses several global flags to guard one-time TDX initialization
flows and prevent them from being repeated.
When the TDX module is updated, all of those states must be reset so that
the module can be initialized again. Today those states are kept as
separate global variables, which makes the reset path awkward and easy to
miss when a new state is added.
Group the states into a single structure so they can be reset together, for
example with memset(), and so a newly added state won't be missed.
Drop the __ro_after_init annotation from tdx_module_initialized because
the other two states do not have it. And with TDX module update support,
all the states need to be writable at runtime.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-4-chao.gao@intel.com
