- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1
is present, guaranteeing that EL2-specific register bits are stateful
in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope
of those instructions remains 'in host' (i.e. L1 kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly in
the VGIC, our in-kernel generator of bugs (and sometimes interrupts)
-----BEGIN PGP SIGNATURE-----
iI0EABYKADUWIQSNXHjWXuzMZutrKNKivnWIJHzdFgUCaoZ/khccb2xpdmVyLnVw
dG9uQGxpbnV4LmRldgAKCRCivnWIJHzdFkK7AP9zvAcDf+DVte8+mZjCMDrvNB1n
UWsBnQkdZ1nDDYBJlwD8D1pvurK4ZV3UIl68IgpTqYKhMspCuOWDHrnBt1QyqQU=
=2iPX
-----END PGP SIGNATURE-----
Merge tag 'kvmarm-7.3' of https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 changes for 7.3
- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1
is present, guaranteeing that EL2-specific register bits are stateful
in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope
of those instructions remains 'in host' (i.e. L1 kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly in
the VGIC, our in-kernel generator of bugs (and sometimes interrupts)
* kvm-arm64/pkvm-7.3: (26 commits)
: pKVM updates for 7.3
:
: - Avoid name collision on trace_clock() when CONFIG_NVHE_EL2_TRACING is
: disabled (Mostafa Saleh)
:
: - Clean up state tracking for whether the EL2 shadow VM has been
: created (Fuad Tabba)
:
: - Synchronize SCTLR_EL1 when injecting an exception to use current
: PAN/SSBS state (Fuad Tabba)
:
: - Avoid unnecessary cache maintenance when I/D-cache are known to be
: coherent in pKVM (Mostafa Saleh)
:
: - Lazy vCPU context save/restore for pKVM (Fuad Tabba)
:
: - Various fixes to the stage-2 MMU for pKVM (Fuad Tabba)
:
: - Allow counter offsetting of non-protected guests in protected mode
: (Mostafa Saleh)
:
: - Condition the 'broken CNTVOFF' mitigation on a VM actually having a
: nonzero offset, fixing boot failures of pVMs on affected hardware
: (Mostafa Saleh)
KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
KVM: arm64: Fix timer offsets for non-protected VMs
KVM: arm64: Make timer_get_offset() work in all contexts
KVM: arm64: selftests: Add stage-2 block transition test
KVM: arm64: Don't advertise eager page splitting under pKVM
KVM: arm64: Don't WARN on pKVM stage-2 map failures
KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled
KVM: arm64: Top up stage-2 memcache for dirty logging faults
KVM: arm64: Top up the memcache for pKVM permission faults
KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings
KVM: arm64: Implement lazy vCPU state sync for non-protected guests
KVM: arm64: Add primitives to flush/sync the VGIC state at EL2
KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
KVM: arm64: Add host and hypervisor vCPU lookup primitives
KVM: arm64: Move PSCI helper functions to a shared header
KVM: arm64: Factor out reusable vCPU reset helpers
KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
KVM: arm64: Extract MPIDR computation into a shared header
KVM: arm64: selftests: Add a userspace watchpoint test
KVM: arm64: Flush external_mdscr_el1 to the pKVM hyp vCPU
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
* kvm-arm64/pmu-7.3:
: vPMU updates for 7.3
:
: - Support for slot-based PMU events, relying on new UAPI that makes
: selection of a vPMU implementation mandatory (Congkai Tan)
KVM: arm64: Add KVM_ARM_VCPU_PMU_V3_STRICT vCPU feature
KVM: arm64: Ignore writes to PMCR_EL0.N when using strict UAPI
KVM: arm64: Advertise STALL_SLOT* in PMCEID1 under strict PMUv3 UAPI
KVM: arm64: Expose PMMIR_EL1.SLOTS under strict PMUv3 UAPI
Signed-off-by: Oliver Upton <oupton@kernel.org>
- Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs.
- Harden the SNP and TDX "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion.
- Fix a variety of #DB priority bugs.
- Fix a class of races related to enabling Hyper-V emulation on a vCPU after
the vCPU is visible to the rest of KVM.
- Use static calls for nested virtualization ops.
- Move more KVM-internal code out of x86's kvm_host.h.
- Enumerate support for a variety of Zhaoxin instructions that don't require
explicit virtualization.
- Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path.
- Harden kvm_vcpu_map() against double-mapping and thus leaking references.
- Misc fixes and cleanups, e.g. for largely benign syzkaller splats.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp8uZQACgkQOlYIJqCj
N/2Pgw//ZmY9QZSvbSS69yYporgRhlTCMq1k3siLYfT19XShCpjO0KK9QTggs57L
0JXgbv4kwzvWWliwlRVK3e2nw027GI0j0kDRdcI3RAsaufZCjVT300cfUJBy/T4C
EQLULA8J3Q5pXStuW7SS2bMjI/D6J0+h80co1gfN4yeBDuKXUhtpmW9G/frCQyqI
3DduI57IBWO+pHImMWVx5jFc6WeDokQPxVASWE4zAU5PobXZSQ2K7KCbcx/rHxLt
3JZRw9sECF4Rn5DHm9Xag4bfZH0IR1jQ4P5fIzlvvYcW8yhj4CcTK648fVivBCL5
cpVQRCUVM6DyAX7l6tchP44XHv8a+D/bB2qZUlKvrMiSYA3usW7olKEDNUPI0dsm
vviat3br/XBu3msNkMZcYoP2WS+ka9zDAkSw7MkNWg1m62UfzvI70AQC0r7Fs/UG
+bUwMpJ11C/TOjwVL8DNMu0RT2uPgZ2xgtyD/IeVrqlm867xtlGxit+Uqs9V0vro
LC2YhNR2LYk8NhKO/IZsHI/9eTD4xQyPBn13U6dsZy19fqcPftSJM7avZfQNiA+l
nPN3v/FZmM22k32hc4NmGCx1/pBU1QtnTcZ0Z91pVzD5+DrvpYnMJpdhazrLL18j
IN2OK+Vy/WiP2xnAvCjOFcHCVgAe3SCorqIrjRlY2nMbnRmc03E=
=bQJe
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-misc-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM x86 misc changes for 7.3
- Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs.
- Harden the SNP and TDX "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion.
- Fix a variety of #DB priority bugs.
- Fix a class of races related to enabling Hyper-V emulation on a vCPU after
the vCPU is visible to the rest of KVM.
- Use static calls for nested virtualization ops.
- Move more KVM-internal code out of x86's kvm_host.h.
- Enumerate support for a variety of Zhaoxin instructions that don't require
explicit virtualization.
- Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path.
- Harden kvm_vcpu_map() against double-mapping and thus leaking references.
- Misc fixes and cleanups, e.g. for largely benign syzkaller splats.
- Fix several issues with seeding KVM's pRNG, and rework the pRNG APIs to that
the pRNG can be sanely used in host code, not just guest code.
- Add an IRQ test to validate virtual IRQ deliverty for IRQs wired up via
KVM_IRQFD + KVM_SET_GSI_ROUTING, with optional support for triggering IRQs
via writes to an assigned VFIO device.
- Add syscall wrappers to assert success on a variety of pthreads and CPU
affinity APIs.
- Set vCPU pthread affinity as early as possible to reduce contention issues
that were surfaced by PREEMPT_LAZY, which result in runtimes of over a
minute on large hosts, versus the expected ~5 seconds.
- Rework the PMU counters test to run each testcase using a single VM with
many vCPUs for each sub-testcase, instead of using a unique VM for each
sub-testcase. This cuts the runtime by ~20x.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp8yNoACgkQOlYIJqCj
N/2s+w/+KtexoVLUzAy5DwIpm2g0pHqPAm5me7lTj1IuKyPxCksmmskNu3Nrfpn+
0fp0QyB3gfUFscwKgqq8waLHBRyszQvTgfF43h/Bvu7QoNgoBViu9ybkd91REkSM
CZj6ltUiHYJ1AmyB2khc5kPisLdZvEewSr1A2I9PudaPDa7DlOzN7DOCq4yePYkq
4Bt7oFKDB/BH1Cv/QlCpUbseKLydBX8Zgd9QZy8bTqRdEeHywXfbKv1b1AkX8Vc+
C0puEKrnJ6gluBTERYZ8+ibLBdewt/Hr+JdqlVtVkjSF2esGAWW//RuDlRj2zRzl
MTj5BK8MXotmrHp0RZD+QTcG6JEOVM5XtRTstJP4taKV1jKXxHcCa3+9jOSFbcU1
0E7cklQOwbgXIGw/Tc2fQ1wHu2E4Ee/r7C29qGxx6xUZQ6krUYBqRUZMZ0e5isK1
GW2EPpb8AXC6+XaoNeCP2rPhu6mB+GhC2Q4k9pKWjasDTLes6CkBfDf/+i5+LEXk
oQ3UTP1qGqUmm1y5m9h7+jUy4CdlzIbcLIUUt4X0naNJbpJsZqRvX3D4SKLf/wtV
ZPmmih2ha01gnErm3Znn8rxIIodEiwwXIRSSKi1IzBmghOJzgSXtynMzXmwOaQtW
4vdT22Yf1ip3pREfvtbL0pPDdxaJI4ngmm8Ia/ORHLYFbJmA5aM=
=fs2O
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-selftests2-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM selftests changes for 7.3, part 2
- Fix several issues with seeding KVM's pRNG, and rework the pRNG APIs to that
the pRNG can be sanely used in host code, not just guest code.
- Add an IRQ test to validate virtual IRQ deliverty for IRQs wired up via
KVM_IRQFD + KVM_SET_GSI_ROUTING, with optional support for triggering IRQs
via writes to an assigned VFIO device.
- Add syscall wrappers to assert success on a variety of pthreads and CPU
affinity APIs.
- Set vCPU pthread affinity as early as possible to reduce contention issues
that were surfaced by PREEMPT_LAZY, which result in runtimes of over a
minute on large hosts, versus the expected ~5 seconds.
- Rework the PMU counters test to run each testcase using a single VM with
many vCPUs for each sub-testcase, instead of using a unique VM for each
sub-testcase. This cuts the runtime by ~20x.
- Clean up nested SVM's handling of GPRs on L2<=>L1 transitions, reuse the
functionality for nested VMX, and drop the ucall hack that was fudging
around the lack of GPR switching on nVMX.
- Add a stress test to verify KVM doesn't clobber/drop #PF state, e.g. CR2,
across save/restore, including when L2 is active.
- Add a test to verify KVM_CREATE_VM accepts exactly what is reported by
KVM_CAP_VM_TYPES.
- Misc selftests fixes and cleanups
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmp8wrYACgkQOlYIJqCj
N/2XHxAAgqT2MgtdpST9/n7pW1ifgiaRsxqFI09c2YheFVmpYzILVio4qaCLrO1e
92whJbOuiXgPWqPOU0FyS0wHICo5Q4aMJG3ECoAufeBifHM6+pR/mNE2q4U95rkw
jbLhEaRgWgeiHrRoPZo8KtH/gbzaXCJZJIkVfsRVFY10wPiymM/OSsSHeBHYsJAN
L9vmoJFa14tFu5c2rwPcpqgczWj2mKgVnlGVPRX3MizVXUuOwfatdChFOEEDxKCC
xsGieTbu19MBZuqcwzeIWLEw1e4TJcsi1YsCiwFGyV5r3eq2ZFvjMoLBJ7drLERs
awHaK8kG3qOo1KMZO0fVKi7gOq41h8VqsGDx/RKr3sKdd4aO67X5Sw5YZWB4xy68
TJduTwUn8DFHzl9VJK5iyyIw8R8E2EqlxlybMPh2B+luulzo5DkCD6XBjDo5TD1A
m2Ky7BzTzGOVlHpSj6uqf5+654BW/nEDMAekXFD7NkQfdZ8Bz27PpdA7l1sijFU0
0kDCRroAUqj/IVw+j2Ptnh+YbYYlKq/8OVu7RJmKw/nFSMSFLAiRn1U3xZedDrIW
HU08F3xZ/FxNIqu9a8biYJsLJovUFG5l2V2NGDy8POOkPLQ+bERjq366h50LHY5d
PPD8erp8cQwaX6Ypa3/K5lezUMTdDZowXevMfFt3/nV5gp37+Es=
=VqlG
-----END PGP SIGNATURE-----
Merge tag 'kvm-x86-selftests-7.3' of https://github.com/kvm-x86/linux into HEAD
KVM selftests changes for 7.3, part 1
- Clean up nested SVM's handling of GPRs on L2<=>L1 transitions, reuse the
functionality for nested VMX, and drop the ucall hack that was fudging
around the lack of GPR switching on nVMX.
- Add a stress test to verify KVM doesn't clobber/drop #PF state, e.g. CR2,
across save/restore, including when L2 is active.
- Add a test to verify KVM_CREATE_VM accepts exactly what is reported by
KVM_CAP_VM_TYPES.
- Misc selftests fixes and cleanups
1. Advertise already-supported capabilities.
2. Some bug fixes about timer and MMIO.
3. Some hardening about interrupt injection.
4. Replace kvm_err() with kvm_pr_unimpl().
5. Add FPU/LSX/LASX test cases for selftests.
-----BEGIN PGP SIGNATURE-----
iQJKBAABCAA0FiEEzOlt8mkP+tbeiYy5AoYrw/LiJnoFAmp9ca4WHGNoZW5odWFj
YWlAa2VybmVsLm9yZwAKCRAChivD8uImelAuD/4jhprQWR9b8RMQ9bGYaBDml+98
fCRW8NPyicnct+97U/E9EXJrrMyyDOqJtOSnNQOVmTjz3IGQtzLoHULaJPqUv0Dh
7xUbpmb7qkqxrykUEUa81tDnClpISfLcgUBWKsg0VGLyF7MIEnbtECTadbzQEd10
hNXy/Pv8OiAIZe4rtlGWxOU5JxU6OVf0zEmXiS6cbqXrwxVl4STbF97/5ULqjU+D
4KMMs1IcOdf4klQeoE1oKvWsCiedZQBkKbtsv2hpDMmY4lUdheJg9+0atr8Oh7VS
TtlO6Fmt7ilK9eFOCapDcs4gf5WQgs8WXKAyVUYSoteyEXQ9kSF2bHMYiC+WaKhb
kMRR9ZyJa086FwDvW9/TyMrP8HLR4ORa7ksOeBnGt4Uv9A92ZbGqvEqLILPoKsIR
LBJUM17XuLlR9J56sYNtid3Rdq88VLo84Wfc0hW966b+6oNdOVQBPI2YRpk4wDT1
Mtra60dtzwQ3jCW/GTjJThueEC4lqOFKHpYPIP3rd4aL8oCvfgLYlYZ4TiXHUjpX
MgQ9rm8nDZAWipsjvt2yADFnv3YW4kD6hrn1S0EaDJXsTtT7pB6Mr0jlxQE/ZJdA
ibXSubO5cakIdpMnAylL78ZyGZOWn6YOSG1pEjHjkI+in73sZ5a89k4s5ErxKxtB
Hnc/FXwVWbylUdJkKw==
=eMRy
-----END PGP SIGNATURE-----
Merge tag 'loongarch-kvm-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson into HEAD
LoongArch KVM changes for v7.3
1. Advertise already-supported capabilities.
2. Some bug fixes about timer and MMIO.
3. Some hardening about interrupt injection.
4. Replace kvm_err() with kvm_pr_unimpl().
5. Add FPU/LSX/LASX test cases for selftests.
KVM/riscv changes for 7.3
- Svadu/Zicfiss/Zicfilp FWFT support for Guest
- Use try_cmpxchg for IMSIC MRIF RMW
- More arch-specific tracepoints in KVM RISC-V
- Eager Page Splitting for KVM RISC-V
- Optimize hfence request handling for SMP Guests
- Improve dirty log clearing by skipping zero bits in mask
- Guard HFENCE range loops against overflow
- CPU PM notifiers in KVM RISC-V for non-retentive idle states
- Fix kernel-mode vector context save/restore for Guest
To reduce the runtime of the PMU counters test by an order of magnitude,
create one VM per category of test (arch events, GP PMCs, fixed PMCs),
with N vCPUs per VM, where 'N' is the number of testscases to run per
category. The vast majority of the test's runtime is spent creating VMs,
because while creating a VM only takes a few milliseconds, the total time
adds up when creating hundreds of VMs. The only reason the test creates
so many VMs is because KVM disallows changing the virtual PMU model after
a vCPU runs, so rather than create an entirely new VM just to get a "fresh"
vCPUs, simply use a different vCPU.
On an Emerald Rapids host, this reduces the runtime from 75+ seconds to
less than 6 seconds.
Opportunistically fix a typo in the comment about unavailable_masks pointed
out by Like Xu[1].
Link: https://lore.kernel.org/all/20260625090155.6326-1-likexu@tencent.com[1]
Link: https://patch.msgid.link/20260804210046.3413149-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add FPU/LSX/LASX test cases and verify FPU register get and set APIs,
the FPU width supports 64/128/256 bits.
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Add LSX and LASX features when VM is created. These features are detected
firstly, enable it if it is available on host machine.
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Tidy up the x86 vPMU emulation code and its selftests by fixing comments
that no longer reflect the code along with assorted spelling and grammar
mistakes, and by dropping redundant blank lines reported by checkpatch.
No functional change intended.
Signed-off-by: Like Xu <likexu@tencent.com>
Link: https://patch.msgid.link/20260625090155.6326-1-likexu@tencent.com
[sean: keep quotes around "hit", shove sequences fix into separate patch]
Signed-off-by: Sean Christopherson <seanjc@google.com>
To significantly reduce the PMU counter test's runtime, without sacrificing
test coverage in the aggregate, test a random GP counter in the arch events
testcase instead of testing every possible GP counter. Testing every PMC
in every run of the test significantly increases the runtime of the test,
without providing an equivalent increase in validation coverage, as the
odds of a KVM having a bug that only affected a subset of counters and only
when testing all other counters are extremely low.
Opportunistically clean up kvm_random_u64_in_range() to eliminate
unnecessary newlines.
Link: https://patch.msgid.link/20260804210046.3413149-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
KVM_CAP_VM_TYPES advertises the bitmap of VM types that KVM_CREATE_VM
accepts, but nothing verified that the ioctl actually enforces it: that
every advertised type can be created and every non-advertised type is
rejected. sev_init2_tests carried a TODO for this ("check that
unsupported types cannot be created. Probably a separate selftest"),
but the check is not specific to SEV or KVM_SEV_INIT2, and not even to
x86.
Add a standalone test that walks the type space and, for each value,
asserts that KVM_CREATE_VM succeeds iff the corresponding bit is set in
KVM_CAP_VM_TYPES, and otherwise fails with -EINVAL. The walk extends
past bit 31 so that out-of-range type values, which can never be
advertised in the u32 bitmap, are also confirmed to be rejected. The
test only depends on KVM_CAP_VM_TYPES, so it lives in the common set and
is skipped on architectures that don't advertise the capability.
Drop the now-addressed TODO from sev_init2_tests.c.
Tested on an AMD SEV-SNP capable host. With KVM_CAP_VM_TYPES=0x15
(DEFAULT/SEV/SNP), only the advertised types are created and everything
else is rejected:
$ strace -e trace=ioctl ./vm_types_test 2>&1 | grep KVM_CREATE_VM
ioctl(3, KVM_CREATE_VM, 0) = 4 # DEFAULT
ioctl(3, KVM_CREATE_VM, 0x1) = -1 EINVAL # SW_PROTECTED
ioctl(3, KVM_CREATE_VM, 0x2) = 4 # SEV
ioctl(3, KVM_CREATE_VM, 0x3) = -1 EINVAL # SEV-ES
ioctl(3, KVM_CREATE_VM, 0x4) = 4 # SNP
ioctl(3, KVM_CREATE_VM, 0x5) = -1 EINVAL # TDX
... 0x6..0x3f all -1 EINVAL ...
Reloading kvm_amd with sev_snp=0 drops the bitmap to 0x5 and only types
0 and 2 are then created, confirming the test tracks the advertised set
rather than hard-coded types.
Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com>
Link: https://patch.msgid.link/20260710050442.826777-1-hemanth.selam@gmail.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Extend the invalid nVMX guest state to cover RSM, i.e. to validate that KVM
synthesizes SHUTDOWN for L1 if SMRAM is clobbered with invalid guest state
during an L2 => SMI => RSM => L2 sequence.
Note, unlike the existing testcase, clobbering SMRAM should result in L1,
not L2, getting SHUTDOWN / TRIPLE_FAULT, as RSM is architecturally defined
to trigger shutdown if the CPU detects invalid state.
Signed-off-by: Hao Zhang <hao_zhang_kdev@163.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Link: https://patch.msgid.link/20260731173340.2644656-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
In the invalid nVMX guest state test, extract the creation of the VM and
initial running of the vCPU to get to L2 into helpers so that the common
code can be reused to extend the test to also cover RSM.
Eliminate the unnecessary global "vm", and opportunistically free the VM
after the testcase as there's zero reason not to.
Opportunistically assert that L2 is never resumed after the I/O exit to L1,
e.g. to guard against false passes.
Link: https://patch.msgid.link/20260731173340.2644656-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Use port 0x80 instead of 0x2000 to trigger an L2 => L0 VM-Exit in the nVMX
invalid guest state test, so that the test can be expanded to cover RSM in
the L2 => SMI => L2 path without having to manually encode a MOV to DX (the
immediate form of IN only supports an imm8).
Use port 0x80 as it's a well-known port and is used by other tests, e.g. in
the coalesced I/O test.
Link: https://patch.msgid.link/20260731173340.2644656-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add and use KVM wrappers for pthread_{g,s}etaffinity_np() so that selftests
don't need to manually assert that the syscalls succeeded, and so that they
don't need to manually pass in sizeof(cpu_set_t) for the size.
Link: https://patch.msgid.link/20260731195612.2697986-12-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Rename the global constants in the hardware disable test to better
capture what they control, and open code the literal number of sleeping
tasks and iterations instead of using power-of-2 math to express values
that are 100% arbitrary.
Opportunistically tag the global semaphore with "static".
No functional change intended.
Link: https://patch.msgid.link/20260731195612.2697986-13-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Deduplicate test code that does cancel()+join() on pthreads by adding a
collection of syscall wrappers. The tests that asynchronously cancel
pthreads, i.e. use PTHREAD_CANCEL_ASYNCHRONOUS, are particularly ugly,
especially since it's not immediately obvious why those tests assert on
the thread being CANCELED, whereas others do not.
No functional change intended.
Link: https://patch.msgid.link/20260731195612.2697986-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add and use KVM wrappers for pthread_{cancel,join}() so that selftests
don't need to manually assert that the syscalls succeeded.
Note, the vast majority tests don't actually assert success, but they all
obviously rely on the syscall to succeed.
Other than explicitly failing if a syscall fails, no functional change
intended.
Link: https://patch.msgid.link/20260731195612.2697986-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add and use a KVM wrapper for pthread_create() syscall so that selftests
don't need to manually assert that the syscall succeeded.
Note, most tests don't actually assert success, but they all obviously
rely on the syscall to succeed.
Other than explicitly failing if pthread_create() fails, no functional
change intended.
Link: https://patch.msgid.link/20260731195612.2697986-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Drop all of the code that is unreachable in the hardware disable test, as
it adds a lot of noise that makes the test seem far more complicated than
it actually is.
Link: https://patch.msgid.link/20260731195612.2697986-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Affine the worker threads to random CPUs in the hardware disable test, and
honor the starting CPU set in the process. Hardcoding to CPUs 0-3 is both
wrong and less interesting than running on random CPUs.
Opportunistically convert the local 'i' to an int, as the macros it tests
against are signed, and using 'i' as an iterator is conventionally declared
as an int.
Link: https://patch.msgid.link/20260731195612.2697986-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Extract kvm_pick_random_cpu() out of pin_task_to_random_cpu() so that tests
can choose a random CPU without having to immediately pin a task to that
CPU.
No functional change intended.
Link: https://patch.msgid.link/20260731195612.2697986-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When pinning a task to a random CPU, return which CPU the task was pinned
to so that the caller can do things like avoid running other tasks on the
target CPU.
Link: https://patch.msgid.link/20260731195612.2697986-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
When building against a libc that provides pthread_attr_setaffinity_np(),
pre-set the grandchildren threads CPU affinity in the hardware disable test
to further reduce the perceived latency of pthread_create(). On large NUMA
systems, this reduces the average runtime from ~10s to ~5s.
Link: https://patch.msgid.link/20260731195612.2697986-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Affine worker threads to their target pCPU set *before* doing any work in
the hardware disable test. When running a preemptible kernel on a NUMA
system with a large number of CPUs (100+ of CPUs), waiting to set the
affinity of the grandchildren until after the grandchild is fully spun up
results in total test runtimes of over a minute, presumably due to
cross-node NUMA penalties and scheduler lock contention.
E.g. from the perspective of the caller, when waiting to affine the tasks,
pthread_create() takes ~100x longer.
Link: https://patch.msgid.link/20260731195612.2697986-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add and use a KVM wrapper for sched_setaffinity() so that selftests don't
need to manually assert that the syscall succeeded.
No functional change intended.
Link: https://patch.msgid.link/20260731195612.2697986-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Link to the VFIO library only on architectures where it's actually needed,
i.e. only on architectures that support irq_test, which is currently
x86-only. The irq_test series, which pulled in VFIO support, effectively
raced with commit e65f1bf8a2 ("vfio: selftests: Extend container/iommufd
setup for passing vf_token"). I.e. when the KVM test was posted, pulling
in libvfio.mk didn't require linking to any new libraries other than VFIO
itself.
Now that libvfio.mk pulls in luuid, unconditionally linking in support for
VFIO is quite onerous, especially when cross-compiling, as it requires
installing what is effectively an unused library on four architectures.
Eventually KVM selftests will likely support irq_test and thus need to link
to VFIO on architectures other than x86, but at least then there will be
some amount of gain for the pain.
Fixes: a262fc49e0 ("KVM: selftests: Build and link selftests/vfio/lib into KVM selftests")
Cc: David Matlack <dmatlack@google.com>
Cc: Josh Hilke <jrhilke@google.com>
Link: https://patch.msgid.link/20260730165159.1988017-1-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
Since we now expose a new system register on NV3-capable HW when
the guest is exposed to it, add it to the existing checker. FWIW.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-29-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
Add a test for the two stage-2 granularity changes dirty logging forces
at fault time, asserting the guest completes with no KVM_RUN error. The
first scenario collapses a page into a hugetlb-backed block: it writes
under logging, re-write-protects the page via GET_DIRTY_LOG, then writes
again with logging off. The second splits blocks: it faults in several
non-executable 2M blocks, enables logging, then executes in each block so
an execute permission fault splits it. It is skipped when CTR_EL0.DIC is
set, since mappings are then executable and no execute fault occurs.
Both paths make the fault handler allocate under mmu_lock, so a backend
that fails to stage that memory returns a KVM_RUN error or crashes the
host. The property holds on any host. On a pKVM host, where a
non-protected guest uses the pkvm_pgtable_*() backend, it also guards
that backend's fault-time staging.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-8-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
debug-exceptions covers guest self-hosted debug and userspace
single-step, but not a userspace (KVM_GUESTDBG_USE_HW) watchpoint,
whose KVM_EXIT_DEBUG reports the accessed address in debug.arch.far.
Add a test that installs a host-directed write watchpoint and checks
that the reported address matches the accessed variable.
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260726143643.303456-3-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
Introduce a new field pmmir_slots in struct kvm_arch to store
PMMIR_EL1.SLOTS. It only saves the actual hardware PMU value when
the VMM explicitly selects a PMU under KVM_ARM_VCPU_PMU_V3_STRICT.
Otherwise, it stays 0 after allocation.
Use this field to implement guest access, userspace get, and userspace
set for PMMIR_EL1:
- access_pmmir(): uses the value in kvm->arch.pmmir_slots directly. If
the VMM selected a PMU and KVM_ARM_VCPU_PMU_V3_STRICT is set, the guest
can correctly read the underlying core's SLOTS. Otherwise, it continues
to read 0 since the true SLOTS value can be nondeterministic.
- get_pmmir(): same as access_pmmir().
- set_pmmir(): only the SLOTS field is writable; a value setting any
other bit is rejected with -EINVAL, since get_pmmir() returns SLOTS
zero-extended. A value of 0 resets kvm->arch.pmmir_slots to 0 for
backward compatibility, as the register is RAZ in older KVM, a value
matching the current SLOTS is accepted as a no-op, and anything else is
rejected with -EINVAL. Once the VM has run PMMIR_EL1 is immutable, so a
mismatching write then returns -EBUSY.
The register is now exposed via KVM_GET_REG_LIST for PMUv3 vCPUs, so add
it to the get-reg-list selftest's PMU register list.
Signed-off-by: Congkai Tan <congkai@amazon.com>
Reviewed-by: Geoff Blake <blakgeof@amazon.com>
Reviewed-by: Haris Okanovic <harisokn@amazon.com>
Reviewed-by: Stanislav Spassov <stanspas@amazon.de>
Co-developed-by: Oliver Upton <oupton@kernel.org>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260722202702.4165917-2-congkai@amazon.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
Extend the testing coverage in L2 by forcing a nested VM-Exit from L2 to
L1 right after restore on every other iteration. Forcing a nested
VM-Exit while L0 has control (e.g. without explicitly running L2 and
making a hypercall) is valuable, as it often happens during live
migration (e.g. L1 timer interrupt fires by the time the VM lands on the
destination).
To force the nested VM-Exit inject a #UD in to the saved vCPU state, and
intercept #UD from L1.
With this change, the test reliably reproduces the CR2 bug fixed by
commit 5c247d08bc ("KVM: nSVM: Use vcpu->arch.cr2 when updating vmcb12
on nested #VMEXIT") -- at least on Milan, Genoa, and Turin CPUs.
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-14-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Extend the stress test to allow running the access+#PF code in L2
instead of L1 by adding proper L1 guest code to bootstrap L2. By
default, the test runs in L2 after running in L1 if nested is supported.
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-13-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Instead of an explicit GUEST_SYNC() after each access+#PF, run another
thread that keeps sending SIGUSR to the vCPU thread, essentially
triggering exits to userspace and save+restore on random points in guest
execution. This makes the test a lot more meaningful as it opens the
door to exercising race conditions between #PF handling in the guest
and save+restore in the host.
The signals are ignored using SIG_IGN outside of __vcpu_run() to avoid
interrupting other ioctls/sysctls performed by the test.
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-12-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a basic stress test for handling #PFs in a guest while the host is
doing save+restore cycles. The guest periodically accesses non-present
memory causing a #PF, and the #PF handler walks the page tables and
updates the PTE to be present, like a proper #PF handler.
After every access (and #PF), the guest triggers a sync and the test
performs save+restore of the VM. This is not very meaningful as
save+restore are performed after the access and #PF handling complete,
but following changes will change that.
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-11-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
init_vmcs_control_fields() sets PFEC_MASK and PFEC_MATCH so that they
never match, which reverses the meaning of the PF_VECTOR bit in
EXCEPTION_BITMAP (which is zeroed), effectively enabling #PF
interception by default.
The relevant part of the SDM describes this:
When a page fault occurs, a processor consults (1) bit 14 of the
exception bitmap; (2) the error code produced with the page fault [PFEC];
(3) the page-fault error-code mask field [PFEC_MASK]; and (4) the
page-fault error-code match field [PFEC_MATCH].
It checks if PFEC & PFEC_MASK = PFEC_MATCH. If there is equality, the
specification of bit 14 in the exception bitmap is followed (for example,
a VM exit occurs if that bit is set). If there is inequality, the meaning
of that bit is reversed (for example, a VM exit occurs if that bit is
clear).
Clear PFEC_MATCH such that there is equality, and the #PF bit in the
exception bitmap is followed, so that #PFs are not intercepted by default,
same as every other exception. This also allows tests to set the #PF bit
in the exception bitmap if they want to intercept #PFs, without having to
muck with the PFEC_{MASK,MATCH} fields.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-10-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Expose a guest_mmu to the guest to allow guest code to use the PTE masks
for page table manipulation. Since guest page tables are not mapped in
the guest by default, zero the PGD in guest_mmu in an attempt to make it
more difficult for new tests to shoot themselves in the foot and assume
that page tables can be immediately used by guest code.
Ultimately, guest code can read CR3 any way, so guest_mmu.pgd doesn't
add a lot of value.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-9-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Add a blank line for visual separation, which is especially useful for
tests that use carriage returns for same-line printing, in case a
failure occurs before a blank line is ever logged by the test.
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-8-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Now that nVMX test codes preserves GPRs across nested VM-Exits
(specifically RBP, RDX, and RDI among others), drop the ucall-specific
hack to avoid clobbering these registers.
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-7-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Reuse the GPR switching logic for nVMX by defining VMX_SWITCH_GPRS_ASM,
which is essentially the same as SVM_SWITCH_GPRS_ASM but also switches
RAX and doesn't switch RFLAGS, replacing the push/pop of a subset of the
registers.
The long clobber list of registers is no longer needed as registers are
saved and restored appropriately (and not clobbered by L2).
Define VMX_SWITCH_GPRS_ASM before including evmcs.h, such that it can be
used by evmcs_vmlaunch() and evmcs_vmresume().
This replaces the apparently thread-safe push/pop sequence with the
global GPR switching logic used by SVM, which isn't thread-safe at all.
However this is still an improvement because:
- The VMX logic is half-baked and prompts the UCALL clobber hack as it
doesn't properly save/restore everything. Reusing the GPR switching
logic used by SVM allows for dropping that hack.
- Hitting a problem due to half-baked GPR save/restore logic is arguably
more likely than thread-safety. Evidently, adding more involved stress
tests fails on VMX with the existing push/pop sequence. OTOH, there
are no known failures on SVM due to lack of thread-safety fo
save/restore. Only one test currently uses more than one vCPU with
nested (the memstress test).
The logical next step is to move the guest_regs to be per-vCPU,
making it thread-safe for both VMX and SVM in a proper way.
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-6-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Instead of handling rflags separately, add it to guest_regs. No
functional change intended.
Assisted-by: Gemini:Gemini-Next
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-5-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
The assembly code defined by SAVE_GPR_C uses the wrong offsets for some
registers in guest_regs. For example, the offset of RCX should be 0x08
not 0x10. Also, the last offset in the struct (R15) is 0x78, not 0x80,
so the code actually saves and restore beyond the end of gpr64_regs.
Eliminate hardcoded offsets by dynamically generating offsets using
offset_of() and using macros to pass the offsets to assembly as asm
constraints.
To avoid register conflicts in inline assembly (since almost all GPRs are
context-switched), access guest_regs via absolute symbol addressing
(guest_regs + offset) rather than using a base register which could
get overwritten mid-assembly.
While at it, rename SAVE_GPR_C and LOAD_GPR_C to a single macro,
SVM_SWITCH_GPRS_ASM, rename gpr64_regs to guest_regs, and expose it in
processor.h (in preparation for reusing it for VMX).
Assisted-by: Gemini:Gemini-Next
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-4-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
The offsets used (0x170 and 0x1f8) are offsets within vmcb_save_area,
not vmcb. The correct offsets should include the base of vmcb_save_area
within vmcb (which is 0x400 -- so 0x570 and 0x5f8).
Instead of just correcting the offsets, use vmcb->save.rax and
vmcb->save.rflags as parameters to the asm block and avoid hardcoding
offsets completely. While at it, also use guest_regs.rax directly
instead of assuming it's at offset 0 of guest_regs.
Note: "+m" must be used for vmcb_rax and vmcb_rflags, as caching those
fields in registers would be wrong as the underlying KVM will update
them in memory.
The same problem was recently fixed (differently) for kvm-unit-tests
[1].
[1]https://lore.kernel.org/all/20260521092311.86030-1-pbonzini@redhat.com/
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260518202514.2037078-1-yosry%40kernel.org?part=1
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-3-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
Drop the custom defined XSTR() macros in KVM selftests and use
__stringify() instead. Include stringify.h in test_util.h to make it
available for all tests instead of including it in all the tests that
need it, as more tests will start using it.
No functional change intended.
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Link: https://patch.msgid.link/20260728174232.2423257-2-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
