* Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
* Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
* Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1
is present, guaranteeing that EL2-specific register bits are stateful
in the EL1 counterpart
* Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope
of those instructions remains 'in host' (i.e. L1 kernel/userspace)
* Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
* Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
* Considerable pile of LLM-assisted fixes around the shop but mostly in
the VGIC, our in-kernel generator of bugs (and sometimes interrupts)
LoongArch:
* Advertise already-supported capabilities.
* Some bug fixes about timer and MMIO.
* Some hardening about interrupt injection.
* Replace kvm_err() with kvm_pr_unimpl().
* Add FPU/LSX/LASX test cases for selftests.
RISC-V:
* Svadu/Zicfiss/Zicfilp FWFT support for Guest
* Use try_cmpxchg for IMSIC MRIF RMW
* More arch-specific tracepoints in KVM RISC-V
* Eager page splitting when enabling dirty logging
* Optimize hfence request handling for SMP Guests
* Improve dirty log clearing by skipping zero bits in mask
* Guard HFENCE range loops against overflow
* CPU PM notifiers in KVM RISC-V for non-retentive idle states
* Fix kernel-mode vector context save/restore for Guest
s390:
* Fixes for vfio-ap
* Fixes for the gmap rework
* Fixes for vsie
* AI triggered fixes all over
* diag9c tracing
* code move preparation for the additional arm64 support
* enable CONTEXT_ANALYSIS
x86:
* Perform spring cleaning on x86.{c,h} and asm/kvm_host.h, by adding regs.c
(the kvm_cache_regs.h => regs.h is already applied) and msrs.{c,h}, and moving
relevant code out of x86.c.
* Split kvm_mmu in three parts, respectively to describe the format of page
tables, walking the guest page tables and building the page tables. Always
use the same page table walker kvm->arch.gva_walk as the entry point to
convert a guest's virtual address, where the previous code used two
different kvm_mmu structs depending on whether the walk included nested
EPT/NPT or not. Make page fault vmexits reuse the permission checking
machinery that is used for guest page faults. This is both a cleanup
and a baby step towards supporting XS/XU memory permissions.
* Document some of the "fun" gotchas with the APIC base when creating IRQCHIPs
on x86.
* Remove a defunct masterclock update from kvm_xen_shared_info_init(). It
could result in incorrect kvmclock due to triggering an unnecessary
switch to/from masterclock mode.
* Skip Xen runstate time updates if time has effectively gone backwards, so
that the guest doesn't report 100% steal time for a very, very long time.
* Drop KVM's runtime updates of the Xen PV timing CPUID leaf, as KVM was
updating the wrong sub-leaf, and upstream KVM will soon provide all the
information needed by userspace to populate the CPUID field itself.
* Fix a bug where KVM would walk a newly created rmap without holding the rmap
lock (or mmu_lock) during aging.
* Fix a bug where aging TDP MMU SPTEs could clobber FROZEN SPTEs.
* Fix a variety of #DB priority bugs.
* Fix a class of races related to enabling Hyper-V emulation on a vCPU after
the vCPU is visible to the rest of KVM.
* Use static calls for nested virtualization ops.
* Move more KVM-internal code out of x86's kvm_host.h.
* Enumerate support for a variety of Zhaoxin instructions that don't require
explicit virtualization.
* Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path.
* Harden kvm_vcpu_map() against double-mapping and thus leaking references.
* Misc fixes and cleanups, e.g. for largely benign syzkaller splats.
x86 (Intel):
* Zero a vCPU's entry in VMX's Posted Interrupt Descriptor table used for IPI
virtualization when the vCPU is freed, to fix a use-after-free where hardware
will write to a freed vCPU's PID.
* Service local TLB flushes on a failed nested VM-Enter to fix a bug where KVM
could miss a TLB on a future, successful VM-Enter with the same L2 VPID.
* Cap the maximum value shoved into the VMX Preemption Timer to workaround an
erratum that affects all existing Intel CPUs that support CPUID 0x15.
* Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs.
* Harden the TDX "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion.
x86 (AMD):
* Forcefully invalidate SNP VMSA pages if their backing guest_memfd page is
zapped/invalidated, e.g. due to a PUNCH_HOLE in response to a Page-State
Change request.
* Remove a dying VM from the GA Log notifier list before the VM is actually
destroyed, to fix a potential use-after-free.
* While FOLL_WRITE was needed in the past to trigger CoW unsharing, nowadays
FOLL_LONGTERM does that already even without FOLL_WRITE, and in fact,
get_user_pages() actually disallows FOLL_WRITE together with FOLL_LONGTERM.
So don't pass FOLL_WRITE when registering encrypted memory regions, i.e. when
pinning SEV/SEV-ES guest memory, to fix a regression with file-backed memory
introduced by KVM's (correct) usage of long-term pins.
(This was reviewed by mm maintainers; for more information, see commit
ee1a586dd1).
* Allocate full pages for SEV/SEV-ES {DE,EN}CRYPT ops on SNP-enabled hosts to
fix a data corruption issue due to the PSP driver assigning to-be-written
pages to firmware (as required by the SNP specs).
* Unconditionally intercept ICBEP so that KVM generates the correct guest RIP
when handling an ICEBP-induced TASK_SWITCH #VMEXIT.
* Harden the SNP "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion.
Generic:
* Remove kvm_debugfs_dir if kvm_init() fails after creating KVM's debugfs.
* Add a per-VM bitmap to track which vCPU IDs have been "claimed" but for
which the vCPU isn't yet online, and use the bitmap to reject duplicate IDs
before calling into arch code. This allows arch code to consume vcpu_id
without having to worry about cross-vCPU clobbering (at least s390 and x86
have had related bugs).
* Rework the so called "prepare" and "invalidate" guest_memfd hooks to prepare
for in-place private<=>shared conversion, and clean up a few warts along the
way.
Selftests:
* Automatically allocate a full page for L2 guest stacks on x86 instead of
requiring test-specific L1 guest code to carve out a portion of the L1
stack for L2 usage, and to ensure the L2 stack also adheres to the x86-64
calling convention ABI.
* Add a selftest to verify {Guest,Host}-Only behavior in x86's mediated PMU.
* Clean up nested SVM's handling of GPRs on L2<=>L1 transitions, reuse the
functionality for nested VMX, and drop the ucall hack that was fudging
around the lack of GPR switching on nVMX.
* Add a stress test to verify KVM doesn't clobber/drop #PF state, e.g. CR2,
across save/restore, including when L2 is active.
* Add a test to verify KVM_CREATE_VM accepts exactly what is reported by
KVM_CAP_VM_TYPES.
* Misc selftests fixes and cleanups
* Fix several issues with seeding the pRNG, and rework the pRNG APIs to that
the pRNG can be sanely used in host code, not just guest code.
* Add an IRQ test to validate virtual IRQ deliverty for IRQs wired up via
KVM_IRQFD + KVM_SET_GSI_ROUTING, with optional support for triggering IRQs
via writes to an assigned VFIO device.
* Add syscall wrappers to assert success on a variety of pthreads and CPU
affinity APIs.
* Set vCPU pthread affinity as early as possible to reduce contention issues
that were surfaced by PREEMPT_LAZY, which result in runtimes of over a
minute on large hosts, versus the expected ~5 seconds.
* Rework the PMU counters test to run each testcase using a single VM with
many vCPUs for each sub-testcase, instead of using a unique VM for each
sub-testcase. This cuts the runtime by ~20x.
Miscellaneous:
* MAINTAINERS updates for vfio-ap, guest_memfd, kvm-x86. Mostly representing
the status quo more accurately, but also... welcome David Hildenbrand
as guest_memfd reviewer!
-----BEGIN PGP SIGNATURE-----
iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmqMgosUHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroP8bwf+ORImBMDM3QEmybZM3I+N2+xqSuHP
QHttbmqGbsFK/RUeH96/X/+P9waqaz3uVeUQ6Qp2r0ryqwKtLt8YvIxKDp+M0vVJ
n+iukk1xulBEc28aGdKHn9G4wayAwDA/9f7CvJ23hojaJfLScbF3OlFkDd7y5DpO
x15Rtg9folYUjjop3LDML4N9/9Qmk4KRvVZ4ZVv6IB4uGJJ72fLd5dbBMyDk+BLl
Lz9N1xVTXcnJXJmrjMB4/QNt/HiQJdun8LcokJZyykta7Xx6aY7OGZv+VmCeq47K
e9Hk8mD4AyAdVbVvIntROSeBJOrlsgWJXAPHea6wEdRQnS3XVEBuZC0yDA==
=rfLh
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm updates from Paolo Bonzini:
"ARM64:
- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when
FEAT_NV2p1 is present, guaranteeing that EL2-specific register bits
are stateful in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the
scope of those instructions remains 'in host' (i.e. L1
kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly
in the VGIC, our in-kernel generator of bugs (and sometimes
interrupts)
LoongArch:
- Advertise already-supported capabilities
- Some bug fixes about timer and MMIO
- Some hardening about interrupt injection
- Replace kvm_err() with kvm_pr_unimpl()
- Add FPU/LSX/LASX test cases for selftests
RISC-V:
- Svadu/Zicfiss/Zicfilp FWFT support for Guest
- Use try_cmpxchg for IMSIC MRIF RMW
- More arch-specific tracepoints in KVM RISC-V
- Eager page splitting when enabling dirty logging
- Optimize hfence request handling for SMP Guests
- Improve dirty log clearing by skipping zero bits in mask
- Guard HFENCE range loops against overflow
- CPU PM notifiers in KVM RISC-V for non-retentive idle states
- Fix kernel-mode vector context save/restore for Guest
s390:
- Fixes for vfio-ap
- Fixes for the gmap rework
- Fixes for vsie
- AI triggered fixes all over
- diag9c tracing
- code move preparation for the additional arm64 support
- enable CONTEXT_ANALYSIS
x86:
- Perform spring cleaning on x86.{c,h} and asm/kvm_host.h, by adding
regs.c (the kvm_cache_regs.h => regs.h is already applied) and
msrs.{c,h}, and moving relevant code out of x86.c
- Split kvm_mmu in three parts, respectively to describe the format
of page tables, walking the guest page tables and building the page
tables. Always use the same page table walker kvm->arch.gva_walk as
the entry point to convert a guest's virtual address, where the
previous code used two different kvm_mmu structs depending on
whether the walk included nested EPT/NPT or not. Make page fault
vmexits reuse the permission checking machinery that is used for
guest page faults. This is both a cleanup and a baby step towards
supporting XS/XU memory permissions
- Document some of the "fun" gotchas with the APIC base when creating
IRQCHIPs on x86
- Remove a defunct masterclock update from kvm_xen_shared_info_init().
It could result in incorrect kvmclock due to triggering an
unnecessary switch to/from masterclock mode
- Skip Xen runstate time updates if time has effectively gone
backwards, so that the guest doesn't report 100% steal time for
a very, very long time
- Drop KVM's runtime updates of the Xen PV timing CPUID leaf, as KVM
was updating the wrong sub-leaf, and upstream KVM will soon provide
all the information needed by userspace to populate the CPUID field
itself
- Fix a bug where KVM would walk a newly created rmap without holding
the rmap lock (or mmu_lock) during aging
- Fix a bug where aging TDP MMU SPTEs could clobber FROZEN SPTEs
- Fix a variety of #DB priority bugs
- Fix a class of races related to enabling Hyper-V emulation on a
vCPU after the vCPU is visible to the rest of KVM
- Use static calls for nested virtualization ops
- Move more KVM-internal code out of x86's kvm_host.h
- Enumerate support for a variety of Zhaoxin instructions that don't
require explicit virtualization
- Fix missing EFER validation bugs, including in the KVM_SET_SREGS*
path
- Harden kvm_vcpu_map() against double-mapping and thus leaking
references
- Misc fixes and cleanups, e.g. for largely benign syzkaller splats
x86 (Intel):
- Zero a vCPU's entry in VMX's Posted Interrupt Descriptor table used
for IPI virtualization when the vCPU is freed, to fix a
use-after-free where hardware will write to a freed vCPU's PID
- Service local TLB flushes on a failed nested VM-Enter to fix a bug
where KVM could miss a TLB on a future, successful VM-Enter with
the same L2 VPID
- Cap the maximum value shoved into the VMX Preemption Timer to
workaround an erratum that affects all existing Intel CPUs that
support CPUID 0x15
- Fix VPID virtualization bugs where KVM would fail to flush hardware
TLBs
- Harden the TDX "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion
x86 (AMD):
- Forcefully invalidate SNP VMSA pages if their backing guest_memfd
page is zapped/invalidated, e.g. due to a PUNCH_HOLE in response to
a Page-State Change request
- Remove a dying VM from the GA Log notifier list before the VM is
actually destroyed, to fix a potential use-after-free
- While FOLL_WRITE was needed in the past to trigger CoW unsharing,
nowadays FOLL_LONGTERM does that already even without FOLL_WRITE,
and in fact, get_user_pages() actually disallows FOLL_WRITE
together with FOLL_LONGTERM. So don't pass FOLL_WRITE when
registering encrypted memory regions, i.e. when pinning SEV/SEV-ES
guest memory, to fix a regression with file-backed memory
introduced by KVM's (correct) usage of long-term pins
(This was reviewed by mm maintainers; for more information, see
commit ee1a586dd1 "KVM: SEV: Drop FOLL_WRITE for encrypted region
registration")
- Allocate full pages for SEV/SEV-ES {DE,EN}CRYPT ops on SNP-enabled
hosts to fix a data corruption issue due to the PSP driver
assigning to-be-written pages to firmware (as required by the SNP
specs)
- Unconditionally intercept ICBEP so that KVM generates the correct
guest RIP when handling an ICEBP-induced TASK_SWITCH #VMEXIT
- Harden the SNP "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion
Generic:
- Remove kvm_debugfs_dir if kvm_init() fails after creating KVM's
debugfs
- Add a per-VM bitmap to track which vCPU IDs have been "claimed" but
for which the vCPU isn't yet online, and use the bitmap to reject
duplicate IDs before calling into arch code. This allows arch code
to consume vcpu_id without having to worry about cross-vCPU
clobbering (at least s390 and x86 have had related bugs)
- Rework the so called "prepare" and "invalidate" guest_memfd hooks
to prepare for in-place private<=>shared conversion, and clean up a
few warts along the way
Selftests:
- Automatically allocate a full page for L2 guest stacks on x86
instead of requiring test-specific L1 guest code to carve out a
portion of the L1 stack for L2 usage, and to ensure the L2 stack
also adheres to the x86-64 calling convention ABI
- Add a selftest to verify {Guest,Host}-Only behavior in x86's
mediated PMU
- Clean up nested SVM's handling of GPRs on L2<=>L1 transitions,
reuse the functionality for nested VMX, and drop the ucall hack
that was fudging around the lack of GPR switching on nVMX
- Add a stress test to verify KVM doesn't clobber/drop #PF state,
e.g. CR2, across save/restore, including when L2 is active
- Add a test to verify KVM_CREATE_VM accepts exactly what is reported
by KVM_CAP_VM_TYPES
- Misc selftests fixes and cleanups
- Fix several issues with seeding the pRNG, and rework the pRNG APIs
to that the pRNG can be sanely used in host code, not just guest
code
- Add an IRQ test to validate virtual IRQ deliverty for IRQs wired up
via KVM_IRQFD + KVM_SET_GSI_ROUTING, with optional support for
triggering IRQs via writes to an assigned VFIO device
- Add syscall wrappers to assert success on a variety of pthreads and
CPU affinity APIs
- Set vCPU pthread affinity as early as possible to reduce contention
issues that were surfaced by PREEMPT_LAZY, which result in runtimes
of over a minute on large hosts, versus the expected ~5 seconds
- Rework the PMU counters test to run each testcase using a single VM
with many vCPUs for each sub-testcase, instead of using a unique VM
for each sub-testcase. This cuts the runtime by ~20x
Miscellaneous:
- MAINTAINERS updates for vfio-ap, guest_memfd, kvm-x86. Mostly
representing the status quo more accurately, but also... welcome
David Hildenbrand as guest_memfd reviewer!"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (413 commits)
KVM: arm64: Validate GICv5 timer PPIs before claiming ownership
KVM: arm64: vgic: Reject out-of-range GICv5 PPI IDs
KVM: arm64: vgic: Prevent speculative SPI array underflow
KVM: arm64: vgic: Free gic_kvm_info on initialization failure
KVM: arm64: Avoid mismatched accesses to 'struct kvm_nvhe_init_params'
s390/vfio-ap: Fix NULL deref in status_show() during queue probe
s390/vfio-ap: Fix hot-unplug skipped when last AP adapter or domain removed
s390/vfio-ap: fix potential use of uninitialized apm_filtered bitmap
s390/vfio-ap: Fix control domain removal in vfio_ap_mdev_cfg_remove
s390/vfio-ap: Fix required lock not held during update of ap_matrix_mdev object
s390/vfio-ap: Fix missing lock required to access list of ap_matrix_mdev objects
s390/vfio-ap: Fix dereference matrix_mdev->kvm without checking for NULL
s390/vfio-ap: Fix stale do_remove flag across iterations in vfio_ap_mdev_cfg_remove
RISC-V: KVM: fix vcpu vector context handling for kernel-mode vector
riscv: vector: allow non-preemptible kernel-mode vector with IRQs off
riscv: vector: refactor riscv_v_start_kernel_context
KVM: s390: gmap: Make prefix handling optional
KVM: s390: gmap: Make CMMA optional
KVM: s390: gmap: Make storage keys optional
KVM: s390: Prepare gmap for a second KVM implementation
...
7bb6284aa7