mirror of
https://github.com/torvalds/linux.git
synced 2026-10-16 12:50:02 +02:00
master
405 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Fuad Tabba
|
04447b95c6 |
KVM: arm64: Use the host's HCR_EL2 for non-protected VMs in pKVM
For each vCPU, pKVM's EL2 builds its own HCR_EL2 and takes only TWI, TWE
and VSE from the host's value on each entry. For a non-protected VM it
has fallen behind the host's: on a CPU with MTE the guest can read
GMID_EL1, on one with FEAT_EVT2 but no FGT it can execute a TLBI OS its
ID registers hide, an interrupt injected without a vGIC (VI, VF) never
arrives, and set/way emulation loses the TVM trap it relies on.
For a non-protected VM, also take the bits the host varies with the VM's
configuration or at runtime: VI, VF, TVM, TID2, TID4, TID5 and TTLBOS.
They only pend interrupts for, or add traps to, a VM the host controls.
The traps exit to the host, which handles them as it does without pKVM.
The bits that change what EL2 does on entry and exit (E2H, RW, API/APK)
or depend only on the CPU (TEA, TERR, FWB) stay EL2's. A protected VM
still takes only TWI, TWE and VSE.
EL2 now sets TID2 or TID4 only for a protected VM, and never sets ATA,
as the host rejects KVM_CAP_ARM_MTE in pKVM.
Fixes:
|
||
|
Fuad Tabba
|
cfe80c3837 |
KVM: arm64: Check every private mapping is hyp-owned at pKVM init
fix_host_ownership() transfers only what it walks, so a hyp mapping outside the linear map is not manipulated by the walk. Walk the quarter of the VA space holding the private range and the vmemmap once the transfer is done, and fail init unless every valid leaf is hyp-owned: in the vmemmap when the page is memory, and in the host stage-2, where hyp text may instead be mapped without write access. A leaf that is not memory has no vmemmap entry and is checked against the host stage-2 alone. Hyp text is matched by physical address, since the only executable mapping in the range is the Spectre-v3a vectors, whose VA is a private allocation, and an executable mapping of anything else must not be host-readable. The vmemmap can be block-mapped, so the walker checks each page of a leaf. Suggested-by: Will Deacon <will@kernel.org> Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev> Reviewed-by: Vincent Donnefort <vdonnefort@google.com> Tested-by: Vincent Donnefort <vdonnefort@google.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Link: https://patch.msgid.link/20260908110713.1540304-5-fuad.tabba@linux.dev Signed-off-by: Oliver Upton <oupton@kernel.org> |
||
|
Fuad Tabba
|
5a8b505ede |
KVM: arm64: Match hyp text by physical address in fix_host_ownership()
On a non-hVHE host, fix_host_ownership_walker()'s test for PAGE_HYP_EXEC
never matches: KVM_PGTABLE_PROT_UX is cleared at map time and only PX is
reported on read-back. Hyp text is therefore donated rather than left
read-only in the host stage-2, and the instruction dump in
nvhe_hyp_panic_handler() reads a page the host has no access to.
Match the text by physical address instead, in a helper a later patch
reuses. A test on the permissions would leave any other executable
mapping host-readable too.
Fixes:
|
||
|
Oliver Upton
|
5f44f603b0 |
Merge branch 'kvm-arm64/feat-nv3' into next
* kvm-arm64/feat-nv3: (29 commits)
: Support for FEAT_NV2p1 and FEAT_NV3, courtesy of Marc Zyngier
:
: Couple of accelerations for nested virtualization:
:
: - Elide traps for EL1 alias registers when hardware guarantees that
: EL2-only bits are stateful (NV2p1)
:
: - Use TGE-aware hardware to avoid unnecesary ERET/TLBI traps that
: remain 'in-context' (i.e. L1 kernel/userspace)
arm64: Add override for ID_AA64MMFR4_EL1.NV_frac
KVM: arm64: selftest: Add NVHCR_EL2 to get-reg-list
KVM: arm64: Expose FEAT_NV3 to guests
KVM: arm64: Add FEAT_NV3 detection
KVM: arm64: Engage NV3 TLBI trap elision
KVM: arm64: Engage NV3 ERET trap elision
KVM: arm64: Add NVHCR_EL2 context switching
KVM: arm64: Add routing for NVHCR_EL2 trap
KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
KVM: arm64: Add sanitisation for NVHCR_EL2
KVM: arm64: Make HCR_EL2 a non-VNCR register
KVM: arm64: Add kvm_has_nv{2,3}() predicates
KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
KVM: arm64: Split NV-specific exit fixups from the non-NV handling
arm64: Add ARM64_HAS_NV3 capability
arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
arm64: Add FEAT_NV2p1 detection
KVM: arm64: Expose FEAT_NV2p1 to NV guests
KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
||
|
Oliver Upton
|
5279b2f422 |
Merge branch 'kvm-arm64/pkvm-7.3' into next
* kvm-arm64/pkvm-7.3: (26 commits)
: pKVM updates for 7.3
:
: - Avoid name collision on trace_clock() when CONFIG_NVHE_EL2_TRACING is
: disabled (Mostafa Saleh)
:
: - Clean up state tracking for whether the EL2 shadow VM has been
: created (Fuad Tabba)
:
: - Synchronize SCTLR_EL1 when injecting an exception to use current
: PAN/SSBS state (Fuad Tabba)
:
: - Avoid unnecessary cache maintenance when I/D-cache are known to be
: coherent in pKVM (Mostafa Saleh)
:
: - Lazy vCPU context save/restore for pKVM (Fuad Tabba)
:
: - Various fixes to the stage-2 MMU for pKVM (Fuad Tabba)
:
: - Allow counter offsetting of non-protected guests in protected mode
: (Mostafa Saleh)
:
: - Condition the 'broken CNTVOFF' mitigation on a VM actually having a
: nonzero offset, fixing boot failures of pVMs on affected hardware
: (Mostafa Saleh)
KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
KVM: arm64: Fix timer offsets for non-protected VMs
KVM: arm64: Make timer_get_offset() work in all contexts
KVM: arm64: selftests: Add stage-2 block transition test
KVM: arm64: Don't advertise eager page splitting under pKVM
KVM: arm64: Don't WARN on pKVM stage-2 map failures
KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled
KVM: arm64: Top up stage-2 memcache for dirty logging faults
KVM: arm64: Top up the memcache for pKVM permission faults
KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings
KVM: arm64: Implement lazy vCPU state sync for non-protected guests
KVM: arm64: Add primitives to flush/sync the VGIC state at EL2
KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
KVM: arm64: Add host and hypervisor vCPU lookup primitives
KVM: arm64: Move PSCI helper functions to a shared header
KVM: arm64: Factor out reusable vCPU reset helpers
KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
KVM: arm64: Extract MPIDR computation into a shared header
KVM: arm64: selftests: Add a userspace watchpoint test
KVM: arm64: Flush external_mdscr_el1 to the pKVM hyp vCPU
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
||
|
Marc Zyngier
|
2858600ecd |
KVM: arm64: Make timer_get_offset() work in all contexts
We currently have two implementations of get_timer offset(), one in arm_arch_timer.h, and another one in switch.h. These two only differ by a pair of kern_hyp_va(), which seems a pretty weak reason to open-code it. Turn this function into a macro to avoid the include dependency hell on kern_hyp_va(), and make it work correctly in all contexts. Signed-off-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Mostafa Saleh <smostafa@google.com> Link: https://patch.msgid.link/20260808085824.732659-2-smostafa@google.com Signed-off-by: Oliver Upton <oupton@kernel.org> |
||
|
Marc Zyngier
|
62ed180231 |
KVM: arm64: Add NVHCR_EL2 context switching
Since NVHCR_EL2 represents the HCR_EL2 state of the EL1 guest, it must be dealt with in some particular way: - for a guest in hyp context (an L1 by definition), NVHCR_EL2 directly reflects HCR_EL2 as read and written by the guest itself. It must therefore be eagerly synced back with the emulation code which only knows about HCR_EL2. This is unconditional if NV3 is available on the host. - For an L2 guest, NVHCR_EL2 is controlled by the L1 guest, and we just context switch it like any other EL1 register. Yes, EL1, as that's where this thing runs from the PoV of L1. This is conditioned on the guest using NV3. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://patch.msgid.link/20260730071022.296811-24-maz@kernel.org Signed-off-by: Oliver Upton <oupton@kernel.org> |
||
|
Marc Zyngier
|
649ba04326 |
KVM: arm64: Relax CPTR_EL2 handling when FEAT_NV2p1 is present
With FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful. Prevent such trapping and context switch CPACTR_EL1 in NV contexts when NV2P1 is present. Reviewed-by: Joey Gouly <joey.gouly@arm.com> Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://patch.msgid.link/20260730071022.296811-10-maz@kernel.org Signed-off-by: Oliver Upton <oupton@kernel.org> |
||
|
Marc Zyngier
|
4422d8b750 |
KVM: arm64: Merge guest's HCRX_EL2 using NV_HCRX_GUEST_EXCLUDE
The way we merge the guest-provided HCRX_EL2 value with the host's is bonkers. We try to make it look like the FGT registers by using positive and negative polarities for traps, but most of these bits are not strictly about trapping, as they actively change the way some architectural state is managed. It would be far better to deal with these bits like we do for HCR_EL2, by enumerating the list of bits we don't allow the guest to override. This is simplified by the fact that HCRX_EL2 only affects EL1, and not EL2. Re-jig the HCRX_EL2 handling with a macro that list the bits excluded from the merge (TMEA, PTTWI, EnIDCP128). Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Reviewed-by: Joey Gouly <joey.gouly@arm.com> Link: https://patch.msgid.link/20260730071022.296811-4-maz@kernel.org Signed-off-by: Oliver Upton <oupton@kernel.org> |
||
|
Paolo Bonzini
|
5b0d0464f6 |
KVM/arm64 fixes for 7.2, take #1
- Fix an accounting buglet when reclaiming pages from a protected guest - Fix a bunch of architectural compliance issues when injecting a synthesised exception, most of which were missing the PSTATE.IL bit indicating a 32bit-wide instruction - Another set of fixes addressing issues with translation of VNCR_EL2, including corner cases where the guest point that register at a RO page... - Don't warn when trapping accesses to ZCR_EL2 from an L2 guest, as that's not unexpected at all - Address a bunch of races with LPI migration vs LPIs being disabled - Fix a total howler of a bug combining FEAT_MOPS and NV, resulting in exception returning in the wrong place... - Coerce Fuad Tabba into a reviewer role, and may his Inbox catch fire! -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEn9UcU+C1Yxj9lZw9I9DQutE9ekMFAmo+fUoACgkQI9DQutE9 ekOgBA/8D1XhSpPjYu7NDBOf6+62n02BRhihlFgA3KQkFmiVwUoeiAGXfDyZqQiM ms1LfUmU+7Ry7zoeasHzVi5k/FEk+gMR911eEcciJF7C6BbC3uHUOgEg30DWQo9J bb23PgeAcYnAqgjRw8vcXsvArK6YcY9Y6Qvp3SA1fM8Ls62P0+OdxCE/HM4TE1Ny 1t1vokNHaJCAkf0h6gAI12yEZGhK2bFZsVnkOYgkIuqKI5730/B2hbjqK+hcMPHL TO7tVeRk8qQ0ZFJq3hpnOcRjO6uLA/f/6IHkYej/ADxaqH2uzOQeKY3awoS1UtNJ okRKS3OQjFXW8nsarVRIzGDMp41EjydVB8po6HXWg1zhiD6PEH92W+rnVPq273xh gKi0KOMchgBr+61zeZU/wdPZiU/wnMTxukcpqyCSR5WdoOxwMpGZvdCwHzGOyg7v 8QebZzkAxSB854aaHf5JjdTH9zI/HOk184fxtdgYbxGKMKaZ/wdTM7kos1bBHt8t BP/yu7/jLBxEvrO9iLBHYzlk40i1oqyM4sZpM2Z2w8MBbOaHni1xm1URwFwTX5aM FJ8BxObnnUcB8ARzRHafPcLwgyC0LubEVYsbddppijEhBu8f/wF/sMcIoHNppVCR Vp9KK9+BH6OyYbG7TsKj2zy2S8KqgVonehoDrr66cPySo0l9KWY= =mmLT -----END PGP SIGNATURE----- Merge tag 'kvmarm-fixes-7.2-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD KVM/arm64 fixes for 7.2, take #1 - Fix an accounting buglet when reclaiming pages from a protected guest - Fix a bunch of architectural compliance issues when injecting a synthesised exception, most of which were missing the PSTATE.IL bit indicating a 32bit-wide instruction - Another set of fixes addressing issues with translation of VNCR_EL2, including corner cases where the guest point that register at a RO page... - Don't warn when trapping accesses to ZCR_EL2 from an L2 guest, as that's not unexpected at all - Address a bunch of races with LPI migration vs LPIs being disabled - Fix a total howler of a bug combining FEAT_MOPS and NV, resulting in exception returning in the wrong place... - Coerce Fuad Tabba into a reviewer role, and may his Inbox catch fire! |
||
|
Mostafa Saleh
|
007e0f0fd9 |
KVM: arm64: Avoid naming collision in tracing
When the hypervisor tracing (CONFIG_NVHE_EL2_TRACING) is disabled, it
defines a static inline stub for trace_clock().
However, trace_clock() is already declared as an extern function in
linux/trace_clock.h which is pulled in EL2 compilation.
If the file <nvhe/clock.h> is included when CONFIG_NVHE_EL2_TRACING
is disabled (by including it manually in setup.c) it will cause:
In file included from arch/arm64/kvm/hyp/nvhe/setup.c:22:
./arch/arm64/kvm/hyp/include/nvhe/clock.h:14:19: error: static declaration of ‘trace_clock’ follows non-static declaration
14 | static inline u64 trace_clock(void) { return 0; }
| ^~~~~~~~~~~
on GCC and a linker error on LLVM (it seems to change the linkage to
global)
Although that is not a problem at the moment, as no other files
include <nvhe/clock.h>. That does not seem to be the intent of
this code and that will cause issues with more users as the SMMUv3
driver.
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Link: https://patch.msgid.link/20260713141320.4065600-1-smostafa@google.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
||
|
Linus Torvalds
|
c98d767b34 |
Arm:
"This is a bit of an odd merge window on the KVM/arm64 front. There is
absolutely no new feature in the pull request. It is purely fixes,
because it is simply becoming too hard to review new stuff when so
many AI-fuelled fixes hit the list".
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX)
that caused the FPU load request.
- Some enhancements about interrupt injection.
- Some bug fixes and other small changes.
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic
PTE updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on
a memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes.
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace.
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root.
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both
of these are preparatory changes for upcoming APX support.
- Clean up KVM's register tracking and storage, primarily to prepare for
APX support, which expands the maximum number of GPRs from 16 to 32.
- Keep a single copy of the PDPTRs rather than two, since architecturally
there is just one.
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in shadow page table handling, where KVM fails to recursively
zap nested EPT/NPT shadow page tables when the nested hypervisor tears
down its own EPT/NPT page tables from the bottom up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in the
generic section; also run with MBEC enabled even for non-nested mode.
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Fix a largely benign bug where KVM TDX would incorrectly state it could
emulate several x2APIC MSRs.
- Use the "safe" WRMSR API when proxying LBR MSR writes as the to-be-written
value is guest controlled and completely unvalidated.
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in the
generic section; also run with GMET enabled even for non-nested mode.
- Fixes and minor cleanups to GHCB handling, on top of the earlier work
already merged into 7.1-rc.
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers.
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
- Don't advertise support for unusable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to follow
the kernel's nomenclature, e.g. for mmu_notifiers.
- Use guard() to cleanup up various KVM+VFIO flows.
- Minor cleanups.
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a gmem
range to multiple memslots, and fix the test that was supposed to ensure
KVM returns -EEXIST.
- Treat memslot binding offsets and sizes as unsigned values to fix a bug
where KVM interprets a large "offset + size" as a negative value and allows
a nonsensical offset.
- Use the inode number instead of the page offset for the NUMA interleaving
index to fix a bug where the effective index would jump by two for
consecutive pages (the caller also adds in the page offset).
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the first
pass, as always waiting only 1ms hid a KVM RISC-V bug as the test reaped the
bitmap before KVM could build up enough state to hit the bug.
- A pile of one-off fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmoypI0UHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroNCSAf8CyuvicojnZGFsNcoY4DZ0whBj5l8
IwVkHaptuyGLSwPAebC/DwtU2NBY4CQdbtLjGmxenf+fSQOtcz1YWKbdY4Nr0hJG
euAb/wO47RqKp1C4WPi7lL2P4XHgmUgD5XZ1qDnRV9gnGddZi3QLLmKgAtNq9Z4m
+pAdGD2/RyZeI0Dx61+5ufE/z8YBlH6GrzgBTxfvvRNKqNDFykBenxYHI7seEfHC
BPz0LxjHDD8BMDUo86qaylZnaeQSmFTGO/eTgvCvabxG6WbM1s0w3K1owmUmP7o3
p9o8274Xh53iK2JYD/qGOtAJXejIwbYKT5bnvUQX0O1EH2+ouvSFapLQ9Q==
=sSqM
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
|
||
|
Oliver Upton
|
9f1667098c |
KVM: arm64: nv: Drop bogus WARN for write to ZCR_EL2
It is entirely possible for a guest to write to the ZCR_EL2 sysreg alias
while in a nested context, as it is expected if FEAT_NV2 is advertised
to the L1 hypervisor.
Get rid of the bogus WARN which, since the hyp vectors were installed at
this point, has the effect of a hyp_panic...
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Weiming Shi
|
ff1022c3de |
KVM: arm64: nv: Fix SPSR_EL2 restore in kvm_hyp_handle_mops()
kvm_hyp_handle_mops() resets the single-step state machine as part of
rewinding state for a MOPS exception by modifying vcpu_cpsr() and
writing the result directly into hardware.
In the case of nested virtualization, vcpu_cpsr() is a synthetic value
such that the rest of KVM can deal with vEL2 cleanly. That means the
value requires translation before being written into hardware, which is
unfortunately missing from the MOPS handler.
Fix it by directly modifying SPSR_EL2 and avoiding the synthetic state
altogether, which will be resynchronized on the next 'full' exit back
to KVM.
Fixes:
|
||
|
Linus Torvalds
|
80476f22b8 |
arm64 updates for 7.2
CPU errata handling:
- Extend CnP disabling workaround to HiSilicon HIP09 hardware.
- Work around eternally broken broadcast TLB invalidation on more CPUs.
- Documentation and code cleanups.
CPU features:
- Add new hwcaps for the 2025 dpISA extensions.
Floating point / SVE / SME:
- Significant cleanup to the low-level state management code in the core
architecture code and KVM.
- Use correct register widths during SVE/SME save/restore assembly.
- Expose SVE/SME save/restore memory accesses to sanitisers.
Memory management:
- Preparatory work for unmapping the kernel data and bss sections from
the linear map.
Miscellaneous:
- Inline DAIF manipulation helpers so they can be used safely from
non-instrumentable code.
- Fix handling of the 'nosmp' cmdline option to avoid marking secondary
cores as "possible".
MPAM:
- Add support for v0.1 of the MPAM architecture.
Perf:
- Update HiSilicon PMU MAINTAINERS entry.
- Fix event encodings for the DVM node in the CMN driver.
Selftests:
- Extend sigframe tests to cover POE context.
- Add coverage for the newly added 2025 dpISA hwcaps.
System registers:
- Add new registers and ESR encodings for the HDBSS feature.
Plus minor fixes and cleanups across the board.
-----BEGIN PGP SIGNATURE-----
iQFEBAABCgAuFiEEPxTL6PPUbjXGY88ct6xw3ITBYzQFAmoun+0QHHdpbGxAa2Vy
bmVsLm9yZwAKCRC3rHDchMFjNFEvB/9scDYXHUCxQOqoRB5TPUi63pgl5vl83IfI
TWE2r2wNsrjqXlAV2FDbzuTWxLpACS+vpk5ltsb57aUfENQ5wZEFCDvJlbpNZuHS
5/zAQmCajPpgC5TFDo7rbgSv2BSR4TXuFfvGS349+a82c9C12EUulBFvR1DDVElh
FCYT5PfHNu1Khas4YiUln+9WSqS8qSHfYsz4Qb/lDaZcAmh22jQBPZQFU7fMlZST
Fr/nzSy6cxkARotqU7RlRQLgtDHBEqTYrNHCP0WSQ2aPJN8SeFUyjEcQas/Zak+S
Bme8OHcqn4pZPll5TfHUTaJp91oIJAA9uX5RSwE+G3NvAGNbptuw
=LXuh
-----END PGP SIGNATURE-----
Merge tag 'arm64-upstream' of gitolite.kernel.org:pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Will Deacon:
"It feels like the new world of AI tooling has slowed us down a little
on the feature side when compared to the fixes side. The extra rounds
of Sashiko review have also pushed a few things out until next time.
Still, there's some good foundational stuff here for the fpsimd code
and hardening work towards removing the predictable linear alias of
the kernel image.
CPU errata handling:
- Extend CnP disabling workaround to HiSilicon HIP09 hardware.
- Work around eternally broken broadcast TLB invalidation on more
CPUs.
- Documentation and code cleanups.
CPU features:
- Add new hwcaps for the 2025 dpISA extensions.
Floating point / SVE / SME:
- Significant cleanup to the low-level state management code in the
core architecture code and KVM.
- Use correct register widths during SVE/SME save/restore assembly.
- Expose SVE/SME save/restore memory accesses to sanitisers.
Memory management:
- Preparatory work for unmapping the kernel data and bss sections
from the linear map.
Miscellaneous:
- Inline DAIF manipulation helpers so they can be used safely from
non-instrumentable code.
- Fix handling of the 'nosmp' cmdline option to avoid marking
secondary cores as "possible".
MPAM:
- Add support for v0.1 of the MPAM architecture.
Perf:
- Update HiSilicon PMU MAINTAINERS entry.
- Fix event encodings for the DVM node in the CMN driver.
Selftests:
- Extend sigframe tests to cover POE context.
- Add coverage for the newly added 2025 dpISA hwcaps.
System registers:
- Add new registers and ESR encodings for the HDBSS feature.
Plus minor fixes and cleanups across the board"
* tag 'arm64-upstream' of gitolite.kernel.org:pub/scm/linux/kernel/git/arm64/linux: (73 commits)
arm64: errata: Mitigate TLBI errata on Microsoft Azure Cobalt 100 CPU
arm64: errata: Mitigate TLBI errata on NVIDIA Olympus CPU
arm64: errata: Mitigate TLBI errata on various Arm CPUs
arm64: cputype: Add C1-Premium definitions
arm64: cputype: Add C1-Ultra definitions
Revert "arm64: mm: Unmap kernel data/bss entirely from the linear map"
Revert "arm64: mm: Defer remap of linear alias of data/bss"
arm64: arch_timer: reuse arch_timer_read_cnt{p,v}ct_el0() helpers
arm64/mm: Rename ptdesc_t
arm64: mm: Defer remap of linear alias of data/bss
KVM: arm64: Omit tag sync on stage-2 mappings of the zero page
arm64: Avoid double evaluation of __ptep_get()
kasan: Move generic KASAN page tables out of BSS too
arm64: Rename page table BSS section to .bss..pgtbl
arm64: patching: replace min_t with min in __text_poke
perf/arm-cmn: Fix DVM node events
arm64: fpsimd: Remove <asm/fpsimdmacros.h>
arm64: fpsimd: Move SME save/restore inline
arm64: fpsimd: Move sve_flush_live() inline
arm64: fpsimd: Move SVE save/restore inline
...
|
||
|
Paolo Bonzini
|
751d041a13 |
KVM/arm64 updates for 7.2
* New features:
- None. Zilch. Nada. Que dalle.
* Fixes and other improvements:
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEn9UcU+C1Yxj9lZw9I9DQutE9ekMFAmorw+MACgkQI9DQutE9
ekNZSw//bDv7l2dz3nUbtAuDIX9bQfbyq25a/R1SCC5om8yKOY0v0DmEAju5iu8T
y7n2G/ME+GeeAvRo8thWaDftqJTKD73sXll0F7AhRSdnWsPD4k5B6E4m62cwb8jd
oabUlf/ruKq0ftcwuJy3TWUNB0XVuO4Ep6zwcnrLSmyOUPCLqfbvWWKujQTy/Yec
zj9/FolfpgBla0cdZ1GwrBSppfPuT9rQtFrHwA91b66yNoDc+DxGln6/b0vDsN79
GcLJSoYeUQTXBdy4SH6oYLhhCi30+CFtSdkzA/vv3INX269vyBOmgMPJDnoquWPW
oH41sYIhjcy08zvbOzi/P7/mDUNRgpMGpzUZ/rKAZPvpWrQYCQfPYWSeRHCjBwQd
lGTJ2pXu0UcLoLweshaKEcuO6bq2TKtf3mCMMZWyN3pnLp/CBCIfroQr3RYmrWTT
GPoBTN8epnOJuhfOpmbfDWjm2vIJ6z/B0zFTOtUZstybxm9sOg6NnPr7WoaT1LEV
FvD+4Oc53h7Py0NGGlk5iqlRYgTULjvCQ4rcPUs2tU8GYXy3MA4l0Erj1FsEwmXG
eX4CTbCEI2FCaXuVK7zz+YngilNERFFD+ebwWgzCbZmA+w6pA4fWT1T+LywV24lu
zANT0Id4ogoI3RxxHLG61TH9xy6+OW56PYGTnZedurxyXsxxod0=
=ER2M
-----END PGP SIGNATURE-----
Merge tag 'kvmarm-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 updates for 7.2
* New features:
- None. Zilch. Nada. Que dalle.
* Fixes and other improvements:
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
|
||
|
Marc Zyngier
|
1ee27dacbe |
Merge branch kvm-arm64/nv-mmu-7.2 into kvmarm-master/next
* kvm-arm64/nv-mmu-7.2: : . : Assorted collection of fixes for NV MMU bugs : : - Correctly plug AT S1E1A handling in the emulation backend : : - Make CPTR_EL2.E0POE depend on FEAT_S1POE : : - Drop the reference on the page if the VNCR translation : races with an MMU notifier : : - Correctly synthesise an SEA if a page table walk fails due : to a guest error : : - Fully invalidate the VNCR TLB and fixmap when translating : for a new VNCR : : - Restart S1 walk when the S2 walk fails due to a race condition : : - Correctly return -EAGAIN when a S1 walk fails : : - Fix block mapping validity check in stage-1 walker for 64kB pages : : - Fix potential NULL dereference when performing an EL2 TLBI targeting : the VNCR page : : - Hold kvm->mmu_lock while initialising the vncr_tlb pointer : . KVM: arm64: nv: Hold kvm->mmu_lock while initialising vcpu->arch.vncr_tlb KVM: arm64: nv: Avoid dereferencing NULL VNCR pseudo-TLB KVM: arm64: Fix block mapping validity check in stage-1 walker KVM: arm64: nv: Restart stage-1 walk if stage-2 desc update fails KVM: arm64: Restart instruction upon race in __kvm_at_s12() KVM: arm64: nv: Inject SEA TTW when desc update can't write to GPA KVM: arm64: nv: Fully update VNCR fixmap state in kvm_translate_vncr() KVM: arm64: Don't leak PFN when kvm_translate_vncr() races MMU notifier arm64: cpufeature: Expose ID_AA64ISAR2_EL1.ATS1A to KVM KVM: arm64: Wire AT S1E1A in the system instruction handling table KVM: arm64: Key CPTR_EL2.E0POE propagation on FEAT_S1POE Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Marc Zyngier
|
bd2ed0733b |
Merge branch kvm-arm64/misc-7.2 into kvmarm-master/next
* kvm-arm64/misc-7.2: : . : - Check for a valid vcpu pointer upon deactivating traps when handling : a HYP panic in VHE mode : : - Make the __deactivate_fgt() macro use its arguments instead of the : surrounding context : : - Don't bother with initialising TPIDR_EL2 in the hyp stubs, as this : is already taken care of in more obvious places : : - Drop the unused kvm_arch pointer passed to __load_stage2() : : - Return -EOPNOTSUPP when a hypercall fails for some reason, instead of : returning whatever was in the result structure : : - Make the ITS ABI selection helpers return void, which avoids wondering : about the nature of the return code (always 0) : . KVM: arm64: vgic-its: Make ABI commit helpers return void KVM: arm64: Set a Linux errno on SMCCC error in kvm_call_hyp_nvhe() KVM: arm64: Remove @arch from __load_stage2() KVM: arm64: Don't populate TPIDR_EL2 in finalise_el2() KVM: arm64: Fix __deactivate_fgt macro parameter typo KVM: arm64: Guard against NULL vcpu on VHE hyp panic path Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Marc Zyngier
|
35c060e8d6 |
Merge branch kvm-arm64/pkvm-fixes-7.2 into kvmarm-master/next
* kvm-arm64/pkvm-fixes-7.2: : . : Assorted pKVM fixes for 7.2: : : - Ensure that the vcpu memcache is filled in a number of cases (donate, : share, selftest) : : - Fix vmemmap page order handling by resetting it when initialising the : memory pool : : - Don't leak page references on failed memory donation : : - Add sanity-check for refcounted pages when donating/sharing pages : : - Clear __hyp_running_vcpu on state flush : : - Check LR upper bound against a trusted value : : - Assorted fixes for the host-side tracking of the pages shared with : EL2 as a result of some Sashiko testing from Fuad : : - Correctly forward HCR_EL2.VSE from host to guest, so that protected : guests can see SErrors : . KVM: arm64: Roll back partial shares on kvm_share_hyp() failure KVM: arm64: Avoid host/hyp share desync on unshare hypercall failure KVM: arm64: Free hyp-share tracking node when share hypercall fails KVM: arm64: Flush HCR_EL2.VSE to deliver SErrors to pKVM guests KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU KVM: arm64: Clear __hyp_running_vcpu when flushing the pKVM hyp vCPU KVM: arm64: Pre-check vcpu memcache for host->guest donate KVM: arm64: Pre-check vcpu memcache for host->guest share KVM: arm64: Seed pkvm_ownership_selftest vcpu memcache KVM: arm64: Add fail-safe for refcounted pages in __pkvm_hyp_donate_host KVM: arm64: Fix __pkvm_init_vm error path KVM: arm64: Reset page order in pKVM hyp_pool Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Joey Gouly
|
cbaffe843a |
KVM: arm64: Restore POR_EL0 access to host EL0
CPTR_EL2.E0POE was being cleared in __deactivate_cptr_traps_vhe(), which meant that any accesses to POR_EL0 from host EL0 would trap and be reported to userspace as an Illegal instruction. This would happen after running any VM, regardless if it used POE or not. Signed-off-by: Joey Gouly <joey.gouly@arm.com> Link: https://sashiko.dev/#/patchset/20260602155430.2088142-1-maz@kernel.org?part=1 Link: https://patch.msgid.link/20260604105434.2297268-1-joey.gouly@arm.com Signed-off-by: Marc Zyngier <maz@kernel.org> Cc: stable@vger,kernel.org |
||
|
Marc Zyngier
|
978cd6b2ad |
KVM: arm64: Key CPTR_EL2.E0POE propagation on FEAT_S1POE
We propagate CPTR_EL2.E0POE from a L1 into the L0 configuration, but
we key this on the L1 guest supporting FEAT_S2POE. This is obviously
wrong, as this bit is solely concerned with Stage-1 translation.
Fix this by making the update depend on FEAT_S1POE.
Fixes:
|
||
|
Mark Rutland
|
2768101b39 |
arm64: fpsimd: Move SVE save/restore inline
Currently the SVE register save/restore sequences are written in out-of-line assembly routines. While this works, it's somewhat painful: * As KVM needs to be able to use the sequences in hyp code, separate assembly files are used for the regular kernel and KVM code. While the common logic is shared in assembly macros, this still requires some duplication, and has lead to some trivial divergence. * As the SVE LDR/STR instrucitons have limited addressing modes, the assembly macros use an awkward pattern requiring negative offsets. This could be written more clearly with addresses being generated in C code. * As the FFR does not always exist in streaming mode, some awkward conditional branching has been written in assembly which could be clearer in C (and would permit the compiler to optimize out unnecessary branches in some cases). * For historical reasons, the assembly macros take some register arguments as numerical indices (e.g. "sve_save 0, x1" uses x0 and x1), which is simply confusing. * For historical reasons, the SVE save/restore code and FPSIMD save/restore code have a distinct sequences for FPSR and FPCR. Ideally this logic would be shared. * The assembly sequences can't be instrumented, and so it's harder than necessary to catch memory safety issues. To handle the above, move the SVE register save/restore sequences to inline assembly. Neither GCC nor LLVM instrument memory arguments to inline assembly, so explicit instrumentation is added in the same manner as other assembly routines. This instrumentation is implicitly disabled by Kbuild for nVHE hyp code. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> |
||
|
Mark Rutland
|
e1b163e405 |
arm64: fpsimd: Use opaque type for SVE state
As the SVE state size can vary at runtime, we don't have a concrete type for the in-memory SVE state, and pass this around using a pointer to void. The functions which save/restore the SVE state have a very unusual calling convention, expecting a pointer to the FFR *in the middle of* the in-memory SVE state, which is also passed as a pointer to void. Passing a pointer to the FFR also requires that callers find the live VL and perform some arithmetic, which callers implement differently. Using pointer to void means that it's very easy to introduce errors that cannot be caught by the compiler (e.g. as 'void **' can be assigned to 'void *'). In general this is unnecessarily confusing and fragile. Improve this by adding an opaque 'struct arm64_sve_state', and consistently passing a pointer to this, performing the necessary offsetting *within* the save/restore functions. For the moment, the offsetting is performed in a new '_sve_pffr' assembly macro, using the ADDVL and ADDPL instructions. These add a multiple of the live vector length and predicate length respectively. The ADDVL immediate range cannot encode 32, so this is split into two increments of 16. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> |
||
|
Mark Rutland
|
890712d450 |
arm64: fpsimd: Move fpsimd save/restore inline
Currently the FPSIMD register save/restore sequences are written in
out-of-line assembly routines. While this works, it's somewhat painful:
* As KVM needs to be able to use the sequences in hyp code, separate
assembly files are used for the regular kernel and KVM code. While the
common logic is shared in assembly macros, this still requires some
duplication, and has lead to some trivial divergence.
* For historical reasons, the assembly macros take some register
arguments as numerical indices (e.g. "fpsimd_save x0, 8" uses x0 and
x8), which is simply confusing.
* For historical reasons, the SVE save/restore code and FPSIMD
save/restore code have distinct sequences for FPSR and FPCR. Ideally
this logic would be shared.
* The assembly sequences can't be instrumented, and so it's harder than
necessary to catch memory safety issues.
To handle the above, move the FPSIMD register save/restore sequences to
inline assembly, and share the FPSR+FPCR save/restore with SVE.
Neither GCC nor LLVM instrument memory arguments to inline assembly, so
explicit instrumentation is added in the same manner as other assembly
routines. This instrumentation is implicitly disabled by Kbuild for nVHE
hyp code.
I've used the SVE sequence for restoring FPCR, which uses an
unconditional write to FPCR, rather than the conditional write used by
the FPSIMD assembly sequence. I believe that in practice, this doesn't
matter to a real workload, and given it's possible for the mis-predicted
branch to cost more than the necessary micro-architectural
synchronization, I strongly suspect any performance impact is within the
noise.
Looking at the history, the FPSIMD assembly sequence was changed to use
a conditional write to FPCR since 2014 in commit:
|
||
|
Mark Rutland
|
1277531fca |
arm64: fpsimd: Split FPSR/FPCR from SVE save/restore
Regardless of whether the vector registers are saved in FPSIMD or SVE
format, we store FPSR and FPCR in user_fpsimd_state::{fpsr,fpcr}.
For historical reasons, the functions which save/restore SVE context
take a pointer to user_fpsimd_state::fpsr, and use this to access both
user_fpsimd_state::fpsr and user_fpsimd_state::fpcr. This is
unnecessarily fragile.
Move the save/restore of FPSR and FPCR into separate helper functions
which take a pointer to user_fpsimd_state. I've used read_sysreg_s() and
write_sysreg_s() as contemporary versions of LLVM will refuse to
directly assemble accesses to FPCR or FPSR unless the "fp" arch
extension is enabled.
For the moment, fpsimd_save_state() and fpsimd_load_state() are left
as-is with their own logic to save/restore FPSR and FPCR. This will be
unified in subsequent patches.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Fuad Tabba <tabba@google.com>
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Oliver Upton <oupton@kernel.org>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
|
||
|
Mark Rutland
|
afd7af2b56 |
KVM: arm64: pkvm: Remove struct cpu_sve_state
There's no need for struct cpu_sve_state. Code would be simpler and more robust without it, and removing it will simplify further cleanups (e.g. adding an opaque type for the sve register state). Protected KVM stores most of the host's system register state in kvm_host_data::host_ctxt, which is an instance of struct kvm_cpu_context. As kvm_cpu_context::sys_regs[] has a slot for ZCR_EL1, we can store the host's ZCR_EL1 there. While kvm_cpu_context::sys_regs doesn't have slots for FPSR and FPCR, these are usually expected to be stored in struct user_fpsimd_state. For historical reasons, __sve_save_state and __sve_restore_state() expect a pointer to fpsr *within* struct user_fpsimd_state, assuming the fpcr will immediately follow, as per the order within struct user_fpsimd_state. We currently match this ordering in struct cpu_sve_state, but it would be simpler and more robust to use struct user_fpsimd_state directly. After moving ZCR_EL1, FPSR, and FPCR out of struct cpu_sve_state, all that's left is sve_regs, which can be represented as a pointer without need for a container struct. This is kept as a pointer to u8 (matching the array type), as this permits the compiler to catch unbalanced referencing/dereferencing, which is not possible for pointers to void. Apply the above changes, and remove cpu_sve_state. I've dropped the comment regarding buffer alignment as AFAICT this was never necessary. The LDR/STR (vector) instructions only require this alignment when SCTLR_ELx.A==1, which is not the case for the kernel or hyp code. Nothing else depends on the alignment. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> |
||
|
Mark Rutland
|
da20bb4bc5 |
KVM: arm64: pkvm: Save host FPMR in host cpu context
Protected KVM stores most of the host's system register state in kvm_host_data::host_ctxt, which is an instance of struct kvm_cpu_context. As kvm_cpu_context::sys_regs[] has a slot for FPMR, we can store the host's FPMR there. Do so, and remove kvm_host_data::fpmr. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Fuad Tabba <tabba@google.com> Cc: James Morse <james.morse@arm.com> Cc: Marc Zyngier <maz@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org> |
||
|
Mark Brown
|
8372633074 |
KVM: arm64: Correctly cap ZCR_EL2 provided by a guest hypervisor
ZCR_EL2 can be updated by a VHE guest hypervisor either using ZCR_EL2
(which traps) or ZCR_EL1 (which does not trap). KVM handles both in
different way:
- on ZCR_EL2 trap, ZCR_EL2.LEN is immediately capped at the VM's own
VL limit. This has the potential to break existing SW that relies
on the full LEN field to be stateful.
- on ZCR_EL1 access, we do absolutely nothing.
On restoring the SVE context for an L2 guest, we directly restore the
guest hypervisor's view of ZCR_EL2 into the physical ZCR_EL2. If the
guest's view of the register was updated using the ZCR_EL2 accessor,
the value has already been sanitised (with the caveat mentioned above).
But if the guest used ZCR_EL1, the raw value is written into the HW,
and the L2 guest can now access VLs that it shouldn't.
Fix all the above by moving the VL capping to the restore points,
ensuring that:
- the HW is always programmed with a capped value, irrespective of
the accessor being used,
- the ZCR_EL2.LEN field is always completely stateful, irrespective
of the accessor being used.
Additionally, move ZCR_EL2 to be a sanitised register, ensuring that
only the LEN field is actually stateful. This requires some creative
construction of the RES0 mask, as the sysreg generation script does
not yet generate RAZ/WI fields.
Fixes:
|
||
|
Zenghui Yu (Huawei)
|
e72b425f61 |
KVM: arm64: Remove @arch from __load_stage2()
Since commit
|
||
|
Fuad Tabba
|
1bdcdc84f9 |
KVM: arm64: Fix __deactivate_fgt macro parameter typo
__deactivate_fgt() declares its first parameter as "htcxt" but the body
references "hctxt". The parameter is unused; the macro silently captures
"hctxt" from the enclosing scope. Both existing callers
(__deactivate_traps_hfgxtr() and __deactivate_traps_ich_hfgxtr()) happen
to define a local "struct kvm_cpu_context *hctxt", so the macro works
by coincidence.
A future caller without an "hctxt" local in scope, or naming it
differently, would compile but bind to the wrong context. Align the
parameter name with the sibling __activate_fgt() macro.
The "vcpu" parameter remains unused in the body, kept for API symmetry
with __activate_fgt() (which uses it).
Fixes:
|
||
|
Vincent Donnefort
|
20d2753295 |
KVM: arm64: Fix __pkvm_init_vm error path
In the unlikely case where insert_vm_table_entry fails, __pkvm_init_vm
release the memory donated by the host for the PGD, but as the stage-2
is still set-up the hypervisor keeps a refcount on those pages,
effectively leaking the references.
Fix the rollback with the newly added kvm_guest_destroy_stage2().
Fixes:
|
||
|
Fuad Tabba
|
d4d215e5b8 |
KVM: arm64: Fix __deactivate_fgt macro parameter typo
__deactivate_fgt() declares its first parameter as "htcxt" but the body
references "hctxt". The parameter is unused; the macro silently captures
"hctxt" from the enclosing scope. Both existing callers
(__deactivate_traps_hfgxtr() and __deactivate_traps_ich_hfgxtr()) happen
to define a local "struct kvm_cpu_context *hctxt", so the macro works
by coincidence.
A future caller without an "hctxt" local in scope, or naming it
differently, would compile but bind to the wrong context. Align the
parameter name with the sibling __activate_fgt() macro.
The "vcpu" parameter remains unused in the body, kept for API symmetry
with __activate_fgt() (which uses it).
Fixes:
|
||
|
Linus Torvalds
|
01f492e181 |
Arm:
- Add support for tracing in the standalone EL2 hypervisor code, which should help both debugging and performance analysis. This uses the new infrastructure for 'remote' trace buffers that can be exposed by non-kernel entities such as firmware, and which came through the tracing tree. - Add support for GICv5 Per Processor Interrupts (PPIs), as the starting point for supporting the new GIC architecture in KVM. - Finally add support for pKVM protected guests, where pages are unmapped from the host as they are faulted into the guest and can be shared back from the guest using pKVM hypercalls. Protected guests are created using a new machine type identifier. As the elusive guestmem has not yet delivered on its promises, anonymous memory is also supported. This is only a first step towards full isolation from the host; for example, the CPU register state and DMA accesses are not yet isolated. Because this does not really yet bring fully what it promises, it is hidden behind CONFIG_ARM_PKVM_GUEST + 'kvm-arm.mode=protected', and also triggers TAINT_USER when a VM is created. Caveat emptor. - Rework the dreaded user_mem_abort() function to make it more maintainable, reducing the amount of state being exposed to the various helpers and rendering a substantial amount of state immutable. - Expand the Stage-2 page table dumper to support NV shadow page tables on a per-VM basis. - Tidy up the pKVM PSCI proxy code to be slightly less hard to follow. - Fix both SPE and TRBE in non-VHE configurations so that they do not generate spurious, out of context table walks that ultimately lead to very bad HW lockups. - A small set of patches fixing the Stage-2 MMU freeing in error cases. - Tighten-up accepted SMC immediate value to be only #0 for host SMCCC calls. - The usual cleanups and other selftest churn. LoongArch: - Use CSR_CRMD_PLV for kvm_arch_vcpu_in_kernel(). - Add DMSINTC irqchip in kernel support. RISC-V: - Fix steal time shared memory alignment checks - Fix vector context allocation leak - Fix array out-of-bounds in pmu_ctr_read() and pmu_fw_ctr_read_hi() - Fix double-free of sdata in kvm_pmu_clear_snapshot_area() - Fix integer overflow in kvm_pmu_validate_counter_mask() - Fix shift-out-of-bounds in make_xfence_request() - Fix lost write protection on huge pages during dirty logging - Split huge pages during fault handling for dirty logging - Skip CSR restore if VCPU is reloaded on the same core - Implement kvm_arch_has_default_irqchip() for KVM selftests - Factored-out ISA checks into separate sources - Added hideleg to struct kvm_vcpu_config - Factored-out VCPU config into separate sources - Support configuration of per-VM HGATP mode from KVM user space s390: - Support for ESA (31-bit) guests inside nested hypervisors. - Remove restriction on memslot alignment, which is not needed anymore with the new gmap code. - Fix LPSW/E to update the bear (which of course is the breaking event address register). x86: - Shut up various UBSAN warnings on reading module parameter before they were initialized. - Don't zero-allocate page tables that are used for splitting hugepages in the TDP MMU, as KVM is guaranteed to set all SPTEs in the page table and thus write all bytes. - As an optimization, bail early when trying to unsync 4KiB mappings if the target gfn can just be mapped with a 2MiB hugepage. x86 generic: - Copy single-chunk MMIO write values into struct kvm_vcpu (more precisely struct kvm_mmio_fragment) to fix use-after-free stack bugs where KVM would dereference stack pointer after an exit to userspace. - Clean up and comment the emulated MMIO code to try to make it easier to maintain (not necessarily "easy", but "easier"). - Move VMXON+VMXOFF and EFER.SVME toggling out of KVM (not *all* of VMX and SVM enabling) as it is needed for trusted I/O. - Advertise support for AVX512 Bit Matrix Multiply (BMM) instructions - Immediately fail the build if a required #define is missing in one of KVM's headers that is included multiple times. - Reject SET_GUEST_DEBUG with -EBUSY if there's an already injected exception, mostly to prevent syzkaller from abusing the uAPI to trigger WARNs, but also because it can help prevent userspace from unintentionally crashing the VM. - Exempt SMM from CPUID faulting on Intel, as per the spec. - Misc hardening and cleanup changes. x86 (AMD): - Fix and optimize IRQ window inhibit handling for AVIC; make it per-vCPU so that KVM doesn't prematurely re-enable AVIC if multiple vCPUs have to-be-injected IRQs. - Clean up and optimize the OSVW handling, avoiding a bug in which KVM would overwrite state when enabling virtualization on multiple CPUs in parallel. This should not be a problem because OSVW should usually be the same for all CPUs. - Drop a WARN in KVM_MEMORY_ENCRYPT_REG_REGION where KVM complains about a "too large" size based purely on user input. - Clean up and harden the pinning code for KVM_MEMORY_ENCRYPT_REG_REGION. - Disallow synchronizing a VMSA of an already-launched/encrypted vCPU, as doing so for an SNP guest will crash the host due to an RMP violation page fault. - Overhaul KVM's APIs for detecting SEV+ guests so that VM-scoped queries are required to hold kvm->lock, and enforce it by lockdep. Fix various bugs where sev_guest() was not ensured to be stable for the whole duration of a function or ioctl. - Convert a pile of kvm->lock SEV code to guard(). - Play nicer with userspace that does not enable KVM_CAP_EXCEPTION_PAYLOAD, for which KVM needs to set CR2 and DR6 as a response to ioctls such as KVM_GET_VCPU_EVENTS (even if the payload would end up in EXITINFO2 rather than CR2, for example). Only set CR2 and DR6 when consumption of the payload is imminent, but on the other hand force delivery of the payload in all paths where userspace retrieves CR2 or DR6. - Use vcpu->arch.cr2 when updating vmcb12's CR2 on nested #VMEXIT instead of vmcb02->save.cr2. The value is out of sync after a save/restore or after a #PF is injected into L2. - Fix a class of nSVM bugs where some fields written by the CPU are not synchronized from vmcb02 to cached vmcb12 after VMRUN, and so are not up-to-date when saved by KVM_GET_NESTED_STATE. - Fix a class of bugs where the ordering between KVM_SET_NESTED_STATE and KVM_SET_{S}REGS could cause vmcb02 to be incorrectly initialized after save+restore. - Add a variety of missing nSVM consistency checks. - Fix several bugs where KVM failed to correctly update VMCB fields on nested #VMEXIT. - Fix several bugs where KVM failed to correctly synthesize #UD or #GP for SVM-related instructions. - Add support for save+restore of virtualized LBRs (on SVM). - Refactor various helpers and macros to improve clarity and (hopefully) make the code easier to maintain. - Aggressively sanitize fields when copying from vmcb12, to guard against unintentionally allowing L1 to utilize yet-to-be-defined features. - Fix several bugs where KVM botched rAX legality checks when emulating SVM instructions. There are remaining issues in that KVM doesn't handle size prefix overrides for 64-bit guests. - Fail emulation of VMRUN/VMLOAD/VMSAVE if mapping vmcb12 fails instead of somewhat arbitrarily synthesizing #GP (i.e. don't double down on AMD's architectural but sketchy behavior of generating #GP for "unsupported" addresses). - Cache all used vmcb12 fields to further harden against TOCTOU bugs. x86 (Intel): - Drop obsolete branch hint prefixes from the VMX instruction macros. - Use ASM_INPUT_RM() in __vmcs_writel() to coerce clang into using a register input when appropriate. - Code cleanups. guest_memfd: - Don't mark guest_memfd folios as accessed, as guest_memfd doesn't support reclaim, the memory is unevictable, and there is no storage to write back to. LoongArch selftests: - Add KVM PMU test cases s390 selftests: - Enable more memory selftests. x86 selftests: - Add support for Hygon CPUs in KVM selftests. - Fix a bug in the MSR test where it would get false failures on AMD/Hygon CPUs with exactly one of RDPID or RDTSCP. - Add an MADV_COLLAPSE testcase for guest_memfd as a regression test for a bug where the kernel would attempt to collapse guest_memfd folios against KVM's will. -----BEGIN PGP SIGNATURE----- iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmnftRQUHHBib256aW5p QHJlZGhhdC5jb20ACgkQv/vSX3jHroPAzwf+NKO4Ktv+7A22ImN0SBl0nlUuulsz vTcw3+hxdRoIw83GdNS+hG5js0wrpMDnbv3t4+VliDNBSSxrBzcSWX2wpilW0Xtw qGo1MWhs2lKPy1NlaRVOwPS6j7uF3AR0TQ1iQLGMedQuCU9WpiKJxyhNXJdbLrt3 8EgFzsvtEsv+jKNRUNDf9+d0j4gZsFyIe+Brhianbw+u3/UCiUClLCdsKPc4+5ZX 08otYXytacGNIf/5Ev1vT4pHkHL0yqKXAtX7LEtaS3+0KrPuLjV4slemivzE9vf5 Evafm5AhA4wpaNMb1ZerhY3T94lsMaJpWxotjR//0Q7C9B59pCQnXCm8mg== =CcE0 -----END PGP SIGNATURE----- Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm Pull kvm updates from Paolo Bonzini: "Arm: - Add support for tracing in the standalone EL2 hypervisor code, which should help both debugging and performance analysis. This uses the new infrastructure for 'remote' trace buffers that can be exposed by non-kernel entities such as firmware, and which came through the tracing tree - Add support for GICv5 Per Processor Interrupts (PPIs), as the starting point for supporting the new GIC architecture in KVM - Finally add support for pKVM protected guests, where pages are unmapped from the host as they are faulted into the guest and can be shared back from the guest using pKVM hypercalls. Protected guests are created using a new machine type identifier. As the elusive guestmem has not yet delivered on its promises, anonymous memory is also supported This is only a first step towards full isolation from the host; for example, the CPU register state and DMA accesses are not yet isolated. Because this does not really yet bring fully what it promises, it is hidden behind CONFIG_ARM_PKVM_GUEST + 'kvm-arm.mode=protected', and also triggers TAINT_USER when a VM is created. Caveat emptor - Rework the dreaded user_mem_abort() function to make it more maintainable, reducing the amount of state being exposed to the various helpers and rendering a substantial amount of state immutable - Expand the Stage-2 page table dumper to support NV shadow page tables on a per-VM basis - Tidy up the pKVM PSCI proxy code to be slightly less hard to follow - Fix both SPE and TRBE in non-VHE configurations so that they do not generate spurious, out of context table walks that ultimately lead to very bad HW lockups - A small set of patches fixing the Stage-2 MMU freeing in error cases - Tighten-up accepted SMC immediate value to be only #0 for host SMCCC calls - The usual cleanups and other selftest churn LoongArch: - Use CSR_CRMD_PLV for kvm_arch_vcpu_in_kernel() - Add DMSINTC irqchip in kernel support RISC-V: - Fix steal time shared memory alignment checks - Fix vector context allocation leak - Fix array out-of-bounds in pmu_ctr_read() and pmu_fw_ctr_read_hi() - Fix double-free of sdata in kvm_pmu_clear_snapshot_area() - Fix integer overflow in kvm_pmu_validate_counter_mask() - Fix shift-out-of-bounds in make_xfence_request() - Fix lost write protection on huge pages during dirty logging - Split huge pages during fault handling for dirty logging - Skip CSR restore if VCPU is reloaded on the same core - Implement kvm_arch_has_default_irqchip() for KVM selftests - Factored-out ISA checks into separate sources - Added hideleg to struct kvm_vcpu_config - Factored-out VCPU config into separate sources - Support configuration of per-VM HGATP mode from KVM user space s390: - Support for ESA (31-bit) guests inside nested hypervisors - Remove restriction on memslot alignment, which is not needed anymore with the new gmap code - Fix LPSW/E to update the bear (which of course is the breaking event address register) x86: - Shut up various UBSAN warnings on reading module parameter before they were initialized - Don't zero-allocate page tables that are used for splitting hugepages in the TDP MMU, as KVM is guaranteed to set all SPTEs in the page table and thus write all bytes - As an optimization, bail early when trying to unsync 4KiB mappings if the target gfn can just be mapped with a 2MiB hugepage x86 generic: - Copy single-chunk MMIO write values into struct kvm_vcpu (more precisely struct kvm_mmio_fragment) to fix use-after-free stack bugs where KVM would dereference stack pointer after an exit to userspace - Clean up and comment the emulated MMIO code to try to make it easier to maintain (not necessarily "easy", but "easier") - Move VMXON+VMXOFF and EFER.SVME toggling out of KVM (not *all* of VMX and SVM enabling) as it is needed for trusted I/O - Advertise support for AVX512 Bit Matrix Multiply (BMM) instructions - Immediately fail the build if a required #define is missing in one of KVM's headers that is included multiple times - Reject SET_GUEST_DEBUG with -EBUSY if there's an already injected exception, mostly to prevent syzkaller from abusing the uAPI to trigger WARNs, but also because it can help prevent userspace from unintentionally crashing the VM - Exempt SMM from CPUID faulting on Intel, as per the spec - Misc hardening and cleanup changes x86 (AMD): - Fix and optimize IRQ window inhibit handling for AVIC; make it per-vCPU so that KVM doesn't prematurely re-enable AVIC if multiple vCPUs have to-be-injected IRQs - Clean up and optimize the OSVW handling, avoiding a bug in which KVM would overwrite state when enabling virtualization on multiple CPUs in parallel. This should not be a problem because OSVW should usually be the same for all CPUs - Drop a WARN in KVM_MEMORY_ENCRYPT_REG_REGION where KVM complains about a "too large" size based purely on user input - Clean up and harden the pinning code for KVM_MEMORY_ENCRYPT_REG_REGION - Disallow synchronizing a VMSA of an already-launched/encrypted vCPU, as doing so for an SNP guest will crash the host due to an RMP violation page fault - Overhaul KVM's APIs for detecting SEV+ guests so that VM-scoped queries are required to hold kvm->lock, and enforce it by lockdep. Fix various bugs where sev_guest() was not ensured to be stable for the whole duration of a function or ioctl - Convert a pile of kvm->lock SEV code to guard() - Play nicer with userspace that does not enable KVM_CAP_EXCEPTION_PAYLOAD, for which KVM needs to set CR2 and DR6 as a response to ioctls such as KVM_GET_VCPU_EVENTS (even if the payload would end up in EXITINFO2 rather than CR2, for example). Only set CR2 and DR6 when consumption of the payload is imminent, but on the other hand force delivery of the payload in all paths where userspace retrieves CR2 or DR6 - Use vcpu->arch.cr2 when updating vmcb12's CR2 on nested #VMEXIT instead of vmcb02->save.cr2. The value is out of sync after a save/restore or after a #PF is injected into L2 - Fix a class of nSVM bugs where some fields written by the CPU are not synchronized from vmcb02 to cached vmcb12 after VMRUN, and so are not up-to-date when saved by KVM_GET_NESTED_STATE - Fix a class of bugs where the ordering between KVM_SET_NESTED_STATE and KVM_SET_{S}REGS could cause vmcb02 to be incorrectly initialized after save+restore - Add a variety of missing nSVM consistency checks - Fix several bugs where KVM failed to correctly update VMCB fields on nested #VMEXIT - Fix several bugs where KVM failed to correctly synthesize #UD or #GP for SVM-related instructions - Add support for save+restore of virtualized LBRs (on SVM) - Refactor various helpers and macros to improve clarity and (hopefully) make the code easier to maintain - Aggressively sanitize fields when copying from vmcb12, to guard against unintentionally allowing L1 to utilize yet-to-be-defined features - Fix several bugs where KVM botched rAX legality checks when emulating SVM instructions. There are remaining issues in that KVM doesn't handle size prefix overrides for 64-bit guests - Fail emulation of VMRUN/VMLOAD/VMSAVE if mapping vmcb12 fails instead of somewhat arbitrarily synthesizing #GP (i.e. don't double down on AMD's architectural but sketchy behavior of generating #GP for "unsupported" addresses) - Cache all used vmcb12 fields to further harden against TOCTOU bugs x86 (Intel): - Drop obsolete branch hint prefixes from the VMX instruction macros - Use ASM_INPUT_RM() in __vmcs_writel() to coerce clang into using a register input when appropriate - Code cleanups guest_memfd: - Don't mark guest_memfd folios as accessed, as guest_memfd doesn't support reclaim, the memory is unevictable, and there is no storage to write back to LoongArch selftests: - Add KVM PMU test cases s390 selftests: - Enable more memory selftests x86 selftests: - Add support for Hygon CPUs in KVM selftests - Fix a bug in the MSR test where it would get false failures on AMD/Hygon CPUs with exactly one of RDPID or RDTSCP - Add an MADV_COLLAPSE testcase for guest_memfd as a regression test for a bug where the kernel would attempt to collapse guest_memfd folios against KVM's will" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (373 commits) KVM: x86: use inlines instead of macros for is_sev_*guest x86/virt: Treat SVM as unsupported when running as an SEV+ guest KVM: SEV: Goto an existing error label if charging misc_cg for an ASID fails KVM: SVM: Move lock-protected allocation of SEV ASID into a separate helper KVM: SEV: use mutex guard in snp_handle_guest_req() KVM: SEV: use mutex guard in sev_mem_enc_unregister_region() KVM: SEV: use mutex guard in sev_mem_enc_ioctl() KVM: SEV: use mutex guard in snp_launch_update() KVM: SEV: Assert that kvm->lock is held when querying SEV+ support KVM: SEV: Document that checking for SEV+ guests when reclaiming memory is "safe" KVM: SEV: Hide "struct kvm_sev_info" behind CONFIG_KVM_AMD_SEV=y KVM: SEV: WARN on unhandled VM type when initializing VM KVM: LoongArch: selftests: Add PMU overflow interrupt test KVM: LoongArch: selftests: Add basic PMU event counting test KVM: LoongArch: selftests: Add cpucfg read/write helpers LoongArch: KVM: Add DMSINTC inject msi to vCPU LoongArch: KVM: Add DMSINTC device support LoongArch: KVM: Make vcpu_is_preempted() as a macro rather than function LoongArch: KVM: Move host CSR_GSTAT save and restore in context switch LoongArch: KVM: Move host CSR_EENTRY save and restore in context switch ... |
||
|
Marc Zyngier
|
83a3980750 |
Merge branch kvm-arm64/pkvm-protected-guest into kvmarm-master/next
* kvm-arm64/pkvm-protected-guest: (41 commits) : . : pKVM support for protected guests, implementing the very long : awaited support for anonymous memory, as the elusive guestmem : has failed to deliver on its promises despite a multi-year : effort. Patches courtesy of Will Deacon. From the initial cover : letter: : : "[...] this patch series implements support for protected guest : memory with pKVM, where pages are unmapped from the host as they are : faulted into the guest and can be shared back from the guest using pKVM : hypercalls. Protected guests are created using a new machine type : identifier and can be booted to a shell using the kvmtool patches : available at [2], which finally means that we are able to test the pVM : logic in pKVM. Since this is an incremental step towards full isolation : from the host (for example, the CPU register state and DMA accesses are : not yet isolated), creating a pVM requires a developer Kconfig option to : be enabled in addition to booting with 'kvm-arm.mode=protected' and : results in a kernel taint." : . KVM: arm64: Don't hold 'vm_table_lock' across guest page reclaim KVM: arm64: Allow get_pkvm_hyp_vm() to take a reference to a dying VM KVM: arm64: Prevent teardown finalisation of referenced 'hyp_vm' drivers/virt: pkvm: Add Kconfig dependency on DMA_RESTRICTED_POOL KVM: arm64: Rename PKVM_PAGE_STATE_MASK KVM: arm64: Extend pKVM page ownership selftests to cover guest hvcs KVM: arm64: Extend pKVM page ownership selftests to cover forced reclaim KVM: arm64: Register 'selftest_vm' in the VM table KVM: arm64: Extend pKVM page ownership selftests to cover guest donation KVM: arm64: Add some initial documentation for pKVM KVM: arm64: Allow userspace to create protected VMs when pKVM is enabled KVM: arm64: Implement the MEM_UNSHARE hypercall for protected VMs KVM: arm64: Implement the MEM_SHARE hypercall for protected VMs KVM: arm64: Add hvc handler at EL2 for hypercalls from protected VMs KVM: arm64: Return -EFAULT from VCPU_RUN on access to a poisoned pte KVM: arm64: Reclaim faulting page from pKVM in spurious fault handler KVM: arm64: Introduce hypercall to force reclaim of a protected page KVM: arm64: Annotate guest donations with handle and gfn in host stage-2 KVM: arm64: Change 'pkvm_handle_t' to u16 KVM: arm64: Introduce host_stage2_set_owner_metadata_locked() ... Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Marc Zyngier
|
f8078d51ee |
Merge branch kvm-arm64/vgic-v5-ppi into kvmarm-master/next
* kvm-arm64/vgic-v5-ppi: (40 commits) : . : Add initial GICv5 support for KVM guests, only adding PPI support : for the time being. Patches courtesy of Sascha Bischoff. : : From the cover letter: : : "This is v7 of the patch series to add the virtual GICv5 [1] device : (vgic_v5). Only PPIs are supported by this initial series, and the : vgic_v5 implementation is restricted to the CPU interface, : only. Further patch series are to follow in due course, and will add : support for SPIs, LPIs, the GICv5 IRS, and the GICv5 ITS." : . KVM: arm64: selftests: Add no-vgic-v5 selftest KVM: arm64: selftests: Introduce a minimal GICv5 PPI selftest KVM: arm64: gic-v5: Communicate userspace-driveable PPIs via a UAPI Documentation: KVM: Introduce documentation for VGICv5 KVM: arm64: gic-v5: Probe for GICv5 device KVM: arm64: gic-v5: Set ICH_VCTLR_EL2.En on boot KVM: arm64: gic-v5: Introduce kvm_arm_vgic_v5_ops and register them KVM: arm64: gic-v5: Hide FEAT_GCIE from NV GICv5 guests KVM: arm64: gic: Hide GICv5 for protected guests KVM: arm64: gic-v5: Mandate architected PPI for PMU emulation on GICv5 KVM: arm64: gic-v5: Enlighten arch timer for GICv5 irqchip/gic-v5: Introduce minimal irq_set_type() for PPIs KVM: arm64: gic-v5: Initialise ID and priority bits when resetting vcpu KVM: arm64: gic-v5: Create and initialise vgic_v5 KVM: arm64: gic-v5: Support GICv5 interrupts with KVM_IRQ_LINE KVM: arm64: gic-v5: Implement direct injection of PPIs KVM: arm64: Introduce set_direct_injection irq_op KVM: arm64: gic-v5: Trap and mask guest ICC_PPI_ENABLERx_EL1 writes KVM: arm64: gic-v5: Check for pending PPIs KVM: arm64: gic-v5: Clear TWI if single task running ... Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
5bae7bc636 |
KVM: arm64: Rename PKVM_PAGE_STATE_MASK
Rename PKVM_PAGE_STATE_MASK to PKVM_PAGE_STATE_VMEMMAP_MASK to make it clear that the mask applies to the page state recorded in the entries of the 'hyp_vmemmap', rather than page states stored elsewhere (e.g. in the ptes). Suggested-by: Alexandru Elisei <alexandru.elisei@arm.com> Reviewed-by: Fuad Tabba <tabba@google.com> Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-38-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
8972a99160 |
KVM: arm64: Register 'selftest_vm' in the VM table
In preparation for extending the pKVM page ownership selftests to cover forceful reclaim of donated pages, rework the creation of the 'selftest_vm' so that it is registered in the VM table while the tests are running. Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-35-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
246c976c37 |
KVM: arm64: Implement the MEM_UNSHARE hypercall for protected VMs
Implement the ARM_SMCCC_KVM_FUNC_MEM_UNSHARE hypercall to allow protected VMs to unshare memory that was previously shared with the host using the ARM_SMCCC_KVM_FUNC_MEM_SHARE hypercall. Reviewed-by: Vincent Donnefort <vdonnefort@google.com> Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-31-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
03313efed5 |
KVM: arm64: Implement the MEM_SHARE hypercall for protected VMs
Implement the ARM_SMCCC_KVM_FUNC_MEM_SHARE hypercall to allow protected VMs to share memory (e.g. the swiotlb bounce buffers) back to the host. Reviewed-by: Vincent Donnefort <vdonnefort@google.com> Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-30-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
94c5250515 |
KVM: arm64: Add hvc handler at EL2 for hypercalls from protected VMs
Add a hypercall handler at EL2 for hypercalls originating from protected VMs. For now, this implements only the FEATURES and MEMINFO calls, but subsequent patches will implement the SHARE and UNSHARE functions necessary for virtio. Unhandled hypercalls (including PSCI) are passed back to the host. Reviewed-by: Vincent Donnefort <vdonnefort@google.com> Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-29-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
5991916392 |
KVM: arm64: Return -EFAULT from VCPU_RUN on access to a poisoned pte
If a protected vCPU faults on an IPA which appears to be mapped, query the hypervisor to determine whether or not the faulting pte has been poisoned by a forceful reclaim. If the pte has been poisoned, return -EFAULT back to userspace rather than retrying the instruction forever. Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-28-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
56080f53a6 |
KVM: arm64: Introduce hypercall to force reclaim of a protected page
Introduce a new hypercall, __pkvm_force_reclaim_guest_page(), to allow the host to forcefully reclaim a physical page that was previous donated to a protected guest. This results in the page being zeroed and the previous guest mapping being poisoned so that new pages cannot be subsequently donated at the same IPA. Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-26-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
be9ed3529e |
KVM: arm64: Support translation faults in inject_host_exception()
Extend inject_host_exception() to support the injection of translation faults on both the data and instruction side to 32-bit and 64-bit EL0 as well as 64-bit EL1. This will be used in a subsequent patch when resolving an unhandled host stage-2 abort. Cc: Fuad Tabba <tabba@google.com> Reviewed-by: Fuad Tabba <tabba@google.com> Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-19-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
0bf5f4d400 |
KVM: arm64: Introduce __pkvm_reclaim_dying_guest_page()
To enable reclaim of pages from a protected VM during teardown, introduce a new hypercall to reclaim a single page from a protected guest that is in the dying state. Since the EL2 code is non-preemptible, the new hypercall deliberately acts on a single page at a time so as to allow EL1 to reschedule frequently during the teardown operation. Reviewed-by: Vincent Donnefort <vdonnefort@google.com> Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Co-developed-by: Quentin Perret <qperret@google.com> Signed-off-by: Quentin Perret <qperret@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-16-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
1e579adca1 |
KVM: arm64: Introduce __pkvm_host_donate_guest()
In preparation for supporting protected VMs, whose memory pages are isolated from the host, introduce a new pKVM hypercall to allow the donation of pages to a guest. Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-13-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
6c58f914eb |
KVM: arm64: Split teardown hypercall into two phases
In preparation for reclaiming protected guest VM pages from the host during teardown, split the current 'pkvm_teardown_vm' hypercall into separate 'start' and 'finalise' calls. The 'pkvm_start_teardown_vm' hypercall puts the VM into a new 'is_dying' state, which is a point of no return past which no vCPU of the pVM is allowed to run any more. Once in this new state, 'pkvm_finalize_teardown_vm' can be used to reclaim meta-data and page-table pages from the VM. A subsequent patch will add support for reclaiming the individual guest memory pages. Reviewed-by: Fuad Tabba <tabba@google.com> Tested-by: Fuad Tabba <tabba@google.com> Tested-by: Mostafa Saleh <smostafa@google.com> Co-developed-by: Quentin Perret <qperret@google.com> Signed-off-by: Quentin Perret <qperret@google.com> Signed-off-by: Will Deacon <will@kernel.org> Link: https://patch.msgid.link/20260330144841.26181-12-will@kernel.org Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Will Deacon
|
660b208e8b |
KVM: arm64: Remove unused PKVM_ID_FFA definition
Commit
|
||
|
Ben Horgan
|
eda1cd1f9d |
KVM: arm64: Preserve host MPAM configuration when changing traps
When KVM enables or disables MPAM traps to EL2 it clears all other bits in MPAM2_EL2. Notably, it clears the partition ids (PARTIDs) and performance monitoring groups (PMGs). Avoid changing these bits in anticipation of adding support for MPAM in the kernel. Otherwise, on a VHE system with the host running at EL2 where MPAM2_EL2 and MPAM1_EL1 access the same register, any attempt to use MPAM to monitor or partition resources for kernel space would be foiled by running a KVM guest. Additionally, MPAM2_EL2.EnMPAMSM is always set to 0 which causes MPAMSM_EL1 to always trap. Keep EnMPAMSM set to 1 when not in a guest so that the kernel can use MPAMSM_EL1. Tested-by: Gavin Shan <gshan@redhat.com> Tested-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com> Tested-by: Peter Newman <peternewman@google.com> Tested-by: Zeng Heng <zengheng4@huawei.com> Tested-by: Punit Agrawal <punit.agrawal@oss.qualcomm.com> Tested-by: Jesse Chick <jessechick@os.amperecomputing.com> Reviewed-by: Zeng Heng <zengheng4@huawei.com> Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Gavin Shan <gshan@redhat.com> Acked-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Ben Horgan <ben.horgan@arm.com> Signed-off-by: James Morse <james.morse@arm.com> |
||
|
Sascha Bischoff
|
9d6d9514c0 |
KVM: arm64: gic-v5: Support GICv5 FGTs & FGUs
Extend the existing FGT/FGU infrastructure to include the GICv5 trap registers (ICH_HFGRTR_EL2, ICH_HFGWTR_EL2, ICH_HFGITR_EL2). This involves mapping the trap registers and their bits to the corresponding feature that introduces them (FEAT_GCIE for all, in this case), and mapping each trap bit to the system register/instruction controlled by it. As of this change, none of the GICv5 instructions or register accesses are being trapped. Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Link: https://patch.msgid.link/20260319154937.3619520-14-sascha.bischoff@arm.com Signed-off-by: Marc Zyngier <maz@kernel.org> |
||
|
Vincent Donnefort
|
696dfec22b |
KVM: arm64: Add hyp_enter/hyp_exit events to nVHE/pKVM hyp
The hyp_enter and hyp_exit events are logged by the hypervisor any time it is entered and exited. Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Link: https://patch.msgid.link/20260309162516.2623589-29-vdonnefort@google.com Signed-off-by: Marc Zyngier <maz@kernel.org> |