The user cannot use MTE on VMAs created by mapping a guest_memfd file,
as arch_calc_vm_flag_bits() does not set VM_MTE_ALLOWED.
When creating a guest_memfd backed memslot,
kvm_arch_prepare_memory_region() rejects the memslot if MTE is enabled for
the VM and if guest_memfd has been mapped in a VMA that intersects the
memslot.
However, the documentation for KVM_SET_USER_MEMORY_REGION2 explicitly
states that the only condition for userspace_addr is for it to be a legal
userspace address, but the mapping is not required to be valid nor
populated at memslot creation.
If userspace sets userspace_addr to an address that hasn't been mapped, or
if userspace_addr belongs to a VMA that isn't backed by the guest_memfd
file, or if the VMA doesn't intersect the memslot, memslot creation is
successful and KVM ends up with a VM with MTE and guest_memfd-backed
memslots.
The same happens if the order is reversed: when userspace enables MTE, KVM
does not check if memslots backed by guest_memfd are already present.
Fix both issues by rejecting guest_memfd-backed memslots when MTE is
enabled, and by rejecting MTE when guest_memfd-backed memslots are already
present.
Fixes: 32e200bd6e ("KVM: arm64: Enable support for guest_memfd backed memory")
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com>
Link: https://patch.msgid.link/20260722090354.94245-1-alexandru.elisei@arm.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Add a missing hypervisor event call for hyp_enter on sysreg trapping,
causing an unbalanced hyp_enter/hyp_exit.
The enum hyp_enter_exit_reason is not ABI, so we can keep the ERET
reasons at the end for clarity.
Fixes: 696dfec22b ("KVM: arm64: Add hyp_enter/hyp_exit events to nVHE/pKVM hyp")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260617095238.1530121-1-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
The footprint calculated for struct hyp_trace_desc sizes only
trace_buffer_desc and do not take into account the other fields. It
worked so far thanks to the follow-up PAGE_ALIGN().
Fix the descriptor size and while at it, enforce an overflow check after
PAGE_ALIGN().
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 3aed038aac ("KVM: arm64: Add trace remote for the nVHE/pKVM hyp")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260710114819.2689386-3-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
In the very unlikely event of a failure in __map_hyp, the allocated
backing pages are leaked in hyp_trace_buffer_alloc_bpages_backing(). Fix
this by freeing the pages on error.
Fixes: 3aed038aac ("KVM: arm64: Add trace remote for the nVHE/pKVM hyp")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Link: https://patch.msgid.link/20260710114819.2689386-2-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Fix the disable path in hyp_trace_clock_enable(), which fell through to
re-initialize and reschedule the clock after cancelling the work. Return
early instead.
While at it, cleanup hyp_trace_clock::lock which is unused and
hyp_trace_clock::running which is redundant: the trace_remote framework
already serializes calls to the callback enable_tracing.
Fixes: b22888917f ("KVM: arm64: Sync boot clock with the nVHE/pKVM hyp")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev> (✓ DKIM/linux.dev)
Link: https://patch.msgid.link/20260715105100.3178255-1-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Mitigate a potential failure when inserting a new LPI into the VGIC LPI
xarray.
When vgic_add_lpi() is preparing to register a new LPI, it pre-allocates
an xarray entry using xa_reserve_irq(), so that it can later perform the
insertion under the xarray lock without allocating.
However, since xa_reserve_irq() is called before acquiring such lock,
there is a potential race where xa_reserve_irq() observes a populated
entry, thus not performing the allocation, and another CPU removes that
entry before the xarray lock is grabbed to perform the insertion.
CPU0 (Adding new LPI) CPU1 (Releasing LPI)
===================== ===================
vgic_add_lpi()
/* Entry populated, does not allocate */
xa_reserve_irq(.., intid, ..)
vgic_release_deleted_lpis()
xa_lock_irqsave()
vgic_release_lpi_locked()
xarray node freed --> __xa_erase(.., intid)
xa_unlock_irqrestore()
xa_lock_irqsave()
xa_load(.., intid) == NULL
vgic_try_get_irq_ref(NULL) == false
__xa_store(.., intid, irq, 0) <-- xarray node was freed, gfp=0
cannot allocate, returns -ENOMEM
This can happen e.g. if the guest issues a DISCARD while the LPI is
still referenced from a vCPU's active-pending list (ap_list), and the
same INTID is re-mapped via MAPTI.
Mitigate this by passing GFP_NOWAIT to __xa_store(), so that the
allocation can happen under the lock in the rare case that this
condition is hit. Add __GFP_ACCOUNT as well to match xa_reserve_irq()'s
flags.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 1d6f83f60f ("KVM: arm64: vgic: Store LPIs in an xarray")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260715105137.3973823-5-clopez@suse.de
Signed-off-by: Marc Zyngier <maz@kernel.org>
Fix a potential race between decrementing an LPI's reference count and
evicting that structure from the LPI xarray.
LPI structures are maintained in the VGIC LPI xarray (dist->lpi_xa).
When the reference count of an LPI structure drops to zero,
vgic_release_lpi_locked() removes the structure from the xarray and
frees it under the xarray lock.
However, the release of an LPI can race with a concurrent LPI
re-registration with the same INTID via vgic_add_lpi() on another CPU,
since the reference count drop and the xarray eviction are not performed
in a single atomic step. This can happen e.g. if the guest issues a
DISCARD while the LPI is still referenced from a vCPU's active-pending
list (ap_list), and the same INTID is re-mapped via MAPTI.
Particularly, vgic_release_lpi_locked() is called from two distinct
paths: direct release via vgic_put_irq(), and deferred release via
vgic_release_deleted_lpis(). During direct release, the issue can result
in deleting a newly registered LPI from the xarray:
CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq()
__vgic_put_irq()
refcount_dec_and_test()
vgic_add_lpi()
xa_lock_irqsave()
old_irq = xa_load(.., intid)
vgic_try_get_irq_ref(old_irq) == false
new IRQ inserted --> __xa_store(.., intid, ..)
xa_unlock_irqrestore()
xa_lock_irqsave();
vgic_release_lpi_locked()
__xa_erase(.., irq->intid) <-- BUG: new IRQ is erased
kfree_rcu(old_irq)
During the deferred release path, the old IRQ can be leaked:
CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq_norelease()
__vgic_put_irq()
refcount_dec_and_test()
irq->pending_release = true
vgic_add_lpi()
xa_lock_irqsave()
old_irq = xa_load(.., intid)
vgic_try_get_irq_ref(oldirq) == false
BUG: old IRQ overwritten --> __xa_store(.., intid, ..)
xa_unlock_irqrestore()
vgic_release_deleted_lpis()
xa_lock_irqsave()
xa_for_each() { .. } <-- old IRQ with pending_release = true
is gone, so it cannot be released
To fix the direct release path, move the reference count drop inside
the xarray lock, making sure that vgic_add_lpi() never encounters the
to-be-released LPI.
In the deferred release path, the refcount drop must happen under a raw
spinlock, so the xarray lock cannot be grabbed, and the same solution
does not work. Instead, update vgic_add_lpi(), so that if it evicts
an LPI from the xarray, it takes on the responsibility of freeing it.
Consequently, an LPI may now be freed concurrently after a deferred
release drops the refcount, so accessing the pending_release field is no
longer safe from use-after-free. Delete all uses of the flag, and update
vgic_release_deleted_lpis() to identify orphaned LPIs purely based on
their refcount.
Reported-by: Claude:claude-opus-4-6
Fixes: 3a08a6ca7c ("KVM: arm64: vgic-v3: Use bare refcount for VGIC LPIs")
Fixes: d54594accf ("KVM: arm64: vgic-v3: Erase LPIs from xarray outside of raw spinlocks")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260715105137.3973823-4-clopez@suse.de
Signed-off-by: Marc Zyngier <maz@kernel.org>
In the nested state, the physical interrupt has already been
deactivated through the HW bit in the LR. The extra deactivation
would be harmless but can hit an errata case on AmpereOne, so
avoid it here.
On AmpereOne, deactivating a physical interrupt through
ICC_DIR_EL1 or ICC_EOIR1_EL1 (depending on EOImode) which is not
active, but is the highest priority pending interrupt causes the
cpu to lose the interrupt pending state and also prevents the
delivery of future interrupts.
Fixes: 6dd333c894 ("KVM: arm64: GICv3: nv: Plug L1 LR sync into deactivation primitive")
Signed-off-by: D Scott Phillips <scott@os.amperecomputing.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/linux-arm-kernel/20260710222128.416581-1-scott@os.amperecomputing.com/
Link: https://patch.msgid.link/20260714231158.496808-1-scott@os.amperecomputing.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
- Move locking for kvm_io_bus_get_dev() into the caller, ensuring
race-free checks that the returned object is of the correct type
- Fix initialisation of the page-table walk level when relaxing
permissions
- Correctly update the XN attribute when relaxing permissions
- Fix the sign extension of loads from emulated MMIO regions
- Assorted collection of fixes for pKVM's FFA proxy, together with a
couple of FFA driver adjustments
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEn9UcU+C1Yxj9lZw9I9DQutE9ekMFAmpOLOIACgkQI9DQutE9
ekPH5Q//VrbqIVfJekxRoI0B8zWpnG52q9ILdZBEhTL+xHEP+Z8WawzbpfHzv9LQ
tt7rUk8gmkGBvMXdY3fZaLkaQRDGvg51SKclxIaoNdmrav7Sscz/sar7P8QWUuIb
d0EQ3hqV+2hvNFqUVy0cTAFoog0U00KDpdBV136ePTYgT7/N0B8fiFStdHzVP80H
0WbeWnB0LB15mnkCkO0QY4QmEozfnjKYPrhv52fZFSmnc0BLY3muSR0y+lZ1FtN0
c2aUiYH/iyIkmmpEZyMYzzhwnMkPwwTIof1OgQ93//JUPkmEUtun5SNQ9YaQw/cd
/IKDo5xhEn/3jH6Zo2th8oKhH4q5mSOxBd8WH/hkwnTpZYbUZ5Q0647QHHEtWWEp
mEdQFroaORrPXtHfKSeltkr5JKWK9vweARwbTcLmNwbGfLqZGgJe8/FgOw8Q3lXB
wZcmNc5TqGe4QWiatH7qKTdqulWdQMrRE2XRAinfrfOfZMg69cnNizWAdNPVhPfw
B5yewkPrPwZNNXGksZOXfbFN7TnIM2M8j7uwXl5D3rGi/ebDp5ByuVKkP5ZNLnz3
D666cORhX1zdCfqbLx4mhPs+vcqmL8HHMASlaJg+hH1se/veLbkEZ9GIrjw8Y0iO
SMUJvwJ5Z2SZpvCKGsC85Wdm/pIBLLh5ISIW8wDIIw+7BoMrXFY=
=RmEk
-----END PGP SIGNATURE-----
Merge tag 'kvmarm-fixes-7.2-2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 fixes for 7.2, take #2
- Move locking for kvm_io_bus_get_dev() into the caller, ensuring
race-free checks that the returned object is of the correct type
- Fix initialisation of the page-table walk level when relaxing
permissions
- Correctly update the XN attribute when relaxing permissions
- Fix the sign extension of loads from emulated MMIO regions
- Assorted collection of fixes for pKVM's FFA proxy, together with a
couple of FFA driver adjustments
- Fix an accounting buglet when reclaiming pages from a protected
guest
- Fix a bunch of architectural compliance issues when injecting a
synthesised exception, most of which were missing the PSTATE.IL bit
indicating a 32bit-wide instruction
- Another set of fixes addressing issues with translation of VNCR_EL2,
including corner cases where the guest point that register at a RO
page...
- Don't warn when trapping accesses to ZCR_EL2 from an L2 guest, as
that's not unexpected at all
- Address a bunch of races with LPI migration vs LPIs being disabled
- Fix a total howler of a bug combining FEAT_MOPS and NV, resulting in
exception returning in the wrong place...
- Coerce Fuad Tabba into a reviewer role, and may his Inbox catch
fire!
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEn9UcU+C1Yxj9lZw9I9DQutE9ekMFAmo+fUoACgkQI9DQutE9
ekOgBA/8D1XhSpPjYu7NDBOf6+62n02BRhihlFgA3KQkFmiVwUoeiAGXfDyZqQiM
ms1LfUmU+7Ry7zoeasHzVi5k/FEk+gMR911eEcciJF7C6BbC3uHUOgEg30DWQo9J
bb23PgeAcYnAqgjRw8vcXsvArK6YcY9Y6Qvp3SA1fM8Ls62P0+OdxCE/HM4TE1Ny
1t1vokNHaJCAkf0h6gAI12yEZGhK2bFZsVnkOYgkIuqKI5730/B2hbjqK+hcMPHL
TO7tVeRk8qQ0ZFJq3hpnOcRjO6uLA/f/6IHkYej/ADxaqH2uzOQeKY3awoS1UtNJ
okRKS3OQjFXW8nsarVRIzGDMp41EjydVB8po6HXWg1zhiD6PEH92W+rnVPq273xh
gKi0KOMchgBr+61zeZU/wdPZiU/wnMTxukcpqyCSR5WdoOxwMpGZvdCwHzGOyg7v
8QebZzkAxSB854aaHf5JjdTH9zI/HOk184fxtdgYbxGKMKaZ/wdTM7kos1bBHt8t
BP/yu7/jLBxEvrO9iLBHYzlk40i1oqyM4sZpM2Z2w8MBbOaHni1xm1URwFwTX5aM
FJ8BxObnnUcB8ARzRHafPcLwgyC0LubEVYsbddppijEhBu8f/wF/sMcIoHNppVCR
Vp9KK9+BH6OyYbG7TsKj2zy2S8KqgVonehoDrr66cPySo0l9KWY=
=mmLT
-----END PGP SIGNATURE-----
Merge tag 'kvmarm-fixes-7.2-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 fixes for 7.2, take #1
- Fix an accounting buglet when reclaiming pages from a protected
guest
- Fix a bunch of architectural compliance issues when injecting a
synthesised exception, most of which were missing the PSTATE.IL bit
indicating a 32bit-wide instruction
- Another set of fixes addressing issues with translation of VNCR_EL2,
including corner cases where the guest point that register at a RO
page...
- Don't warn when trapping accesses to ZCR_EL2 from an L2 guest, as
that's not unexpected at all
- Address a bunch of races with LPI migration vs LPIs being disabled
- Fix a total howler of a bug combining FEAT_MOPS and NV, resulting in
exception returning in the wrong place...
- Coerce Fuad Tabba into a reviewer role, and may his Inbox catch
fire!
Assigning the invalidation level (an s8 value) with TLBI_TTL_UNKNOWN
(a 32bit signed value) is not ideal, to say the least. Instead of
this, only pass TLBI_TTL_UNKNOWN to __kvm_tlb_flush_vmid_ipa_nsh()
when we know for sure that we don't have a provided level.
Fixes: 100baf0184 ("KVM: arm64: Ensure level is always initialized when relaxing perms")
Reported-by: Mark Brown <broonie@kernel.org>
Reviewed-by: Oliver Upton <oupton@kernel.org>
Link: https://lore.kernel.org/r/akztC7H2IsEKaq4i@sirena.org.uk
Link: https://patch.msgid.link/20260707162935.1900874-1-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
Harden the check for the constituent memory region page alignment
to prevent over-sharing when the negotiated FFA_PAGE_SIZE size is
smaller than the system PAGE_SIZE.
At the moment we only check that the size of the range is page
aligned, and truncate the address to the page boundary which can
annotate more memory than needed as being used by the FF-A.
Fixes: 4360900017 ("KVM: arm64: Handle FFA_MEM_SHARE calls from the host")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Link: https://patch.msgid.link/20260702103848.1647249-6-sebastianene@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Prevent the pKVM hypervisor from making assumptions that the
endpoint memory access descriptor (EMAD) comes right after the
FF-A memory region header.
Prior to FF-A version 1.1 the header of the memory region
didn't contain an offset to the endpoint memory access descriptor.
The layout of a memory transaction looks like this from 1.1 onward:
Type | Field name | Offset
[ Header | ffa_mem_region | 0
EMAD 1 | ffa_mem_region_attributes) | ffa_mem_region.ep_mem_offset
]
Verify that the offset to the first endpoint memory access descriptor
is within the mailbox buffer bounds.
Also, fix one hardcoded sizeof(struct ffa_mem_region_attributes) that
should be replaced ffa_emad_size_get() for compatibility with FFA v1.0.
Fixes: 42fb33dde4 ("KVM: arm64: Use FF-A 1.1 with pKVM")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Link: https://patch.msgid.link/20260702103848.1647249-5-sebastianene@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Sashiko (locally) reports out of bound write possiblity if SPMD
returns an invalid data.
While SPMD is considered trusted, pKVM does some basic checks,
for offset to be less than or equal len.
However, that is incorrect as even if the offset is smaller than
len pKVM can still access out of bound memory in the next
ffa_host_unshare_ranges().
Split this check into 2:
1- Check that the fixed portion of the descriptor fits.
2- After getting reg, check the variable array size addr_range_cnt
fits.
Also, drop the WARN_ONs as that will panic the kernel and in the
next checks there are no WARNs, so that makes it consistent.
Fixes: 0a9f15fd56 ("KVM: arm64: pkvm: Add support for fragmented FF-A descriptors")
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Sebastian Ene <sebastianene@google.com>
Link: https://patch.msgid.link/20260702103848.1647249-4-sebastianene@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
A sign-extending load (LDRSB, LDRSH, LDRSW) from MMIO returns a
zero-extended value to the guest. The architecture performs such a load
as a memory read of the access size, then a sign-extension to the
register width. For LDRSH (DDI 0487 M.b C6.2.225, with the Mem accessor
at J1.2.3.111):
data = Mem{16}(address, accdesc);
X{regsize}(t) = SignExtend{regsize}(data);
The byte order is handled inside the Mem accessor, keyed on the access
size; the register width is separate, applied afterwards by SignExtend().
kvm_handle_mmio_return() runs these in the wrong order: it sign-extends
the access-width data, then calls vcpu_data_host_to_guest(), which masks
the value back to the access width (the size-keyed byte-order step). The
mask drops the sign bits that sign-extension produced.
Reorder so vcpu_data_host_to_guest() runs first, with the sign-extension
to register width after it. trace_kvm_mmio() moves with it and now logs
the access-width data before sign-extension.
Fixes: b30070862e ("ARM64: KVM: MMIO support BE host running LE code")
Reviewed-by: Oliver Upton <oupton@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260706115522.954913-2-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
On systems without DIC, KVM lazily grants execute permission to stage-2
translations after taking an instruction abort due to a permission
fault, allowing it to defer I-cache invalidations to the point they're
absolutely required.
If a data abort happens later down the line to such a translation, KVM
will not request execute permissions as part of the S2 relaxation on the
assumption that kvm_pgtable_stage2_relax_perms() does exactly what the
name implies and adds the requested permissions to the pre-existing
ones.
Avoid taking unintended execute permission faults by only preparing the
XN attribute if KVM_PGTABLE_PROT_X is set.
Fixes: 2608563b46 ("KVM: arm64: Add support for FEAT_XNX stage-2 permissions")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Reviewed-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://patch.msgid.link/20260701231620.3300204-3-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
stage2_update_leaf_attrs() returns early before writing to @level if the
table walker returned an error. At the same time,
kvm_pgtable_stage2_relax_perms() uses the level as a TLBI TTL hint when the
error was EAGAIN, indicating the vCPU raced with a table update and the TLB
entry it hit is now stale.
Fall back to an unknown TTL if none was provided by the walk.
Cc: stable@vger.kernel.org
Fixes: be097997a2 ("KVM: arm64: Always invalidate TLB for stage-2 permission faults")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Reviewed-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://patch.msgid.link/20260701231620.3300204-2-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
kvm_io_bus_get_dev() returns a device that is only matched by the
address, and nothing else. This can cause a lifetime issue if
the matched device is not the expected type, as by the time
the caller can introspect the object, it might be gone (the srcu
lock having been dropped).
Given that there is only a single user of this helper, the simplest
option is to move the locking responsibility to the caller, which
can keep the srcu lock held for as long as it wants.
Note that this aligns with other kvm_io_bus*() helpers, which
already require the srcu lock to be held by the callers.
Reported-by: Will Deacon <will@kernel.org>
Fixes: 8a39d00670 ("KVM: kvm_io_bus: Add kvm_io_bus_get_dev() call")
Link: https://lore.kernel.org/all/20260626111344.802555-1-maz@kernel.org
Cc: stable@vger.kernel.org
Reviewed-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260627105105.1005990-1-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
Protected guest faults charge long term pins to the VM's mm. Teardown
can run later from file release, where current->mm may be unrelated.
Drop the charge from kvm->mm instead.
Fixes: 4e6e03f9ea ("KVM: arm64: Hook up reclaim hypercall to pkvm_pgtable_stage2_destroy()")
Signed-off-by: Bradley Morgan <include@grrlz.net>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260621213155.6019-1-include@grrlz.net
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
__pkvm_memshare_page_req() constructs a fake DABT ESR_EL2 to exit to
the host without setting IL. The ESR has ISV=0, so IL must be 1 per the
architecture. The host does not read IL on this path, but the
constructed syndrome should still be architecturally valid.
Set ESR_ELx_IL.
Fixes: 03313efed5 ("KVM: arm64: Implement the MEM_SHARE hypercall for protected VMs")
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260618121643.4105064-8-tabba@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
The FPAC syndrome constructed during nested ERET emulation does not set
IL. For FPAC (EC=0x1C), IL reflects the instruction length. ERET and
its authenticated variants are always A64 32-bit instructions, so IL
must be 1.
Fixes: 213b3d1ea1 ("KVM: arm64: nv: Handle ERETA[AB] instructions")
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260618121643.4105064-5-tabba@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
inject_abt64() derives IL from the triggering trap's instruction length
(kvm_vcpu_trap_il_is32bit()), but the IL of the injected abort is fixed
by its EC, not by the triggering instruction. The architecture mandates
IL=1 for Instruction Aborts unconditionally and for Data Aborts with
ISV=0, and this function never sets ISV (the FSC is always EXTABT or
SEA_TTW). For a 16-bit T32 trap (a 32-bit EL0 task under an AArch64 EL1
guest) the trap has IL=0, so the abort is injected with the wrong IL.
Set ESR_ELx_IL unconditionally.
Fixes: aa8eff9bfb ("arm64: KVM: fault injection into a guest")
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260618121643.4105064-4-tabba@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
inject_undef64() derives IL from the triggering trap's instruction
length (kvm_vcpu_trap_il_is32bit()), but the IL of the injected
exception is fixed by its EC, not by the triggering instruction. The
architecture mandates IL=1 for EC=0 (Unknown) unconditionally, so the
conditional is wrong. The undef-injection paths are not reached from
16-bit instructions, so there is no functional change today, but the
logic should not rely on that.
Set ESR_ELx_IL unconditionally.
Fixes: aa8eff9bfb ("arm64: KVM: fault injection into a guest")
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260618121643.4105064-3-tabba@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
inject_undef64() constructs an ESR with EC=0 (Unknown) but does not set
IL. The architecture mandates IL=1 for EC=0 unconditionally (ARM DDI
0487, ESR_ELx.IL description), so the injected syndrome is one that
conforming hardware cannot produce.
Set ESR_ELx_IL in the constructed syndrome.
Fixes: e5d40a5a97 ("KVM: arm64: pkvm: Add a generic synchronous exception injection primitive")
Reported-by: sashiko <sashiko@sashiko.dev>
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260618121643.4105064-2-tabba@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
KVM is unlikely to resolve an unexpected VNCR abort, meaning that
returning to the guest will likely leave the vCPU stuck in an abort
loop. Bug the VM and exit to userspace instead.
Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260618234207.1063941-6-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
When constructing an L1 VNCR mapping, KVM unconditionally uses cacheable
memory attributes, even if the underlying PFN isn't memory. This gets
particularly hairy if the endpoint doesn't support cacheable memory
attributes, potentially throwing an SError on writeback...
While KVM does permit cacheable memory attributes on certain PFNMAP
VMAs, kvm_translate_vncr() isn't currently grabbing the VMA. So do the
simpler thing for now and just reject everything that isn't memory.
Cc: stable@vger.kernel.org
Fixes: 2a359e0725 ("KVM: arm64: nv: Handle mapping of VNCR_EL2 at EL2")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260618234207.1063941-5-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
KVM faults in the VNCR page with FOLL_WRITE whenever the guest aborts
for a write, similar to how a regular stage-2 mapping is handled. It is
entirely possible that the guest reads from the VNCR before writing to
it, in which case the PFN could only be read-only.
Invalidate the VNCR TLB and re-fetch the translation upon taking a VNCR
abort, allowing the host mapping to be faulted in for write the second
time around. Interestingly enough, this also satisfies the ordering
requirements of FEAT_ETS2/3 between descriptor updates and MMU faults.
Cc: stable@vger.kernel.org
Fixes: 2a359e0725 ("KVM: arm64: nv: Handle mapping of VNCR_EL2 at EL2")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260618234207.1063941-4-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
kvm_handle_vncr_abort() assumes that s1_walk_result conveys an abort
when kvm_translate_vncr() returns -EFAULT. This is not always the case
as it's possible to encounter 'late' failures on the output of S1
translation, e.g. a GFN outside of the memslots.
Fix it by preparing an external abort before returning from
kvm_translate_vncr(). Get rid of the BUG_ON() in the fault injection
path while at it.
Cc: stable@vger.kernel.org
Fixes: 2a359e0725 ("KVM: arm64: nv: Handle mapping of VNCR_EL2 at EL2")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260618234207.1063941-3-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
KVM currently maps the L1 VNCR into the host stage-1 by relying entirely
on the permissions of the guest stage-1. At the same time, it is
entirely possible that the backing PFN is read-only (e.g. RO memslot),
meaning that the L1 VNCR should use at most a read-only mapping.
Cache the writability of the PFN in the VNCR TLB and use it to constrain
the resulting fixmap permissions. Promote VNCR permission faults to an
SEA in the case where the guest attempts to write to a read-only
endpoint. Conveniently, this also plugs a page leak found by Sashiko [*]
resulting from the early return for a read-only PFN.
Cc: stable@vger.kernel.org
Fixes: 2a359e0725 ("KVM: arm64: nv: Handle mapping of VNCR_EL2 at EL2")
Link: https://lore.kernel.org/kvm/20260608082603.16AEC1F00893@smtp.kernel.org/
Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260618234207.1063941-2-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
"This is a bit of an odd merge window on the KVM/arm64 front. There is
absolutely no new feature in the pull request. It is purely fixes,
because it is simply becoming too hard to review new stuff when so
many AI-fuelled fixes hit the list".
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX)
that caused the FPU load request.
- Some enhancements about interrupt injection.
- Some bug fixes and other small changes.
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic
PTE updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on
a memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes.
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace.
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root.
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both
of these are preparatory changes for upcoming APX support.
- Clean up KVM's register tracking and storage, primarily to prepare for
APX support, which expands the maximum number of GPRs from 16 to 32.
- Keep a single copy of the PDPTRs rather than two, since architecturally
there is just one.
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor code
gets a chance to handle things like reaping the PML buffer.
- Update KVM's view of PV async enabling if and only if the MSR write fully
succeeds.
- Fix a variety of issues where the emulator doesn't honor guest-debug state,
and clean up related code along the way.
- Synthesize EPT Violation and #NPF "error code" bits when injecting faults
into L1 that didn't originate in hardware (in which case the VMCS/VMCB
doesn't hold relevant information).
- Add support for virtualizing (well, emulating) AMD's flavor of CPL>0 CPUID
faulting.
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and fix a
variety of minor bugs along the way.
- Fix an OOB memory access due to not checking the VP ID when handling a
Hyper-V PV TLB flush for L2.
- Fix a bug in the mediated PMU's handling of fixed counters that allowed the
guest to bypass the PMU event filter.
- Allow userspace to return EAGAIN when handling SNP and TDX hypercalls, so
the KVM can forward a "retry" status code to the guest, and reserve all
unused error codes for future usage.
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in shadow page table handling, where KVM fails to recursively
zap nested EPT/NPT shadow page tables when the nested hypervisor tears
down its own EPT/NPT page tables from the bottom up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in the
generic section; also run with MBEC enabled even for non-nested mode.
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Fix a largely benign bug where KVM TDX would incorrectly state it could
emulate several x2APIC MSRs.
- Use the "safe" WRMSR API when proxying LBR MSR writes as the to-be-written
value is guest controlled and completely unvalidated.
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in the
generic section; also run with GMET enabled even for non-nested mode.
- Fixes and minor cleanups to GHCB handling, on top of the earlier work
already merged into 7.1-rc.
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers.
- Add support for virtualizing gPAT (KVM previously just used L1's PAT when
running L2).
- Fix goofs where KVM mishandles side effects (e.g. single-step and PMC
updates) when emulating VMRUN.
- Fix a variety of bugs in AVIC's handling of x2APIC MSR interception, most
notably where KVM didn't disable interception of IRR, ISR, and TMR regs.
- Add support for virtualizing Host-Only/Guest-Only bits in the mediated PMU.
- Don't advertise support for unusable VM types, and account for VM types
that are disabled by firmware, e.g. to mitigate security vulnerabilities.
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with bugs and
unnecessarily complicated, and add comprehensive tests.
- Clean up and deduplicate the SEV page pinning code.
- Fix minor goofs related to writing back CPUID information after firmware
rejects a CPUID page for an SNP vCPU.
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to follow
the kernel's nomenclature, e.g. for mmu_notifiers.
- Use guard() to cleanup up various KVM+VFIO flows.
- Minor cleanups.
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a gmem
range to multiple memslots, and fix the test that was supposed to ensure
KVM returns -EEXIST.
- Treat memslot binding offsets and sizes as unsigned values to fix a bug
where KVM interprets a large "offset + size" as a negative value and allows
a nonsensical offset.
- Use the inode number instead of the page offset for the NUMA interleaving
index to fix a bug where the effective index would jump by two for
consecutive pages (the caller also adds in the page offset).
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the first
pass, as always waiting only 1ms hid a KVM RISC-V bug as the test reaped the
bitmap before KVM could build up enough state to hit the bug.
- A pile of one-off fixes and cleanups.
-----BEGIN PGP SIGNATURE-----
iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmoypI0UHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroNCSAf8CyuvicojnZGFsNcoY4DZ0whBj5l8
IwVkHaptuyGLSwPAebC/DwtU2NBY4CQdbtLjGmxenf+fSQOtcz1YWKbdY4Nr0hJG
euAb/wO47RqKp1C4WPi7lL2P4XHgmUgD5XZ1qDnRV9gnGddZi3QLLmKgAtNq9Z4m
+pAdGD2/RyZeI0Dx61+5ufE/z8YBlH6GrzgBTxfvvRNKqNDFykBenxYHI7seEfHC
BPz0LxjHDD8BMDUo86qaylZnaeQSmFTGO/eTgvCvabxG6WbM1s0w3K1owmUmP7o3
p9o8274Xh53iK2JYD/qGOtAJXejIwbYKT5bnvUQX0O1EH2+ouvSFapLQ9Q==
=sSqM
-----END PGP SIGNATURE-----
Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
kvm_check_illegal_exception_return() sourced the flags {N,Z,C,V} and
masks {D,A,I,F} of the resulting PSTATE from the current PSTATE, but
R_VWJHB takes them from the SPSR being returned to and leaves
PSTATE.{EL,SP,nRW} (and EXLOCK when FEAT_GCS) unchanged. PAN, ALLINT
and PM were not applied at all.
Build the PSTATE by taking those fields from the SPSR while preserving
EL, SP, nRW and EXLOCK from the current PSTATE, then set IL.
Fixes: 47f3a2fc76 ("KVM: arm64: nv: Support virtual EL2 exceptions")
Suggested-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/all/86wlvxs5r0.wl-maz@kernel.org/
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260617144907.2972095-1-tabba@google.com
[maz: tidied things a bit]
Signed-off-by: Marc Zyngier <maz@kernel.org>
When pKVM injects a synchronous exception into a protected guest, it
re-enters without restoring the guest's EL1 sysregs and writes the EL1
exception registers to hardware by hand: ESR_EL1 and ELR_EL1, but not
SPSR_EL1. enter_exception64() sets SPSR_EL1 (the interrupted PSTATE)
only in memory, so the guest's handler reads a stale SPSR_EL1 and
restores the wrong PSTATE on eret.
Write SPSR_EL1 alongside the other exception registers.
Fixes: 6c30bfb18d ("KVM: arm64: Add handlers for protected VM System Registers")
Reported-by: sashiko <sashiko@sashiko.dev>
Signed-off-by: Fuad Tabba <tabba@google.com>
Acked-by: Will Deacon <will@kernel.org>
Link: https://patch.msgid.link/20260612113414.1022901-1-tabba@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
kvm_inject_el2_exception() writes ESR_EL2 for synchronous exceptions
but not for SError. enter_exception64() does not write ESR_ELx for any
exception type, so the constructed syndrome is dropped. A guest L2
hypervisor taking a nested SError observes stale ESR_EL2.
This affects both kvm_inject_nested_serror() and the EASE path in
kvm_inject_nested_sea().
Write ESR_EL2 for except_type_serror, matching except_type_sync.
Fixes: 77ee70a073 ("KVM: arm64: nv: Honor SError exception routing / masking")
Reported-by: sashiko <sashiko@sashiko.dev>
Signed-off-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20260615131116.390977-1-tabba@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
It is entirely possible for a guest to write to the ZCR_EL2 sysreg alias
while in a nested context, as it is expected if FEAT_NV2 is advertised
to the L1 hypervisor.
Get rid of the bogus WARN which, since the hyp vectors were installed at
this point, has the effect of a hyp_panic...
Cc: stable@vger.kernel.org
Fixes: 0cfc85b8f5 ("KVM: arm64: nv: Load guest FP state for ZCR_EL2 trap")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260615051324.830045-1-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
kvm_hyp_handle_mops() resets the single-step state machine as part of
rewinding state for a MOPS exception by modifying vcpu_cpsr() and
writing the result directly into hardware.
In the case of nested virtualization, vcpu_cpsr() is a synthetic value
such that the rest of KVM can deal with vEL2 cleanly. That means the
value requires translation before being written into hardware, which is
unfortunately missing from the MOPS handler.
Fix it by directly modifying SPSR_EL2 and avoiding the synthetic state
altogether, which will be resynchronized on the next 'full' exit back
to KVM.
Fixes: 2de451a329 ("KVM: arm64: Add handler for MOPS exceptions")
Reported-by: Zhong Wang <wangzhong.c0ss4ck@bytedance.com>
Reported-by: Xuanqing Shi <shixuanqing.11@bytedance.com>
Link: https://lore.kernel.org/all/ajE4lHQevXNHpl1M@Air.local/
Cc: stable@vger.kernel.org
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
Link: https://patch.msgid.link/20260617040820.2194831-2-bestswngs@gmail.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
vgic_prune_ap_list() drops both ap_list_lock and irq_lock while migrating
an interrupt to another vCPU. After reacquiring the locks it only checks
that the affinity is unchanged (target_vcpu == vgic_target_oracle(irq))
before moving the interrupt, which assumes that an interrupt whose affinity
is preserved is still queued on this vCPU's ap_list.
That assumption no longer holds if the interrupt is taken off the ap_list
while the locks are dropped. vgic_flush_pending_lpis() removes the
interrupt from the list and sets irq->vcpu to NULL, but leaves
enabled/pending/target_vcpu untouched. As the interrupt is still enabled
and pending, vgic_target_oracle() returns the same target_vcpu, so the
affinity check passes and list_del() is run a second time on an entry that
has already been removed.
Also check that the interrupt is still assigned to this vCPU
(irq->vcpu == vcpu) before moving it.
Fixes: 0919e84c0f ("KVM: arm/arm64: vgic-new: Add IRQ sync/flush framework")
Signed-off-by: Hyunwoo Kim <imv4bel@gmail.com>
Link: https://patch.msgid.link/aiHnI1mu6SGQrgnz@v4bel
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Hyunwoo Kim reports some really bad races should the following
situation occur:
- LPI-I is pending in vcpu-B's AP list
- vcpu-A writes to vcpu-B's RD to disable its LPIs
- vcpu-C moves I from B to C
If the last two race nicely enough, vgic_prune_ap_list() can drop
the irq and AP list locks, reacquire them, and in the interval
the irq has been freed. UAF follows.
The fix is two-fold:
- Before dropping the irq and ap_list locks, take a reference on
the irq
- Do not try to handle migration of the pending bit: there is no
expectation that this state is retained, as per the architecture
With that, we're sure that the interrupt is still around, and we
safely remove it from the AP list as it has no target at this
stage (unless another interrupt fires, but that's another story).
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Tested-by: Hyunwoo Kim <imv4bel@gmail.com>
Link: https://lore.kernel.org/r/ailsCnyoS82r_QRz@v4bel
Link: https://patch.msgid.link/20260615181625.3029352-1-maz@kernel.org
Fixes: 5dd4b924e3 ("KVM: arm/arm64: vgic: Add refcounting for IRQs")
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
CPU errata handling:
- Extend CnP disabling workaround to HiSilicon HIP09 hardware.
- Work around eternally broken broadcast TLB invalidation on more CPUs.
- Documentation and code cleanups.
CPU features:
- Add new hwcaps for the 2025 dpISA extensions.
Floating point / SVE / SME:
- Significant cleanup to the low-level state management code in the core
architecture code and KVM.
- Use correct register widths during SVE/SME save/restore assembly.
- Expose SVE/SME save/restore memory accesses to sanitisers.
Memory management:
- Preparatory work for unmapping the kernel data and bss sections from
the linear map.
Miscellaneous:
- Inline DAIF manipulation helpers so they can be used safely from
non-instrumentable code.
- Fix handling of the 'nosmp' cmdline option to avoid marking secondary
cores as "possible".
MPAM:
- Add support for v0.1 of the MPAM architecture.
Perf:
- Update HiSilicon PMU MAINTAINERS entry.
- Fix event encodings for the DVM node in the CMN driver.
Selftests:
- Extend sigframe tests to cover POE context.
- Add coverage for the newly added 2025 dpISA hwcaps.
System registers:
- Add new registers and ESR encodings for the HDBSS feature.
Plus minor fixes and cleanups across the board.
-----BEGIN PGP SIGNATURE-----
iQFEBAABCgAuFiEEPxTL6PPUbjXGY88ct6xw3ITBYzQFAmoun+0QHHdpbGxAa2Vy
bmVsLm9yZwAKCRC3rHDchMFjNFEvB/9scDYXHUCxQOqoRB5TPUi63pgl5vl83IfI
TWE2r2wNsrjqXlAV2FDbzuTWxLpACS+vpk5ltsb57aUfENQ5wZEFCDvJlbpNZuHS
5/zAQmCajPpgC5TFDo7rbgSv2BSR4TXuFfvGS349+a82c9C12EUulBFvR1DDVElh
FCYT5PfHNu1Khas4YiUln+9WSqS8qSHfYsz4Qb/lDaZcAmh22jQBPZQFU7fMlZST
Fr/nzSy6cxkARotqU7RlRQLgtDHBEqTYrNHCP0WSQ2aPJN8SeFUyjEcQas/Zak+S
Bme8OHcqn4pZPll5TfHUTaJp91oIJAA9uX5RSwE+G3NvAGNbptuw
=LXuh
-----END PGP SIGNATURE-----
Merge tag 'arm64-upstream' of gitolite.kernel.org:pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Will Deacon:
"It feels like the new world of AI tooling has slowed us down a little
on the feature side when compared to the fixes side. The extra rounds
of Sashiko review have also pushed a few things out until next time.
Still, there's some good foundational stuff here for the fpsimd code
and hardening work towards removing the predictable linear alias of
the kernel image.
CPU errata handling:
- Extend CnP disabling workaround to HiSilicon HIP09 hardware.
- Work around eternally broken broadcast TLB invalidation on more
CPUs.
- Documentation and code cleanups.
CPU features:
- Add new hwcaps for the 2025 dpISA extensions.
Floating point / SVE / SME:
- Significant cleanup to the low-level state management code in the
core architecture code and KVM.
- Use correct register widths during SVE/SME save/restore assembly.
- Expose SVE/SME save/restore memory accesses to sanitisers.
Memory management:
- Preparatory work for unmapping the kernel data and bss sections
from the linear map.
Miscellaneous:
- Inline DAIF manipulation helpers so they can be used safely from
non-instrumentable code.
- Fix handling of the 'nosmp' cmdline option to avoid marking
secondary cores as "possible".
MPAM:
- Add support for v0.1 of the MPAM architecture.
Perf:
- Update HiSilicon PMU MAINTAINERS entry.
- Fix event encodings for the DVM node in the CMN driver.
Selftests:
- Extend sigframe tests to cover POE context.
- Add coverage for the newly added 2025 dpISA hwcaps.
System registers:
- Add new registers and ESR encodings for the HDBSS feature.
Plus minor fixes and cleanups across the board"
* tag 'arm64-upstream' of gitolite.kernel.org:pub/scm/linux/kernel/git/arm64/linux: (73 commits)
arm64: errata: Mitigate TLBI errata on Microsoft Azure Cobalt 100 CPU
arm64: errata: Mitigate TLBI errata on NVIDIA Olympus CPU
arm64: errata: Mitigate TLBI errata on various Arm CPUs
arm64: cputype: Add C1-Premium definitions
arm64: cputype: Add C1-Ultra definitions
Revert "arm64: mm: Unmap kernel data/bss entirely from the linear map"
Revert "arm64: mm: Defer remap of linear alias of data/bss"
arm64: arch_timer: reuse arch_timer_read_cnt{p,v}ct_el0() helpers
arm64/mm: Rename ptdesc_t
arm64: mm: Defer remap of linear alias of data/bss
KVM: arm64: Omit tag sync on stage-2 mappings of the zero page
arm64: Avoid double evaluation of __ptep_get()
kasan: Move generic KASAN page tables out of BSS too
arm64: Rename page table BSS section to .bss..pgtbl
arm64: patching: replace min_t with min in __text_poke
perf/arm-cmn: Fix DVM node events
arm64: fpsimd: Remove <asm/fpsimdmacros.h>
arm64: fpsimd: Move SME save/restore inline
arm64: fpsimd: Move sve_flush_live() inline
arm64: fpsimd: Move SVE save/restore inline
...
- Expand timekeeping snapshot mechanisms
The various snapshot functions are mostly used for PTP to collect
"atomic" snapshots of various involved clocks.
They lack support for the recently introduced AUX clocks and do not
provide the underlying counter value (e.g. TSC) to user space. Exposing
the counter value snapshot allows for better control and steering.
Convert the hard wired ktime_get_snapshot() to take a clock ID, which
allows the caller to select the clock ID to be captured along with
CLOCK_MONONOTONIC_RAW. Additionally capture the underlying hardware
counter value and the clock source ID of the counter.
Expand the hardware based snapshot capture where devices provide a
mechanism to snapshot the hardware PTP clock and the system counter
(usually via PCI/PTM) to support AUX clocks and also provide the
captured counter value back to the caller and not only the clock
timestamps derived from it.
- Add a new optional read_snapshot() callback to clocksources
That is required to capture atomic snapshots from clocksources which
are derived from TSC with a scaling mechanism (e.g. Hyper-V, KVMclock).
The value pair is handed back in the snapshot structure to the callers,
so they can do the necessary correlations in a more precise way.
This touches usage sites of the affected functions and data structure all
over the tree, but stays fully backwards compatible for the existing user
space exposed interfaces. New PTP IOCTLs will provide access to the
extended functionality in later kernel versions.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotoSAQHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoYWbEACv/g3pGDxWxfzOI2h6vQgGxDvD3LwmdhPE
bzXRRaxp3/J0rZTQmCghknVGDPVjepNQgKkUMXfaFG2UZmiPHG5qVTXO6DddguS4
cQc0SUO3e422lUPCoBmTULZ+vlctb4LJsWXPQHYNKC73KqMJtWte7T2HBiFDK5RB
O0S34DZtkvOW4tHIu0RwlwCXZ0gcO+zsjxKA8K/P6sMtKBQU1/rRkZZCx2KCvq0F
Rx3NTGoY4if/C83YBq1cEn8BvXrcQQH4ZOOWuySsLGJGRPZ1dXGP+JtfRWutk/f9
HZztlaXcEz71dJXlhBxc0Eb/86uC3POEq7ZYvQdzLbsSZ/3AbalksL9CLyxgdHtc
U964SuwOVPcYfEytd4TWb1nu7JgOR0olYK+l4AbCt4EdKst5TADCJ7rtlZV3Idp+
Yg1GN3TwJcKItUNX9Szk+7MbvB8EWOEl7Obahfm48qDK1pqFe08qhOzSCeRXu+Bb
QiupC3ndzUB1Yjf3DPV6wQl4Fl/TscrAVrPlnGCOJEKXtUKFxvcKquy/W29UD//w
NuxKO2zK05UDsbBEwnZiCrdSGGNiLBYUbHfx2UvA7M0rfrrbjmG4rCFPotxhNb54
UuqgdM8G45MkyBV3qSSh3VC0XeD7UqzQtMYgUjjhvtapLlsri69vzL2DnQUcajSG
dgjzIg9O3g==
=kBrz
-----END PGP SIGNATURE-----
Merge tag 'timers-ptp-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull timekeeping updates from Thomas Gleixner:
"Updates for NTP/timekeeping and PTP:
- Expand timekeeping snapshot mechanisms
The various snapshot functions are mostly used for PTP to collect
"atomic" snapshots of various involved clocks.
They lack support for the recently introduced AUX clocks and do not
provide the underlying counter value (e.g. TSC) to user space.
Exposing the counter value snapshot allows for better control and
steering.
Convert the hard wired ktime_get_snapshot() to take a clock ID,
which allows the caller to select the clock ID to be captured along
with CLOCK_MONONOTONIC_RAW. Additionally capture the underlying
hardware counter value and the clock source ID of the counter.
Expand the hardware based snapshot capture where devices provide a
mechanism to snapshot the hardware PTP clock and the system counter
(usually via PCI/PTM) to support AUX clocks and also provide the
captured counter value back to the caller and not only the clock
timestamps derived from it.
- Add a new optional read_snapshot() callback to clocksources
That is required to capture atomic snapshots from clocksources
which are derived from TSC with a scaling mechanism (e.g. Hyper-V,
KVMclock).
The value pair is handed back in the snapshot structure to the
callers, so they can do the necessary correlations in a more
precise way.
This touches usage sites of the affected functions and data structure
all over the tree, but stays fully backwards compatible for the
existing user space exposed interfaces. New PTP IOCTLs will provide
access to the extended functionality in later kernel versions"
* tag 'timers-ptp-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (28 commits)
ptp: vmclock: Use hw_cycles from snapshot for precise TSC pairing
x86/kvmclock: Implement read_snapshot() for kvmclock clocksource
clocksource/hyperv: Implement read_snapshot() for TSC page clocksource
timekeeping: Add clocksource read_snapshot() method and hw_cycles to snapshot
ptp: Switch to ktime_get_snapshot_id() for pre/post timestamps
timekeeping: Add support for AUX clock cross timestamping
timekeeping: Remove system_device_crosststamp::sys_realtime
ALSA: hda/common: Use system_device_crosststamp::sys_systime
wifi: iwlwifi: Use system_device_crosststamp::sys_systime
ptp: Use system_device_crosststamp::sys_systime
timekeeping: Prepare for cross timestamps on arbitrary clock IDs
timekeeping: Remove ktime_get_snapshot()
virtio_rtc: Use provided clock ID for history snapshot
net/mlx5: Use provided clock ID for history snapshot
igc: Use provided clock ID for history snapshot
ice/ptp: Use provided clock ID for history snapshot
wifi: iwlwifi: Adopt PTP cross timestamps to core changes
timekeeping: Add CLOCK ID to system_device_crosststamp
timekeeping: Add system_counterval_t to struct system_device_crosststamp
timekeeping: Add CLOCK_AUX support for ktime_get_snapshot_id()
...
* for-next/mm: (24 commits)
Revert "arm64: mm: Unmap kernel data/bss entirely from the linear map"
Revert "arm64: mm: Defer remap of linear alias of data/bss"
arm64/mm: Rename ptdesc_t
arm64: mm: Defer remap of linear alias of data/bss
KVM: arm64: Omit tag sync on stage-2 mappings of the zero page
arm64: Avoid double evaluation of __ptep_get()
kasan: Move generic KASAN page tables out of BSS too
arm64: Rename page table BSS section to .bss..pgtbl
arm64: mm: Unmap kernel data/bss entirely from the linear map
arm64: mm: Map the kernel data/bss read-only in the linear map
mm: Make empty_zero_page[] const
sh: Drop cache flush of the zero page at boot
powerpc/code-patching: Avoid r/w mapping of the zero page
arm64: mm: Don't abuse memblock NOMAP to check for overlaps
arm64: Move fixmap and kasan page tables to end of kernel image
arm64: mm: Permit contiguous attribute for preliminary mappings
arm64: kfence: Avoid NOMAP tricks when mapping the early pool
arm64: mm: Permit contiguous descriptors to be manipulated
arm64: mm: Preserve non-contiguous descriptors when mapping DRAM
arm64: mm: Preserve existing table mappings when mapping DRAM
...
* New features:
- None. Zilch. Nada. Que dalle.
* Fixes and other improvements:
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEn9UcU+C1Yxj9lZw9I9DQutE9ekMFAmorw+MACgkQI9DQutE9
ekNZSw//bDv7l2dz3nUbtAuDIX9bQfbyq25a/R1SCC5om8yKOY0v0DmEAju5iu8T
y7n2G/ME+GeeAvRo8thWaDftqJTKD73sXll0F7AhRSdnWsPD4k5B6E4m62cwb8jd
oabUlf/ruKq0ftcwuJy3TWUNB0XVuO4Ep6zwcnrLSmyOUPCLqfbvWWKujQTy/Yec
zj9/FolfpgBla0cdZ1GwrBSppfPuT9rQtFrHwA91b66yNoDc+DxGln6/b0vDsN79
GcLJSoYeUQTXBdy4SH6oYLhhCi30+CFtSdkzA/vv3INX269vyBOmgMPJDnoquWPW
oH41sYIhjcy08zvbOzi/P7/mDUNRgpMGpzUZ/rKAZPvpWrQYCQfPYWSeRHCjBwQd
lGTJ2pXu0UcLoLweshaKEcuO6bq2TKtf3mCMMZWyN3pnLp/CBCIfroQr3RYmrWTT
GPoBTN8epnOJuhfOpmbfDWjm2vIJ6z/B0zFTOtUZstybxm9sOg6NnPr7WoaT1LEV
FvD+4Oc53h7Py0NGGlk5iqlRYgTULjvCQ4rcPUs2tU8GYXy3MA4l0Erj1FsEwmXG
eX4CTbCEI2FCaXuVK7zz+YngilNERFFD+ebwWgzCbZmA+w6pA4fWT1T+LywV24lu
zANT0Id4ogoI3RxxHLG61TH9xy6+OW56PYGTnZedurxyXsxxod0=
=ER2M
-----END PGP SIGNATURE-----
Merge tag 'kvmarm-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 updates for 7.2
* New features:
- None. Zilch. Nada. Que dalle.
* Fixes and other improvements:
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
* kvm-arm64/nv-mmu-7.2:
: .
: Assorted collection of fixes for NV MMU bugs
:
: - Correctly plug AT S1E1A handling in the emulation backend
:
: - Make CPTR_EL2.E0POE depend on FEAT_S1POE
:
: - Drop the reference on the page if the VNCR translation
: races with an MMU notifier
:
: - Correctly synthesise an SEA if a page table walk fails due
: to a guest error
:
: - Fully invalidate the VNCR TLB and fixmap when translating
: for a new VNCR
:
: - Restart S1 walk when the S2 walk fails due to a race condition
:
: - Correctly return -EAGAIN when a S1 walk fails
:
: - Fix block mapping validity check in stage-1 walker for 64kB pages
:
: - Fix potential NULL dereference when performing an EL2 TLBI targeting
: the VNCR page
:
: - Hold kvm->mmu_lock while initialising the vncr_tlb pointer
: .
KVM: arm64: nv: Hold kvm->mmu_lock while initialising vcpu->arch.vncr_tlb
KVM: arm64: nv: Avoid dereferencing NULL VNCR pseudo-TLB
KVM: arm64: Fix block mapping validity check in stage-1 walker
KVM: arm64: nv: Restart stage-1 walk if stage-2 desc update fails
KVM: arm64: Restart instruction upon race in __kvm_at_s12()
KVM: arm64: nv: Inject SEA TTW when desc update can't write to GPA
KVM: arm64: nv: Fully update VNCR fixmap state in kvm_translate_vncr()
KVM: arm64: Don't leak PFN when kvm_translate_vncr() races MMU notifier
arm64: cpufeature: Expose ID_AA64ISAR2_EL1.ATS1A to KVM
KVM: arm64: Wire AT S1E1A in the system instruction handling table
KVM: arm64: Key CPTR_EL2.E0POE propagation on FEAT_S1POE
Signed-off-by: Marc Zyngier <maz@kernel.org>
* kvm-arm64/misc-7.2:
: .
: - Check for a valid vcpu pointer upon deactivating traps when handling
: a HYP panic in VHE mode
:
: - Make the __deactivate_fgt() macro use its arguments instead of the
: surrounding context
:
: - Don't bother with initialising TPIDR_EL2 in the hyp stubs, as this
: is already taken care of in more obvious places
:
: - Drop the unused kvm_arch pointer passed to __load_stage2()
:
: - Return -EOPNOTSUPP when a hypercall fails for some reason, instead of
: returning whatever was in the result structure
:
: - Make the ITS ABI selection helpers return void, which avoids wondering
: about the nature of the return code (always 0)
: .
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: arm64: Set a Linux errno on SMCCC error in kvm_call_hyp_nvhe()
KVM: arm64: Remove @arch from __load_stage2()
KVM: arm64: Don't populate TPIDR_EL2 in finalise_el2()
KVM: arm64: Fix __deactivate_fgt macro parameter typo
KVM: arm64: Guard against NULL vcpu on VHE hyp panic path
Signed-off-by: Marc Zyngier <maz@kernel.org>
* kvm-arm64/vgic-v5-PPI-fixes:
: .
: Substantial cleanup of the vgic-v5 PPI support. From the original
: cover letter:
:
: "With the GICv5 PPi support merged in, it has become obvious that a few
: things could be improved, both from the correctness and maintainability
: angles."
: .
KVM: arm64: Fix arch timer interrupts for GICv3-on-GICv5 guests
irqchip/gic-v5: Immediately exec priority drop following activate
Documentation: KVM: Clarify that PMU_V3_IRQ IntID requirements for GICv5
Documentation: KVM: Fix typos in VGICv5 documentation
KVM: arm64: selftests: Improve error handling for GICv5 PPI selftest
KVM: arm64: selftests: Cleanup unused vars in GICv5 PPI selftest
KVM: arm64: selftests: Add missing GIC CDEN to no-vgic-v5 selftest
KVM: arm64: vgic-v5: Atomically assign bits to PPI DVI bitmap
KVM: arm64: vgic-v5: Add missing trap handing for NV triage
KVM: arm64: vgic-v5: Limit support to 64 PPIs
KVM: arm64: vgic: Rationalise per-CPU irq accessor
KVM: arm64: vgic-v5: Drop defensive checks from vgic_v5_ppi_queue_irq_unlock()
KVM: arm64: vgic: Consolidate vgic_allocate_private_irqs_locked()
KVM: arm64: vgic: Constify struct irq_ops usage
KVM: arm64: vgic-v5: Drop pointless ARM64_HAS_GICV5_CPUIF check
KVM: arm64: vgic-v5: Remove use of __assign_bit() with a constant
KVM: arm64: vgic-v5: Move PPI caps into kvm_vgic_global_state
KVM: arm64: vgic-v5: Add for_each_visible_v5_ppi() iterator
Signed-off-by: Marc Zyngier <maz@kernel.org>
* kvm-arm64/pkvm-fixes-7.2:
: .
: Assorted pKVM fixes for 7.2:
:
: - Ensure that the vcpu memcache is filled in a number of cases (donate,
: share, selftest)
:
: - Fix vmemmap page order handling by resetting it when initialising the
: memory pool
:
: - Don't leak page references on failed memory donation
:
: - Add sanity-check for refcounted pages when donating/sharing pages
:
: - Clear __hyp_running_vcpu on state flush
:
: - Check LR upper bound against a trusted value
:
: - Assorted fixes for the host-side tracking of the pages shared with
: EL2 as a result of some Sashiko testing from Fuad
:
: - Correctly forward HCR_EL2.VSE from host to guest, so that protected
: guests can see SErrors
: .
KVM: arm64: Roll back partial shares on kvm_share_hyp() failure
KVM: arm64: Avoid host/hyp share desync on unshare hypercall failure
KVM: arm64: Free hyp-share tracking node when share hypercall fails
KVM: arm64: Flush HCR_EL2.VSE to deliver SErrors to pKVM guests
KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU
KVM: arm64: Clear __hyp_running_vcpu when flushing the pKVM hyp vCPU
KVM: arm64: Pre-check vcpu memcache for host->guest donate
KVM: arm64: Pre-check vcpu memcache for host->guest share
KVM: arm64: Seed pkvm_ownership_selftest vcpu memcache
KVM: arm64: Add fail-safe for refcounted pages in __pkvm_hyp_donate_host
KVM: arm64: Fix __pkvm_init_vm error path
KVM: arm64: Reset page order in pKVM hyp_pool
Signed-off-by: Marc Zyngier <maz@kernel.org>
* kvm-arm64/nv-granule-sizes:
: .
: Tidying up of the behaviour when the selected page size in not
: implemented, courtesy of Wei-Lin Chang. From the initial cover
: letter:
:
: "This small series fixes the granule size selection for software stage-1
: and stage-2 walks. Previously we treat the guest's TCR/VTCR.TGx as-is
: and use the encoded granule size for the walks. However this is
: incorrect if the granule sizes are not advertised in the guest's
: ID_AA64MMFR0_EL1.TGRAN*. The architecture specifies that when an
: unsupported size is programed in TGx, it must be treated as an
: implemented size. Fix this by choosing an available one while
: prioritizing PAGE_SIZE."
: .
KVM: arm64: Fallback to a supported value for unsupported guest TGx
KVM: arm64: nv: Use literal granule size in TLBI range calculation
KVM: arm64: Factor out TG0/1 decoding of VTCR and TCR
KVM: arm64: nv: Rename vtcr_to_walk_info() to setup_s2_walk()
Signed-off-by: Marc Zyngier <maz@kernel.org>
* kvm-arm64/nv-fp-elision:
: .
: Significantly reduce the overhead of the context switch between L1 and
: L2 guests by eliding the save/restore of the FP/SIMD/SVE registers, as
: this state is shared between the two guests, and therefore can be left
: live.
: .
KVM: arm64: nv: Don't save/restore FP register during a nested ERET or exception
KVM: arm64: nv: Track L2 to L1 exception emulation
Signed-off-by: Marc Zyngier <maz@kernel.org>
