From 98a843669196ace8789a859796e45caffb970e40 Mon Sep 17 00:00:00 2001 From: Mikhail Malyshev Date: Sun, 20 Sep 2026 07:24:59 +0000 Subject: [PATCH] KVM: x86/mmu: Convert can't-happen fault path EFAULTs to KVM_BUG_ON() + -EIO Four -EFAULT returns in x86's page fault path are guarded by WARN_ON_ONCE() because they describe KVM bugs, not conditions the guest or userspace can reach: - direct_map() and FNAME(fetch)() completing the walk at a level other than the goal level, i.e. KVM's shadow page table walk disagreeing with the mapping level KVM itself just computed. - a CR2 with bits 63:32 set on 32-bit KVM, which the hardware cannot produce. - PFERR_PRIVATE_ACCESS set on a reserved-bit fault. KVM sets that synthetic bit itself, and only when PFERR_RSVD_MASK is clear. Returning -EFAULT for a KVM bug conflicts with KVM_CAP_MEMORY_FAULT_INFO, which promises that an -EFAULT out of KVM_RUN on a guest page fault VM-Exit is accompanied by kvm_run.memory_fault. Describing a broken KVM invariant as a memory fault would be actively misleading, as there is no guest access for userspace to resolve and retry. Convert the four sites to KVM_BUG_ON() + -EIO, the established way to report that KVM is hosed, so that -EFAULT out of the fault path always means "guest memory access KVM could not resolve". Suggested-by: Sean Christopherson Link: https://lore.kernel.org/all/Zr-8M9rYplgN6IS3@google.com/ Signed-off-by: Mikhail Malyshev Link: https://patch.msgid.link/20260920072459.3485710-3-mike.malyshev@gmail.com Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 12 ++++++------ arch/x86/kvm/mmu/paging_tmpl.h | 4 ++-- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2bf6ed878589..f14440a2b547 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3577,8 +3577,8 @@ static int direct_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) fault->req_level >= it.level); } - if (WARN_ON_ONCE(it.level != fault->goal_level)) - return -EFAULT; + if (KVM_BUG_ON(it.level != fault->goal_level, vcpu->kvm)) + return -EIO; ret = mmu_set_spte(vcpu, fault->slot, it.sptep, access, base_gfn, fault->pfn, fault); @@ -4941,8 +4941,8 @@ int kvm_handle_page_fault(struct kvm_vcpu *vcpu, u64 error_code, #ifndef CONFIG_X86_64 /* A 64-bit CR2 should be impossible on 32-bit KVM. */ - if (WARN_ON_ONCE(fault_address >> 32)) - return -EFAULT; + if (KVM_BUG_ON(fault_address >> 32, vcpu->kvm)) + return -EIO; #endif /* * Legacy #PF exception only have a 32-bit error code. Simply drop the @@ -6651,8 +6651,8 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 err r = RET_PF_INVALID; if (unlikely(error_code & PFERR_RSVD_MASK)) { - if (WARN_ON_ONCE(error_code & PFERR_PRIVATE_ACCESS)) - return -EFAULT; + if (KVM_BUG_ON(error_code & PFERR_PRIVATE_ACCESS, vcpu->kvm)) + return -EIO; r = handle_mmio_page_fault(vcpu, cr2_or_gpa, direct); if (r == RET_PF_EMULATE) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 27427e7f22fa..4f5369cf3753 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -775,8 +775,8 @@ static int FNAME(fetch)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, fault->req_level >= it.level); } - if (WARN_ON_ONCE(it.level != fault->goal_level)) - return -EFAULT; + if (KVM_BUG_ON(it.level != fault->goal_level, vcpu->kvm)) + return -EIO; ret = mmu_set_spte(vcpu, fault->slot, it.sptep, gw->pte_access, base_gfn, fault->pfn, fault);