mirror of
https://github.com/torvalds/linux.git
synced 2026-08-07 07:54:18 +02:00
4286a3ec25
Add support for "fast" aging of SPTEs in both the TDP MMU and Shadow MMU, where "fast" means "without holding mmu_lock". Not taking mmu_lock allows multiple aging actions to run in parallel, and more importantly avoids stalling vCPUs, e.g. due to holding mmu_lock for an extended duration while a vCPU is faulting in memory. For the TDP MMU, protect aging via RCU; the page tables are RCU-protected and KVM doesn't need to access any metadata to age SPTEs. For the Shadow MMU, use bit 1 of rmap pointers (bit 0 is used to terminate a list of rmaps) to implement a per-rmap single-bit spinlock. When aging a gfn, acquire the rmap's spinlock with read-only permissions, which allows hardening and optimizing the locking and aging, e.g. locking an rmap for write requires mmu_lock to also be held. The lock is NOT a true R/W spinlock, i.e. multiple concurrent readers aren't supported. To avoid forcing all SPTE updates to use atomic operations (clearing the Accessed bit out of mmu_lock makes it inherently volatile), rework and rename spte_has_volatile_bits() to spte_needs_atomic_update() and deliberately exclude the Accessed bit. KVM (and mm/) already tolerates false positives/negatives for Accessed information, and all testing has shown that reducing the latency of aging is far more beneficial to overall system performance than providing "perfect" young/old information. -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEKTobbabEP7vbhhN9OlYIJqCjN/0FAmfZizUACgkQOlYIJqCj N/1XkBAAwLxK3uKwkIIgzu+V6NPMiqPBsNtGiiQgRYMfvCwMW2vU6ztsBgNgs6zI eMnOCCo6fQaxPFvpKue8VN7TD33BcjzKZaPiuHZrzQIa/oYQeOlZ4oaN8lr9F9Ec 5l1Lg/p2+z1GGDhWc2opNpg48sCtX7IQ0Tx46LkoB3VSDFP33GwW4+Ht2r71rNeL ofKB+T0hU5HOry5j0w0nTVwOEoNzlm1sVFqOHzgnK18Lmqw2CfOPm+46K+w8nOh+ v+rwGuGa//1kcCjNCcGP1OuJdNAMgXBxND/l6LAkWcHfffIRbXlO07O05dAGqPeF rRn5JUl02OkI6lq99+935OmtEROe6mt+Bx0dhAzk4Z0CD6JY34ShZSAADSnltQlK 2a1E95t63v8a7ZM5dwED7os2HBhxODoyeWQAlIHpkVdmeTJstkyvjPhubJc13+Js oDL6ehs3hhZ171ePn2aXo0NobA5fe7xbl4wugP3hNmBXjLvu04D+llcDmC095nBk ICuzFqFXCXzdjEwgWwPzTseWOCoWTkoRqeJ9lch4UD3mMMcmK0MbK6joocGvCFto cL/eZdElnf1MZwWYdo44X+NEc1jItZVvktkRrllpwCtpRSDINO6RYZGcRf/g0Lha XmaU7jICfi3AKc4N3S2l4KIkd/AeJQySM+kGArxIOYoaqFCe2Mc= =Iy57 -----END PGP SIGNATURE----- Merge tag 'kvm-x86-mmu-6.15' of https://github.com/kvm-x86/linux into HEAD KVM x86/mmu changes for 6.15 Add support for "fast" aging of SPTEs in both the TDP MMU and Shadow MMU, where "fast" means "without holding mmu_lock". Not taking mmu_lock allows multiple aging actions to run in parallel, and more importantly avoids stalling vCPUs, e.g. due to holding mmu_lock for an extended duration while a vCPU is faulting in memory. For the TDP MMU, protect aging via RCU; the page tables are RCU-protected and KVM doesn't need to access any metadata to age SPTEs. For the Shadow MMU, use bit 1 of rmap pointers (bit 0 is used to terminate a list of rmaps) to implement a per-rmap single-bit spinlock. When aging a gfn, acquire the rmap's spinlock with read-only permissions, which allows hardening and optimizing the locking and aging, e.g. locking an rmap for write requires mmu_lock to also be held. The lock is NOT a true R/W spinlock, i.e. multiple concurrent readers aren't supported. To avoid forcing all SPTE updates to use atomic operations (clearing the Accessed bit out of mmu_lock makes it inherently volatile), rework and rename spte_has_volatile_bits() to spte_needs_atomic_update() and deliberately exclude the Accessed bit. KVM (and mm/) already tolerates false positives/negatives for Accessed information, and all testing has shown that reducing the latency of aging is far more beneficial to overall system performance than providing "perfect" young/old information. |
||
|---|---|---|
| .. | ||
| ABI | ||
| accel | ||
| accounting | ||
| admin-guide | ||
| arch | ||
| block | ||
| bpf | ||
| cdrom | ||
| core-api | ||
| cpu-freq | ||
| crypto | ||
| dev-tools | ||
| devicetree | ||
| doc-guide | ||
| driver-api | ||
| fault-injection | ||
| fb | ||
| features | ||
| filesystems | ||
| firmware_class | ||
| firmware-guide | ||
| fpga | ||
| gpu | ||
| hid | ||
| hwmon | ||
| i2c | ||
| iio | ||
| images | ||
| infiniband | ||
| input | ||
| isdn | ||
| kbuild | ||
| kernel-hacking | ||
| leds | ||
| litmus-tests | ||
| livepatch | ||
| locking | ||
| maintainer | ||
| mhi | ||
| misc-devices | ||
| mm | ||
| netlabel | ||
| netlink | ||
| networking | ||
| nvdimm | ||
| nvme | ||
| PCI | ||
| pcmcia | ||
| peci | ||
| power | ||
| process | ||
| RCU | ||
| rust | ||
| scheduler | ||
| scsi | ||
| security | ||
| sound | ||
| sphinx | ||
| sphinx-static | ||
| spi | ||
| staging | ||
| sunrpc/xdr | ||
| target | ||
| tee | ||
| timers | ||
| tools | ||
| trace | ||
| translations | ||
| usb | ||
| userspace-api | ||
| virt | ||
| w1 | ||
| watchdog | ||
| wmi | ||
| .gitignore | ||
| atomic_bitops.txt | ||
| atomic_t.txt | ||
| Changes | ||
| CodingStyle | ||
| conf.py | ||
| docutils.conf | ||
| index.rst | ||
| Kconfig | ||
| Makefile | ||
| memory-barriers.txt | ||
| SubmittingPatches | ||
| subsystem-apis.rst | ||