github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-14 16:10:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph
9b20885f14
linux/mm
History
Andy Lutomirski 478a1c3abe mm: fix incorrect flush address in direct page table reclaim 
When zap_pte_range reclaims a page table, it does:

    pte_free_tlb(tlb, pmd_pgtable(pmdval), addr);

and this is unconditionally wrong: if this code executes, addr *always*
points one past the end of the range covered by the table.  The addr
parameter is used to flush the TLB (really the paging-structure-cache)
to drop references to the to-be-freed table, and any architecture that
cares about the parameter will flush the wrong address.  (But they'll
still free the correct page).

I think it's worth contemplating why the kernel works at all.

If we hit the offending line of code, we will first clear the PMD entry
(line 1954, zap_empty_pte_table), then we will issue pending flushes if
force_flush is set (tlb_flush_mmu_tlbonly(tlb)), then we will skip the
retry on line 1979 (phew!), and then we will do the offending
pte_free_tlb call.  *Or* we will clear the PMD entry immediately before
pte_free_tlb (line 1983, zap_pte_table_if_empty).

If we have any pending flushes (i.e. we actually zapped any last-level
entries) at the time we clear the PMD entry, then the flush really ought
to flush all references to the table (Linus certainly seems to think it
will on all architectures [0]).

The condition under which we have no accumulated flushes at the time of
the clear is very complex (the whole zap_pte_range function has absurdly
complex control flow).  If we do hit the bad case, then we will end up
clearing the PMD entry after the last time the range is flushed, and any
CPU is free to cache a reference to the (empty) page table.  If this
happens due to an ordinary read or write, it would segfault, so it would
be rare.  But the cache could be speculatively filled as well.  Then
we'll flush the wrong address and then free and possibly reuse the
table.

On x86, even flushing the wrong address works on non-KPTI Intel systems
because INVLPG flushes *all* paging-structure-caches, not just the ones
for the target address.  But INVPCID does not, and flush_tlb_one_user
will use INVPCID if it's available.  And then we're toast.  AMD systems
are more susceptible: we set the EFER.TCE bit, which makes even INVLPG
only flush the target address.

I think this might fix an issue in ripgrep reported here:
https://github.com/BurntSushi/ripgrep/issues/3494

[0] https://lore.kernel.org/all/CA+55aFzBggoXtNXQeng5d_mRoDnaMBE5Y+URs+PHR67nUpMtaw@mail.gmail.com/T/#u

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Fixes: 4c640eb418 ("mm: move pte table reclaim code to memory.c")
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Liam R. Howlett <Liam.Howlett@oracle.com>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: stable@vger.kernel.org
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-08-05 08:00:25 -07:00
..
damon mm/damon/core: disallow overlapping input ranges for damon_set_regions() 2026-07-09 15:48:55 -07:00
kasan mm.git review status for mm-hotfixes-stable..mm-stable 2026-06-19 10:14:34 -07:00
kfence slab changes for 7.2 - part 2 2026-06-22 08:28:48 -07:00
kmsan Convert 'alloc_obj' family to use the new default GFP_KERNEL argument 2026-02-21 17:09:51 -08:00
tests sparc/mm: export symbols for lazy_mmu_mode KUnit tests 2026-01-31 14:22:40 -08:00
backing-dev.c mm: blk-cgroup: fix use-after-free in cgwb_release_workfn() 2026-04-18 23:24:27 -07:00
balloon.c mm: rename CONFIG_BALLOON_COMPACTION to CONFIG_BALLOON_MIGRATION 2026-01-31 14:22:36 -08:00
bootmem_info.c mm/bootmem_info: stop marking mem_section_usage as MIX_SECTION_INFO 2026-06-02 15:22:11 -07:00
bpf_memcontrol.c bpf: Revert "bpf: drop KF_ACQUIRE flag on BPF kfunc bpf_get_root_mem_cgroup()" 2026-01-21 09:38:16 -08:00
cma_debug.c mm/cma_debug: fix invalid accesses for inactive CMA areas 2026-05-28 20:50:33 -07:00
cma_sysfs.c mm/cma_sysfs: skip inactive CMA areas in sysfs 2026-06-03 16:25:49 -07:00
cma.c mm/cma: fix reserved page leak on activation failure 2026-05-28 20:50:34 -07:00
cma.h mm: cma: set early_pfn and bitmap as a union in cma_memrange 2025-05-22 14:55:36 -07:00
compaction.c mm/compaction: handle free_pages_prepare() properly in compaction_free() 2026-07-01 19:02:53 -07:00
debug_page_alloc.c mm/debug_page_alloc: improve error message for invalid guardpage minorder 2025-05-12 23:50:38 -07:00
debug_page_ref.c
debug_vm_pgtable.c mm/debug_vm_pgtable: replace WRITE_ONCE() with pxd_clear() 2026-04-05 13:53:11 -07:00
debug.c mm: constify __dump_folio() arguments 2025-11-20 13:43:57 -08:00
dmapool_test.c Convert 'alloc_obj' family to use the new default GFP_KERNEL argument 2026-02-21 17:09:51 -08:00
dmapool.c docs: dma-api: replace consistent with coherent 2025-07-01 13:25:36 -06:00
early_ioremap.c mm/early_ioremap: clean up the use of WARN() for debugging 2026-01-26 20:02:26 -08:00
execmem.c mm/execmem: make the populate and alloc atomic 2026-04-05 13:53:34 -07:00
fadvise.c mm/fadvise: validate offset in generic_fadvise 2026-04-05 13:52:53 -07:00
fail_page_alloc.c fault-inject: improve build for CONFIG_FAULT_INJECTION=n 2024-09-01 20:43:33 -07:00
failslab.c fault-inject: improve build for CONFIG_FAULT_INJECTION=n 2024-09-01 20:43:33 -07:00
filemap.c mm: do file ownership checks with the proper mount idmap 2026-07-01 19:02:56 -07:00
folio-compat.c mm: add SPDX id lines to some mm source files 2026-02-06 15:47:16 -08:00
gup_test.c mm/gup_test: reject wrapped user ranges 2026-06-21 11:37:37 -07:00
gup_test.h
gup.c mm/gup: cleanup pgtable entry accessors 2026-05-28 21:04:41 -07:00
highmem.c mm/highmem: fix __kmap_to_page() build error 2026-01-31 14:22:38 -08:00
hmm.c driver core: Replace dev->dma_skip_sync with dev_dma_skip_sync() 2026-04-26 23:38:57 +02:00
huge_memory.c mm/huge_memory: unlock i_mmap_rwsem before releasing after-split folios 2026-07-28 17:37:32 -07:00
hugetlb_cgroup.c Convert 'alloc_flex' family to use the new default GFP_KERNEL argument 2026-02-21 17:09:51 -08:00
hugetlb_cma.c mm/hugetlb_cma: restrict hugetlb_cma parameter to gigantic-page alignment 2026-06-02 15:22:16 -07:00
hugetlb_cma.h mm: hugetlb: allocate frozen pages for gigantic allocation 2026-01-26 20:02:28 -08:00
hugetlb_internal.h mm/hugetlb: extract sysctl into hugetlb_sysctl.c 2025-11-20 13:43:57 -08:00
hugetlb_sysctl.c mm, hugetlb: implement movable_gigantic_pages sysctl 2026-01-20 19:24:50 -08:00
hugetlb_sysfs.c mm/hugetlb: extract sysfs into hugetlb_sysfs.c 2025-11-20 13:43:57 -08:00
hugetlb_vmemmap.c mm/hugetlb_vmemmap: fix incorrect vmemmap restore in rollback 2026-05-31 21:50:24 -07:00
hugetlb_vmemmap.h mm/hugetlb: do pre-HVO for bootmem allocated pages 2025-03-16 22:06:29 -07:00
hugetlb.c mm/hugetlb: fix list corruption in allocate_file_region_entries() 2026-07-20 17:41:28 -07:00
hwpoison-inject.c mm/hwpoison: decouple hwpoison_filter from mm/memory-failure.c 2025-09-21 14:22:21 -07:00
init-mm.c exec_state: relocate dumpable information 2026-05-26 11:02:01 +02:00
internal.h mm: switch deferred split shrinker to list_lru 2026-06-08 18:21:25 -07:00
interval_tree.c mm/memory: simplify calculation in unmap_mapping_range_tree() 2026-04-05 13:53:13 -07:00
ioremap.c mm/ioremap: pass pgprot_t to ioremap_prot() instead of unsigned long 2025-03-16 22:06:23 -07:00
Kconfig mm.git review status for linus..mm-stable 2026-06-23 12:03:44 -07:00
Kconfig.debug mm: kmemleak: add CONFIG_DEBUG_KMEMLEAK_VERBOSE build option 2026-04-18 00:10:48 -07:00
khugepaged.c mm/khugepaged: avoid underflow in madvise_collapse for sub-PMD MADV_COLLAPSE 2026-06-21 11:37:31 -07:00
kmemleak.c mm/kmemleak: fix checksum computation for per-cpu objects 2026-07-09 15:48:55 -07:00
ksm.c mm: convert do_brk_flags() to use vma_flags_t 2026-04-05 13:53:40 -07:00
list_lru.c mm.git review status for mm-hotfixes-stable..mm-stable 2026-06-19 10:14:34 -07:00
maccess.c mm: unexport globally copy_to_kernel_nofault 2025-07-09 22:42:22 -07:00
madvise.c mm: do file ownership checks with the proper mount idmap 2026-07-01 19:02:56 -07:00
Makefile mm/memcg: remove no longer used swap cgroup array 2026-06-02 15:22:23 -07:00
mapping_dirty_helpers.c mm/dirty: replace READ_ONCE() with pudp_get() 2025-11-16 17:27:58 -08:00
memblock.c kho: fix deferred initialization of scratch areas 2026-06-01 02:31:37 +03:00
memcontrol-v1.c mm/memcg: remove no longer used swap cgroup array 2026-06-02 15:22:23 -07:00
memcontrol-v1.h mm: memcg: initialize *locked in memcg1_oom_prepare() stub 2026-07-28 17:37:31 -07:00
memcontrol.c slab changes for 7.2 - part 2 2026-06-22 08:28:48 -07:00
memfd_luo.c mm/memfd_luo: document preservation of file seals 2026-05-04 14:03:16 +00:00
memfd.c memfd: deny writeable mappings when implying SEAL_WRITE 2026-05-21 19:06:11 -07:00
memory_hotplug.c drivers/base/memory: make memory block get/put explicit 2026-06-02 15:22:12 -07:00
memory-failure.c mm/memory-failure: use zone_pcp_disable() for poison handling 2026-06-02 15:22:33 -07:00
memory-tiers.c mm: introduce CONFIG_NUMA_MIGRATION and simplify CONFIG_MIGRATION 2026-04-05 13:53:33 -07:00
memory.c mm: fix incorrect flush address in direct page table reclaim 2026-08-05 08:00:25 -07:00
mempolicy.c mm/nodemask: correctly describe nodemask operation return types 2026-06-08 18:21:23 -07:00
mempool.c mm: simplify the mempool_alloc_bulk API 2026-06-03 18:20:47 +02:00
memremap.c mm: decrement MTHP_STAT_NR_ANON in free_zone_device_folio() 2026-07-28 17:37:31 -07:00
memtest.c mm/memtest: add underflow detection for size calculation 2026-01-09 11:53:51 +02:00
migrate_device.c mm: migrate_device: fix pte_pfn/pte_dirty called on non-present PTE 2026-07-20 17:41:27 -07:00
migrate.c mm/migrate: exclude hugetlb folios from MTHP_STAT_NR_ANON accounting 2026-07-28 17:37:31 -07:00
mincore.c mm: do file ownership checks with the proper mount idmap 2026-07-01 19:02:56 -07:00
mlock.c mm: rename unlock_page_lruvec_irq and its variants 2026-04-18 00:10:44 -07:00
mm_init.c mm/vmstat: fold stranded per-cpu node stats when a node comes online 2026-07-20 17:41:25 -07:00
mm_slot.h
mmap_lock.c mm/vma: improve and document __is_vma_write_locked() 2026-01-31 14:22:51 -08:00
mmap.c mm: fix mmap errno value when MAP_DROPPABLE is not supported 2026-05-28 21:04:48 -07:00
mmu_gather.c mm/mmu_gather: replace IPI with synchronize_rcu() when batch allocation fails 2026-04-05 13:53:05 -07:00
mmu_notifier.c mm.git review status for linus..mm-stable 2026-04-15 12:59:16 -07:00
mmzone.c mm: introduce memdesc_flags_t 2025-09-13 16:55:07 -07:00
mprotect.c mm/mprotect: special-case small folios when applying permissions 2026-04-18 00:10:55 -07:00
mremap.c mm: convert do_brk_flags() to use vma_flags_t 2026-04-05 13:53:40 -07:00
mseal.c mm/mseal: use min/max in mseal_apply 2026-06-02 15:22:15 -07:00
msync.c
nommu.c mm: abstract reading sysctl_max_map_count, and READ_ONCE() 2026-04-05 13:53:28 -07:00
numa_emulation.c mm/fake-numa: fix under-allocation detection in uniform split 2026-06-02 08:34:03 +03:00
numa_memblks.c memblock: numa_memblks: fix detection of NUMA node for CXL windows 2026-02-21 09:58:22 -08:00
numa.c mm/numa: remove unnecessary local variable in alloc_node_data() 2025-05-12 23:50:38 -07:00
oom_kill.c mm/oom_kill.c: simpilfy rcu call with guard(rcu) 2026-04-05 13:53:17 -07:00
page_alloc.c mm/page_alloc: only update NUMA min ratios on sysctl write 2026-06-21 11:31:29 -07:00
page_counter.c page_counter: track failcnt only for legacy cgroups 2025-03-17 00:05:35 -07:00
page_ext.c mm/page_ext: Add page_ext_get_from_phys() 2026-01-21 12:51:48 +01:00
page_frag_cache.c mm/page_alloc: export free_frozen_pages() instead of free_unref_page() 2025-01-13 22:40:31 -08:00
page_idle.c mm/page_idle.c: remove redundant mmu notifier in aging code 2026-04-05 13:53:02 -07:00
page_io.c block-7.2-20260625 2026-06-25 09:56:47 -07:00
page_isolation.c mm: page_isolation: avoid unsafe folio reads while scanning compound pages 2026-06-21 11:37:36 -07:00
page_owner.c mm/page_owner: fix %pGp format specifier argument type 2026-05-28 21:04:48 -07:00
page_poison.c
page_reporting.c mm/page_reporting: use system_freezable_wq to fix UAF during suspend 2026-07-28 17:37:33 -07:00
page_reporting.h
page_table_check.c mm/page_table_check: Pass mm_struct to pxx_user_accessible_page() 2026-03-13 00:07:47 +01:00
page_vma_mapped.c mm/page_vma_mapped: fix device-private PMD handling 2026-07-09 15:48:54 -07:00
page-writeback.c vfs-7.2-rc1.writeback 2026-06-15 03:30:45 +05:30
pagewalk.c mm/pagewalk: fix race between concurrent split and refault 2026-04-05 13:53:37 -07:00
percpu-internal.h mm/percpu-internal.h: optimise pcpu_chunk struct to save memory 2026-06-02 15:22:13 -07:00
percpu-km.c mm/percpu-km: fix bitmap overflow and accounting in pcpu_create_chunk() 2026-07-20 17:41:27 -07:00
percpu-stats.c mm: remove outdated filename comment in percpu-stats.c 2025-07-13 16:38:23 -07:00
percpu-vm.c kmsan: remove hard-coded GFP_KERNEL flags 2025-11-16 17:27:54 -08:00
percpu.c mm: memcontrol: return root object cgroup for root memory cgroup 2026-04-18 00:10:44 -07:00
pgalloc-track.h
pgtable-generic.c mm: change to return bool for pmdp_clear_flush_young() 2026-04-05 13:53:35 -07:00
process_vm_access.c mm: refactor mm_access() to not return NULL 2024-11-05 16:56:23 -08:00
ptdump.c mm/ptdump: replace READ_ONCE() with standard page table accessors 2025-11-16 17:27:52 -08:00
readahead.c Revert "mm: limit filemap_fault readahead to VMA boundaries" 2026-06-21 11:37:38 -07:00
rmap.c mm: remove page_mapped() 2026-05-28 21:04:57 -07:00
rodata_test.c mm/rodata_test: verify test data is unchanged, rather than non-zero 2025-01-13 22:40:38 -08:00
secretmem.c libfs: drop redundant SB_I_NOEXEC/SB_I_NODEV in init_pseudo() callers 2026-06-04 10:10:49 +02:00
shmem_quota.c treewide: Replace kmalloc with kmalloc_obj for non-scalar types 2026-02-21 01:02:28 -08:00
shmem.c fs: push nr_cached_objects memcg gating into individual filesystems 2026-07-23 11:35:02 +02:00
show_mem.c mm: add gpu active/reclaim per-node stat counters (v2) 2026-04-08 06:52:47 +10:00
shrinker_debug.c mm: shrinker: fix NULL pointer dereference in debugfs 2026-07-01 19:02:52 -07:00
shrinker.c mm: shrinker: fix shrinker_info teardown race with expansion 2026-07-01 19:02:52 -07:00
shuffle.c
shuffle.h
slab_common.c mm/slab: prevent unbounded recursion in free path with new kmalloc type 2026-07-14 17:22:40 +02:00
slab.h mm/slab: prevent unbounded recursion in free path with new kmalloc type 2026-07-14 17:22:40 +02:00
slub.c mm/slab: prevent unbounded recursion in free path with new kmalloc type 2026-07-14 17:22:40 +02:00
sparse-vmemmap.c mm/sparse-vmemmap: flush_cache_vmap() after hotplugging vmemmap 2026-07-13 11:29:52 -06:00
sparse.c mm/sparse: remove sparse buffer pre-allocation mechanism 2026-05-28 21:04:49 -07:00
swap_state.c mm: switch deferred split shrinker to list_lru 2026-06-08 18:21:25 -07:00
swap_table.h mm, swap: merge zeromap into swap table 2026-06-02 15:22:23 -07:00
swap.c mm/mglru: use folio_mark_accessed to replace folio_set_active 2026-06-04 14:45:02 -07:00
swap.h mm, swap: merge zeromap into swap table 2026-06-02 15:22:23 -07:00
swapfile.c mm/swap: remove redundant swap device reference in alloc/free 2026-06-08 18:21:32 -07:00
truncate.c mm/truncate: use folio_split() in truncate_inode_partial_folio() 2026-06-21 11:37:16 -07:00
usercopy.c usercopy: Remove folio references from check_heap_object() 2025-11-13 11:01:08 +01:00
userfaultfd.c userfaultfd: prevent registration of special VMAs 2026-07-09 15:48:53 -07:00
util.c mm/util: don't read __page_2 for order-1 folios in snapshot_page() 2026-07-20 17:41:27 -07:00
vma_exec.c mm: convert do_brk_flags() to use vma_flags_t 2026-04-05 13:53:40 -07:00
vma_init.c Summary of significant series in this pull request: 2025-10-02 18:18:33 -07:00
vma_internal.h mm: relocate the page table ceiling and floor definitions 2026-02-12 15:42:53 -08:00
vma.c drivers/char/mem: eliminate unnecessary use of success_hook 2026-06-04 14:44:59 -07:00
vma.h mm: allow handling of stacked mmap_prepare hooks in more drivers 2026-04-05 13:53:44 -07:00
vmalloc.c mm/vmalloc: free unused pages on vrealloc() shrink 2026-06-02 15:22:32 -07:00
vmpressure.c mm/vmpressure: skip socket pressure for costly order reclaim 2026-05-28 21:04:50 -07:00
vmscan.c mm: vmscan: abort proactive reclaim early when freezing for suspend 2026-07-28 17:37:33 -07:00
vmstat.c mm.git review status for linus..mm-stable 2026-04-19 08:01:17 -07:00
workingset.c mm/mglru: use folio_mark_accessed to replace folio_set_active 2026-06-04 14:45:02 -07:00
zpdesc.h mm: zpdesc: minor naming and comment corrections 2025-09-21 14:21:59 -07:00
zsmalloc.c zsmalloc: simplify data output in zs_stats_size_show() 2026-06-21 11:31:28 -07:00
zswap.c mm, swap: add support for stable large allocation in swap cache directly 2026-06-02 15:22:21 -07:00