From ea3034b2b00fa50c8d2518d0804c9d427bbafa86 Mon Sep 17 00:00:00 2001 From: Gregory Price Date: Sat, 27 Jun 2026 16:22:43 -0400 Subject: [PATCH 01/13] mm/vmstat: fold stranded per-cpu node stats when a node comes online A per-node vmstat counter is pgdat->vm_stat[] plus per-cpu deltas. A balanced counter can sit split as global=+N / per-cpu=-N. The folds reconciling the split only walk online nodes, so when try_offline_node() marks a node offline the per-cpu deltas are stranded. A subsequent online resets the per-cpu area but not pgdat->vm_stat[], orphaning the +N permanently. All NR_VM_NODE_STAT_ITEMS are affected. The existing code zeroes the per-cpu counters and causes a permanent skew. Fold the stranded deltas instead, before the node rejoins the online set. The node is not online yet and the hotplug lock is held, so the remote access to per-cpu values is safe. Discovered when node compaction hung for a nearly empty node, as the math to determine throttling broke. Reproduced by repeated memory hotplug/unplug cycles on a node under pressure: NR_ISOLATED_ANON ratchets up and never returns to zero. Link: https://lore.kernel.org/20260627202243.758289-1-gourry@gourry.net Fixes: 75ef71840539 ("mm, vmstat: add infrastructure for per-node vmstats") Signed-off-by: Gregory Price Cc: Johannes Weiner Cc: Mel Gorman Cc: Mike Rapoport Cc: Vlastimil Babka Cc: Signed-off-by: Andrew Morton --- mm/mm_init.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/mm_init.c b/mm/mm_init.c index 0f64909e8d20..498d62c4ece3 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1540,7 +1540,7 @@ void __ref free_area_init_core_hotplug(struct pglist_data *pgdat) { int nid = pgdat->node_id; enum zone_type z; - int cpu; + int cpu, i; pgdat_init_internals(pgdat); @@ -1558,10 +1558,17 @@ void __ref free_area_init_core_hotplug(struct pglist_data *pgdat) pgdat->node_start_pfn = 0; pgdat->node_present_pages = 0; - for_each_online_cpu(cpu) { - struct per_cpu_nodestat *p; + /* + * Hot-unplug can leave per-cpu vmstat deltas unfolded (folders skip + * offline nodes) - reconcile this at online. Foreign access to counters + * is safe: the node is not online yet and we hold the hotplug lock. + */ + for_each_possible_cpu(cpu) { + struct per_cpu_nodestat *p = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu); - p = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu); + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) + if (p->vm_node_stat_diff[i]) + node_page_state_add(p->vm_node_stat_diff[i], pgdat, i); memset(p, 0, sizeof(*p)); } From 4fc089235378d1f9ddb6bcbe67c192ffdcaae0ed Mon Sep 17 00:00:00 2001 From: Stanislav Kinsburskii Date: Mon, 29 Jun 2026 16:30:14 -0700 Subject: [PATCH 02/13] lib: test_hmm: use device devt for coherent device range selection Commit af69016dab96 ("lib: test_hmm: implement a device release method") moved the initial dmirror_allocate_chunk() call before cdev_device_add(). That means the struct cdev has not been added yet, so cdev_add() has not initialized mdevice->cdevice.dev. The coherent-device range selection uses the device minor to choose between spm_addr_dev0 and spm_addr_dev1. Reading MINOR(mdevice->cdevice.dev) before cdev_add() therefore always sees an uninitialized dev_t. As a result, both coherent devices select the same physical range, and adding the second device fails due to the overlapping dev_pagemap range. Use mdevice->device.devt instead. It is initialized in dmirror_device_init() before dmirror_allocate_chunk() is called and is the same dev_t later passed to cdev_device_add(). Link: https://lore.kernel.org/178277581197.172200.16265155329935822153.stgit@skinsburskii Fixes: af69016dab96 ("lib: test_hmm: implement a device release method") Signed-off-by: Stanislav Kinsburskii Reviewed-by: Alistair Popple Cc: Balbir Singh Cc: Zenghui Yu (Huawei) Cc: Jason Gunthorpe Cc: Leon Romanovsky Cc: Signed-off-by: Andrew Morton --- lib/test_hmm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/test_hmm.c b/lib/test_hmm.c index 9c59d1ceb5b5..c4adbf98fac7 100644 --- a/lib/test_hmm.c +++ b/lib/test_hmm.c @@ -581,7 +581,7 @@ static int dmirror_allocate_chunk(struct dmirror_device *mdevice, devmem->pagemap.type = MEMORY_DEVICE_PRIVATE; break; case HMM_DMIRROR_MEMORY_DEVICE_COHERENT: - devmem->pagemap.range.start = (MINOR(mdevice->cdevice.dev) - 2) ? + devmem->pagemap.range.start = (MINOR(mdevice->device.devt) - 2) ? spm_addr_dev0 : spm_addr_dev1; devmem->pagemap.range.end = devmem->pagemap.range.start + From 4165b7d1c45c2da0dfefe528f8d1fb7d79f0d344 Mon Sep 17 00:00:00 2001 From: Usama Arif Date: Sun, 5 Jul 2026 06:12:31 -0700 Subject: [PATCH 03/13] userfaultfd: wait on source PMD during UFFDIO_MOVE move_pages_huge_pmd() snapshots src_pmdval under src_ptl, drops the lock, and, for migration entries, waits with pmd_migration_entry_wait(). Passing &src_pmdval is wrong. pmd_migration_entry_wait() must lock and re-read the real page-table PMD; on split-PMD-lock kernels, a stack address also resolves to the wrong lock. softleaf_entry_wait_on_locked() then waits without a folio reference, which is safe only while serialized against migration-entry removal by the real PT lock. Pass src_pmd, matching __handle_mm_fault() and hmm_vma_walk_pmd(). Link: https://lore.kernel.org/20260705131231.1499198-1-usama.arif@linux.dev Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI") Reported-by: sashiko-bot Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=8 Signed-off-by: Usama Arif Reviewed-by: Rik van Riel Reviewed-by: Baolin Wang Reviewed-by: Lance Yang Acked-by: David Hildenbrand (Arm) Reviewed-by: Lance Yang Reviewed-by: Lorenzo Stoakes Cc: Andrea Arcangeli Cc: Barry Song Cc: Dev Jain Cc: Johannes Weiner Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Shakeel Butt Cc: Zi Yan Cc: Signed-off-by: Andrew Morton --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b5d1e9d4463d..032702a4637b 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2774,7 +2774,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm if (!pmd_trans_huge(src_pmdval)) { spin_unlock(src_ptl); if (pmd_is_migration_entry(src_pmdval)) { - pmd_migration_entry_wait(mm, &src_pmdval); + pmd_migration_entry_wait(mm, src_pmd); return -EAGAIN; } return -ENOENT; From 07b4377bdbe74a3ec0c8da5849d014f70e003384 Mon Sep 17 00:00:00 2001 From: "Kiryl Shutsemau (Meta)" Date: Tue, 7 Jul 2026 16:13:49 +0100 Subject: [PATCH 04/13] fs/proc/task_mmu: fix PAGEMAP_SCAN written state for unpopulated ptes PAGEMAP_SCAN reports an unpopulated pte differently depending on which path serves the request. The PAGE_IS_WRITTEN fast path in pagemap_scan_pmd_entry() reports a pte_none as written (and, under PM_SCAN_WP_MATCHING, arms a marker); pagemap_page_category() returns 0 for the same pte_none. A request that cannot take the fast path (an extra category bit, category_anyof_mask or category_inverted) therefore reports the pte as clean and skips arming it. A range that was populated and then MADV_DONTNEED'd reads as written via one mask and clean via another, and in the latter case is not re-armed for the next round -- an incremental-dump consumer (e.g. CRIU) using a richer mask drops the zapped range and stops tracking writes to it. Report pte_none as written in pagemap_page_category() too. A pte_none carries no uffd-wp marker, i.e. it is not write-protected -- the same condition under which the present and swap cases already report PAGE_IS_WRITTEN. The fast path applies no VMA test, so neither does this. The hugetlb and fully-unpopulated-PMD (no page table) scans have no PAGE_IS_WRITTEN fast path, so they do not exhibit the per-entry divergence and are left unchanged. Add a pagemap_ioctl selftest that populates a range, drops it with MADV_DONTNEED, and checks that the fast path and the generic (category_anyof_mask) path both report every page written. Link: https://lore.kernel.org/20260707151349.92143-1-kirill@shutemov.name Fixes: 12f6b01a0bcb ("fs/proc/task_mmu: add fast paths to get/clear PAGE_IS_WRITTEN flag") Signed-off-by: Kiryl Shutsemau Cc: Muhammad Usama Anjum Cc: David Hildenbrand Cc: Jann Horn Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Pedro Falcato Cc: Peter Xu Cc: Shuah Khan Cc: Suren Baghdasaryan Cc: Vlastimil Babka Assisted-by: Claude:claude-fable-5 Cc: Signed-off-by: Andrew Morton --- fs/proc/task_mmu.c | 14 +++++- tools/testing/selftests/mm/pagemap_ioctl.c | 56 +++++++++++++++++++++- 2 files changed, 67 insertions(+), 3 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index d32408f7cd5e..d45c729ab6bb 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2432,8 +2432,18 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p, { unsigned long categories; - if (pte_none(pte)) - return 0; + if (pte_none(pte)) { + /* + * An unpopulated pte carries no uffd-wp marker, i.e. it is not + * write-protected, the same condition under which the present + * and swap cases below report PAGE_IS_WRITTEN. Report it here + * too so this generic path agrees with the PAGE_IS_WRITTEN fast + * path in pagemap_scan_pmd_entry(), which reports pte_none as + * written and, under PM_SCAN_WP_MATCHING, arms a marker. The + * fast path applies no VMA test, so neither does this. + */ + return PAGE_IS_WRITTEN; + } if (pte_present(pte)) { struct page *page; diff --git a/tools/testing/selftests/mm/pagemap_ioctl.c b/tools/testing/selftests/mm/pagemap_ioctl.c index 6f8971d5b3ce..f9bcff8e78fa 100644 --- a/tools/testing/selftests/mm/pagemap_ioctl.c +++ b/tools/testing/selftests/mm/pagemap_ioctl.c @@ -1051,6 +1051,57 @@ static void test_simple(void) ksft_test_result(i == TEST_ITERATIONS, "Test %s\n", __func__); } +/* + * A range that was populated and then MADV_DONTNEED'd is genuine pte_none + * with no uffd-wp marker. Such a pte must read the same regardless of which + * PAGEMAP_SCAN path serves the request: both the PAGE_IS_WRITTEN fast path and + * the generic path (reached e.g. via category_anyof_mask) must report every + * page written. + */ +static void unpopulated_scan_test(void) +{ + int npages = 16, i; + long mem_size = npages * page_size; + struct page_region regions[16]; + long fast = 0, slow = 0, ret; + char *mem; + + mem = mmap(NULL, mem_size, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (mem == MAP_FAILED) + ksft_exit_fail_msg("%s mmap failed\n", __func__); + + wp_init(mem, mem_size); + + /* Populate, then drop: the ptes become pte_none without a marker. */ + memset(mem, 1, mem_size); + if (madvise(mem, mem_size, MADV_DONTNEED)) + ksft_exit_fail_msg("%s MADV_DONTNEED failed\n", __func__); + + /* Fast path: category_mask == return_mask == PAGE_IS_WRITTEN. */ + ret = pagemap_ioctl(mem, mem_size, regions, npages, 0, 0, + PAGE_IS_WRITTEN, 0, 0, PAGE_IS_WRITTEN); + if (ret < 0) + ksft_exit_fail_msg("%s fast scan failed\n", __func__); + for (i = 0; i < ret; i++) + fast += LEN(regions[i]); + + /* Generic path: same query expressed via category_anyof_mask. */ + ret = pagemap_ioctl(mem, mem_size, regions, npages, 0, 0, + 0, PAGE_IS_WRITTEN, 0, PAGE_IS_WRITTEN); + if (ret < 0) + ksft_exit_fail_msg("%s generic scan failed\n", __func__); + for (i = 0; i < ret; i++) + slow += LEN(regions[i]); + + ksft_test_result(fast == npages && slow == npages, + "%s unpopulated ptes reported written by both paths (%ld, %ld of %d)\n", + __func__, fast, slow, npages); + + wp_free(mem, mem_size); + munmap(mem, mem_size); +} + int sanity_tests(void) { unsigned long long mem_size, vec_size; @@ -1559,7 +1610,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) if (!hugetlb_setup_default(4)) ksft_print_msg("HugeTLB test will be skipped\n"); - ksft_set_plan(117); + ksft_set_plan(118); page_size = getpagesize(); hpage_size = read_pmd_pagesize(); @@ -1737,6 +1788,9 @@ int main(int __attribute__((unused)) argc, char *argv[]) /* 17. ZEROPFN tests */ zeropfn_tests(); + /* 18. Unpopulated pte scan-path consistency */ + unpopulated_scan_test(); + close(pagemap_fd); ksft_finished(); } From 63867c82d0c0c2d182016a32b1cc0103116b0ea5 Mon Sep 17 00:00:00 2001 From: Kefeng Wang Date: Mon, 6 Jul 2026 19:19:58 +0800 Subject: [PATCH 05/13] mm: migrate_device: fix pte_pfn/pte_dirty called on non-present PTE pte_pfn() and pte_dirty() have undefined behaviour when called on a non-present PTE. In migrate_vma_collect_pmd(), these functions may be invoked on non-present entries (e.g., device-private entries), leading to potential crashes from pte_pfn() or incorrect dirty folio accounting from pte_dirty(). Fix both by guarding with pte_present() checks. Link: https://lore.kernel.org/20260708003955.4024340-1-wangkefeng.wang@huawei.com Link: https://lore.kernel.org/20260706111958.3649651-1-wangkefeng.wang@huawei.com Fixes: fd35ca3d12cc ("mm/migrate_device.c: copy pte dirty bit to page") Fixes: 6c287605fd56 ("mm: remember exclusively mapped anonymous pages with PG_anon_exclusive") Signed-off-by: Kefeng Wang Reviewed-by: Balbir Singh Acked-by: Zi Yan Cc: Alistair Popple Cc: Byungchul Park Cc: David Hildenbrand Cc: Gregory Price Cc: "Huang, Ying" Cc: Joshua Hahn Cc: Matthew Brost Cc: Rakie Kim Cc: Ying Huang Cc: Signed-off-by: Andrew Morton --- mm/migrate_device.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 554754eb26ff..908d2d4ec43a 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -401,7 +401,8 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, bool anon_exclusive; pte_t swp_pte; - flush_cache_page(vma, addr, pte_pfn(pte)); + if (pte_present(pte)) + flush_cache_page(vma, addr, pte_pfn(pte)); anon_exclusive = folio_test_anon(folio) && PageAnonExclusive(page); if (anon_exclusive) { @@ -422,7 +423,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, migrate->cpages++; /* Set the dirty flag on the folio now the pte is gone. */ - if (pte_dirty(pte)) + if (pte_present(pte) && pte_dirty(pte)) folio_mark_dirty(folio); /* Setup special migration page table entry */ From 83abe2fd5b3aeb3123b5408a5a91709c5538fb23 Mon Sep 17 00:00:00 2001 From: "Kiryl Shutsemau (Meta)" Date: Wed, 8 Jul 2026 10:01:10 +0100 Subject: [PATCH 06/13] mm/hugetlb: fix swap entry corruption when clearing uffd-wp at fork() copy_hugetlb_page_range() clears the uffd-wp bit of migration and hwpoison entries with huge_pte_clear_uffd_wp(), which operates on the present-PTE bit position. Swap entries keep the uffd-wp state elsewhere -- the migration branch reads and sets it with pte_swp_uffd_wp() and pte_swp_mkuffd_wp() -- and the present-PTE position falls into the swap payload. On x86-64 it lands in the inverted swap offset, where a naturally-aligned hugetlb PFN always has the affected bit set, so the clear advances the encoded PFN by two pages. No userfaultfd needs to be involved: the clear is guarded only by the child VMA not being uffd-wp registered, so a plain fork() with an in-flight hugetlb migration entry (or a poisoned hugetlb page) corrupts the entry copied into the child. Instrumenting the clear and forking after MADV_HWPOISON on a 2MB anon hugetlb page shows: offset before=120e00 offset after =120e02 The fallout is mostly latent: rmap walks match migration entries by folio range and remove_migration_pte() rebuilds the PTE from the folio, so a within-folio PFN skew heals once migration completes. But any path that re-encodes the corrupted offset -- e.g. hugetlb_change_protection() rewriting a writable migration entry via make_readable_migration_entry(swp_offset(entry)) -- propagates it. Migration entries legitimately carry uffd-wp, so clear it with pte_swp_clear_uffd_wp(), matching copy_nonpresent_pte() and move_huge_pte(). A hwpoison entry, on the other hand, never carries the uffd-wp bit: it is installed fresh by make_hwpoison_entry() (try_to_unmap_one() does not preserve uffd-wp on the hwpoison path) and hugetlb_change_protection() leaves hwpoison entries untouched. There was nothing to clear there, only the corruption, so drop the clear entirely. Link: https://lore.kernel.org/20260708090110.136162-1-kirill@shutemov.name Fixes: bc70fbf269fd ("mm/hugetlb: handle uffd-wp during fork()") Signed-off-by: Kiryl Shutsemau Reported-by: Sashiko AI review Closes: https://lore.kernel.org/all/20260703140011.99E601F000E9@smtp.kernel.org/ Suggested-by: David Hildenbrand Acked-by: David Hildenbrand (Arm) Assisted-by: Claude:claude-fable-5 Cc: Muchun Song Cc: Oscar Salvador Cc: Peter Xu Cc: Signed-off-by: Andrew Morton --- mm/hugetlb.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 571212b80835..bca2707d02e3 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -4917,8 +4917,12 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, softleaf = softleaf_from_pte(entry); if (unlikely(softleaf_is_hwpoison(softleaf))) { - if (!userfaultfd_wp(dst_vma)) - entry = huge_pte_clear_uffd_wp(entry); + /* + * A hwpoison entry never carries the uffd-wp bit: it is + * installed fresh by make_hwpoison_entry() and + * hugetlb_change_protection() leaves it untouched, so + * there is nothing to clear for the child. + */ set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(softleaf_is_migration(softleaf))) { bool uffd_wp = pte_swp_uffd_wp(entry); @@ -4936,7 +4940,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, set_huge_pte_at(src, addr, src_pte, entry, sz); } if (!userfaultfd_wp(dst_vma)) - entry = huge_pte_clear_uffd_wp(entry); + entry = pte_swp_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(pte_is_marker(entry))) { const pte_marker marker = copy_pte_marker(softleaf, dst_vma); From 7441d6348c70738e9ed307510db171c7a9b3f4bf Mon Sep 17 00:00:00 2001 From: Aboorva Devarajan Date: Thu, 9 Jul 2026 01:49:54 +0530 Subject: [PATCH 07/13] mm/util: don't read __page_2 for order-1 folios in snapshot_page() snapshot_page() currently reads __page_2 after checking nr_pages > 1, but it should only do so when nr_pages > 2. If an order-1 folio is allocated at the end of a vmemmap section, __page_2 will not exist and reading it will cause a fault. During DLPAR memory remove on a 22 TB ppc64le LPAR, snapshot_page() oopsed on the page isolation path while reading an order-1 folio's __page_2 from an adjacent absent section (unmapped vmemmap). Fix this to avoid reading memmap that doesn't exist (e.g., a vmemmap hole). Link: https://lore.kernel.org/20260708201954.686111-1-aboorvad@linux.ibm.com Fixes: 31a31da8a618 ("mm: move _pincount in folio to page[2] on 32bit") Signed-off-by: Aboorva Devarajan Reported-by: Sourabh Jain Acked-by: David Hildenbrand (Arm) Reviewed-by: Lorenzo Stoakes Reviewed-by: Matthew Wilcox (Oracle) Reviewed-by: Luiz Capitulino Cc: Liam R. Howlett Cc: Michal Hocko Cc: Mike Rapoport Cc: "Ritesh Harjani (IBM)" Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: # v6.15+ Signed-off-by: Andrew Morton --- mm/util.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/util.c b/mm/util.c index af2c2103f0d9..34cb43b3eaa4 100644 --- a/mm/util.c +++ b/mm/util.c @@ -1353,7 +1353,7 @@ void snapshot_page(struct page_snapshot *ps, const struct page *page) if (ps->idx < MAX_FOLIO_NR_PAGES) { memcpy(&ps->folio_snapshot, foliop, 2 * sizeof(struct page)); nr_pages = folio_nr_pages(&ps->folio_snapshot); - if (nr_pages > 1) + if (nr_pages > 2) memcpy(&ps->folio_snapshot.__page_2, &foliop->__page_2, sizeof(struct page)); set_ps_flags(ps, foliop, page); From 89b1b79c308818a715e75f28744b70d8940a07c9 Mon Sep 17 00:00:00 2001 From: Zi Yan Date: Thu, 9 Jul 2026 15:12:01 -0400 Subject: [PATCH 08/13] mm/percpu-km: fix bitmap overflow and accounting in pcpu_create_chunk() In pcpu_create_chunk(), nr_pages is the total contiguous backing allocation, i.e., nr_units * pcpu_unit_pages, but pcpu_chunk_populated() uses it to set chunk->populated, whose size is pcpu_unit_pages, bitmap. Since bit N in chunk->populated means page offset N inside every unit is backed. When nr_units > 1, the function writes beyond chunk->populated. Fix it by using chunk->nr_pages. It also fixes the global pcpu_nr_empty_pop_pages accounting, since pcpu_balance_free() only iterates up to chunk->nr_pages. Commit a63d4ac4ab609 ("percpu: make percpu-km set chunk->populated bitmap properly") introduced the bitmap overflow issue. Later, commit b539b87fed37f ("percpu: implmeent pcpu_nr_empty_pop_pages and chunk->nr_populated") added pcpu_nr_empty_pop_pages and caused the accounting issue. Link: https://lore.kernel.org/20260709-fix-pcpu_create_chunk-in-percpu-km-v1-1-1f64745a84cc@nvidia.com Fixes: a63d4ac4ab609 ("percpu: make percpu-km set chunk->populated bitmap properly") Reported-by: Sashiko Closes: https://sashiko.dev/#/patchset/20260703-keep-subpage-private-zero-at-free-v2-0-2970fe777dd6%40nvidia.com?part=1 Assisted-by: Codex:GPT-5 Signed-off-by: Zi Yan Acked-by: Dennis Zhou Cc: Christoph Lameter Cc: Tejun Heo Cc: Zi Yan Cc: Signed-off-by: Andrew Morton --- mm/percpu-km.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/percpu-km.c b/mm/percpu-km.c index 4efa74a495cb..dc096b5a6ce4 100644 --- a/mm/percpu-km.c +++ b/mm/percpu-km.c @@ -75,7 +75,7 @@ static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp) chunk->base_addr = page_address(pages); spin_lock_irqsave(&pcpu_lock, flags); - pcpu_chunk_populated(chunk, 0, nr_pages); + pcpu_chunk_populated(chunk, 0, chunk->nr_pages); spin_unlock_irqrestore(&pcpu_lock, flags); pcpu_stats_chunk_alloc(); From df8ce7ab48d01ac4f247599b35f0506d95ff57e1 Mon Sep 17 00:00:00 2001 From: Joseph Qi Date: Fri, 10 Jul 2026 12:05:12 +0800 Subject: [PATCH 09/13] ocfs2: fix boundary check in ocfs2_check_dir_entry() to use buffer offset Commit 390ac56cf0f6 ("ocfs2: add boundary check to ocfs2_check_dir_entry()") added an out-of-bounds guard using the caller-supplied 'offset' argument: if (offset > size - OCFS2_DIR_REC_LEN(1)) return 0; However, 'offset' and 'size' are not measured against the same base for all callers. In the block-based lookup path, ocfs2_find_entry_el() passes 'offset' as an absolute offset into the whole directory: i = ocfs2_search_dirblock(bh, dir, name, namelen, block << sb->s_blocksize_bits, bh->b_data, sb->s_blocksize, res_dir); while 'size' is a single block size (sb->s_blocksize). For any directory entry located in the second or later block, 'offset' is >= sb->s_blocksize, so the guard rejects every such entry even though it is perfectly valid and lies entirely within its block buffer. This makes mounting fail for filesystems whose system directory spans more than one block, e.g. a volume formatted with a small block size: mkfs.ocfs2 -b 512 -C 4096 -N 2 -T datafiles --fs-features=usrquota,grpquota ocfs2_check_dir_entry:314 ERROR: directory entry (#18: offset=512) too close to end or out-of-bounds ocfs2_init_local_system_inodes:496 ERROR: status=-22, sysfile=12, slot=0 ocfs2_mount_volume:1757 ERROR: status = -22 The dirent's position within the buffer being validated is ((char *)de - buf), which is what the rest of the function already uses (via next_offset) and what must be bounds-checked against 'size'. Compute that buffer-relative offset and use it for the guard. The subtraction is reordered to size - buf_offset < OCFS2_DIR_REC_LEN(1) to avoid an unsigned underflow when size is smaller than the minimal record length. Link: https://lore.kernel.org/20260710040512.3310736-1-joseph.qi@linux.alibaba.com Fixes: 390ac56cf0f6 ("ocfs2: add boundary check to ocfs2_check_dir_entry()") Signed-off-by: Joseph Qi Reviewed-by: Dmitry Antipov Tested-by: Dmitry Antipov Cc: Mark Fasheh Cc: Joel Becker Cc: Junxiao Bi Cc: Changwei Ge Cc: Jun Piao Cc: Heming Zhao Cc: Signed-off-by: Andrew Morton --- fs/ocfs2/dir.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/ocfs2/dir.c b/fs/ocfs2/dir.c index 8e6b03238327..d7fc3cccf2f4 100644 --- a/fs/ocfs2/dir.c +++ b/fs/ocfs2/dir.c @@ -302,10 +302,11 @@ static int ocfs2_check_dir_entry(struct inode *dir, unsigned long offset) { const char *error_msg = NULL; + unsigned long buf_offset = (char *)de - buf; unsigned long next_offset; int rlen; - if (offset > size - OCFS2_DIR_REC_LEN(1)) { + if (buf_offset > size || size - buf_offset < OCFS2_DIR_REC_LEN(1)) { /* Dirent is (maybe partially) beyond the buffer * boundaries so touching 'de' members is unsafe. */ @@ -316,7 +317,7 @@ static int ocfs2_check_dir_entry(struct inode *dir, } rlen = le16_to_cpu(de->rec_len); - next_offset = ((char *) de - buf) + rlen; + next_offset = buf_offset + rlen; if (unlikely(rlen < OCFS2_DIR_REC_LEN(1))) error_msg = "rec_len is smaller than minimal"; From e3a0127eee04db8769e53c8102c4e76aa49be8c3 Mon Sep 17 00:00:00 2001 From: Jori Koolstra Date: Fri, 10 Jul 2026 19:17:35 +0200 Subject: [PATCH 10/13] selftest: fix headers in fclog.c fclog.c does not compile because it is missing fcntl.h, needed for O_RDONLY etc. There are also some redundant includes that are also in kselftest_harness.h. Link: https://lore.kernel.org/20260710171741.837308-1-jkoolstra@xs4all.nl Signed-off-by: Jori Koolstra Cc: Aleksa Sarai Cc: Shuah Khan Cc: Wei Yang Cc: Christian Brauner Cc: Signed-off-by: Andrew Morton --- tools/testing/selftests/filesystems/fclog.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/tools/testing/selftests/filesystems/fclog.c b/tools/testing/selftests/filesystems/fclog.c index 551c4a0f395a..593a5136e991 100644 --- a/tools/testing/selftests/filesystems/fclog.c +++ b/tools/testing/selftests/filesystems/fclog.c @@ -6,10 +6,8 @@ #include #include +#include #include -#include -#include -#include #include #include From de4660898b7aa7e03d3b120a6bfa6b26211e4e77 Mon Sep 17 00:00:00 2001 From: Qi Zheng Date: Fri, 10 Jul 2026 23:43:18 +0800 Subject: [PATCH 11/13] mm: mglru: fix stale batch updates after memcg reparenting The mglru page table walker batches per-generation size deltas in walk->nr_pages while walking page tables without holding the lruvec lock. The reset_batch_size() later folds those deltas into walk->lruvec under the lruvec lock. The page table walker can run concurrently with the memcg reparenting path as follows: CPU0 CPU1 ==== ==== walk_mm --> walk_page_range --> update_batch_size --> walk->nr_pages += delta mem_cgroup_css_offline --> memcg_reparent_objcgs --> lock lruvec lru_gen_reparent_memcg --> reparent child folios to parent unlock lruvec lock lruvec reset_batch_size --> child lrugen->nr_pages += delta This will trigger the following warning in lru_gen_exit_memcg(): VM_WARN_ON_ONCE(memchr_inv(lruvec->lrugen.nr_pages, 0, sizeof(lruvec->lrugen.nr_pages))); And the user-visible impact of underestimated nr_pages in MGLRU was premature OOMs because MGLRU does not try to reclaim memory when nr_pages reaches zero, but there are still more pages. To fix it, make reset_batch_size() check CSS_DYING under RCU before flushing the pending batch. A non-dying memcg keeps the original lruvec stable against RCU-delayed offlining; a dying memcg redirects the deltas to the first non-dying ancestor. Link: https://lore.kernel.org/20260710154318.75388-1-qi.zheng@linux.dev Fixes: f304652609ea ("mm: vmscan: prepare for reparenting MGLRU folios") Signed-off-by: Qi Zheng Reported-by: Peiyang He Closes: https://lore.kernel.org/all/5A9E929D82717101+12fcf643-efb8-4b9a-a53a-1e28cc894f0b@smail.nju.edu.cn Reviewed-by: Harry Yoo (Oracle) Acked-by: Johannes Weiner Acked-by: Shakeel Butt Cc: Axel Rasmussen Cc: Barry Song Cc: David Hildenbrand Cc: Kairui Song Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Muchun Song Cc: Roman Gushchin Cc: Wei Xu Cc: Yuanchu Xie Cc: Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 25 +++++++++++++++++++++++++ mm/vmscan.c | 11 ++++------- 2 files changed, 29 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index e1f46a0016fc..957260677678 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1472,6 +1472,31 @@ static inline void lruvec_lock_irq(struct lruvec *lruvec) spin_lock_irq(&lruvec->lru_lock); } +static inline struct lruvec *lruvec_live_lock_irq(struct lruvec *lruvec) +{ +#ifdef CONFIG_MEMCG + struct pglist_data *pgdat = lruvec_pgdat(lruvec); + struct mem_cgroup *memcg = lruvec_memcg(lruvec); + + rcu_read_lock(); + + /* + * The memcg can be NULL when the memory controller is disabled. + * Otherwise, the caller keeps the memcg owning @lruvec alive. + */ + while (unlikely(memcg && css_is_dying(&memcg->css))) { + memcg = parent_mem_cgroup(memcg); + lruvec = mem_cgroup_lruvec(memcg, pgdat); + } + + spin_lock_irq(&lruvec->lru_lock); +#else + lruvec_lock_irq(lruvec); +#endif + + return lruvec; +} + static inline void lruvec_unlock(struct lruvec *lruvec) { spin_unlock(&lruvec->lru_lock); diff --git a/mm/vmscan.c b/mm/vmscan.c index 35c3bb15ae96..1a142c58700d 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3265,7 +3265,7 @@ static void update_batch_size(struct lru_gen_mm_walk *walk, struct folio *folio, static void reset_batch_size(struct lru_gen_mm_walk *walk) { int gen, type, zone; - struct lruvec *lruvec = walk->lruvec; + struct lruvec *lruvec = lruvec_live_lock_irq(walk->lruvec); struct lru_gen_folio *lrugen = &lruvec->lrugen; walk->batched = 0; @@ -3285,6 +3285,8 @@ static void reset_batch_size(struct lru_gen_mm_walk *walk) lru += LRU_ACTIVE; __update_lru_size(lruvec, lru, zone, delta); } + + lruvec_unlock_irq(lruvec); } static int should_skip_vma(unsigned long start, unsigned long end, struct mm_walk *args) @@ -3779,11 +3781,8 @@ static void walk_mm(struct mm_struct *mm, struct lru_gen_mm_walk *walk) mmap_read_unlock(mm); } - if (walk->batched) { - lruvec_lock_irq(lruvec); + if (walk->batched) reset_batch_size(walk); - lruvec_unlock_irq(lruvec); - } cond_resched(); } while (err == -EAGAIN); @@ -4867,9 +4866,7 @@ static int evict_folios(unsigned long nr_to_scan, struct lruvec *lruvec, walk = current->reclaim_state->mm_walk; if (walk && walk->batched) { walk->lruvec = lruvec; - lruvec_lock_irq(lruvec); reset_batch_size(walk); - lruvec_unlock_irq(lruvec); } mod_lruvec_state(lruvec, PGDEMOTE_KSWAPD + reclaimer_offset(sc), From dd9623f58ec702a07b2d67179d6fcea79c52231a Mon Sep 17 00:00:00 2001 From: Xiangfeng Cai Date: Tue, 14 Jul 2026 01:14:55 +0800 Subject: [PATCH 12/13] mm/hugetlb: fix list corruption in allocate_file_region_entries() allocate_file_region_entries() tops up resv->region_cache with freshly allocated file_region descriptors. The allocation uses GFP_KERNEL, so resv->lock is dropped around it: the new entries are gathered on a stack-local list head, allocated_regions, and spliced into resv->region_cache once the lock is re-acquired. The splice used list_splice(), which moves the entries but does not re-initialize the source head, so allocated_regions is left pointing at an entry that now lives on resv->region_cache. The top-up runs in a while loop that re-checks the cache deficit after re-acquiring the lock. For a shared mapping the resv_map is shared by every mapper of the hugetlbfs inode, so a concurrent region_chg()/region_add()/region_del() on the same resv_map can consume cache entries during the unlocked window and force a second iteration. That iteration calls list_add() on the stale head and corrupts the list; with CONFIG_DEBUG_LIST the __list_add_valid() check trips: list_add corruption. next->prev should be prev (ffffc900011ff7f8), but was ffff88814c281460. (next=ffff88814c545640). kernel BUG at lib/list_debug.c:31! allocate_file_region_entries+0x191/0x420 region_chg+0x267/0x300 hugetlb_reserve_pages+0x387/0xc80 hugetlbfs_file_mmap+0x2ce/0x3f0 mmap_region+0x1348/0x1a80 do_mmap+0x85e/0xb90 vm_mmap_pgoff+0x18c/0x330 ksys_mmap_pgoff+0x2a1/0x3e0 do_syscall_64+0xd7/0x420 Without CONFIG_DEBUG_LIST the bad list_add() silently links a kernel-stack address into resv->region_cache, leading to later use-after-free. This was observed as a real host panic on a dense KVM host where a QEMU guest-RAM hugetlbfs file was mapped MAP_SHARED by both QEMU and a separate SPDK/DPDK vhost-user target, generating concurrent region_* traffic on one shared resv_map. Use list_splice_init() so the source head is re-initialized empty after each splice, making the retry loop safe. Link: https://lore.kernel.org/20260713171456.300518-2-caixiangfeng@bytedance.com Fixes: d3ec7b6e09e5 ("mm/hugetlb: use list_splice to merge two list at once") Signed-off-by: Xiangfeng Cai Reviewed-by: Muchun Song Cc: Baoquan He Cc: David Hildenbrand Cc: Oscar Salvador Cc: Shuah Khan Cc: Wei Yang Cc: Signed-off-by: Andrew Morton --- mm/hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index bca2707d02e3..e93c4d2456aa 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -693,7 +693,7 @@ static int allocate_file_region_entries(struct resv_map *resv, spin_lock(&resv->lock); - list_splice(&allocated_regions, &resv->region_cache); + list_splice_init(&allocated_regions, &resv->region_cache); resv->region_cache_count += to_allocate; } From 40de8160ca7f67d14619ee0351ce5d68fc4a237a Mon Sep 17 00:00:00 2001 From: "Kiryl Shutsemau (Meta)" Date: Wed, 15 Jul 2026 15:42:33 +0100 Subject: [PATCH 13/13] fs/proc/task_mmu: fix PAGEMAP_SCAN written state for PMD holes PAGEMAP_SCAN reports an unpopulated PTE in a uffd-wp VMA as written, but a range with no page table at all -- a PMD hole -- is skipped: pagemap_scan_pte_hole() tests p->cur_vma_category, which never carries PAGE_IS_WRITTEN, so the hole is neither reported nor (under PM_SCAN_WP_MATCHING) armed. In a uffd-wp VMA, WP_UNPOPULATED installs uffd-wp markers when protecting a range, allocating page tables as needed, so an unpopulated slot is treated as written -- see the pte_none() handling in pagemap_page_category(). A missing marker therefore means the range was zapped, e.g. via MADV_DONTNEED. This applies to anon and shmem VMAs. An anonymous THP is write-protected in place as a huge PMD, so a full-PMD MADV_DONTNEED clears it to pmd_none -- a hole with no page table -- and pagemap_scan_pte_hole() misses it. For a MAP_PRIVATE|MAP_ANON mapping MADV_DONTNEED has fill-with-zeros semantics, so a write-tracking checkpoint/migration tool (e.g. CRIU) treats the range as unchanged and keeps its previous contents; after restore or live migration the process reads stale data instead of zeroes -- data corruption. Report a hole in a non-hugetlb uffd-wp VMA as written, matching the pte_none handling in pagemap_page_category(); the existing PM_SCAN_WP_MATCHING path then arms it via uffd_wp_range(). hugetlb is excluded: pagemap_hugetlb_category() reports an empty hugetlb entry (huge_pte_none) as not-written, unlike pagemap_page_category(), which reports pte_none as written. pagemap_scan_pte_hole() fires for a hugetlb slot only when it has no page table; keeping that not-written matches how an allocated-but-empty hugetlb entry reads, so the hole and the empty-entry cases agree within the VMA. Link: https://lore.kernel.org/20260715144234.442721-2-kirill@shutemov.name Fixes: 2bad466cc9d9 ("mm/uffd: UFFD_FEATURE_WP_UNPOPULATED") Signed-off-by: Kiryl Shutsemau Reported-by: Sashiko AI review Closes: https://sashiko.dev/#/patchset/20260707151349.92143-1-kirill@shutemov.name Tested-by: Muhammad Usama Anjum Acked-by: David Hildenbrand (Arm) Cc: Peter Xu Cc: Jann Horn Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Pedro Falcato Cc: Shuah Khan Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Zenghui Yu Assisted-by: Claude:claude-fable-5 Cc: Signed-off-by: Andrew Morton --- fs/proc/task_mmu.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index d45c729ab6bb..229d1fc3d7f1 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3049,12 +3049,28 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end, { struct pagemap_scan_private *p = walk->private; struct vm_area_struct *vma = walk->vma; + unsigned long categories; int ret, err; - if (!vma || !pagemap_scan_is_interesting_page(p->cur_vma_category, p)) + if (!vma) return 0; - ret = pagemap_scan_output(p->cur_vma_category, p, addr, &end); + /* + * In a uffd-wp VMA an unpopulated range is treated as written: + * uffd-wp registration populates page tables and installs markers + * with WP_UNPOPULATED, so a missing marker means the range was + * zapped. See the pte_none() handling in pagemap_page_category(). + * + * hugetlb differs, see pagemap_hugetlb_category(). + */ + categories = p->cur_vma_category; + if (userfaultfd_wp(vma) && !is_vm_hugetlb_page(vma)) + categories |= PAGE_IS_WRITTEN; + + if (!pagemap_scan_is_interesting_page(categories, p)) + return 0; + + ret = pagemap_scan_output(categories, p, addr, &end); if (addr == end) return ret;