github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph
1,505,200 Commits 1 Branch 952 Tags 16 GiB
v7.4-rc1
Commit Graph

275 Commits

Author SHA1 Message Date
Linus Torvalds
8a36f53fd5 mm.git review status for linus..mm-stable 
Everything:
 
 Total patches:       213
 Fixes: tags:         69
 cc:stable patches:   43
 Reviews/patch:       2.04
 Reviewed rate:       86%
 
 Excluding DAMON:
 
 Total patches:       192
 Fixes: tags:         63
 cc:stable patches:   39
 Reviews/patch:       2.18
 Reviewed rate:       89%
 
 Excluding DAMON and selftests:
 
 Total patches:       182
 Fixes: tags:         57
 cc:stable patches:   39
 Reviews/patch:       2.23
 Reviewed rate:       89%
 
 Summary of series in this merge:
 
 - "mm: make VMA flag semantics explicit, eliminate VM_SPECIAL"
   (Lorenzo Stoakes (ARM)):
 
   Replace VM_SPECIAL with helpers describing mapping ownership and
   behavior.  Enforce the mmap flag rules and convert affected drivers,
   removing ambiguous tests shared by driver mappings and ordinary memory.
 
 - "support kselftest on nommu platform" (Hajime Tazaki):
 
   Add opt-in mmap and mremap tests for NOMMU systems.  Teach the runner to
   avoid its fork-dependent timeout command and document the build and
   runtime requirements.
 
 - "mm/collapse: separate a collapse from its callers"
   (Kiryl Shutsemau (Meta)):
 
   Separate huge-page collapse from caller policy.  Replace khugepaged
   identity checks with explicit permissions, and separate scanning from
   collapse with the mmap_lock drop in one place.
 
 - "mm: implement and use vma_has_anon_rmap(), silence KCSAN"
   (Lorenzo Stoakes (ARM)):
 
   Provide a common helper for testing whether a VMA has anonymous reverse
   mappings.  Document the locking rules, annotate intentional concurrent
   reads for KCSAN, and convert the core MM callers.
 
 - "mm/sparse: remove SECTION_MARKED_PRESENT and further cleanups"
   (David Hildenbrand (Arm)):
 
   Remove SECTION_MARKED_PRESENT in favor of the early-section flag.
   Simplify sparse memory initialization, section metadata handling and
   related PFN validity checks.
 
 - "sh: remove NUMA and SPARSEMEM support" (Mike Rapoport (Microsoft)):
 
   Remove SuperH NUMA and SPARSEMEM support.  The extra memory banks have
   been unusable since the NO_BOOTMEM conversion, leaving only the code and
   data overhead.
 
 - "mm: Unify device DAX and HugeTLB vmemmap population paths"
   (Muchun Song):
 
   Unify the generic device DAX and HugeTLB vmemmap population paths and
   shared tail-page handling.  Remove the duplicate DAX machinery while
   retaining the powerpc radix implementation.
 
 - "kselftest: mm: fix some failure of split_huge_page_test" (Yeoreum Yun):
 
   Fix huge-page split tests racing with khugepaged recollapse.  Use pagemap
   and page flags instead of VMA-boundary assumptions which fail when glibc
   has already enabled THP.
 
 - "memcg: move memcgid refcount to objcg to unpin dying memcgs"
   (Bingfang Guo):
 
   Let dying memcgs be freed while swapped-out shmem still holds their IDs.
   Move ID references to reparentable objcgs, with the necessary swap
   accounting and RCU lifetime changes.
 
 - "mm/truncate: fix data loss when truncating straddling large folios"
   (Zhang Yi):
 
   Preserve out-of-range data when truncating straddling large folios.  Fix
   a lookup race and split-failure handling, and handle minimum folio orders
   for the forthcoming ext4 iomap buffered I/O path.
 
 - "mm/damon: introduce damos quota goal target metric complement flag"
   (SJ Park):
 
   Let DAMOS quota goals use a metric's complement, so utilization can also
   express free-memory goals without adding separate metrics.  Add the sysfs
   interface, documentation and tests.
 
 - "arm64/mm: Standardize printing for pgtable entries" (Anshuman Khandual):
 
   Use the generic formatter for arm64 page-table entries, ready for future
   wider entries.  Move the safe-PTE-update check out of the header to avoid
   an include cycle.
 
 - "mm: kmsan: fix iounmap metadata teardown" (Dima Koziuk):
 
   Fix KMSAN metadata leaks on iounmap and failed ioremap allocations.  Free
   backing pages before removing their mappings, and share the teardown with
   partial-allocation cleanup.
 
 - "mm/damon/core: preserve quota state when constructing schemes"
   (Karl Mehltretter):
 
   Preserve running DAMOS quota state while validating parameter updates.
   Even rejected updates could reset accounting and exceed the quota window;
   add tests for both accepted and rejected updates.
 
 - "mm/vmalloc: fix vmalloc_dump_obj VA lookup" (Ye Liu):
 
   Fix vmalloc_dump_obj() lookups for last-page addresses and allocations
   spanning vmap zones.  These allocations were being missed in diagnostic
   output.
 
 - "mm: various (urgent?) fixes" (David Hildenbrand (Arm)):
 
   Fix mmap charge leaks and hugetlb use-after-free, wrong-address
   unmapping, stuck locks and endless faults.  Also preserve reservations on
   failed mremap and handle stale userfaultfd bits.
 
 - "maple_tree: fix maple_range_64 crashes in RCU mode" (Dimitris Charisis):
 
   Fix pointer and metadata confusion during RCU destruction of
   maple_range_64 nodes, avoiding double frees and invalid accesses.  Add a
   regression test; no in-tree user currently uses this combination.
 
 - "mm: fix hugetlb NR_HUGETLB accounting on folio migration" (Hongfu Li):
 
   Fix hugetlb migration and preallocation accounting so /proc/vmstat
   retains the correct count and memory.numa_stat follows the folio to its
   new node.
 
 - "mm/damon: fix a minor bug, clean code, and add a test" (SJ Park):
 
   Fix 32-bit DAMON age overflow which made old regions look young.  Share
   hugetlb folio acquisition code and add a regression test for the
   previously fixed last-region skipping bug.
 
 - "mm/damon/core: cleanup code, reduce stack usage, and add kunit"
   (SJ Park):
 
   Reduce kdamond stack usage and remove unused enums.  Add deterministic
   PSI quota-goal tests which do not depend on the machine's current memory
   pressure.
 
 - "mm/damon: fix the temporal goal tuner's size quota conversion"
   (Donggeun Yoo):
 
   Bound the temporal tuner's quota conversion to prevent overflow and zero
   effective quotas, which could stop schemes making progress.  Add boundary
   tests.
 
 - "Don't use GFP_DMA when calling dma_alloc_coherent" (Baoquan He):
 
   Remove redundant GFP_DMA flags from coherent DMA allocation calls.  The
   DMA API already selects the zone from device limits and ignores the
   caller's flag.
 
 - "kselftest: mm: fix intermittent failure khugepaged test" (Yeoreum Yun):
 
   Propagate child-process failures in the khugepaged tests and prevent
   background collapse from racing their checks.  This removes false passes
   and intermittent failures.
 
 - "MAINTAINERS: cover all hugetlb headers under HUGETLB SUBSYSTEM"
   (Enze Li):
 
   Cover all hugetlb headers in MAINTAINERS so get_maintainer.pl finds the
   right maintainers.  Also label the closing hugetlb_cgroup.h include
   guard.
 
 Summary of the 65 singletons in this merge:
 
 The singletons include a few worthwhile performance fixes. Avoiding
 unnecessary reverse-map tree updates gave a reported 15.9% improvement
 in UnixBench's execl test on a 192-core x86 system. A reclaim retry fix
 stops defrag_mode from spinning under memory.low protection; a reported
 file-creation stress test ran 5.7 times faster, with far less memory
 pressure. Narrower hugetlb invalidations also spare KVM guests needless
 refaults: the reported Redis-in-VM P99 QPS penalty while punching holes
 fell from 7.09% to 1.45%.
 
 Several fixes prevent lost data or corrupted memory. Lazyfree huge-page
 races with mremap()/munmap() could lose data or access freed memory, and
 MADV_COLD could lose a concurrent write's dirty bit. Another fix
 prevents stale migration entries from outliving large folios. On
 filesystems with blocks smaller than a page, extending a file by 4 GiB
 or more could expose bytes written past the old EOF through mmap instead
 of zeroes. That is fixed, along with use-after-free races in
 PAGEMAP_SCAN during process exit and in memory-cgroup accounting on
 PREEMPT_RT kernels.
 
 Other fixes stop UFFDIO_MOVE spinning forever on pinned large folios,
 which could wedge other memory operations, and stop mremap() leaking
 commit charges until allocations fail under strict overcommit. A further
 fix keeps userfaultfd tracking intact on neighboring mappings. NUMA
 binding no longer prevents long-term pins when the chosen node has only
 movable memory, fixing QEMU/VFIO startup and io_uring buffer
 registration failures. MGLRU also gets a fix for early-boot crashes with
 memory cgroups disabled at boot, notably in small crash kernels.
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCat66sAAKCRDdBJ7gKXxA
 jrQrAP99kgSXRdlmRZbHeRBHQ6KvI4XnLvQ78MDIzZ3zmBaOugEA5jiIsK2/snFD
 3VaV0oipVZpuwJkVJE6zFkh0tcHgTAo=
 =DHrN
 -----END PGP SIGNATURE-----

Merge tag 'mm-stable-2026-10-25-19-27' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull more MM updates from Andrew Morton:

 - "mm: make VMA flag semantics explicit, eliminate VM_SPECIAL" (Lorenzo
   Stoakes):

   Replace VM_SPECIAL with helpers describing mapping ownership and
   behavior. Enforce the mmap flag rules and convert affected drivers,
   removing ambiguous tests shared by driver mappings and ordinary
   memory.

 - "support kselftest on nommu platform" (Hajime Tazaki):

   Add opt-in mmap and mremap tests for NOMMU systems. Teach the runner
   to avoid its fork-dependent timeout command and document the build
   and runtime requirements.

 - "mm/collapse: separate a collapse from its callers" (Kiryl
   Shutsemau):

   Separate huge-page collapse from caller policy. Replace khugepaged
   identity checks with explicit permissions, and separate scanning from
   collapse with the mmap_lock drop in one place.

 - "mm: implement and use vma_has_anon_rmap(), silence KCSAN" (Lorenzo
   Stoakes):

   Provide a common helper for testing whether a VMA has anonymous
   reverse mappings. Document the locking rules, annotate intentional
   concurrent reads for KCSAN, and convert the core MM callers.

 - "mm/sparse: remove SECTION_MARKED_PRESENT and further cleanups"
   (David Hildenbrand):

   Remove SECTION_MARKED_PRESENT in favor of the early-section flag.
   Simplify sparse memory initialization, section metadata handling and
   related PFN validity checks.

 - "sh: remove NUMA and SPARSEMEM support" (Mike Rapoport):

   Remove SuperH NUMA and SPARSEMEM support. The extra memory banks have
   been unusable since the NO_BOOTMEM conversion, leaving only the code
   and data overhead.

 - "mm: Unify device DAX and HugeTLB vmemmap population paths" (Muchun
   Song):

   Unify the generic device DAX and HugeTLB vmemmap population paths and
   shared tail-page handling. Remove the duplicate DAX machinery while
   retaining the powerpc radix implementation.

 - "kselftest: mm: fix some failure of split_huge_page_test" (Yeoreum
   Yun):

   Fix huge-page split tests racing with khugepaged recollapse. Use
   pagemap and page flags instead of VMA-boundary assumptions which fail
   when glibc has already enabled THP.

 - "memcg: move memcgid refcount to objcg to unpin dying memcgs"
   (Bingfang Guo):

   Let dying memcgs be freed while swapped-out shmem still holds their
   IDs. Move ID references to reparentable objcgs, with the necessary
   swap accounting and RCU lifetime changes.

 - "mm/truncate: fix data loss when truncating straddling large folios"
   (Zhang Yi):

   Preserve out-of-range data when truncating straddling large folios.
   Fix a lookup race and split-failure handling, and handle minimum
   folio orders for the forthcoming ext4 iomap buffered I/O path.

 - "mm/damon: introduce damos quota goal target metric complement flag"
   (SJ Park):

   Let DAMOS quota goals use a metric's complement, so utilization can
   also express free-memory goals without adding separate metrics. Add
   the sysfs interface, documentation and tests.

 - "arm64/mm: Standardize printing for pgtable entries" (Anshuman
   Khandual):

   Use the generic formatter for arm64 page-table entries, ready for
   future wider entries. Move the safe-PTE-update check out of the
   header to avoid an include cycle.

 - "mm: kmsan: fix iounmap metadata teardown" (Dima Koziuk):

   Fix KMSAN metadata leaks on iounmap and failed ioremap allocations.
   Free backing pages before removing their mappings, and share the
   teardown with partial-allocation cleanup.

 - "mm/damon/core: preserve quota state when constructing schemes" (Karl
   Mehltretter):

   Preserve running DAMOS quota state while validating parameter
   updates. Even rejected updates could reset accounting and exceed the
   quota window; add tests for both accepted and rejected updates.

 - "mm/vmalloc: fix vmalloc_dump_obj VA lookup" (Ye Liu):

   Fix vmalloc_dump_obj() lookups for last-page addresses and
   allocations spanning vmap zones. These allocations were being missed
   in diagnostic output.

 - "mm: various (urgent?) fixes" (David Hildenbrand):

   Fix mmap charge leaks and hugetlb use-after-free, wrong-address
   unmapping, stuck locks and endless faults. Also preserve reservations
   on failed mremap and handle stale userfaultfd bits.

 - "maple_tree: fix maple_range_64 crashes in RCU mode" (Dimitris
   Charisis):

   Fix pointer and metadata confusion during RCU destruction of
   maple_range_64 nodes, avoiding double frees and invalid accesses. Add
   a regression test; no in-tree user currently uses this combination.

 - "mm: fix hugetlb NR_HUGETLB accounting on folio migration" (Hongfu
   Li):

   Fix hugetlb migration and preallocation accounting so /proc/vmstat
   retains the correct count and memory.numa_stat follows the folio to
   its new node.

 - "mm/damon: fix a minor bug, clean code, and add a test" (SJ Park):

   Fix 32-bit DAMON age overflow which made old regions look young.
   Share hugetlb folio acquisition code and add a regression test for
   the previously fixed last-region skipping bug.

 - "mm/damon/core: cleanup code, reduce stack usage, and add kunit" (SJ
   Park):

   Reduce kdamond stack usage and remove unused enums. Add deterministic
   PSI quota-goal tests which do not depend on the machine's current
   memory pressure.

 - "mm/damon: fix the temporal goal tuner's size quota conversion"
   (Donggeun Yoo):

   Bound the temporal tuner's quota conversion to prevent overflow and
   zero effective quotas, which could stop schemes making progress. Add
   boundary tests.

 - "Don't use GFP_DMA when calling dma_alloc_coherent" (Baoquan He):

   Remove redundant GFP_DMA flags from coherent DMA allocation calls.
   The DMA API already selects the zone from device limits and ignores
   the caller's flag.

 - "kselftest: mm: fix intermittent failure khugepaged test" (Yeoreum
   Yun):

   Propagate child-process failures in the khugepaged tests and prevent
   background collapse from racing their checks. This removes false
   passes and intermittent failures.

 - "MAINTAINERS: cover all hugetlb headers under HUGETLB SUBSYSTEM"
   (Enze Li):

   Cover all hugetlb headers in MAINTAINERS so get_maintainer.pl finds
   the right maintainers. Also label the closing hugetlb_cgroup.h
   include guard.

Summary of the 65 singletons in this merge:

  The singletons include a few worthwhile performance fixes. Avoiding
  unnecessary reverse-map tree updates gave a reported 15.9% improvement
  in UnixBench's execl test on a 192-core x86 system. A reclaim retry
  fix stops defrag_mode from spinning under memory.low protection; a
  reported file-creation stress test ran 5.7 times faster, with far less
  memory pressure. Narrower hugetlb invalidations also spare KVM guests
  needless refaults: the reported Redis-in-VM P99 QPS penalty while
  punching holes fell from 7.09% to 1.45%.

  Several fixes prevent lost data or corrupted memory. Lazyfree
  huge-page races with mremap()/munmap() could lose data or access freed
  memory, and MADV_COLD could lose a concurrent write's dirty bit.
  Another fix prevents stale migration entries from outliving large
  folios. On filesystems with blocks smaller than a page, extending a
  file by 4 GiB or more could expose bytes written past the old EOF
  through mmap instead of zeroes. That is fixed, along with
  use-after-free races in PAGEMAP_SCAN during process exit and in
  memory-cgroup accounting on PREEMPT_RT kernels.

  Other fixes stop UFFDIO_MOVE spinning forever on pinned large folios,
  which could wedge other memory operations, and stop mremap() leaking
  commit charges until allocations fail under strict overcommit. A
  further fix keeps userfaultfd tracking intact on neighboring mappings.
  NUMA binding no longer prevents long-term pins when the chosen node
  has only movable memory, fixing QEMU/VFIO startup and io_uring buffer
  registration failures. MGLRU also gets a fix for early-boot crashes
  with memory cgroups disabled at boot, notably in small crash kernels.

* tag 'mm-stable-2026-10-25-19-27' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (213 commits)
  mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline
  mm/page_counter: use WRITE_ONCE() for emin/elow when parent is the root
  mm/memcontrol: publish null pointer before dropping css reference
  test_maple_tree: test maple_range_64 node in RCU mode
  maple_tree: fix maple_range_64 crashes in RCU mode
  mm/vma: avoid broken uffd merge on mremap
  tools/testing/shared: catch a failed posix_memalign()
  mm/damon/core: remove unreachable damos charge_{target,addr}_from
  mm/shrinker: allocate shrinker_info_unit from a dedicated kmem_cache
  riscv: mm: exclude invalid THP PMDs from page table check
  mm/damon/core: filter out damos address filter-split failed region
  fs/proc/task_mmu: pin mm_users in do_pagemap_scan()
  mm/mempolicy: apply bind policies against the scoped allocation zone
  mm/vmscan: reclaim every zone in per-node proactive reclaim
  mm/mremap: correctly uncharge accountable VMAs in error path
  mm/mremap: populate calculated fields in vrm state correctly
  MAINTAINERS: add an entry for MEMORY MANAGEMENT - MEMORY MAP
  mm: bdi: reset min_ratio after the sysfs attributes are gone
  mm/page_vma_mapped: continue past nonmatching huge PMDs
  selftests/mm: skip hugetlb cgroup v2 tests without the needed controllers
  ...
 2026-10-27 12:23:17 -10:00
Linus Torvalds
394023fbe3 memblock: updates for 7.4-rc1 
Performance improvement:
 
 * Use binary search in memblock_add_range() and memblock_isolate_range()
 
 Cleanups:
 
 * Several simplifications in memblock and mm_init that remvove redundand
   code.
 * After rework of sparsemem buffer preallocation, memmap_alloc() does not
   require exact_nid parameter. Remove that parameter, drop now unused
   memblock_alloc_exact_nid_raw() and ajust memblock tests accordingly.
 
 memblock test updates:
 
 * add coverage for lowmem allocations
 * fix 32-bit build warinings
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEeOVYVaWZL5900a/pOQOGJssO/ZEFAmrcgaQACgkQOQOGJssO
 /ZHVcAf9HoX58N9mA76ESc5Aogkg1PKdTFweeR4nXLTTz10amxvH0O1aERyZE8nO
 4lbqa7fUPV/0gPBtFVu7ZcBbgGZR0nmb5atKJWfKoZ373cXqdqVhN0brULLTyRPr
 18dlOSX7CmU84m8WiKpVeSZX94xwF5YeCad577y2nmEyWyQ1aZ1iqjzV97cDTcg3
 bAzWUEsLdiO2OYoyiasMQIJdk+xeFTtT61so8c3/W1gcUTZHwW1jh0gQ+hKj2E8G
 GXfkafy+nUtuvzKP7iOqc8uUIW/j3dLlO3tVp+KFjcHVP+0TLWQzB80oAmvgxuxK
 AZ7+xsxhwIxXiS9by2o5huV/oPhU3Q==
 =A8E9
 -----END PGP SIGNATURE-----

Merge tag 'memblock-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mm/memblock

Pull memblock updates from Mike Rapoport:
 "Performance improvement:

   - Use binary search in memblock_add_range() and
     memblock_isolate_range()

  Cleanups:

   - Several simplifications in memblock and mm_init that remove
     redundant code

   - After rework of sparsemem buffer preallocation, memmap_alloc() does
     not require exact_nid parameter. Remove that parameter, drop now
     unused memblock_alloc_exact_nid_raw() and adjust memblock tests
     accordingly

  memblock test updates:

   - add coverage for lowmem allocations

   - fix 32-bit build warnings"

* tag 'memblock-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mm/memblock:
  memblock: use binary search to locate candidate regions
  memblock: drop for_each_memblock_type() and open code its users
  memblock: Remove unused memblock_alloc_exact_nid_raw()
  memblock tests: Move exact nid tests to memblock_alloc_range_nid()
  mm: Remove unused exact_nid parameter from memmap_alloc()
  memblock: initialize the region merge end index
  tools: use __pa() in virt_to_phys()
  memblock tests: zero-extend simulated memory addresses
  memblock tests: use physical addresses for region removal
  memblock: reject non-page-aligned reserve_mem regions
  memblock tests: cover allocations below the low address limit
  memblock tests: model the low allocation limit within dummy memory
  mm/mm_init: remove zone_absent_pages_in_node()
  memblock: remove conditional return with no effect
  memblock: simplify reset_all_zones_managed_pages()
 2026-10-25 17:27:19 -10:00
David Hildenbrand (Arm)
0632840dba mm/sparse: remove flags parameter from sparse_init_one_section() 
The last user of the flags parameter that used to pass SECTION_IS_EARLY
was removed, so let's remove the parameter.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-11-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:09 -07:00
David Hildenbrand (Arm)
da7acb8982 mm/sparse: remove SECTION_MARKED_PRESENT 
All present section iterators run before memory hotplug added any further
memory sections, Therefore, we can simply use the SECTION_IS_EARLY flag by
setting that flag earlier in sparse_sections_init().

Get rid of SECTION_MARKED_PRESENT entirely and rename
for_each_present_section_nr() to for_each_early_section_nr().

Take care of the .clang-format for_each_present_section_nr() handling.

No functional change intended.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-10-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:08 -07:00
David Hildenbrand (Arm)
4af04bd812 mm/sparse: move __highest_used_section_nr handling 
In preparation for removing __section_mark_present(), let's move
__highest_used_section_nr handling into its callers.  Verify in
sparse_init_one_section() that it was properly updated.

In sparse_sections_init() we can just set it to the last processed
section_nr.

No functional change intended.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-8-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:08 -07:00
David Hildenbrand (Arm)
a8c5020eeb mm/sparse: rename __highest_present_section_nr to __highest_used_section_nr 
In preparation for getting rid of SECTION_MARKED_PRESENT, rename
__highest_present_section_nr and clarify the comment.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-6-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:07 -07:00
David Hildenbrand (Arm)
31b3829703 mm/sparse: rename and cleanup sparse_init_nid() 
Let's rename it to "sparse_metadata_init_nid", avoid the "pnum"
terminology and drop the function comment.  Further, rename the "map"
variable to "mem_map" for consistency with other functions.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-4-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:07 -07:00
David Hildenbrand (Arm)
d8ab71169c mm/sparse: move initialization of section metadata to sparse_metadata_init() 
Let's move the code responsible for initializing sparse metadata (usemap,
memmap) into a helper.  Cleanup the variable while at it (e.g.,
"map_count").

Drop the rather obvious code comments.

No functional change intended.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-3-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:07 -07:00
David Hildenbrand (Arm)
89e2b6e3d3 mm/sparse: refactor sparse_sections_init() 
memory_present() really identifies+prepares all early sections so the
initialization in sparse_init() can properly iterating them to initialize
metadata.

Let's just inline memory_present() into sparse_sections_init() and
cleaning up the code a bit while at it: make it clear that we are
operating on pfns.

Note that we call set_section_nid() now only if the section was not
already created earlier.  Now, there is no more inconsistency between what
we (temporarily) store in ms->section_mem_map and what we store in our
section->nid array.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-2-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:06 -07:00
David Hildenbrand (Arm)
07c81e0196 mm/sparse: move mem_section init to sparse_extreme_init() 
Patch series "mm/sparse: remove SECTION_MARKED_PRESENT and further
cleanups", v2.

SECTION_MARKED_PRESENT is really only needed during boot, where we can
instead just rely on SECTION_IS_EARLY_BIT by setting that flag earlier.

Some preparations for doing that conversion and some cleanups in the same
(sparse) area.


This patch (of 13):

Let's just avoid another pair of ifdef inside a function.  While at it,
switch to INTERNODE_CACHE_BYTES by just defining a fallback in cache.h as
well, given that the x86 variant already provides one.

Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-0-54d81d65e125@kernel.org
Link: https://lore.kernel.org/20260921-b4-sparsemem_cleanups-v2-1-54d81d65e125@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Kieran Bingham <kbingham@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Rafael J. Wysocki <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
 2026-10-24 01:56:06 -07:00
Kaitao Cheng
063b5e8bbd mm: Remove unused exact_nid parameter from memmap_alloc() 
Commit ffe5539313 ("mm/sparse: remove sparse buffer pre-allocation
mechanism") removed sparse_buffer_init(), which was the only caller of
memmap_alloc() that passed true for exact_nid.

All remaining callers pass false, leaving the exact-node allocation
branch unreachable. Remove the exact_nid parameter and unconditionally
use memblock_alloc_try_nid_raw().

This has no functional impact.

Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
Link: https://patch.msgid.link/20260919135327.74089-2-kaitao.cheng@linux.dev
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
 2026-10-13 12:45:55 +03:00
Muchun Song
76df418d29 mm/sparse-vmemmap: drop unused section_nr_vmemmap_pages() arguments 
section_nr_vmemmap_pages() no longer uses the altmap or pgmap arguments,
so drop them from the helper and its callers.

Link: https://lore.kernel.org/20260930140627.57431-12-songmuchun@bytedance.com
Signed-off-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Lance Yang <lance.yang@linux.dev>
 2026-10-08 22:55:08 -07:00
Muchun Song
7d55d0a118 mm/sparse: remove section_map_size() 
section_map_size() no longer provides any shared logic.

After the sparse-vmemmap changes, its only remaining user is the
!CONFIG_SPARSEMEM_VMEMMAP path in __populate_section_memmap(), which can
compute the size inline with PAGE_ALIGN(sizeof(struct page) *
PAGES_PER_SECTION).

Remove section_map_size() and inline the remaining calculation.

Link: https://lore.kernel.org/20260910063256.64386-14-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
 2026-10-06 17:38:51 -07:00
Muchun Song
1269621f9a mm/sparse: inline usemap allocation into sparse_init_nid() 
After removing SPARSEMEM_VMEMMAP_PREINIT, sparse_init_nid() no longer
needs the transient sparse_usagebuf state and its helper wrappers.

Allocate the usemap buffer directly in sparse_init_nid(), pass it to
sparse_init_one_section(), and drop sparse_usage_init(),
sparse_usage_fini(), and sparse_init_early_section().

Link: https://lore.kernel.org/20260910063256.64386-13-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
 2026-10-06 17:38:51 -07:00
Muchun Song
3286497153 mm/sparse-vmemmap: remove SPARSEMEM_VMEMMAP_PREINIT support 
SPARSEMEM_VMEMMAP_PREINIT existed only to support HugeTLB's early vmemmap
optimization setup.

Now that HugeTLB bootmem vmemmap optimization uses the common
section-based sparse-vmemmap path, the sparse initialization code no
longer needs a separate pre-initialization mechanism for vmemmap
population.

Remove the related Kconfig symbols, section flag, and empty early hook, so
present sections always go through the normal sparse setup path.

Link: https://lore.kernel.org/20260910063256.64386-12-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
 2026-10-06 17:38:51 -07:00
Muchun Song
74d2a4e89f mm/sparse: initialize memory sections earlier 
Upcoming HugeTLB bootmem changes need sparsemem section metadata before
the HugeTLB bootmem allocation path runs.  The memory sections are
initialized from sparse_init(), which is called too late for that setup.

Move the code that initializes sparsemem section metadata for memblock
ranges into mm_core_init_early(), before free_area_init() and the HugeTLB
bootmem setup.  Rename the helper to sparse_sections_init() so the new
caller describes the sparsemem-specific initialization step.

This is a preparatory change.

Link: https://lore.kernel.org/20260910063256.64386-10-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
 2026-10-06 17:38:50 -07:00
Muchun Song
a64b8c60c6 mm/sparse-vmemmap: support section-based vmemmap optimization 
Teach sparse-vmemmap population code to use the compound page order when
deciding whether a vmemmap page can be optimized.

With this information, the common sparse-vmemmap population path can
allocate or reuse shared tail vmemmap pages directly instead of relying on
HugeTLB-specific handling.

This centralizes vmemmap optimization logic in the sparse-vmemmap code,
based on section metadata, and prepares for sharing the same mechanism
across different users of vmemmap optimization, including HugeTLB and DAX.

Link: https://lore.kernel.org/20260910063256.64386-9-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Cc: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
 2026-10-06 17:38:50 -07:00
Muchun Song
48070b721a mm/sparse: relax struct mem_section size constraints 
Patch series "mm: Introduce section-based vmemmap optimization for
HugeTLB", v6.

This series is split out from the earlier, larger series "mm: Generalize
HVO for HugeTLB and device DAX" [1].  While the parent series generalizes
vmemmap optimization across HugeTLB and device DAX, this subset addresses
a single, self-contained step: making the generic sparse-vmemmap code
section-based optimization aware and switching HugeTLB bootmem pages to
this path.

HugeTLB vmemmap optimization currently has its own early boot setup path. 
It pre-populates optimized vmemmap mappings before the normal
sparse-vmemmap population code runs, and sparsemem carries
SPARSEMEM_VMEMMAP_PREINIT only to support that special case.

That makes the HugeTLB vmemmap optimization path harder to share with
other users of sparse-vmemmap optimization and leaves a fair amount of
HugeTLB-specific boot-time state in the generic memory initialization
flow.

This series introduces section-based vmemmap optimization support in the
sparse-vmemmap code and switches HugeTLB bootmem pages over to it. 
Instead of having HugeTLB pre-populate optimized vmemmap mappings itself,
HugeTLB now records the compound page order in the corresponding memory
sections.  The generic sparse-vmemmap population path can then allocate or
reuse shared tail vmemmap pages based on section metadata.

The patches are organized as follows:

  - patches 1-2 prepare sparsemem and vmemmap optimization metadata
  - patches 3-8 teach the common sparse-vmemmap paths to use that state
  - patches 9-10 switch HugeTLB bootmem optimization to the
    section-based path
  - patches 11-17 clean up sparsemem and HugeTLB bootmem code that is no
    longer needed after the conversion

This is intended to be the second smaller step toward the broader HVO
generalization.  The device DAX conversion and the wider HVO consolidation
are left for follow-up series.


This patch (of 17):

struct mem_section is currently forced to a power-of-2 size so the
section-to-root lookup can use a mask instead of a modulo.

That requirement makes future extensions harder than necessary: adding a
small field can require configuration-dependent padding or layout checks
just to preserve the lookup scheme.  Keep the lookup correct for any
struct mem_section size by using a plain modulo instead.

Do not leave the layout entirely unconstrained, though.  Keep struct
mem_section double-word aligned so modest size changes, such as adding
another word-sized field on 64-bit systems, still keep a compact and
efficient layout.  If future fields grow the structure beyond that sweet
spot, the lookup remains correct; only the exact layout efficiency
changes.

Link: https://lore.kernel.org/20260910063256.64386-2-songmuchun@bytedance.com
Link: https://lore.kernel.org/all/20260513130542.35604-1-songmuchun@bytedance.com/ [1]
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Qi Zheng <qi.zheng@linux.dev>
 2026-10-06 17:38:48 -07:00
Muchun Song
e73aeb8a41 mm/sparse: keep mem_section_usage_size() internal 
mem_section_usage_size() is only needed by sparsemem implementation code
after commit ae751d567b ("mm/bootmem_info: stop marking
mem_section_usage as MIX_SECTION_INFO"), so keeping the declaration in
mmzone.h now exposes the helper to all mmzone.h users for no reason.

Move the helper to sparse.h so sparse.c and sparse-vmemmap.c can share it
through the internal header.  While doing so, calculate the allocation
size with struct_size_t(), which ties the expression to the
pageblock_flags trailing array instead of open-coding the struct header
plus bitmap size.

Link: https://lore.kernel.org/20260805022536.1206575-1-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-08-24 18:43:06 -07:00
Sang-Heon Jeon
8380671909 mm/sparse: correct init section annotations 
The !SPARSEMEM_EXTREME stub of sparse_index_init() has no annotation but
the SPARSEMEM_EXTREME variant is __meminit.  So mark the stub __meminit
too.

mminit_validate_memmodel_limits() is only called by memory_present(),
which is __init.  So mark it __init.

sparse_usagebuf and sparse_usagebuf_end are only used by
sparse_init_early_section(), sparse_usage_init() and sparse_usage_fini(),
which are all __init.  So mark them __initdata.

Link: https://lore.kernel.org/20260731164758.1210668-1-ekffu200098@gmail.com
Signed-off-by: Sang-Heon Jeon <ekffu200098@gmail.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-08-24 18:42:58 -07:00
Mike Rapoport (Microsoft)
2b65a42a08 mm: split out sparse declarations from internal.h 
mm/internal.h becomes more and more bloated.

Move declarations related to SPARSE and SPARSE_VMEMMAP memory models to
a new mm/sparse.h header.

No functional changes.

Link: https://lore.kernel.org/20260709-internal-h-v2-2-695631425968@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Muchun Song <muchun.song@linux.dev>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: Pratyush Yadav <pratyush@kernel.org>
Acked-by: SJ Park <sj@kernel.org>
Cc: Alexander Graf <graf@amazon.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Marco Elver <elver@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-08-04 19:18:46 -07:00
Mike Rapoport (Microsoft)
55ed40abb2 mm: split out mm_init and memblock declarations from internal.h 
Patch series "mm: split a couple of headers from internal.h", v2.

mm/internal.h becomes more and more bloated.

Split declarations related to mm_init, memblock, vmalloc and sparse into
new headers.


This patch (of 3):

mm/internal.h becomes more and more bloated.

Move declarations for related to mm/mm_init.c and mm/memblock.c to a new
mm/mm_init.h header.

No functional changes.

[rppt@kernel.org: split stubfs from internal.h to mm_init.h]
  Link: https://lore.kernel.org/alJd1BLypyK9Mpaw@kernel.org
Link: https://lore.kernel.org/20260709-internal-h-v2-0-695631425968@kernel.org
Link: https://lore.kernel.org/20260709-internal-h-v2-1-695631425968@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Muchun Song <muchun.song@linux.dev>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Lorenzo Stoakes <ljs@kernel.org>
Acked-by: Pratyush Yadav <pratyush@kernel.org>
Acked-by: SJ Park <sj@kernel.org>
Cc: Alexander Graf <graf@amazon.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Marco Elver <elver@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-08-04 19:18:45 -07:00
David Hildenbrand (Arm)
4f9ec35df5 mm/sparse: remove bootmem_info.h include 
No longer required, so let's remove it.

Link: https://lore.kernel.org/20260716-bootmem_info_part2-v2-9-4afc76c73d61@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: "Borislav Petkov (AMD)" <bp@alien8.de>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-08-04 19:18:42 -07:00
Hui Zhu
67a06fdfbf mm: fix ASSERT_EXCLUSIVE_BITS by passing memdesc_flags_t by pointer 
KCSAN reports a data race between page_to_nid()/folio_pgdat() reading
page->flags and folio_trylock()/folio_lock() concurrently doing
test_and_set_bit_lock(PG_locked, ...) on the same word, e.g.:

  BUG: KCSAN: data-race in __lruvec_stat_mod_folio / shmem_get_folio_gfp

The race is benign: nid/zone bits are set once at page init and never
overlap with PG_locked.  However, ASSERT_EXCLUSIVE_BITS() inside
memdesc_nid/zonenum() was checking a by-value copy of the flags word, not
the live page->flags, so it failed to annotate the real access.

Change memdesc_nid(), memdesc_zonenum(), memdesc_section(), and
memdesc_is_zone_device() to take a const memdesc_flags_t * and update all
callers to pass &page->flags / &folio->flags, so ASSERT_EXCLUSIVE_BITS()
operates on the actual shared word.

Guard the ASSERT_EXCLUSIVE_BITS() call in memdesc_zonenum() under
ZONES_WIDTH != 0 to avoid a zero-mask check on configs where the zone
field is absent.  memdesc_section() needs no such guard, since
SECTIONS_WIDTH is never 0 wherever SECTION_IN_PAGE_FLAGS is defined. 
Under CONFIG_NUMA=n, memdesc_nid() itself is stubbed to "return 0" instead
of reading page->flags, since NODES_MASK is 0 and the check can never
fire; page_to_nid()/folio_nid() now just call memdesc_nid()
unconditionally and rely on that stub, instead of duplicating the
CONFIG_NUMA split at each call site.

[zhuhui@kylinos.cn: v11]
  Link: https://lore.kernel.org/20260708083308.747930-1-hui.zhu@linux.dev
[ziy@nvidia.com: build fix]
  Link: https://lore.kernel.org/DJSGLP9VFLRI.355AAWR2VHIEL@nvidia.com
Link: https://lore.kernel.org/20260630070810.470763-1-hui.zhu@linux.dev
Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
Co-developed-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-07-30 19:40:31 -07:00
Muchun Song
ec1adc14c1 mm/sparse-vmemmap: remove sparse_vmemmap_init_nid_late() 
hugetlb_vmemmap_init_late() no longer has any users, so the remaining
late-init path in sparse_vmemmap_init_nid_late() is dead code.

Remove sparse_vmemmap_init_nid_late() and its declarations.

Link: https://lore.kernel.org/20260612035903.2468601-18-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Oscar Salvador (SUSE) <osalvador@kernel.org>
Cc: "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-07-28 21:12:00 -07:00
Muchun Song
133b57fee9 mm/sparse: move sparse_vmemmap_init_nid_late() into sparse_init_nid() 
sparse_vmemmap_init_nid_late() is still called separately from
mm_core_init_early(), away from the rest of the sparse initialization
path.

Now that sparse_init() runs after zone initialization, call
sparse_vmemmap_init_nid_late() from sparse_init_nid() instead.  This keeps
both sparse_vmemmap_init_nid_early() and sparse_vmemmap_init_nid_late() in
the sparse setup path.

Link: https://lore.kernel.org/20260612035903.2468601-12-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Oscar Salvador (SUSE) <osalvador@kernel.org>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-07-28 21:11:58 -07:00
Muchun Song
37424eab6d mm/mm_init: remove set_pageblock_order() call from sparse_init() 
free_area_init() already sets pageblock_order before sparse_init() runs
for CONFIG_HUGETLB_PAGE_SIZE_VARIABLE, so sparse_init() does not need to
call set_pageblock_order() again.

With that call removed, set_pageblock_order() is only used in
mm/mm_init.c.  Make it static.

Link: https://lore.kernel.org/20260612035903.2468601-11-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Oscar Salvador (SUSE) <osalvador@suse.de>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Oscar Salvador (SUSE) <osalvador@kernel.org>
Cc: "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-07-28 21:11:58 -07:00
Muchun Song
8e4ee7b1bd mm/sparse: move subsection_map_init() into sparse_init() 
subsection_map_init() is part of sparse memory initialization, but it is
currently called from free_area_init().

Move it into sparse_init() so the sparse-specific setup stays together
instead of being split across the generic free_area_init() path.

Link: https://lore.kernel.org/20260612035903.2468601-8-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Oscar Salvador (SUSE) <osalvador@kernel.org>
Cc: "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-07-28 21:11:57 -07:00
Muchun Song
b3ef855262 mm/sparse: panic on memmap and usemap allocation failure 
When vmemmap or usemap allocation fails, sparse_init_nid() currently marks
the section non-present and continues.  Later boot-time code can still
walk PFNs in that section without checking for this partial setup, which
leads to invalid accesses.  subsection_map_init() can also touch an
unallocated usemap.

Auditing and fixing all early PFN walkers for this case is not worth the
complexity.  These allocation failures are expected to be fatal anyway,
and other memory models already treat them that way.

Make memmap and usemap allocation failures panic immediately instead of
trying to recover and crashing later in less obvious ways.  This is also
consistent with how other memory model configurations handle memmap
allocation failures.

Link: https://lore.kernel.org/20260612035903.2468601-7-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Oscar Salvador <osalvador@suse.de>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Oscar Salvador (SUSE) <osalvador@kernel.org>
Cc: "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-07-28 21:11:57 -07:00
Muchun Song
ffe5539313 mm/sparse: remove sparse buffer pre-allocation mechanism 
Commit 9bdac91424 ("sparsemem: Put mem map for one node together.")
introduced a mechanism to pre-allocate a large memory block to hold all
memmaps for a NUMA node upfront.

However, the original commit message did not clearly state the actual
benefits or the necessity of explicitly pre-allocating a single chunk for
all memmap areas of a given node.

One of the concerns about removing this pre-allocation is that the
subsequent per-section memmap allocations could become scattered around,
and might turn too many memory blocks/sections into an "un-offlinable"
state.  However, tests show that even without the explicit node-wide
pre-allocation, memblock still allocates memory closely and back-to-back. 
When tracing vmemmap_set_pmd allocations, the physical chunks allocated by
memblock are strictly adjacent to each other in a single contiguous
physical range (mapped top-down).  Because they are packed tightly
together naturally, they will at most consume or pollute the exact same
number of memory blocks as the explicit pre-allocation did.

Another concern is the boot performance impact of calling memmap_alloc()
multiple times compared to one large node-wide allocation.  Tests on a
256GB VM showed that memmap allocation time increased from 199,555 ns to
741,292 ns.  Even though it is 3.7x slower, on a 1TB machine, the entire
memory allocation time would only take a few milliseconds.  This boot
performance difference is completely negligible.

Since no negative impact on memory offlining behavior or noticeable boot
performance regression was found, this patch proposes removing the
explicit node-wide memmap pre-allocation mechanism to reduce the
maintenance burden.

Link: https://lore.kernel.org/20260410092419.2446420-1-songmuchun@bytedance.com
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-05-28 21:04:49 -07:00
Sang-Heon Jeon
e0974347f5 mm/sparse: remove unnecessary NULL check before allocating mem_section 
Commit 850ed20539 ("mm: move array mem_section init code out of
memory_present()") moved mem_section allocation logic into
memblocks_present().

Before that move, memory_present() could be called multiple times, so
unlikely() matched the common case, where most calls found mem_section
already allocated.

After that move, memblocks_present() is called exactly once from
sparse_init().  Under CONFIG_SPARSEMEM_EXTREME, mem_section is always NULL
when it is called.

So remove unnecessary NULL check before allocating mem_section.  No
functional change.

Link: https://lore.kernel.org/20260419144225.2875654-1-ekffu200098@gmail.com
Signed-off-by: Sang-Heon Jeon <ekffu200098@gmail.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed by: Donet Tom <donettom@linux.ibm.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-05-28 21:04:47 -07:00
Muchun Song
7cf6d940f4 mm/sparse: fix preinited section_mem_map clobbering on failure path 
sparse_init_nid() is careful to leave alone every section whose vmemmap
has already been set up by sparse_vmemmap_init_nid_early(); it only clears
section_mem_map for the rest:

        if (!preinited_vmemmap_section(ms))
                ms->section_mem_map = 0;

A leftover line after that conditional block

        ms->section_mem_map = 0;

was supposed to be deleted but was missed in the failure path, causing the
field to be overwritten for all sections when memory allocation fails,
effectively destroying the pre-initialization check.

Drop the stray assignment so that preinited sections retain their
already valid state.

Those pre-inited sections (HugeTLB pages) are not activated.  However,
such failures are extremely rare, so I don't see any major userspace
issues.

Link: https://lore.kernel.org/20260331113724.2080833-1-songmuchun@bytedance.com
Fixes: d65917c423 ("mm/sparse: allow for alternate vmemmap section init at boot")
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed by: Donet Tom <donettom@linux.ibm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-18 00:10:52 -07:00
David Hildenbrand (Arm)
738de20c4f mm/sparse: move memory hotplug bits to sparse-vmemmap.c 
Let's move all memory hoptplug related code to sparse-vmemmap.c.

We only have to expose sparse_index_init().  While at it, drop the
definition of sparse_index_init() for !CONFIG_SPARSEMEM, which is unused,
and place the declaration in internal.h.

Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-15-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:33 -07:00
David Hildenbrand (Arm)
08e5f77c37 mm/sparse: move __section_mark_present() to internal.h 
Let's prepare for moving memory hotplug handling from sparse.c to
sparse-vmemmap.c by moving __section_mark_present() to internal.h.

Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-14-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:33 -07:00
David Hildenbrand (Arm)
f62a3bf227 mm/sparse: move sparse_init_one_section() to internal.h 
While at it, convert the BUG_ON to a VM_WARN_ON_ONCE, avoid long lines,
and merge sparse_encode_mem_map() into its only caller
sparse_init_one_section().

Clarify the comment a bit, pointing at page_to_pfn().

[david@kernel.org: s/VM_WARN_ON/VM_WARN_ON_ONCE/]
  Link: https://lkml.kernel.org/r/6b04c1a1-74e7-42e8-8523-a40802e5dacc@kernel.org
Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-13-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:33 -07:00
David Hildenbrand (Arm)
b551ed94d9 mm/sparse: drop set_section_nid() from sparse_add_section() 
CONFIG_MEMORY_HOTPLUG is CONFIG_SPARSEMEM_VMEMMAP-only.  And
CONFIG_SPARSEMEM_VMEMMAP implies that NODE_NOT_IN_PAGE_FLAGS cannot be
set: see include/linux/page-flags-layout.h

	...
	#elif defined(CONFIG_SPARSEMEM_VMEMMAP)
	#error "Vmemmap: No space for nodes field in page flags"
	...

Which implies that the node is always stored in page flags and
NODE_NOT_IN_PAGE_FLAGS cannot be set.  Therefore, set_section_nid() is a
NOP on CONFIG_SPARSEMEM_VMEMMAP.

So let's remove the set_section_nid() call to prepare for moving
CONFIG_MEMORY_HOTPLUG to mm/sparse-vmemmap.c

Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-12-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:32 -07:00
David Hildenbrand (Arm)
fead6dcff8 mm: prepare to move subsection_map_init() to mm/sparse-vmemmap.c 
We want to move subsection_map_init() to mm/sparse-vmemmap.c.

To prepare for getting rid of subsection_map_init() in mm/sparse.c
completely, use a static inline function for !CONFIG_SPARSEMEM_VMEMMAP.

While at it, move the declaration to internal.h and rename it to
"sparse_init_subsection_map()".

Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-11-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:32 -07:00
David Hildenbrand (Arm)
dac89b150b mm/sparse: remove CONFIG_MEMORY_HOTPLUG-specific usemap allocation handling 
In 2008, we added through commit 48c906823f ("memory hotplug: allocate
usemap on the section with pgdat") quite some complexity to try allocating
memory for the "usemap" (storing pageblock information per memory section)
for a memory section close to the memory of the "pgdat" of the node.

The goal was to make memory hotunplug of boot memory more likely to
succeed.  That commit also added some checks for circular dependencies
between two memory sections, whereby two memory sections would contain
each others usemap, turning both boot memory sections un-removable.

However, in 2010, commit a4322e1bad ("sparsemem: Put usemap for one node
together") started allocating the usemap for multiple memory sections on
the same node in one chunk, effectively grouping all usemap allocations of
the same node in a single memblock allocation.

We don't really give guarantees about memory hotunplug of boot memory, and
with the change in 2010, it is impossible in practice to get any circular
dependencies.

So let's simply remove this complexity.

Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-10-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:32 -07:00
David Hildenbrand (Arm)
22688ade3b mm/sparse: remove sparse_decode_mem_map() 
section_deactivate() applies to CONFIG_SPARSEMEM_VMEMMAP only.  So we can
just use pfn_to_page() (after making sure we have the start PFN of the
section), and remove sparse_decode_mem_map().

Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-9-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:32 -07:00
David Hildenbrand (Arm)
119c31caa5 mm/sparse: remove !CONFIG_SPARSEMEM_VMEMMAP leftovers for CONFIG_MEMORY_HOTPLUG 
CONFIG_MEMORY_HOTPLUG now depends on CONFIG_SPARSEMEM_VMEMMAP.  So let's
remove the !CONFIG_SPARSEMEM_VMEMMAP leftovers that are dead code.

Adjust the comment above fill_subsection_map() accordingly.

Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-6-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:32 -07:00
David Hildenbrand (Arm)
e66383b674 mm/sparse: remove WARN_ONs from (online|offline)_mem_sections() 
We do not allow offlining of memory with memory holes, and always hotplug
memory without holes.

Consequently, we cannot end up onlining or offlining memory sections that
have holes (including invalid sections).  That's also why these WARN_ONs
never fired.

Let's remove the WARN_ONs along with the TODO regarding double-checking.
Link: https://lkml.kernel.org/r/20260320-sparsemem_cleanups-v2-3-096addc8800d@kernel.org
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Sidhartha Kumar <sidhartha.kumar@oracle.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:31 -07:00
Kiryl Shutsemau
9f94db4c7e mm/sparse: check memmap alignment for compound_info_has_mask() 
If page->compound_info encodes a mask, it is expected that vmemmap to be
naturally aligned to the maximum folio size.

Add a VM_WARN_ON_ONCE() to check the alignment.

Link: https://lkml.kernel.org/r/20260227194302.274384-9-kas@kernel.org
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Acked-by: Zi Yan <ziy@nvidia.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Baoquan He <bhe@redhat.com>
Cc: Christoph Lameter <cl@gentwo.org>
Cc: David Hildenbrand (arm) <david@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Frank van der Linden <fvdl@google.com>
Cc: Harry Yoo <harry.yoo@oracle.com>
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Usama Arif <usamaarif642@gmail.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: WANG Xuerui <kernel@xen0n.name>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2026-04-05 13:53:08 -07:00
Sumanth Korikkar
300709fbef mm/memory_hotplug: Remove MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers 
MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory notifiers were introduced
to prepare the transition of memory to and from a physically accessible
state. This enhancement was crucial for implementing the "memmap on memory"
feature for s390.

With introduction of dynamic (de)configuration of hotpluggable memory,
memory can be brought to accessible state before add_memory(). Memory
can be brought to inaccessible state before remove_memory(). Hence,
there is no need of MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory
notifiers anymore.

This basically reverts commit
c5f1e2d189 ("mm/memory_hotplug: introduce MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers")
Additionally, apply minor adjustments to the function parameters of
move_pfn_range_to_zone() and mhp_supports_memmap_on_memory() to ensure
compatibility with the latest branch.

Acked-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
 2025-10-14 14:24:53 +02:00
Matthew Wilcox (Oracle)
eb00fdd84d mm: introduce memdesc_nid() 
Remove a conversion from folio to page by passing the folio->flags (which
are a copy of the page->flags) to the new memdesc_nid() function.

Link: https://lkml.kernel.org/r/20250805172307.1302730-4-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2025-09-13 16:55:07 -07:00
Matthew Wilcox (Oracle)
56d578c130 mm: convert page_to_section() to memdesc_section() 
Pass in the memdesc_flags_t instead of a pointer to the page.  This will
allow us to remove a few conversions to struct page in upcoming patches.

Link: https://lkml.kernel.org/r/20250805172307.1302730-3-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2025-09-13 16:55:07 -07:00
Sumanth Korikkar
c3576889d8 mm: fix accounting of memmap pages 
For !CONFIG_SPARSEMEM_VMEMMAP, memmap page accounting is currently done
upfront in sparse_buffer_init().  However, sparse_buffer_alloc() may
return NULL in failure scenario.

Also, memmap pages may be allocated either from the memblock allocator
during early boot or from the buddy allocator.  When removed via
arch_remove_memory(), accounting of memmap pages must reflect the original
allocation source.

To ensure correctness:
* Account memmap pages after successful allocation in sparse_init_nid()
  and section_activate().
* Account memmap pages in section_deactivate() based on allocation
  source.

Link: https://lkml.kernel.org/r/20250807183545.1424509-1-sumanthk@linux.ibm.com
Fixes: 15995a3524 ("mm: report per-page metadata information")
Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
Suggested-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Wei Yang <richard.weiyang@gmail.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2025-08-27 22:45:43 -07:00
Gavin Shan
61659efdb3 drivers/base/memory: improve add_boot_memory_block() 
Patch series "drivers/base/memory: Two cleanups", v3.

Two cleanups to drivers/base/memory.


This patch (of 2)L

It's unnecessary to count the present sections for the specified block
since the block will be added if any section in the block is present. 
Besides, for_each_present_section_nr() can be reused as Andrew Morton
suggested.

Improve by using for_each_present_section_nr() and dropping the
unnecessary @section_count.

No functional changes intended.

Link: https://lkml.kernel.org/r/20250311233045.148943-1-gshan@redhat.com
Link: https://lkml.kernel.org/r/20250311233045.148943-2-gshan@redhat.com
Signed-off-by: Gavin Shan <gshan@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Oscar Salvador <osalvador@suse.de>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2025-03-17 22:07:01 -07:00
Frank van der Linden
d65917c423 mm/sparse: allow for alternate vmemmap section init at boot 
Add functions that are called just before the per-section memmap is
initialized and just before the memmap page structures are initialized. 
They are called sparse_vmemmap_init_nid_early and
sparse_vmemmap_init_nid_late, respectively.

This allows for mm subsystems to add calls to initialize memmap and page
structures in a specific way, if using SPARSEMEM_VMEMMAP.  Specifically,
hugetlb can pre-HVO bootmem allocated pages that way, so that no time and
resources are wasted on allocating vmemmap pages, only to free them later
(and possibly unnecessarily running the system out of memory in the
process).

Refactor some code and export a few convenience functions for external
use.

In sparse_init_nid, skip any sections that are already initialized, e.g. 
they have been initialized by sparse_vmemmap_init_nid_early already.

The hugetlb code to use these functions will be added in a later commit.

Export section_map_size, as any alternate memmap init code will want to
use it.

The internal config option to enable this is SPARSEMEM_VMEMMAP_PREINIT,
which is selected if an architecture-specific option,
ARCH_WANT_HUGETLB_VMEMMAP_PREINIT, is set.  In the future, if other
subsystems want to do preinit too, they can do it in a similar fashion.

The internal config option is there because a section flag is used, and
the number of flags available is architecture-dependent (see mmzone.h). 
Architecures can decide if there is room for the flag when enabling
options that select SPARSEMEM_VMEMMAP_PREINIT.

Fortunately, as of right now, all sparse vmemmap using architectures do
have room.

Link: https://lkml.kernel.org/r/20250228182928.2645936-11-fvdl@google.com
Signed-off-by: Frank van der Linden <fvdl@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Dan Carpenter <dan.carpenter@linaro.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Hildenbrand <david@redhat.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Joao Martins <joao.m.martins@oracle.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Roman Gushchin (Cruise) <roman.gushchin@linux.dev>
Cc: Usama Arif <usamaarif642@gmail.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Yu Zhao <yuzhao@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2025-03-16 22:06:27 -07:00
Guo Weikang
c6f239796b mm/memblock: add memblock_alloc_or_panic interface 
Before SLUB initialization, various subsystems used memblock_alloc to
allocate memory.  In most cases, when memory allocation fails, an
immediate panic is required.  To simplify this behavior and reduce
repetitive checks, introduce `memblock_alloc_or_panic`.  This function
ensures that memory allocation failures result in a panic automatically,
improving code readability and consistency across subsystems that require
this behavior.

[guoweikang.kernel@gmail.com: arch/s390: save_area_alloc default failure behavior changed to panic]
  Link: https://lkml.kernel.org/r/20250109033136.2845676-1-guoweikang.kernel@gmail.com
  Link: https://lore.kernel.org/lkml/Z2fknmnNtiZbCc7x@kernel.org/
Link: https://lkml.kernel.org/r/20250102072528.650926-1-guoweikang.kernel@gmail.com
Signed-off-by: Guo Weikang <guoweikang.kernel@gmail.com>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>	[m68k]
Reviewed-by: Alexander Gordeev <agordeev@linux.ibm.com>	[s390]
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2025-01-25 20:22:38 -08:00
Matthew Wilcox (Oracle)
0386aaa6e9 bootmem: stop using page->index 
Encode the type into the bottom four bits of page->private and the info
into the remaining bits.  Also turn the bootmem type into a named enum.

[arnd@arndb.de: bootmem: add bootmem_type stub function]
  Link: https://lkml.kernel.org/r/20241015143802.577613-1-arnd@kernel.org
[akpm@linux-foundation.org: fix build with !CONFIG_HAVE_BOOTMEM_INFO_NODE]
 Link: https://lore.kernel.org/oe-kbuild-all/202410090311.eaqcL7IZ-lkp@intel.com/
Link: https://lkml.kernel.org/r/20241005200121.3231142-6-willy@infradead.org
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: kernel test robot <lkp@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
 2024-11-07 14:38:07 -08:00