mirror of
https://github.com/torvalds/linux.git
synced 2026-11-06 21:58:01 +01:00
v7.4-rc1
27912 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Linus Torvalds
|
8a36f53fd5 |
mm.git review status for linus..mm-stable
Everything:
Total patches: 213
Fixes: tags: 69
cc:stable patches: 43
Reviews/patch: 2.04
Reviewed rate: 86%
Excluding DAMON:
Total patches: 192
Fixes: tags: 63
cc:stable patches: 39
Reviews/patch: 2.18
Reviewed rate: 89%
Excluding DAMON and selftests:
Total patches: 182
Fixes: tags: 57
cc:stable patches: 39
Reviews/patch: 2.23
Reviewed rate: 89%
Summary of series in this merge:
- "mm: make VMA flag semantics explicit, eliminate VM_SPECIAL"
(Lorenzo Stoakes (ARM)):
Replace VM_SPECIAL with helpers describing mapping ownership and
behavior. Enforce the mmap flag rules and convert affected drivers,
removing ambiguous tests shared by driver mappings and ordinary memory.
- "support kselftest on nommu platform" (Hajime Tazaki):
Add opt-in mmap and mremap tests for NOMMU systems. Teach the runner to
avoid its fork-dependent timeout command and document the build and
runtime requirements.
- "mm/collapse: separate a collapse from its callers"
(Kiryl Shutsemau (Meta)):
Separate huge-page collapse from caller policy. Replace khugepaged
identity checks with explicit permissions, and separate scanning from
collapse with the mmap_lock drop in one place.
- "mm: implement and use vma_has_anon_rmap(), silence KCSAN"
(Lorenzo Stoakes (ARM)):
Provide a common helper for testing whether a VMA has anonymous reverse
mappings. Document the locking rules, annotate intentional concurrent
reads for KCSAN, and convert the core MM callers.
- "mm/sparse: remove SECTION_MARKED_PRESENT and further cleanups"
(David Hildenbrand (Arm)):
Remove SECTION_MARKED_PRESENT in favor of the early-section flag.
Simplify sparse memory initialization, section metadata handling and
related PFN validity checks.
- "sh: remove NUMA and SPARSEMEM support" (Mike Rapoport (Microsoft)):
Remove SuperH NUMA and SPARSEMEM support. The extra memory banks have
been unusable since the NO_BOOTMEM conversion, leaving only the code and
data overhead.
- "mm: Unify device DAX and HugeTLB vmemmap population paths"
(Muchun Song):
Unify the generic device DAX and HugeTLB vmemmap population paths and
shared tail-page handling. Remove the duplicate DAX machinery while
retaining the powerpc radix implementation.
- "kselftest: mm: fix some failure of split_huge_page_test" (Yeoreum Yun):
Fix huge-page split tests racing with khugepaged recollapse. Use pagemap
and page flags instead of VMA-boundary assumptions which fail when glibc
has already enabled THP.
- "memcg: move memcgid refcount to objcg to unpin dying memcgs"
(Bingfang Guo):
Let dying memcgs be freed while swapped-out shmem still holds their IDs.
Move ID references to reparentable objcgs, with the necessary swap
accounting and RCU lifetime changes.
- "mm/truncate: fix data loss when truncating straddling large folios"
(Zhang Yi):
Preserve out-of-range data when truncating straddling large folios. Fix
a lookup race and split-failure handling, and handle minimum folio orders
for the forthcoming ext4 iomap buffered I/O path.
- "mm/damon: introduce damos quota goal target metric complement flag"
(SJ Park):
Let DAMOS quota goals use a metric's complement, so utilization can also
express free-memory goals without adding separate metrics. Add the sysfs
interface, documentation and tests.
- "arm64/mm: Standardize printing for pgtable entries" (Anshuman Khandual):
Use the generic formatter for arm64 page-table entries, ready for future
wider entries. Move the safe-PTE-update check out of the header to avoid
an include cycle.
- "mm: kmsan: fix iounmap metadata teardown" (Dima Koziuk):
Fix KMSAN metadata leaks on iounmap and failed ioremap allocations. Free
backing pages before removing their mappings, and share the teardown with
partial-allocation cleanup.
- "mm/damon/core: preserve quota state when constructing schemes"
(Karl Mehltretter):
Preserve running DAMOS quota state while validating parameter updates.
Even rejected updates could reset accounting and exceed the quota window;
add tests for both accepted and rejected updates.
- "mm/vmalloc: fix vmalloc_dump_obj VA lookup" (Ye Liu):
Fix vmalloc_dump_obj() lookups for last-page addresses and allocations
spanning vmap zones. These allocations were being missed in diagnostic
output.
- "mm: various (urgent?) fixes" (David Hildenbrand (Arm)):
Fix mmap charge leaks and hugetlb use-after-free, wrong-address
unmapping, stuck locks and endless faults. Also preserve reservations on
failed mremap and handle stale userfaultfd bits.
- "maple_tree: fix maple_range_64 crashes in RCU mode" (Dimitris Charisis):
Fix pointer and metadata confusion during RCU destruction of
maple_range_64 nodes, avoiding double frees and invalid accesses. Add a
regression test; no in-tree user currently uses this combination.
- "mm: fix hugetlb NR_HUGETLB accounting on folio migration" (Hongfu Li):
Fix hugetlb migration and preallocation accounting so /proc/vmstat
retains the correct count and memory.numa_stat follows the folio to its
new node.
- "mm/damon: fix a minor bug, clean code, and add a test" (SJ Park):
Fix 32-bit DAMON age overflow which made old regions look young. Share
hugetlb folio acquisition code and add a regression test for the
previously fixed last-region skipping bug.
- "mm/damon/core: cleanup code, reduce stack usage, and add kunit"
(SJ Park):
Reduce kdamond stack usage and remove unused enums. Add deterministic
PSI quota-goal tests which do not depend on the machine's current memory
pressure.
- "mm/damon: fix the temporal goal tuner's size quota conversion"
(Donggeun Yoo):
Bound the temporal tuner's quota conversion to prevent overflow and zero
effective quotas, which could stop schemes making progress. Add boundary
tests.
- "Don't use GFP_DMA when calling dma_alloc_coherent" (Baoquan He):
Remove redundant GFP_DMA flags from coherent DMA allocation calls. The
DMA API already selects the zone from device limits and ignores the
caller's flag.
- "kselftest: mm: fix intermittent failure khugepaged test" (Yeoreum Yun):
Propagate child-process failures in the khugepaged tests and prevent
background collapse from racing their checks. This removes false passes
and intermittent failures.
- "MAINTAINERS: cover all hugetlb headers under HUGETLB SUBSYSTEM"
(Enze Li):
Cover all hugetlb headers in MAINTAINERS so get_maintainer.pl finds the
right maintainers. Also label the closing hugetlb_cgroup.h include
guard.
Summary of the 65 singletons in this merge:
The singletons include a few worthwhile performance fixes. Avoiding
unnecessary reverse-map tree updates gave a reported 15.9% improvement
in UnixBench's execl test on a 192-core x86 system. A reclaim retry fix
stops defrag_mode from spinning under memory.low protection; a reported
file-creation stress test ran 5.7 times faster, with far less memory
pressure. Narrower hugetlb invalidations also spare KVM guests needless
refaults: the reported Redis-in-VM P99 QPS penalty while punching holes
fell from 7.09% to 1.45%.
Several fixes prevent lost data or corrupted memory. Lazyfree huge-page
races with mremap()/munmap() could lose data or access freed memory, and
MADV_COLD could lose a concurrent write's dirty bit. Another fix
prevents stale migration entries from outliving large folios. On
filesystems with blocks smaller than a page, extending a file by 4 GiB
or more could expose bytes written past the old EOF through mmap instead
of zeroes. That is fixed, along with use-after-free races in
PAGEMAP_SCAN during process exit and in memory-cgroup accounting on
PREEMPT_RT kernels.
Other fixes stop UFFDIO_MOVE spinning forever on pinned large folios,
which could wedge other memory operations, and stop mremap() leaking
commit charges until allocations fail under strict overcommit. A further
fix keeps userfaultfd tracking intact on neighboring mappings. NUMA
binding no longer prevents long-term pins when the chosen node has only
movable memory, fixing QEMU/VFIO startup and io_uring buffer
registration failures. MGLRU also gets a fix for early-boot crashes with
memory cgroups disabled at boot, notably in small crash kernels.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCat66sAAKCRDdBJ7gKXxA
jrQrAP99kgSXRdlmRZbHeRBHQ6KvI4XnLvQ78MDIzZ3zmBaOugEA5jiIsK2/snFD
3VaV0oipVZpuwJkVJE6zFkh0tcHgTAo=
=DHrN
-----END PGP SIGNATURE-----
Merge tag 'mm-stable-2026-10-25-19-27' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull more MM updates from Andrew Morton:
- "mm: make VMA flag semantics explicit, eliminate VM_SPECIAL" (Lorenzo
Stoakes):
Replace VM_SPECIAL with helpers describing mapping ownership and
behavior. Enforce the mmap flag rules and convert affected drivers,
removing ambiguous tests shared by driver mappings and ordinary
memory.
- "support kselftest on nommu platform" (Hajime Tazaki):
Add opt-in mmap and mremap tests for NOMMU systems. Teach the runner
to avoid its fork-dependent timeout command and document the build
and runtime requirements.
- "mm/collapse: separate a collapse from its callers" (Kiryl
Shutsemau):
Separate huge-page collapse from caller policy. Replace khugepaged
identity checks with explicit permissions, and separate scanning from
collapse with the mmap_lock drop in one place.
- "mm: implement and use vma_has_anon_rmap(), silence KCSAN" (Lorenzo
Stoakes):
Provide a common helper for testing whether a VMA has anonymous
reverse mappings. Document the locking rules, annotate intentional
concurrent reads for KCSAN, and convert the core MM callers.
- "mm/sparse: remove SECTION_MARKED_PRESENT and further cleanups"
(David Hildenbrand):
Remove SECTION_MARKED_PRESENT in favor of the early-section flag.
Simplify sparse memory initialization, section metadata handling and
related PFN validity checks.
- "sh: remove NUMA and SPARSEMEM support" (Mike Rapoport):
Remove SuperH NUMA and SPARSEMEM support. The extra memory banks have
been unusable since the NO_BOOTMEM conversion, leaving only the code
and data overhead.
- "mm: Unify device DAX and HugeTLB vmemmap population paths" (Muchun
Song):
Unify the generic device DAX and HugeTLB vmemmap population paths and
shared tail-page handling. Remove the duplicate DAX machinery while
retaining the powerpc radix implementation.
- "kselftest: mm: fix some failure of split_huge_page_test" (Yeoreum
Yun):
Fix huge-page split tests racing with khugepaged recollapse. Use
pagemap and page flags instead of VMA-boundary assumptions which fail
when glibc has already enabled THP.
- "memcg: move memcgid refcount to objcg to unpin dying memcgs"
(Bingfang Guo):
Let dying memcgs be freed while swapped-out shmem still holds their
IDs. Move ID references to reparentable objcgs, with the necessary
swap accounting and RCU lifetime changes.
- "mm/truncate: fix data loss when truncating straddling large folios"
(Zhang Yi):
Preserve out-of-range data when truncating straddling large folios.
Fix a lookup race and split-failure handling, and handle minimum
folio orders for the forthcoming ext4 iomap buffered I/O path.
- "mm/damon: introduce damos quota goal target metric complement flag"
(SJ Park):
Let DAMOS quota goals use a metric's complement, so utilization can
also express free-memory goals without adding separate metrics. Add
the sysfs interface, documentation and tests.
- "arm64/mm: Standardize printing for pgtable entries" (Anshuman
Khandual):
Use the generic formatter for arm64 page-table entries, ready for
future wider entries. Move the safe-PTE-update check out of the
header to avoid an include cycle.
- "mm: kmsan: fix iounmap metadata teardown" (Dima Koziuk):
Fix KMSAN metadata leaks on iounmap and failed ioremap allocations.
Free backing pages before removing their mappings, and share the
teardown with partial-allocation cleanup.
- "mm/damon/core: preserve quota state when constructing schemes" (Karl
Mehltretter):
Preserve running DAMOS quota state while validating parameter
updates. Even rejected updates could reset accounting and exceed the
quota window; add tests for both accepted and rejected updates.
- "mm/vmalloc: fix vmalloc_dump_obj VA lookup" (Ye Liu):
Fix vmalloc_dump_obj() lookups for last-page addresses and
allocations spanning vmap zones. These allocations were being missed
in diagnostic output.
- "mm: various (urgent?) fixes" (David Hildenbrand):
Fix mmap charge leaks and hugetlb use-after-free, wrong-address
unmapping, stuck locks and endless faults. Also preserve reservations
on failed mremap and handle stale userfaultfd bits.
- "maple_tree: fix maple_range_64 crashes in RCU mode" (Dimitris
Charisis):
Fix pointer and metadata confusion during RCU destruction of
maple_range_64 nodes, avoiding double frees and invalid accesses. Add
a regression test; no in-tree user currently uses this combination.
- "mm: fix hugetlb NR_HUGETLB accounting on folio migration" (Hongfu
Li):
Fix hugetlb migration and preallocation accounting so /proc/vmstat
retains the correct count and memory.numa_stat follows the folio to
its new node.
- "mm/damon: fix a minor bug, clean code, and add a test" (SJ Park):
Fix 32-bit DAMON age overflow which made old regions look young.
Share hugetlb folio acquisition code and add a regression test for
the previously fixed last-region skipping bug.
- "mm/damon/core: cleanup code, reduce stack usage, and add kunit" (SJ
Park):
Reduce kdamond stack usage and remove unused enums. Add deterministic
PSI quota-goal tests which do not depend on the machine's current
memory pressure.
- "mm/damon: fix the temporal goal tuner's size quota conversion"
(Donggeun Yoo):
Bound the temporal tuner's quota conversion to prevent overflow and
zero effective quotas, which could stop schemes making progress. Add
boundary tests.
- "Don't use GFP_DMA when calling dma_alloc_coherent" (Baoquan He):
Remove redundant GFP_DMA flags from coherent DMA allocation calls.
The DMA API already selects the zone from device limits and ignores
the caller's flag.
- "kselftest: mm: fix intermittent failure khugepaged test" (Yeoreum
Yun):
Propagate child-process failures in the khugepaged tests and prevent
background collapse from racing their checks. This removes false
passes and intermittent failures.
- "MAINTAINERS: cover all hugetlb headers under HUGETLB SUBSYSTEM"
(Enze Li):
Cover all hugetlb headers in MAINTAINERS so get_maintainer.pl finds
the right maintainers. Also label the closing hugetlb_cgroup.h
include guard.
Summary of the 65 singletons in this merge:
The singletons include a few worthwhile performance fixes. Avoiding
unnecessary reverse-map tree updates gave a reported 15.9% improvement
in UnixBench's execl test on a 192-core x86 system. A reclaim retry
fix stops defrag_mode from spinning under memory.low protection; a
reported file-creation stress test ran 5.7 times faster, with far less
memory pressure. Narrower hugetlb invalidations also spare KVM guests
needless refaults: the reported Redis-in-VM P99 QPS penalty while
punching holes fell from 7.09% to 1.45%.
Several fixes prevent lost data or corrupted memory. Lazyfree
huge-page races with mremap()/munmap() could lose data or access freed
memory, and MADV_COLD could lose a concurrent write's dirty bit.
Another fix prevents stale migration entries from outliving large
folios. On filesystems with blocks smaller than a page, extending a
file by 4 GiB or more could expose bytes written past the old EOF
through mmap instead of zeroes. That is fixed, along with
use-after-free races in PAGEMAP_SCAN during process exit and in
memory-cgroup accounting on PREEMPT_RT kernels.
Other fixes stop UFFDIO_MOVE spinning forever on pinned large folios,
which could wedge other memory operations, and stop mremap() leaking
commit charges until allocations fail under strict overcommit. A
further fix keeps userfaultfd tracking intact on neighboring mappings.
NUMA binding no longer prevents long-term pins when the chosen node
has only movable memory, fixing QEMU/VFIO startup and io_uring buffer
registration failures. MGLRU also gets a fix for early-boot crashes
with memory cgroups disabled at boot, notably in small crash kernels.
* tag 'mm-stable-2026-10-25-19-27' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (213 commits)
mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline
mm/page_counter: use WRITE_ONCE() for emin/elow when parent is the root
mm/memcontrol: publish null pointer before dropping css reference
test_maple_tree: test maple_range_64 node in RCU mode
maple_tree: fix maple_range_64 crashes in RCU mode
mm/vma: avoid broken uffd merge on mremap
tools/testing/shared: catch a failed posix_memalign()
mm/damon/core: remove unreachable damos charge_{target,addr}_from
mm/shrinker: allocate shrinker_info_unit from a dedicated kmem_cache
riscv: mm: exclude invalid THP PMDs from page table check
mm/damon/core: filter out damos address filter-split failed region
fs/proc/task_mmu: pin mm_users in do_pagemap_scan()
mm/mempolicy: apply bind policies against the scoped allocation zone
mm/vmscan: reclaim every zone in per-node proactive reclaim
mm/mremap: correctly uncharge accountable VMAs in error path
mm/mremap: populate calculated fields in vrm state correctly
MAINTAINERS: add an entry for MEMORY MANAGEMENT - MEMORY MAP
mm: bdi: reset min_ratio after the sysfs attributes are gone
mm/page_vma_mapped: continue past nonmatching huge PMDs
selftests/mm: skip hugetlb cgroup v2 tests without the needed controllers
...
|
||
|
Linus Torvalds
|
7d1f84f1b9 |
mm.git review status for linus..mm-nonmm-stable
Total patches: 134 Fixes: tags: 86 cc:stable patches: 52 Reviews/patch: 0.73 Reviewed rate: 59% Summary of patch series in this merge: - "ocfs2: exit recovery thread on mount error path" (Joseph Qi): Stop OCFS2 recovery before freeing failed-mount state and replay slots, closing use-after-free and double-free races. Defer suballocator reclaim to break a transaction-lock deadlock with dismount. - "ocfs2: validate suballoc slot and bit of metadata blocks" (Joseph Qi): Validate OCFS2 metadata suballocator slots, bits and directory index block locations. Reject corrupt metadata with filesystem errors instead of BUGs or out-of-bounds accesses. - "ocfs2: deal with legacy signed name hash values" (Joseph Qi): Restore OCFS2 lookups for non-ASCII names stored with legacy signed hashes. Use stored xattr hashes during bucket splits to avoid duplicate entries and spurious ENOSPC failures. - "ocfs2: stabilize dentry names in file and setattr traces" (Cen Zhang): Snapshot OCFS2 names during file and setattr tracing to prevent rename races using freed storage or mismatched names and lengths. Preserve trace formats and the locked release open-count sample. - "panic: fix panic_force_cpu= redirect races and NMI bypass" (Bradley Morgan): Fix panic_force_cpu races and NMI redirection so crash-kernel CPU selection is honored. Preserve variable arguments on failed redirects and use a static message buffer. - "fat: allocate multiple clusters in fallocate" (Matheus Alves de Almeida): Allocate FAT clusters in batches for FALLOC_FL_KEEP_SIZE and batch FAT updates. This removes repeated single-cluster allocation overhead, particularly on synchronous mounts. - "scripts/gdb: fix radix tree lookup and stack depot diagnostic" (Ting-Han Hou): Fix GDB radix-tree lookup and its handling of XArray sibling, retry and zero entries. Also repair the stack-depot diagnostic for an out-of-range pool index. - "hung_task: Improve warning budget handling and task reporting" (Aaron Tomlin): Reset the hung-task warning budget after recovery so later hangs remain visible. Keep the configured limit intact and emit one summary per scan when detailed reports exhaust the budget. - "bootconfig: Reject unexpected data after null character and cleanups" (Masami Hiramatsu (Google)): Reject bootconfig data after an embedded null while accepting trailing padding. Validate the full input in the userspace tool, and retain internal tree sanity checks there rather than in the kernel. - "kbuild: GCOV cleanups following the prefer-atomic fix" (Konstantin Khorenko): Use the actual GCOV build flags in the prefer-atomic compiler probe. Move flag setup into scripts/Makefile.gcov and skip the probe when GCOV is disabled. - "ocfs2: xattr bucket validation fixes" (Joseph Qi): Accept valid OCFS2 xattr buckets spanning 512-byte blocks, avoiding false read-only remounts. Reject misaligned buckets and return errors instead of panicking on corrupt defrag metadata. - "ocfs2: fix metadata reservation for xattr value trees" (Joseph Qi): Reserve enough OCFS2 metadata for new and relocated xattr value trees. Fix shortfalls which could cause ENOSPC, dropped entries and leaked clusters on fragmented filesystems; remove duplicate reservations. - "Introduce arch_do_panic" (Mete Durlu): Add arch_do_panic() and move s390 and sparc handling out of generic panic code. The architecture-specific actions now follow the end-of-panic marker. - "checkpatch: userspace improvements" (Petr Vorel): Improve checkpatch's userspace detection and suppress kernel-only checks. Recognize standard inttypes macros and add explicit --userspace and --no-userspace overrides. - "ocfs2: validate dr_fs_generation of dir index root blocks" (Joseph Qi): Check OCFS2 directory index generations and leaf block numbers. Reject stale or misplaced blocks instead of using metadata left by a previously formatted filesystem. - "selftests/membarrier: Skip an unregistered memory barrier test on Musl" (Chris Gellermann): Skip the unregistered membarrier test when libc has already registered the command. Musl does this during thread creation, causing false failures in the multithreaded test. - "init: fix array boundary bugs in boot parameter parsing" (Wilson Felipe Pereira): Fix boot argument boundary cases which leave stale arguments in init's command line or panic when replacing the final environment variable. Clear the last slot and check duplicates before the limit. Summary of the 87 singletons in this merge: The OCFS2 singletons fix deadlocks between xattr updates and inline truncation, and during inode reclaim. Corrupt quota and extent metadata could also crash the kernel or corrupt memory; these paths now check the layout before using it. Other fixes close use-after-free and publication races around cluster reconfiguration, node failure, quota remount errors and shared refcount trees. New files now retain security labels from all stacked LSMs instead of silently dropping all but one. Valid xattr insertions also stop tripping FORTIFY on builds with counted arrays. Elsewhere, a lost wakeup could leave hardware-monitoring reads stuck in unkillable sleep and block further I2C traffic. That is fixed, as are kernel memory leaks in POSIX message-queue notifications and tmpfs casefold setup which could grow until memory is exhausted. A driver removal race which could crash the kernel is fixed too. The hibernation command through reboot() also stops taking the same mutex twice, avoiding a deadlock before suspension. On the boot side, bzip2 decoding now accepts valid full blocks emitted by lbzip2, fixing failures to boot those compressed kernels. Malformed bzip2 streams can no longer overflow the decode buffer, and LZ4 initrds are checked before the decoder can read past their end. FAT rejects geometry which overflows the data-area calculation on 32-bit systems, and rolls back entries when inode setup fails, preventing failed creations from reappearing after remount. The x86 XOR and RAID6 AVX paths also avoid imposing a slowdown on subsequent SSE code. -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCat57YgAKCRDdBJ7gKXxA jofGAQDBRlA6AI/WIWAuSb9FH4RIMe16EU3klIYoNqgY3eMbLwD/f8J1rm4gBpBl eXV7nwZMb8Ackw1+2CuBtbvpLfjQUA4= =JLjf -----END PGP SIGNATURE----- Merge tag 'mm-nonmm-stable-2026-10-25-14-57' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Pull non-MM updates from Andrew Morton: - "ocfs2: exit recovery thread on mount error path" (Joseph Qi): Stop OCFS2 recovery before freeing failed-mount state and replay slots, closing use-after-free and double-free races. Defer suballocator reclaim to break a transaction-lock deadlock with dismount. - "ocfs2: validate suballoc slot and bit of metadata blocks" (Joseph Qi): Validate OCFS2 metadata suballocator slots, bits and directory index block locations. Reject corrupt metadata with filesystem errors instead of BUGs or out-of-bounds accesses. - "ocfs2: deal with legacy signed name hash values" (Joseph Qi): Restore OCFS2 lookups for non-ASCII names stored with legacy signed hashes. Use stored xattr hashes during bucket splits to avoid duplicate entries and spurious ENOSPC failures. - "ocfs2: stabilize dentry names in file and setattr traces" (Cen Zhang): Snapshot OCFS2 names during file and setattr tracing to prevent rename races using freed storage or mismatched names and lengths. Preserve trace formats and the locked release open-count sample. - "panic: fix panic_force_cpu= redirect races and NMI bypass" (Bradley Morgan): Fix panic_force_cpu races and NMI redirection so crash-kernel CPU selection is honored. Preserve variable arguments on failed redirects and use a static message buffer. - "fat: allocate multiple clusters in fallocate" (Matheus Alves de Almeida): Allocate FAT clusters in batches for FALLOC_FL_KEEP_SIZE and batch FAT updates. This removes repeated single-cluster allocation overhead, particularly on synchronous mounts. - "scripts/gdb: fix radix tree lookup and stack depot diagnostic" (Ting-Han Hou): Fix GDB radix-tree lookup and its handling of XArray sibling, retry and zero entries. Also repair the stack-depot diagnostic for an out-of-range pool index. - "hung_task: Improve warning budget handling and task reporting" (Aaron Tomlin): Reset the hung-task warning budget after recovery so later hangs remain visible. Keep the configured limit intact and emit one summary per scan when detailed reports exhaust the budget. - "bootconfig: Reject unexpected data after null character and cleanups" (Masami Hiramatsu): Reject bootconfig data after an embedded null while accepting trailing padding. Validate the full input in the userspace tool, and retain internal tree sanity checks there rather than in the kernel. - "kbuild: GCOV cleanups following the prefer-atomic fix" (Konstantin Khorenko): Use the actual GCOV build flags in the prefer-atomic compiler probe. Move flag setup into scripts/Makefile.gcov and skip the probe when GCOV is disabled. - "ocfs2: xattr bucket validation fixes" (Joseph Qi): Accept valid OCFS2 xattr buckets spanning 512-byte blocks, avoiding false read-only remounts. Reject misaligned buckets and return errors instead of panicking on corrupt defrag metadata. - "ocfs2: fix metadata reservation for xattr value trees" (Joseph Qi): Reserve enough OCFS2 metadata for new and relocated xattr value trees. Fix shortfalls which could cause ENOSPC, dropped entries and leaked clusters on fragmented filesystems; remove duplicate reservations. - "Introduce arch_do_panic" (Mete Durlu): Add arch_do_panic() and move s390 and sparc handling out of generic panic code. The architecture-specific actions now follow the end-of-panic marker. - "checkpatch: userspace improvements" (Petr Vorel): Improve checkpatch's userspace detection and suppress kernel-only checks. Recognize standard inttypes macros and add explicit --userspace and --no-userspace overrides. - "ocfs2: validate dr_fs_generation of dir index root blocks" (Joseph Qi): Check OCFS2 directory index generations and leaf block numbers. Reject stale or misplaced blocks instead of using metadata left by a previously formatted filesystem. - "selftests/membarrier: Skip an unregistered memory barrier test on Musl" (Chris Gellermann): Skip the unregistered membarrier test when libc has already registered the command. Musl does this during thread creation, causing false failures in the multithreaded test. - "init: fix array boundary bugs in boot parameter parsing" (Wilson Felipe Pereira): Fix boot argument boundary cases which leave stale arguments in init's command line or panic when replacing the final environment variable. Clear the last slot and check duplicates before the limit. Summary of the 87 singletons in this merge: The OCFS2 singletons fix deadlocks between xattr updates and inline truncation, and during inode reclaim. Corrupt quota and extent metadata could also crash the kernel or corrupt memory; these paths now check the layout before using it. Other fixes close use-after-free and publication races around cluster reconfiguration, node failure, quota remount errors and shared refcount trees. New files now retain security labels from all stacked LSMs instead of silently dropping all but one. Valid xattr insertions also stop tripping FORTIFY on builds with counted arrays. Elsewhere, a lost wakeup could leave hardware-monitoring reads stuck in unkillable sleep and block further I2C traffic. That is fixed, as are kernel memory leaks in POSIX message-queue notifications and tmpfs casefold setup which could grow until memory is exhausted. A driver removal race which could crash the kernel is fixed too. The hibernation command through reboot() also stops taking the same mutex twice, avoiding a deadlock before suspension. On the boot side, bzip2 decoding now accepts valid full blocks emitted by lbzip2, fixing failures to boot those compressed kernels. Malformed bzip2 streams can no longer overflow the decode buffer, and LZ4 initrds are checked before the decoder can read past their end. FAT rejects geometry which overflows the data-area calculation on 32-bit systems, and rolls back entries when inode setup fails, preventing failed creations from reappearing after remount. The x86 XOR and RAID6 AVX paths also avoid imposing a slowdown on subsequent SSE code. * tag 'mm-nonmm-stable-2026-10-25-14-57' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (134 commits) scripts/gdb: fix stack depot out-of-bounds diagnostic scripts/gdb: fix radix tree lookup pointer handling ocfs2/cluster: remove zero-address nodes from IP tree on teardown ocfs2: snapshot the dentry name while tracing file release ocfs2: snapshot dentry names in file and setattr traces ocfs2/dlm: drain join handlers before releasing module text MAINTAINERS: update Markus's e-mail address mailmap: add entries for Andrew Jones stackdepot: stop preallocating after the final pool ocfs2: drain quorum work before removing the cluster watchdog: force console verbose when reporting lockups profiling: fix sampling step on 64-bit big-endian systems kernel/reboot: avoid potential deadlocks during suspension compat: avoid double rounding in times() ocfs2: pin owning modules for open debugfs files ocfs2/cluster: initialize heartbeat regions before publishing them selftests/proc: proc-pid-vm: fix unused variable warning on non-x86 ocfs2: pin refcount trees before dropping the lookup lock ocfs2: validate the xattr tree root of an indexed xattr block ocfs2/dlm: hold the lookup reference across convert AST enqueue ... |
||
|
Linus Torvalds
|
126ad898a6 |
liveupdate: updates for 7.4-rc1
* Several small fixes to KHO and LiveUpdate selftest * A series that renames "KHO scratch" to "KHO bootmem" because this memory is way more important in kexec'ed kernels rather than in the original cold boot kernel and the new name reflects it. -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEeOVYVaWZL5900a/pOQOGJssO/ZEFAmrckxoACgkQOQOGJssO /ZFU4Qf/YuuTxIJu0oj9iYnd578+a/9ova4Wkbo6DT01ifG0LVKFPOlxA/+m/7RI T3ZlLQfM2TlpQ4TATXDK9f0g3A1OrgiarskOZOqTlgjFPyMTN4XMxsn0XcwijhV6 p9lEJwARBg7H0CRcmXb+GeWpp42b/HwTLAmePKUDXyLY+rZyuoLUvuCkhysPY92l cLqSSDTTxk4c2Hh+K1pbFihutd0fg+KPqilCieA2O0JX39rIRmJt6BK6IbGYKVrp eDDoLaFdXurr0IxfiGqYZSMsjYhLV4yRtfIX2KXjE9UI2ueI+nCAzYLA22AkIABO eh8kbQQQuXnVXP7uI6941c+gl3743A== =mZMU -----END PGP SIGNATURE----- Merge tag 'liveupdate-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux Pull liveupdate updates from Mike Rapoport: - Several small fixes to KHO and LiveUpdate selftest - A series that renames "KHO scratch" to "KHO bootmem" because this memory is way more important in kexec'ed kernels rather than in the original cold boot kernel and the new name reflects it * tag 'liveupdate-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux: kho: do not reserve empty per-node bootmem areas selftests/liveupdate: Prefer KVM and HVF over TCG in vmtest.sh kho: rename kho_scratch= commandline parameter to kho_bootmem= kho: rename KHO scratch to KHO bootmem memblock: rename KHO_SCRATCH to KHO_NOPRSRV memblock: get rid of CONFIG_MEMBLOCK_KHO_SCRATCH kho: docs: fix scratch_phys debugfs cross-reference kho: do not reserve scratch memory in the kdump kernel |
||
|
Linus Torvalds
|
b3593aba31 |
kexec: updates for 7.4-rc1
* Fix probe error codes and error propagation:
* make architecture functions that probe loaded image consistenly return
-ENOEXEC when they do not recognize the image format
* Make kexec_image_probe_default() to distinguish image format mismatches
(-ENOEXEC) from real errors (e.g. -ENOMEM), propagating real errors
immediately and only continuing to the next loader on -ENOEXEC
* Make sure the next kernel does not land on hardware-poisoned pages
* Remove unused arch_kexec_locate_mem_hole hook
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEeOVYVaWZL5900a/pOQOGJssO/ZEFAmrckAsACgkQOQOGJssO
/ZHlugf+KDVwiBN1i72OU0TKx9clWyixs20ov2yZ6JlrO46lXCLLXeWy+NNFTSyE
sDF7IQ6zrCWQFKizIsrk+KMyH09+LVRl5Lm8+Mc9Y+iVRuM72n9hJdWLNs6PyUhj
GiLwesq5uRPgmGSVaHHVJD4S2kpeNAMdcxSeKOQoYDyxjs/P5ARqbksUWaMMv5jn
uaUXtJpmR4LV/e/m2VrndvLbkekvZdixvFyx6fXJv+2F+g33QOP/OL1ocqqfqvQk
hBrmvEHWzpay4V6NWzhnlYI355/v+xC2fOM6fuwNRmQNLyMN6AYzvRSbF2mNPXVv
P2G6oYDLd5gVMPcwRmiSQIE/SLnlRA==
=Bi00
-----END PGP SIGNATURE-----
Merge tag 'kexec-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux
Pull kexec updates from Mike Rapoport:
- Fix probe error codes and error propagation:
- make architecture functions that probe loaded image consistenly
return -ENOEXEC when they do not recognize the image format
- make kexec_image_probe_default() distinguish image format
mismatches (-ENOEXEC) from real errors (e.g. -ENOMEM),
propagating real errors immediately and only continuing to the
next loader on -ENOEXEC
- Make sure the next kernel does not land on hardware-poisoned pages
- Remove unused arch_kexec_locate_mem_hole hook
* tag 'kexec-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux:
kexec_file: remove unused arch_kexec_locate_mem_hole hook
kexec: fix error propagation in kexec_image_probe_default()
kexec: return -ENOEXEC from image probe functions on mismatch
kexec: keep the next kernel off hardware-poisoned pages
kexec_file: stop the top-down search before it underflows
|
||
|
Linus Torvalds
|
394023fbe3 |
memblock: updates for 7.4-rc1
Performance improvement:
* Use binary search in memblock_add_range() and memblock_isolate_range()
Cleanups:
* Several simplifications in memblock and mm_init that remvove redundand
code.
* After rework of sparsemem buffer preallocation, memmap_alloc() does not
require exact_nid parameter. Remove that parameter, drop now unused
memblock_alloc_exact_nid_raw() and ajust memblock tests accordingly.
memblock test updates:
* add coverage for lowmem allocations
* fix 32-bit build warinings
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEeOVYVaWZL5900a/pOQOGJssO/ZEFAmrcgaQACgkQOQOGJssO
/ZHVcAf9HoX58N9mA76ESc5Aogkg1PKdTFweeR4nXLTTz10amxvH0O1aERyZE8nO
4lbqa7fUPV/0gPBtFVu7ZcBbgGZR0nmb5atKJWfKoZ373cXqdqVhN0brULLTyRPr
18dlOSX7CmU84m8WiKpVeSZX94xwF5YeCad577y2nmEyWyQ1aZ1iqjzV97cDTcg3
bAzWUEsLdiO2OYoyiasMQIJdk+xeFTtT61so8c3/W1gcUTZHwW1jh0gQ+hKj2E8G
GXfkafy+nUtuvzKP7iOqc8uUIW/j3dLlO3tVp+KFjcHVP+0TLWQzB80oAmvgxuxK
AZ7+xsxhwIxXiS9by2o5huV/oPhU3Q==
=A8E9
-----END PGP SIGNATURE-----
Merge tag 'memblock-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mm/memblock
Pull memblock updates from Mike Rapoport:
"Performance improvement:
- Use binary search in memblock_add_range() and
memblock_isolate_range()
Cleanups:
- Several simplifications in memblock and mm_init that remove
redundant code
- After rework of sparsemem buffer preallocation, memmap_alloc() does
not require exact_nid parameter. Remove that parameter, drop now
unused memblock_alloc_exact_nid_raw() and adjust memblock tests
accordingly
memblock test updates:
- add coverage for lowmem allocations
- fix 32-bit build warnings"
* tag 'memblock-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mm/memblock:
memblock: use binary search to locate candidate regions
memblock: drop for_each_memblock_type() and open code its users
memblock: Remove unused memblock_alloc_exact_nid_raw()
memblock tests: Move exact nid tests to memblock_alloc_range_nid()
mm: Remove unused exact_nid parameter from memmap_alloc()
memblock: initialize the region merge end index
tools: use __pa() in virt_to_phys()
memblock tests: zero-extend simulated memory addresses
memblock tests: use physical addresses for region removal
memblock: reject non-page-aligned reserve_mem regions
memblock tests: cover allocations below the low address limit
memblock tests: model the low allocation limit within dummy memory
mm/mm_init: remove zone_absent_pages_in_node()
memblock: remove conditional return with no effect
memblock: simplify reset_all_zones_managed_pages()
|
||
|
Siddharth Chintamaneni
|
183895979c |
mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline
When CONFIG_MEMCG=y, lruvec_pgdat() returns lruvec->pgdat, which for the
per-node pgdat->__lruvec is left NULL by lruvec_init() and only filled in
lazily by mem_cgroup_lruvec(). Normally that happens early, e.g. on the
first folio_add_lru().
With cgroup_disable=memory, lru_gen_shrink_node() passes &pgdat->__lruvec
directly to shrink_one(). If direct reclaim runs before anything has
called mem_cgroup_lruvec() for the node, which can happen during early
boot under memory pressure (e.g. a kdump kernel), get_swappiness() ends
up dereferencing a NULL pgdat:
Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
pc : get_swappiness+0x68/0xb0
lr : try_to_shrink_lruvec+0x54/0x2c0
Call trace:
get_swappiness+0x68/0xb0 (P)
try_to_shrink_lruvec+0x54/0x2c0
shrink_one+0x8c/0x240
shrink_node+0xc6c/0xe08
do_try_to_free_pages+0xdc/0x570
try_to_free_pages+0x124/0x278
__alloc_frozen_pages_noprof+0x6e8/0xf78
alloc_pages_mpol+0xc4/0x1c0
alloc_frozen_pages_noprof+0x54/0xe8
alloc_pages_noprof+0x1c/0x40
atomic_pool_expand+0x98/0x300
__dma_atomic_pool_init+0x58/0xd8
dma_atomic_pool_init+0x154/0x170
do_one_initcall+0x64/0x350
kernel_init_freeable+0x214/0x430
kernel_init+0x2c/0x208
ret_from_fork+0x10/0x20
The classic LRU path and MGLRU before per-node lru_gen_folio lists
obtained the lruvec through mem_cgroup_lruvec(), which sets
lruvec->pgdat.
The above was hit on an arm64 kdump kernel (6.18-based) booted with
cgroup_disable=memory nr_cpus=4. It also reproduces on mainline on an
arm64 VM with CONFIG_ARM64_64K_PAGES=y, CONFIG_MEMCG=y,
CONFIG_LRU_GEN_ENABLED=y and:
mem=160M nr_cpus=4 coherent_pool=512M cgroup_disable=memory
These options only serve to force direct reclaim early in boot; with
160M the boot eventually ends in OOM regardless. On mainline the first
LRU reclaim happens while unpacking the initramfs, and faults the same
way:
Unable to handle kernel NULL pointer dereference at virtual address 0000000000005040
pc : get_swappiness+0xd4/0x150
Call trace:
get_swappiness+0xd4/0x150 (P)
try_to_shrink_lruvec+0x44/0x2b8
shrink_one+0x80/0x1a0
shrink_node+0xd98/0xea8
do_try_to_free_pages+0xf4/0x4b0
try_to_free_pages+0x110/0x360
__alloc_frozen_pages_noprof+0x8b0/0x1718
...
__filemap_get_folio_mpol+0x2ac/0x548
simple_write_begin+0x44/0xf8
page_symlink+0x178/0x198
...
do_populate_rootfs+0x13c/0x200
Link: https://lore.kernel.org/20261017050928.128582-1-sidchintamaneni@gmail.com
Fixes:
|
||
|
Shakeel Butt
|
51542d55bc |
mm/page_counter: use WRITE_ONCE() for emin/elow when parent is the root
page_counter_calculate_protection() runs without locks. Several reclaimers can update the same counter's emin and elow at the same time, and readers such as mem_cgroup_below_min(), mem_cgroup_protection() and the dmem eviction code load them with READ_ONCE(). Commit |
||
|
Joshua Hahn
|
30a397468e |
mm/memcontrol: publish null pointer before dropping css reference
struct memcg_stock_pcp hold references to cgroups that it tracks to make
sure that the memcg never gets freed with unaccounted charges still
living in the stock. When the cached stock goes to 0 for a memcg, its
reference is dropped and the cached memcg pointer is set to NULL.
If that was the last reference, the css_put would free the associated
cgroup before publishing the NULL pointer, which can be racy on
PREEMPT_RT. To summarize Shakeel Butt's analysis in [2],
CPU 1 CPU 2
local_trylock() takes a per-CPU
spinlock, which is an rcu read
lock along with migrate_disable
drain_stock
css_put drops the last reference
release work is queued
kworker runs css_release_work_fn()
call_rcu() starts grace period G
which waits for the refill task.
<preempt>
drain_all_stock
is_memcg_drain_needed()
enters rcu_read_lock(), but
grace period G does not wait
since it started after G did
<prempt>
stock->cached[i] is cleared and
unlocks. Grace period G ends,
old is freed
mem_cgroup_is_descendant() reads
old->css.cgroup which has just
been freed.
To close this gap on PREEMPT_RT, switch the order of the referene drop
and the publishing so that we publish the NULL pointer before dropping
the reference. Note that this can only happen from drain_stock who can
hold the last reference, but not on consume_stock since another
reference will always be held. But fix this in both sites to maintain
consistency.
This is based on a report from Sashiko, who found this pre-existing
issue in my page_counter_stock series [1].
Link: https://lore.kernel.org/20261021010051.1529261-1-joshua.hahnjy@gmail.com
Link: https://sashiko.dev/#/patchset/20260928192349.3432886-1-joshua.hahnjy%40gmail.com [1]
Link: https://lore.kernel.org/all/atfMngq7nyZpt4u-@linux.dev/ [2]
Fixes:
|
||
|
Lorenzo Stoakes (ARM)
|
68cb098ead |
mm/vma: avoid broken uffd merge on mremap
Userfaultfd often causes unexpected interactions within mm, and does so at
the intersection of uffd, mremap() and VMA merging.
When an mremap of a VMA occurs, copy_vma_and_data() calls
mremap_userfaultfd_prep() to notify uffd of this.
However, mremap_userfaultfd_prep() simply drops the UFFD context for the
VMA if the user did not register it for remap, i.e. didn't specify the
UFFD_FEATURE_EVENT_REMAP flag.
This is OK for the VMA itself, as upon page table move, move_ptes() clears
UFFD page table flags, using vma_has_uffd_without_event_remap() to
determine whether to do so.
However, this doesn't account for merging with adjacent VMAs, whose page
tables won't have received this treatment, and thus latent UFFD markers
will be left around.
Worse, it'll break UFFD for any VMAs it is merged with. This is very
surprising behaviour for a user, who will then simply not receive UFFD
events for VMAs they ought to.
Resolve the issue by explicitly checking for this case via
vma_has_uffd_without_event_remap() in vma_merge_copied_range(), the merge
function explicitly designated for mremap() operations.
Also update the userland VMA tests accordingly.
Link: https://lore.kernel.org/20261020-uffd-remap-fix-v1-1-0f34fc181f0a@kernel.org
Fixes:
|
||
|
SJ Park
|
5695abc4e7 |
mm/damon/core: remove unreachable damos charge_{target,addr}_from
damos_quota->charge_{target,addr}_from saves the target and region the
scheme's quota limit is reached. damos_skip_charged_region() skips the
target and regions before the point. The target that was assigned to
charge_target_from could be dynamically removed from the context by the
runtime parameters update (aka commit) feature. In this case,
damos_skip_charged_region() cannot find if the point is passed or not,
so the scheme is effectively disabled.
Similarly, the user could also remove all the regions of the target
using the runtime parameters update feature. It would be uncommon to do
that, but it is a supported feature. Because the skip pointer detection
logic (damos_skip_charged_region()) works for each region of targets,
this result in the logic is entirely skipped and the scheme is
effectively disabled again.
The user impact is that the DAMOS scheme could suddenly stop working
under certain uses of the runtime parameters update feature. It could
surprise users. No crash or leak happens though. Users could also
simply work around by re-installing the scheme.
Catch the corner cases in damos_adjust_quota() and reset the pointers.
Link: https://lore.kernel.org/20261020023958.108276-1-sj@kernel.org
Fixes:
|
||
|
Konstantin Khorenko
|
faaf8b406c |
mm/shrinker: allocate shrinker_info_unit from a dedicated kmem_cache
shrinker_info_unit holds BITS_PER_LONG atomic_long_t of deferred work plus a one-word bitmap, 520 bytes on 64-bit. kzalloc_node() takes it from kmalloc-1k, so almost half of every unit is padding. Allocate the units from a cache of their own, sized to the unit. The cache is not cache-line aligned: the counters of a unit already share cache lines between shrinkers, and aligning the unit would only keep neighbouring units off each other's boundary line, at the cost of part of the saving. Link: https://lore.kernel.org/20261019-vstor-144715-shrinker_info_unit-kmem_cache-v3-1-0fb061e09bf4@virtuozzo.com Signed-off-by: Konstantin Khorenko <khorenko@virtuozzo.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Suggested-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Harry Yoo (Meta) <harry@kernel.org> Reviewed-by: Qi Zheng <qi.zheng@linux.dev> Acked-by: Muchun Song <muchun.song@linux.dev> Assisted-by: LLM Cc: Dave Chinner <david@fromorbit.com> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Shakeel Butt <shakeel.butt@linux.dev> |
||
|
SJ Park
|
bbeec2d832 |
mm/damon/core: filter out damos address filter-split failed region
When an address type DAMOS filter is configured, DAMOS tries to split the
region for the filter range, and apply DAMOS action to the filter-in
region. The split operation (damon_split_region_at()) could fail from
allocation failure under extreme memory pressure. In the case, DAMOS
action could be applied more or less than expected, ignoring the address
filter.
The issue only rarely happens under extreme memory pressure. No crash or
leak happens. Occasionally not applying the action to a few regions
shouldn't be a real problem. But applying the action to a region that the
user has excluded via address filter could be a surprise depending on the
use case. It is only theoretical. No such real case is found yet.
Filter out the entire region if the split is failed. Update
damos_filter_match() signature to receive the error from split. Also
update kunit tests for the updated signature.
The issue was discovered [1] by sashiko.
Link: https://lore.kernel.org/20261020004119.106690-1-sj@kernel.org
Link: https://lore.kernel.org/20260801203121.117449-1-sj@kernel.org [1]
Fixes:
|
||
|
Gregory Price (Meta)
|
d4ea410ce4 |
mm/mempolicy: apply bind policies against the scoped allocation zone
QEMU fails to start a guest with a VFIO device when guest RAM comes
from a memory backend bound to a movable-only node:
vfio_container_dma_map(0x55b1b1317800, 0x0, 0x10000000,
0x7f39a2200000)
= -12 (Cannot allocate memory)
VFIO pins guest RAM when the device is realized, and the pin faults
the pages in. io_uring fixed buffers fail the same way. Faulting the
range in before pinning lets the same pin succeed, since resident
pages are migrated off the node instead.
A long-term pin faults under memalloc_pin_save(). PF_MEMALLOC_PIN makes
current_gfp_context() clear __GFP_MOVABLE, but policy_nodemask() tests
apply_policy_zone() against the caller's gfp.
The bind keeps its movable-only nodemask, and the allocation finds no
usable zone:
page allocation failure:
order:0,
mode:0x140dc2(GFP_HIGHUSER|__GFP_ZERO|__GFP_COMP),
nodemask=1,cpuset=/,mems_allowed=0-1
Call Trace:
warn_alloc+0x136/0x150
__alloc_frozen_pages_noprof+0xed8/0x1720
alloc_pages_mpol+0xaa/0x170
vma_alloc_folio_noprof+0x65/0xd0
__handle_mm_fault+0x137b/0x1ac0
handle_mm_fault+0xa0/0x270
__get_user_pages+0x250/0xfd0
__gup_longterm_locked+0x116/0x870
gup_fast_fallback+0xcfc/0xdf0
pin_user_pages_fast+0x47/0x60
io_pin_pages+0x9a/0x110
Base page faults fail with -ENOMEM.
hugetlb faults fail with -EFAULT.
Check the policy zone against current_gfp_context(gfp), so a pin-scoped
fault falls back off the movable-only nodemask, like any other
non-movable allocation under the same bind.
Tested in qemu with a dax/kmem movable-only node 1. io_uring and VFIO
pins of unfaulted base pages and 2MB hugetlb pages now succeed and land
on node 0 (correct). QEMU now starts with the same backend binding.
Link: https://lore.kernel.org/20261018023349.1697434-1-gourry@gourry.net
Fixes:
|
||
|
Gregory Price (Meta)
|
d0a12d80d2 |
mm/vmscan: reclaim every zone in per-node proactive reclaim
user_proactive_reclaim() limits per-node reclaim to
gfp_zone(GFP_KERNEL), which is ZONE_NORMAL. Proactive reclaim serves no
allocation, but that limit makes shrink_node() skip every ZONE_MOVABLE
folio, so writing to /sys/devices/system/node/nodeN/reclaim never
reclaims a node's movable memory. A movable-only node, such as dax/kmem
or hotplugged memory onlined movable, reclaims nothing and fails with
-EAGAIN.
To reproduce, fill a movable-only node with clean page cache and write
200M to its nodeN/reclaim: the write fails with -EAGAIN and the page
cache stays. The same write to a ZONE_NORMAL node frees 200MB.
Scan every zone, as memory.reclaim does.
Link: https://lore.kernel.org/20261019185002.3428594-1-gourry@gourry.net
Fixes:
|
||
|
Lorenzo Stoakes (ARM)
|
65e722a74d |
mm/mremap: correctly uncharge accountable VMAs in error path
The mremap error path is a fragile and unwieldy beast - it reverses the
page table move then updates vma_remap_struct state to swap VMA, length and
address between old and new VMAs in copy_vma_and_data().
If the mremap() operation didn't specify MREMAP_DONTUNMAP then its caller,
move_vma(), invokes unmap_source_vma(), which unmaps the new VMA rather
than the old one.
For accountable VMAs when MREMAP_DONTUNMAP is not specified,
unmap_source_vma() unsets VMA_ACCOUNT_BIT during this unmap in order that
only the delta of a VMA expansion is charged (performed by
vrm_calc_charge() earlier).
Therefore nothing uncharges the expanded VMA size, and there is a permanent
leak in global Committed_AS.
Resolve the issue by uncharging when MREMAP_DONTUNMAP is not specified.
Link: https://lore.kernel.org/20261019-other-mremap-charge-leak-v1-1-d94477fe6545@kernel.org
Fixes:
|
||
|
Lorenzo Stoakes (ARM)
|
34e753f8d9 |
mm/mremap: populate calculated fields in vrm state correctly
Commit |
||
|
Nickolai Zeldovich
|
b428f19668 |
mm: bdi: reset min_ratio after the sysfs attributes are gone
A write to /sys/class/bdi/<dev>/min_ratio (or min_ratio_fine or min_bytes)
that races with the removal of the device leaves the global sum of minimum
dirty shares charged for a bdi that no longer exists. Until the next
reboot, that share of the dirty threshold is withheld from every other bdi
(__wb_calc_thresh() scales by 100 % minus the sum), and a later min_ratio
store on any bdi that would push the sum to 100 % fails with EINVAL
although every bdi reads 0. Root privilege is needed for the store; the
removal is any del_gendisk() (a disk or loop device going away) or the
unmount of a filesystem with its own bdi. On an x86_64 defconfig kernel
(v7.3-rc7), 2000 rounds of LOOP_CTL_ADD/LOOP_CTL_REMOVE on one loop device
with a second thread writing 50 to its min_ratio leak the share on the
first round the store lands in the window; afterwards `echo 99 >
min_ratio` on another loop device fails with EINVAL. With the fix the
same run keeps the sum at 0 and the store succeeds. Affected since v5.16,
when the reset in bdi_unregister() was added.
bdi_unregister() gives the bdi's share back with bdi_set_min_ratio(bdi, 0)
and only then calls device_unregister(), which removes the sysfs
attributes. A min_ratio_store() that gets its kernfs active reference
before the removal runs to completion after the reset:
__bdi_set_min_ratio() takes bdi_lock, sees bdi->min_ratio == 0, adds the
new value to bdi_min_ratio and stores it in the bdi, which bdi_put() then
frees. Nothing gives that share back.
Move the reset after device_unregister(): once the attributes are gone no
store can run, and bdi_set_min_ratio() needs nothing from the device.
Found while formally verifying bdi_unregister() against the kernel binary.
Link: https://lore.kernel.org/20261017030620.793108-1-nickolai@csail.mit.edu
Fixes:
|
||
|
Kyle Zeng
|
1a2993227e |
mm/page_vma_mapped: continue past nonmatching huge PMDs
A PTE-mapped large folio can straddle PMD boundaries after an unaligned
mremap(). If the first part is discarded and replaced with an unrelated
huge PMD, page_vma_mapped_walk() stops instead of looking for the rest of
the folio in subsequent PMDs.
This also affects migration entries. A walk that stops at an unrelated
huge PMD can leave migration PTEs behind when migration or an anonymous
folio split restores the mappings. Those entries do not hold mapping
references and can outlive the source folio.
This issue can lead to memory corruption and I have a reproducer for
it.
Skip the current PMD rather than ending the walk on a flag or PFN
mismatch. Do the same if the PMD became nonpresent before the locked
recheck. Release the PMD lock, clear the per-PMD state and advance with
step_forward(), so the existing range limit controls termination. This
covers present, migration and device-private huge PMDs while preserving
the matching-PMD and split-to-PTE paths.
Link: https://lore.kernel.org/20261016011549.61146-1-kylebot@openai.com
Fixes:
|
||
|
Muchun Song
|
c82dba3183 |
mm: warn on unaligned compound zone device ranges
A compound page of order N must start at a PFN aligned to 2^N pages. Therefore, a range used to initialize compound pages must start on a compound-page boundary and contain an integral number of compound pages. Two recent Device DAX bugs were caused by ranges that violated this invariant, but neither was detected by memmap_init_zone_device(). Warn if the adjusted zone-device range violates this invariant so future violations are reported when the compound pages are initialized. Link: https://lore.kernel.org/20261016131334.3419393-1-songmuchun@bytedance.com Link: https://lore.kernel.org/r/20261016053944.2579541-1-songmuchun@bytedance.com Link: https://lore.kernel.org/r/20261016054914.2601510-1-songmuchun@bytedance.com Signed-off-by: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Assisted-by: LLM Cc: Mike Rapoport <rppt@kernel.org> Cc: Vishal Verma <vishal.l.verma@intel.com> Cc: Dave Jiang <dave.jiang@intel.com> Cc: Alison Schofield <alison.schofield@intel.com> Cc: Joao Martins <joao.m.martins@oracle.com> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> |
||
|
Nickolai Zeldovich
|
8f90398405 |
mm/truncate: fix the zeroing end of pagecache_isize_extended()
A process that extends a file by 4 GiB or more past a dirty folio, on a file system whose block size is smaller than the page size (ext4 with 1k blocks), reads back the bytes it wrote through a shared mapping beyond the old end of file instead of zeroes, and a later write into that block carries them to the disk. With the example of commit |
||
|
Rik van Riel
|
1503710b3b |
mm/page_alloc: count guard pages as free again when their buddy merges
With debug_guardpage_minorder set, expand() turns the unused halves of a split into guard pages, which page_del_and_expand() leaves out of the free counts. Freeing the allocated half credits only that half, and __free_one_page() then merges the guard buddy with clear_page_guard(), which no longer touches the counters. The merged block goes on a free list with the guard pages never counted, so NR_FREE_PAGES falls behind the free lists by every guard that merges. Before commit |
||
|
Muchun Song
|
3d670c0dce |
mm/mm_init: add zone mismatch warning during page init
For vmemmap-optimized sections, tail struct pages may be backed by shared vmemmap pages. Those shared pages must carry the same page zone ID as the struct pages initialized for the section. Warn in __init_single_page() if the shared tail page has a different page_zone_id(), which would indicate inconsistent initialization. [muchun.song@linux.dev: document ordering of shared and retained struct page initialization] Link: https://lore.kernel.org/20261010115523.4005071-1-songmuchun@bytedance.com Link: https://lore.kernel.org/20261010075755.83339-7-songmuchun@bytedance.com Signed-off-by: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Lance Yang <lance.yang@linux.dev> Acked-by: Qi Zheng <qi.zheng@linux.dev> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Christophe Leroy <chleroy@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> |
||
|
Muchun Song
|
cc1fd8f401 |
mm/sparse-vmemmap: open-code vmemmap_populate_address()
vmemmap_populate_address() no longer has any callers that need the returned PTE. Its only remaining user, vmemmap_populate_range(), only checks whether population succeeded. Open-code vmemmap_populate_address() directly in vmemmap_populate_basepages(), remove the now-redundant range helper, and return -ENOMEM directly on failure. Link: https://lore.kernel.org/20261010075755.83339-6-songmuchun@bytedance.com Signed-off-by: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Lance Yang <lance.yang@linux.dev> Acked-by: Qi Zheng <qi.zheng@linux.dev> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Christophe Leroy <chleroy@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> |
||
|
Muchun Song
|
2815c5d88f |
mm/sparse-vmemmap: remove the unused ptpfn argument
vmemmap_pte_populate() no longer uses ptpfn as an input. Drop the argument to simplify the code. Link: https://lore.kernel.org/20261010075755.83339-5-songmuchun@bytedance.com Signed-off-by: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Lance Yang <lance.yang@linux.dev> Acked-by: Qi Zheng <qi.zheng@linux.dev> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Christophe Leroy <chleroy@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> |
||
|
Muchun Song
|
bd0862382a |
mm/sparse-vmemmap: drop Device DAX-specific population path
The common vmemmap path selects the shared page for optimized mappings itself, so Device DAX no longer needs vmemmap_populate_compound_pages() to find a shared tail page and pass its backing PFN through the generic population helpers. Remove the Device DAX-specific population path and let section memmap population always use vmemmap_populate(). The powerpc retains an architecture-specific compound-page implementation, so select it directly from radix__vmemmap_populate() for optimizable sections. Link: https://lore.kernel.org/20261010075755.83339-4-songmuchun@bytedance.com Signed-off-by: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Lance Yang <lance.yang@linux.dev> Acked-by: Qi Zheng <qi.zheng@linux.dev> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Christophe Leroy <chleroy@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> |
||
|
Muchun Song
|
a123cf7ac6 |
mm/sparse-vmemmap: support device DAX in common vmemmap path
The common vmemmap population path cannot yet handle optimized Device DAX mappings on its own. It uses pfn_to_zone() to find the shared tail page, but Device DAX populates its vmemmap at runtime before the ZONE_DEVICE span is initialized. Teach the common path to use device_zone() for runtime optimized vmemmap population while retaining pfn_to_zone() for early boot. This allows the same path to support both early boot mappings and Device DAX. The backing PFN supplied by the Device DAX-specific population path is no longer used, allowing the redundant lookup and population code to be removed later. Link: https://lore.kernel.org/20261010075755.83339-3-songmuchun@bytedance.com Signed-off-by: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Lance Yang <lance.yang@linux.dev> Acked-by: Qi Zheng <qi.zheng@linux.dev> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Christophe Leroy <chleroy@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> |
||
|
Muchun Song
|
4b77739fe7 |
mm/sparse-vmemmap: drop VMEMMAP_POPULATE_DAX
Patch series "mm: Unify device DAX and HugeTLB vmemmap population paths", v4. This series is split out from the earlier, larger series "mm: Generalize HVO for HugeTLB and device DAX" [2]. While the parent series generalizes vmemmap optimization across HugeTLB and device DAX, this subset addresses a single, self-contained step: unifying their vmemmap population paths. After the preceding Device DAX conversion, both HugeTLB and Device DAX describe optimized vmemmap mappings through memory-section metadata and use per-zone shared tail vmemmap pages. The generic code, however, still carries a Device DAX-specific population flag and compound-page population path, along with arguments and helpers needed only by that path. This series first removes VMEMMAP_POPULATE_DAX and moves selection and reference handling for the shared tail page into the common vmemmap population path. It then removes the generic Device DAX-specific compound-page population path and routes section vmemmap population through vmemmap_populate(). The powerpc radix path continues to use its architecture-specific compound-page population implementation for optimizable sections. The remaining patches remove the unused ptpfn argument, open-code vmemmap_populate_address() now that no caller needs its returned PTE, and add a warning for inconsistent zone initialization of shared tail vmemmap pages. This is the fourth smaller step toward the broader HVO generalization. After this series, HugeTLB and Device DAX use the same population model instead of parallel generic paths, while powerpc keeps its architecture-specific implementation. This patch (of 6): VMEMMAP_POPULATE_DAX currently distinguishes DAX vmemmap population in two places: it keeps allocations on the normal path and takes a reference when a backing page is supplied for reuse. After Device DAX switched to the common per-zone shared tail page, both conditions can be determined locally. DAX supplies ptpfn for every shared tail mapping and requests an allocation only for compound head mappings, whose PFNs are not optimizable. Therefore, vmemmap_optimizable_pfn() alone selects the correct allocation path. When ptpfn is supplied, the caller is reusing an existing backing page. Once the slab allocator is available, take a reference for each reused mapping to balance the release performed by vmemmap_free(). Although the buddy allocator is available before slab, no vmemmap population occurs in that interval. Earlier mappings are backed by memblock/reserved memory and do not need page reference accounting. Remove VMEMMAP_POPULATE_DAX and the flags argument from the vmemmap population helpers. Link: https://lore.kernel.org/20261010075755.83339-1-songmuchun@bytedance.com Link: https://lore.kernel.org/20261010075755.83339-2-songmuchun@bytedance.com Link: https://lore.kernel.org/20260930140627.57431-1-songmuchun@bytedance.com/ [1] Link: https://lore.kernel.org/20260513130542.35604-1-songmuchun@bytedance.com/ [2] Signed-off-by: Muchun Song <muchun.song@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Lance Yang <lance.yang@linux.dev> Acked-by: Qi Zheng <qi.zheng@linux.dev> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Christophe Leroy <chleroy@kernel.org> Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cc: Shrikanth Hegde <sshegde@linux.ibm.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> |
||
|
Jiayuan Chen
|
60f34fedf3 |
mm/swap: fix swap_cache_get_folio() returning a folio that left the swap cache
swap_cache_get_folio() reads the folio from the swap table and takes a
reference with folio_try_get(), without any lock. If the folio is removed
from the swap cache and freed in between, and its memory gets reused,
folio_try_get() still succeeds and we return a page that has nothing to do
with the swap entry:
CPU 0 CPU 1
do_swap_page()
swap_cache_get_folio()
swp_tb = swap_table_get()
folio = swp_tb_to_folio(swp_tb)
madvise(MADV_DONTNEED)
zap_nonpresent_ptes()
swap_put_entries_direct()
swap_put_entries_cluster()
__try_to_reclaim_swap()
swap_cache_del_folio()
folio_put() <- freed
page reused, e.g. as an anon
folio of another task
folio_try_get(folio) <- succeeds
return folio
swap_update_readahead() <- may clear PG_readahead on it
folio_lock_or_retry() <- locks it
folio_matches_swap_entry() fails, unlock and put
filemap_get_entry() does the same lockless lookup for the page cache, and
after folio_try_get() it rechecks the xarray, because the folio may have
been freed and reused in between. swap_cache_get_folio() lost that
recheck when the swap cache moved to the swap table.
So CPU 0 writes to a folio it doesn't own: it may clear PG_readahead and
takes the folio lock of someone else's page, before
folio_matches_swap_entry() finds out the folio is not ours.
When the race occurs, a swap fault can acquire a reference to a
physical page that has been freed and reused for an unrelated purpose,
potentially by another process. The fault handler may then clear that
page's PG_readahead/PG_reclaim flag, wait on or acquire its folio lock,
and potentially attempt to remove it from an unrelated swap cache.
These operations can interfere with unrelated workloads and may cause
unexpected stalls and impaired reclaim. The subsequent swap-entry
validation normally prevents the wrong folio from being mapped into the
faulting process.
It's documented that caller always double checks the folio so this part
is harmless from the aspect of data corruption or crash.
Recheck the slot after folio_try_get() and retry if it no longer holds
this folio.
[akpm@linux-foundation.org: add LLM-generated description of userspace impact]
Link: https://lore.kernel.org/20261015060528.302634-1-jiayuan.chen@linux.dev
Fixes:
|
||
|
Nimrod Oren
|
397e087605 |
mm: stop recalculating min_free_kbytes once set by userspace
min_free_kbytes is recalculated on memory hotplug and on THP sysfs writes. The new value comes from calculate_min_free_kbytes() or, while THP is enabled, from set_recommended_min_free_kbytes(). A value written to the vm.min_free_kbytes sysctl is replaced whenever the new value is higher, so a lower value does not stick. This matters most when THP is enabled on configurations with large pageblocks, where the new value can be excessive [1]. For example, with default arm64 pageblock sizes, the THP recommendation can reach 5.5 GiB per eligible zone with 64 KiB pages vs 22 MiB with 4 KiB pages. This keeps a significant amount of memory free, reducing what applications can use and triggering reclaim much earlier. Users can lower vm.min_free_kbytes to avoid this, but the lower value only sticks until the next recalculation. Stop recalculating min_free_kbytes once a value has been written to the sysctl. Systems where the sysctl is never written keep the current behavior. Link: https://lore.kernel.org/20261014140859.1006967-1-noren@nvidia.com Link: https://lore.kernel.org/r/20260716173504.760369-1-noren@nvidia.com/ [1] Signed-off-by: Nimrod Oren <noren@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Suggested-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Link: https://lore.kernel.org/r/d202dcc3-d7d7-4e98-a9a5-3137cfbfd041@kernel.org/ Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Reviewed-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Shuah Khan <skhan@linuxfoundation.org> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Lance Yang <lance.yang@linux.dev> Cc: Usama Arif <usama.arif@linux.dev> Cc: Kiryl Shutsemau <kas@kernel.org> Cc: Brendan Jackman <brendan.jackman@linux.dev> Cc: David Laight <david.laight.linux@gmail.com> |
||
|
Lorenzo Stoakes (ARM)
|
7135a05838 |
mm/vma: avoid spurious write locks when mm->mm_lock_seq overflows
So far the assumption has been that mm->mm_lock_seq overflow is benign, resulting only in readers spuriously falling back to the mmap lock, however it turns out that this was an unsafe assumption. If a VMA can be spuriously considered write locked that means read lock exclusion does not take place, and thus a VMA can end up write-locked with concurrent readers. Since overflow only occurs every 2^31 write lock/unlock cycles, resolve the issue by resetting every VMA's vma->vm_lock_seq each time such an overflow occurs. This is done in vma_end_write_all(), which is called when the mmap write lock is released, i.e. in mmap_write_unlock() and mmap_write_downgrade(). Care must be taken, as resetting the sequence count at overflow means changing the seqcount of VMAs which are already attached and observable. The mm's seqcount is guaranteed to be odd when a write lock is held, and thus any write locked VMA will have an odd seqcount assigned. Therefore resetting to an even value prevents spurious write locks, since these can only occur with the mmap write lock held, at which point the mmap seqcount will be odd. However, vma_start_read() also checks vma->vm_lock_seq == mm->mm_lock_seq, bailing if so (indicating a write lock). This would cause any even reset value chosen to prevent VMA read locks from being taken - avoid this by also checking whether the VMA seqcount is odd before bailing on the read lock. So far, vma->vm_lock_seq has defaulted to UINT_MAX, but this has been unnecessary since commit |
||
|
Vernon Yang
|
2ca8b38962 |
mm: vmscan: don't count per-node proactive reclaim as memory pressure
When the cgroup has no memory pressure at all, writing to /sys/devices/system/nodeX/reclaim triggers proactive reclaim on NUMA node, causing increase in the writer cgroup's memory PSI. Due to this reclaim is performed in the context of the write(), accounted as memory pressure on the writer, like commit |
||
|
Hui Zhu
|
00a51d0991 |
mm/vmscan: fix missing NR_ISOLATED counter update in MGLRU reclaim path
MGLRU evict_folios() isolates folios from the LRU without updating the NR_ISOLATED_ANON/FILE counters, unlike the legacy shrink_inactive_list() path. This causes compaction's too_many_isolated() check to under-count isolated pages when MGLRU reclaim is active. Add NR_ISOLATED counter updates in evict_folios(): increment after isolate_folios() and decrement after all retry passes complete, using the existing nr_isolated which holds the original isolated count. Link: https://lore.kernel.org/141901355a7d9e4bbca5c4840ae32ea5c65fe3b6.1791968227.git.zhuhui@kylinos.cn Link: https://lore.kernel.org/cover.1791968227.git.zhuhui@kylinos.cn Signed-off-by: Hui Zhu <zhuhui@kylinos.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Barry Song <baohua@kernel.org> Cc: Kairui Song <kasong@tencent.com> Cc: Qi Zheng <qi.zheng@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Wei Xu <weixugc@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: David Hildenbrand <david@kernel.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Lorenzo Stoakes <ljs@kernel.org> |
||
|
Pedro Falcato
|
246c8cc334 |
mm/madvise: atomically clear the young bit on THP MADV_COLD
As-is, madvise_cold_or_pageout_pte_range() discards the hardware young bit
by invalidating the PMD, doing pmd_mkold() and subsequently writing to the
pmd again. However, it does so in a racy manner, looking at a
possibly-stale snapshot. It's entirely possible that a write elsewhere
set the hardware D bit, which this logic now loses.
Fix it by using pmdp_test_and_clear_young(). This is an atomic
modification of the PMD entry, and thus safe against any possible racing
writes. As a (nice) side-effect, this elides one non-required TLB flush
in pmdp_invalidate().
This bug can interact with the swapping system in complicated ways,
leading to data loss. For instance, if a folio is marked lazyfree, but
was later read from, and then MADV_COLD'd while racing a write to that
same THP, dropping the dirty bit may lead to data loss as
reclaim/MADV_PAGEOUT drops the "clean" anon folio:
T0 T1
================================================================
MADV_FREE(thp) |
folio marked lazyfree |
read(thp) |
PMD marked young by hardware |
| MADV_COLD(thp)
write(thp) | if (pmd_young(pmd)) {
PMD marked dirty by hardware | set(pmdp, pmd_mkold(pmd));
MADV_PAGEOUT/reclaim |
dirty bit may or may not be |
set |
reclaim/pageout may drop |
the folio |
Link: https://lore.kernel.org/20261013191719.95502-1-pfalcato@suse.de
Fixes:
|
||
|
Harry Yoo (Meta)
|
c7a467fbcb |
mm/percpu: account full object size for percpu memory allocations
When memory allocation profiling is enabled, the pages backing percpu
memory chunks are attributed to func:pcpu_alloc_pages. However, the
accounting of individual percpu memory allocations via alloc_percpu()
and friends record only the size of a single object rather than its
full size across all CPUs.
As a result, it is often not straightforward to identify which kernel code
is actually responsible for percpu memory, because the size is
under-accounted by a factor of num_possible_cpus().
Account full object size with pcpu_obj_full_size(size) instead, which
multiplies the object size by num_possible_cpus() and adds the objcg
pointer overhead. This is consistent with what memcg already does for
charging percpu allocations.
Verified with 64-CPU VM by spawning 400 processes in a memory cgroup and
measuring mm_init()'s rss_stat counters.
Delta of kernel/fork.c:1142 func:mm_init in /proc/allocinfo:
bytes calls per allocation
unpatched 6320 395 16
patched 418176 396 1056
'percpu' field in memory.stat: 422400 (1056 bytes per process)
Memory allocation profiling now accounts the same amount of memory as
what memcg charges.
Link: https://lore.kernel.org/20261013-fix-alloc-percpu-accounting-v1-1-03e788a1e41f@kernel.org
Fixes:
|
||
|
Cen Zhang (Microsoft)
|
454143aeb4 |
userfaultfd: fix UFFDIO_MOVE livelock on large folio with extra references
To move one page out of a large folio, move_pages_ptes() first has to
split the folio. __folio_split() refuses if the folio has any reference
beyond its page table mappings and the caller's own, and it reports the
refusal as -EAGAIN. move_pages() treats returned -EAGAIN as a transient
race and keeps retrying the same address, holding the VMA read locks the
whole time. This is incorrect since in some cases userspace can hold such
a reference indefinitely, such as by registering the page as an io_uring
buffer.
An unprivileged user can trigger this with three threads: first holds such
a reference on a page of a THP, second calls UFFDIO_MOVE on another page
of the same THP and spins in the kernel forever holding the VMA read
locks, and third calls fork() or mprotect(), which takes mmap_lock for
writing and then waits forever for those read locks. From then on every
mmap_lock user of that mm hangs, including other processes inspecting it
through /proc/<pid>/, until the process is SIGKILLed. The hung task
detector reports the blocked tasks (third one), and with
kernel.hung_task_panic=1 the machine panics:
INFO: task wedge:95 blocked for more than 122 seconds.
task:wedge state:D stack:12528 pid:95 tgid:95 ppid:94
Call Trace:
__schedule+0x596/0x1110
schedule+0x36/0xf0
__vma_start_exclude_readers+0x100/0x170
__vma_start_write+0x51/0xd0
__split_vma+0x208/0x3d0
vma_modify+0x6a8/0x840
vma_modify_flags+0xce/0x110
mprotect_fixup+0xbe/0x480
do_mprotect_pkey+0x2d9/0x580
__x64_sys_mprotect+0x1a/0x30
Fix by returning -EBUSY instead of -EAGAIN in move_pages_ptes(), the
caller of split_folio(), when the split fails. Thus, move_pages() does
not retry -EBUSY and the ioctl drops its locks and returns to userspace
instead of spinning.
Link: https://lore.kernel.org/20261012201514.97015-1-cenzhang@linux.microsoft.com
Fixes:
|
||
|
Nickolai Zeldovich
|
3ae91e3079 |
mm/mlock: use a long for the page count in mlock_fixup()
mlock_fixup() adjusts mm->locked_vm, an unsigned long, by the number of
pages whose VM_LOCKED state it changes, but computes that number in an
int:
int nr_pages;
...
nr_pages = (end - start) >> PAGE_SHIFT;
...
mm->locked_vm += nr_pages;
When a single VMA range of 2^31 pages (8 TiB with 4 KiB pages) or more
is locked or unlocked, the count is truncated to 32 bits and then
sign-extended, so locked_vm is changed by the wrong amount, while
munmap(), mremap() and the other paths that maintain locked_vm use the
full unsigned long page count.
Such a range is reachable on 64-bit architectures with a 47-bit or
larger user address space (x86_64, arm64 with 48-bit VA, riscv64 Sv48)
by a task with CAP_IPC_LOCK or a raised RLIMIT_MEMLOCK, which do_mlock()
checks against the full untruncated length. It costs nothing with
MAP_NORESERVE and MLOCK_ONFAULT/MCL_ONFAULT, since then no page is
populated. On x86_64, after
p = mmap(NULL, 8UL << 40, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
mlock2(p, 8UL << 40, MLOCK_ONFAULT);
VmLck in /proc/self/status reads 18446744065119617024 kB instead of
8589934592 kB; munmap() of the region then subtracts the true size and
leaves VmLck at 18446744056529682432 kB instead of 0. With 16 TiB,
locked_vm is not changed at all, VmLck stays 0 kB, and after the task
drops CAP_IPC_LOCK or lowers RLIMIT_MEMLOCK, mlock(), mmap(MAP_LOCKED)
and the other mlock_future_ok() and account_locked_vm() users enforce
RLIMIT_MEMLOCK against that count, so they let the task lock memory
beyond its limit (or, in the 8 TiB case, refuse every further lock).
The error persists for the life of the mm; unlocking the range in pieces
skews it further. The wrong count is per-mm accounting only: it does
not change which pages are locked.
The int has been there since before git history (the variable was named
"pages" until it was renamed in 2.6.28); every other updater of
locked_vm uses an unsigned long or long count. Make nr_pages a long,
which holds the page count of any user range and its negation, so the
existing sign handling is unchanged.
Found by formal verification of mm/mlock.c against the kernel binary.
Link: https://lore.kernel.org/20261014131358.780240-1-nickolai@csail.mit.edu
Fixes:
|
||
|
Kiryl Shutsemau (Meta)
|
6d06041c83 |
mm/collapse: separate scanning a PTE table from collapsing it
A collapse is two jobs. One reads a PTE table under mmap_lock and decides
whether the range is worth collapsing. The other allocates, isolates,
copies and flushes, and wants the lock given up first.
collapse_single_pmd() did both, so the boundary between them was somewhere
in the middle of a function.
Give each half its own function:
- collapse_scan_pmd() scans one table. It picks the side:
collapse_scan_anon(), the anonymous scan that used to carry the
collapse_scan_pmd() name, or collapse_scan_file().
- collapse_pmd() does the collapse the scan asked for, and is handed
what the scan returned. SCAN_SUCCEED means there is something to
collapse. SCAN_PTE_MAPPED_HUGEPAGE means the page cache already
holds the PMD folio and only the PTE table is left to retract.
Anything else is why there is nothing to do. It calls
collapse_anon(), formerly mthp_collapse(), or collapse_file().
For now collapse_single_pmd() stays, as the two of them with the
mmap_lock drop in between, so its callers see what they saw before.
collapse_control_init() sets a control up before its first scan.
What the scan found and the collapse needs travels in collapse_control.
For an anonymous table that is the orders and the referenced and
swapped-out counts. collapse_anon() and collapse_huge_page() now read
them from there instead of taking them as arguments.
For a file it is the file itself and the offset in it. A file collapse
works on the page cache and never sees a VMA, so the scan takes the
reference while it still has one and the collapse gives it back.
The file scan moves under mmap_lock with the anonymous one, where before
the lock was given up first. The lock is now held over the page cache
walk, an RCU walk over one table's worth of slots with no PTL, and taken
fewer times. collapse_scan_mm_slot() ends its walk whenever the lock was
dropped, so a refused file table used to cost khugepaged an unlock, a
trip back through khugepaged_do_scan(), a relock and a VMA lookup. Now
only a table that goes on to be collapsed does.
Link: https://lore.kernel.org/20261014171145.3292736-10-kas@kernel.org
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Assisted-by: LLM
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Nico Pache <nico.pache@linux.dev>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Jann Horn <jannh@google.com>
|
||
|
Kiryl Shutsemau (Meta)
|
a2d1c9ea76 |
mm/collapse: call collapse_file() from collapse_single_pmd()
collapse_scan_file() reads the page cache to decide whether a table is worth collapsing and, when it is, calls collapse_file() itself. The caller cannot get between the decision and the collapse. Move the collapse_file() call up into collapse_single_pmd(), so the scan stops at the decision. Two things change with it. The writeback retry re-runs collapse_file() alone instead of rescanning first; collapse_file() repeats the scan's checks under the page cache lock anyway. And mm_khugepaged_scan_file fires before the collapse, so for an accepted table its status reads SCAN_SUCCEED; what the collapse made of the table is for mm_khugepaged_collapse_file to report. Preparation for splitting a collapse into a scan under mmap_lock and a run without it. The file scan has to stop where the anonymous one will. Link: https://lore.kernel.org/20261014171145.3292736-9-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
Kiryl Shutsemau (Meta)
|
e7c4148ad1 |
mm/collapse: name the per-table scan reset for what it resets
collapse_control_init_scan() resets what one scan accumulates: the node load, the allocation nodemask and the eligible-PTE bitmap. It runs before every PTE table a scan is given, not once per control, so its name points at the wrong thing. Call it collapse_scan_reset(). Preparation for giving a control a real init, run once for a whole series of scans. Two names a word apart would then stand for two different jobs. No functional change. Link: https://lore.kernel.org/20261014171145.3292736-8-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
Kiryl Shutsemau (Meta)
|
d5dd87dbd6 |
mm/collapse: drop the collapse_possible() wrapper
collapse_possible() only forwards to collapse_possible_orders() and turns its mask into a bool. Its three callers can test the mask themselves. No functional change. Link: https://lore.kernel.org/20261014171145.3292736-7-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
Kiryl Shutsemau (Meta)
|
4460650576 |
mm/collapse: state what a collapse may do in the policy
Tests scattered through the collapse path decide what a collapse is allowed to do by asking whether khugepaged started it. Between them they settle: - which VMAs are eligible, and how hard to try for a folio; - how many empty, swapped-out or shared PTEs a window may contain, and whether a sub-PMD window is held to a stricter rule than a PMD; - whether a range has to look used, and whether a MADV_FREE'd page is left alone; - whether the PMD is mapped as part of the request, and whether dirty pages are worth writing back and retrying. None of those is a fact about khugepaged. Each is something the caller decided before asking, and the collapse code should not have to look up who called to find out. Add struct collapse_policy for the caller to fill: khugepaged from its own settings, MADV_COLLAPSE from the fact that a user asked explicitly. Every test becomes a read of a field, and cc->is_khugepaged goes, having no reader left. The PTE limits come as two sets, one for a PMD-sized window and one for anything smaller, so that the helpers pick a set for the order and read it. khugepaged takes no swapped-out or shared PTE into a sub-PMD window, and empty PTEs only when the knob says all or nothing; the warning for a knob value in between moves to where khugepaged fills its policy. The fields only one side reads say which: anon_ or file_. David Hildenbrand asked for both. khugepaged fills the policy once per scan pass, MADV_COLLAPSE once per call. That is the one change in behaviour. The max_ptes_* limits and the defrag setting behind the allocation mask are sampled once per pass rather than on every table. A table scanned early in a pass and one scanned late are then treated alike. collapse_file() also drops a NULL check on the collapse_control. It has one call site, reached only from collapse_single_pmd(), which dereferences cc unconditionally, so the check was already dead. Link: https://lore.kernel.org/20261014171145.3292736-6-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Tested-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
Kiryl Shutsemau (Meta)
|
81f8bfda35 |
mm/collapse: add collapse.h for the collapse interface
khugepaged.c holds both the users of collapse and the machinery that performs it. The daemon's scan loop, the sysfs tunables, MADV_COLLAPSE's entry point and the collapse itself all sit in one file and reach into each other freely. Nothing marks where a user ends and the engine begins. Start drawing that line. Add mm/collapse.h for what the two sides have to agree on: - enum scan_result - what the engine hands back; - struct collapse_control - the state a request carries. And two constants move with them: - KHUGEPAGED_MAX_PTES_LIMIT -> COLLAPSE_MAX_PTES_LIMIT; - KHUGEPAGED_MIN_MTHP_ORDER -> COLLAPSE_MIN_MTHP_ORDER. Neither is a fact about the daemon, so both lose the KHUGEPAGED_ prefix. No functional change. Link: https://lore.kernel.org/20261014171145.3292736-5-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
Kiryl Shutsemau (Meta)
|
d950ec79fc |
mm/khugepaged: rename mthp_present_ptes bitmap to eligible_ptes
The name says less than the bit means. A set bit means not only that the PTE is present, but also that it passed the other checks: uffd, lazyfree, anonymity, sharing. The PTE can be considered a collapse source. mthp_collapse() then reads the bitmap starting at the PMD order, so the bitmap is not specific to mTHP either. Name it for what a set bit means, and update the comments that named it. No functional change. Link: https://lore.kernel.org/20261014171145.3292736-4-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
Kiryl Shutsemau (Meta)
|
47cda827a0 |
mm/khugepaged: count collapses where khugepaged makes them
collapse_single_pmd() bumps khugepaged_pages_collapsed for its caller, and tests cc->is_khugepaged to know whether it should: the counter belongs to the daemon, and MADV_COLLAPSE must not touch it. The daemon sees every result of every collapse it asks for, so it can keep its own counter without the shared path testing who called. Link: https://lore.kernel.org/20261014171145.3292736-3-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
Kiryl Shutsemau (Meta)
|
b83970d696 |
mm/khugepaged: drop redundant mm_struct pin in madvise_collapse()
Patch series "mm/collapse: separate a collapse from its callers", v5. There is no line between the collapse engine and the callers that ask for a collapse. khugepaged.c holds both, and they reach into each other. - Sixteen tests through the collapse path read cc->is_khugepaged to work out what they are allowed to do, when every one of those decisions was made by the caller before it asked. - collapse_single_pmd() does both halves of a collapse behind one call. Where it gives up mmap_lock between them is buried in the anonymous scan on one side and ahead of the file scan on the other. Start drawing the line. The caller states what it allows in a policy, and the collapse code reads the policy instead of asking who called. The scan of a PTE table and the collapse that follows get a function each, with the mmap_lock drop between them in one place. This patch (of 9): madvise_collapse() holds an mmgrab() reference across its work. It is redundant. Every caller already holds mm_users: - madvise(2) works on current->mm, which lives as long as the task is in the syscall; - process_madvise(2) reaches a remote mm through mm_access(), which takes an mm_users reference and holds it until the syscall returns; - io_uring passes current->mm; - DAMON takes one with get_task_mm() and drops it after the call. Drop the mmgrab()/mmdrop() pair. Link: https://lore.kernel.org/20261014171145.3292736-1-kas@kernel.org Link: https://lore.kernel.org/20261014171145.3292736-2-kas@kernel.org Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Assisted-by: LLM Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Usama Arif <usama.arif@linux.dev> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> |
||
|
David Hildenbrand (Arm)
|
514c9247db |
mm/userfaultfd: only allocate anon_vma in CoW mappings
find_vma_and_prepare_anon() can end up allocating an anon_vma in VMAs that are not CoW mappings, which is wrong. If we have a MAP_SHARED R/O file mapped, we lack VM_SHARED (but do have VM_MAYSHARE) and lack VM_MAYWRITE. While uffd later rejects such mappings (uffd registration requires VM_MAYWRITE, see comment in validate_dst_vma) find_vma_and_prepare_anon() still wrongly allocates an anon vma before it validates the vma. This in turn later confuses copy_page_range()->vma_needs_copy() such that we end up calling copy_hugetlb_page_range() with ranges that actually do not contain *any* per-pagetable information that would require copying, which is the basis for PMD-table sharing in the first place. copy_hugetlb_page_range() should not have to worry about shared PMD tables at all: it doesn't make any sense to copy page tables that are shared, as they can be fully recreated later. Ending up in the function with shareable page tables is the real issue. If a destination PMD table is shared in copy_hugetlb_page_range(), it might get concurrently unshared as we are not holding relevant locks, resulting in us using a page table that is no longer mapped and possibly freed after calling: dst_pte = huge_pte_alloc(dst, dst_vma, addr, sz); To fix it, do not allocate an anon_vma in non-CoW mappings and add a safety guard to copy_hugetlb_page_range(), making sure that we never end up messing with shared PMD tables in there. Before commit |
||
|
Enze Li
|
bd918fca02 |
mm/hugetlb: use sizeof(buf) in string_get_size() calls
Replace the hardcoded 32 with sizeof(buf) at the remaining four call sites, for consistency with the rest of the file. Link: https://lore.kernel.org/20261014031147.444733-1-lienze@kylinos.cn Signed-off-by: Enze Li <lienze@kylinos.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: David Hildenbrand <david@kernel.org> |
||
|
Kiryl Shutsemau (Meta)
|
4cd78a2dcf |
mm: page_alloc: make defrag_mode retries follow the promoted order
Since commit |
||
|
Enze Li
|
1b0f9f70a3 |
mm/hugetlb: move hugetlb_sysfs_initialized into hugetlb_register_all_nodes()
hugetlb_sysfs_init() guards the "hugetlb_sysfs_initialized = true" assignment with an #ifdef CONFIG_NUMA block, while the following hugetlb_register_all_nodes() call is left unguarded and relies on the empty !CONFIG_NUMA stub. Move the assignment into the NUMA version of hugetlb_register_all_nodes(), before it registers the online nodes, so the #ifdef in hugetlb_sysfs_init() can go away. Keep the !CONFIG_NUMA stub, so the call site is unconditional in both configurations. No functional change; this is an #ifdef cleanup. Link: https://lore.kernel.org/20261009142010.552614-1-lienze@kylinos.cn Signed-off-by: Enze Li <lienze@kylinos.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Suggested-by: Muchun Song <muchun.song@linux.dev> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: Enze Li <enze.li@gmx.com> |
||
|
Zi Yan
|
984c070085 |
mm/rmap: don't discard lazyfree THPs at PMD level
__discard_anon_folio_pmd_locked() clears a lazyfree THP's PMD before it
determines whether the folio can be discarded, and restores it if the
folio is dirty or has extra references. It is done under only the PMD
lock and the anon_vma read lock inside try_to_unmap_one() and no mmap lock
is held. The temporarily none PMD causes UAF.
A parallel munmap() or mremap() is supposed to serialize with it on the
PMD lock, but zap_pmd_range() (called by munmap()) and move_page_tables()
(called by mremap()) do not take the lock when they see the temporarily
none PMD and skip the PMD. There is no more page table walk after the
none PMD is restored:
- munmap(): free_pgtables() waits in unlink_anon_vmas() for the anon_vma
lock, then free_pmd_range() drops the restored PMD as bad without
removing the rmap, so the folio keeps a stale mapcount.
- whole-VMA MREMAP_DONTUNMAP: dontunmap_complete() unlinks the source VMA
from its anon_vma while the restored PMD still maps the folio.
- plain mremap(): the THP is not moved but unmapped, so the THP's data is
lost.
In the first two cases, the folio stays mapped, but no VMA that maps it is
linked to its anon_vma. When the anon_vma is freed, it is still reachable
when one rmap walks the folio. Racing lazyfree reclaim against
MREMAP_DONTUNMAP as an unprivileged user reproduces a KASAN use-after-free
in folio_lock_anon_vma_read() as reported in the Closes link.
One solution is to use an invalidated PMD instead of a cleared one until
the discard is committed, but it needs arch fixes first.
1. GUP-fast still follows an invalidated PMD on riscv, LoongArch and s390.
2. sparc64 miscounts thp_pte_count when an invalidated PMD is set again or
cleared.
To keep the fix simple for backport, stop discarding lazyfree THPs at PMD
level instead. try_to_unmap_one() splits the PMD and discards the folio at
PTE level.
unmap_huge_pmd_locked() and __discard_anon_folio_pmd_locked() are left in
place, unused, to minimize this fix for backport and the invalidated PMD
approach is planned as a follow-up series to use them again.
Link: https://lore.kernel.org/20261012-lazyfree-stable-fix-v2-1-6387d6e30063@nvidia.com
Fixes:
|