mirror of
https://github.com/torvalds/linux.git
synced 2026-07-27 17:47:41 +02:00
60bee40e30
1446347 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
SeongJae Park
|
60bee40e30 |
Docs/mm/damon/design: update for context pause/resume feature
Update DAMON design document for the context execution pause/resume feature. Link: https://lore.kernel.org/20260427151231.113429-4-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Shuah Khan <shuah@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
SeongJae Park
|
3375284944 |
mm/damon/sysfs: add pause file under context dir
Add pause DAMON sysfs file under the context directory. It exposes the damon_ctx->pause API parameter to the users so that they can use the pause/resume feature. Link: https://lore.kernel.org/20260427151231.113429-3-sj@kernel.org Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Shuah Khan <shuah@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
SeongJae Park
|
3b9e3cc040 |
mm/damon/core: introduce damon_ctx->paused
Patch series "mm/damon: let DAMON be paused and resumed", v2.
DAMON utilizes a few mechanisms that enhance itself over time. Adaptive
regions adjustment, goal-based DAMOS quota auto-tuning and monitoring
intervals auto-tuning like self-training mechanisms are such examples. It
also adds access frequency stability information (age) to the monitoring
results, which makes it enhanced over time.
Sometimes users have to stop DAMON. In this case, DAMON internal state
that enhanced over the time of the last execution simply goes away.
Restarted DAMON have to train itself and enhance its output from the
scratch. This makes DAMON less useful in such cases. Introducing three
such use cases below.
Investigation of DAMON. It is best to do the investigation online,
especially when it is a production environment. DAMON therefore provides
features for such online investigations, including DAMOS stats, monitoring
result snapshot exposure, and multiple tracepoints. When those are
insufficient, and there are additional clues that could be interfered by
DAMON, users have to temporarily stop DAMON to collect the additional
clues. It is not very useful since many of DAMON internal clues are gone
when DAMON is stopped. The loss of the monitoring results that improved
over time is also problematic, especially in production environments.
Monitoring of workloads that have different user-known phases. For
example, in Android, applications are known to have very different access
patterns and behaviors when they are running on the foreground and the
background. It can therefore be useful to separate monitoring of apps
based on whether they are running on the foreground and on the background.
Having two DAMON threads per application that paused and resumed for the
apps foreground/background switches can be useful for the purpose. But
such pause/resume of the execution is not supported.
Tests of DAMON. A few DAMON selftests are using drgn to dump the internal
DAMON status. The tests show if the dumped status is the same as what the
test code expected. Because DAMON keeps running and modifying its
internal status, there are chances of data races that can cause false test
results. Stopping DAMON can avoid the race. But, since the internal
state of DAMON is dropped, the test coverage will be limited.
Let DAMON execution be paused and resumed without loss of the internal
state, to overhaul the limitations. For this, introduce a new DAMON
context parameter, namely 'pause'. API callers can update it while the
context is running, using the online parameters update functions
(damon_commit_ctx() and damon_call()). Once it is set, kdamond_fn() main
loop will do only limited works excluding the monitoring and DAMOS works,
while sleeping sampling intervals per the work. The limited works include
handling of the online parameters update. Hence users can unset the
'pause' parameter again. Once it is unset, kdamond_fn() main loop will do
all the work again (resumed). Under the paused state, it also does stop
condition checks and handling of it, so that paused DAMON can also be
stopped if needed. Expose the feature to the user space via DAMON sysfs
interface. Also, update existing drgn-based tests to test and use the
feature.
Tests
=====
I confirmed the feature functionality using real time tracing ('perf
trace' or 'trace-cmd stream') of damon:damon_aggregated DAMON tracepoint.
By pausing and resuming the DAMON execution, I was able to see the trace
stops and continued as expected. Note that the pause feature support is
added to DAMON user-space tool (damo) after v3.1.9. Users can use
'--pause_ctx' command line option of damo for that, and I actually used it
for my test. The extended drgn-based selftests are also testing a part of
the functionality.
Patches Sequence
================
Patch 1 introduces the new core API for the pause feature. Patch 2 extend
DAMON sysfs interface for the new parameter. Patches 3-5 update design,
usage and ABI documents for the new sysfs file, respectively. The
following five patches are for tests. Patch 6 implements a new kunit test
for the pause parameter online commitment. Patches 7 and 8 extend DAMON
selftest helpers to support the new feature. Patch 9 extends selftest to
test the commitment of the feature. Finally, patch 10 updates existing
selftest to be safe from the race condition using the pause/resume
feature.
This patch (of 10):
DAMON supports only start and stop of the execution. When it is stopped,
its internal data that it self-trained goes away. It will be useful if
the execution can be paused and resumed with the previous self-trained
data.
Introduce per-context API parameter, 'paused', for the purpose. The
parameter can be set and unset while DAMON is running and paused, using
the online parameters commit helper functions (damon_commit_ctx() and
damon_call()). Once 'paused' is set, the kdamond_fn() main loop does only
limited works with sampling interval sleep during the works. The limited
works include the handling of the online parameters update, so that users
can unset the 'pause' and resume the execution when they want. It also
keep checking DAMON stop conditions and handling of it, so that DAMON can
be stopped while paused if needed.
Link: https://lore.kernel.org/20260427151231.113429-1-sj@kernel.org
Link: https://lore.kernel.org/20260427151231.113429-2-sj@kernel.org
Signed-off-by: SeongJae Park <sj@kernel.org>
Cc: Brendan Higgins <brendan.higgins@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
||
|
Frederick Mayle
|
7b32f64bc5 |
mm: limit filemap_fault readahead to VMA boundaries
When a file mapping covers a strict subset of a file, an access to the
mapping can trigger readahead of file pages outside the mapped region.
Readahead is meant to prefetch pages likely to be accessed soon, but these
pages aren't accessible via the same means, so it fair to say we don't
have a good indicator they'll be accessed soon. Take an ELF file for
example: an access to the end of a program's read-only segment isn't a
sign that nearby file contents will be accessed next (they are likely to
be mapped discontiguously, or not at all). The pressure from loading
these pages into the cache can evict more useful pages.
To improve the behavior, make three changes:
* Introduce a new readahead_control field, max_index, as a hard limit on
the readahead. The existing file_ra_state->size can't be used as a
limit, it is more of a hint and can be increased by various
heuristics.
* Set readahead_control->max_index to the end of the VMA in all of the
readahead paths that can be triggered from a fault on a file mapping
(both "sync" and "async" readahead).
* Limit the read-around range start to the VMA's start.
Note that these changes only affect readahead triggered in the context of
a fault, they do not affect readahead triggered by read syscalls. If a
user mixes the two types of accesses, the behavior is expected to be the
following: if a fault causes readahead and places a PG_readahead marker
and then a read(2) syscall hits the PG_readahead marker, the resulting
async readahead *will not* be limited to the VMA end. Conversely, if a
read(2) syscall places a PG_readahead marker and then a fault hits the
marker, the async readahead *will* be limited to the VMA end.
There is an edge case that the above motivation glosses over: A single
file mapping might be backed by multiple VMAs. For example, a whole file
could be mapped RW, then part of the mapping made RO using mprotect. This
patch would hurt performance of a sequential faulted read of such a
mapping, the degree depending on how fragmented the VMAs are. A usage
pattern like that is likely rare and already suffering from sub-optimal
performance because, e.g., the fragmented VMAs limit the fault-around, so
each VMA boundary in a sequential faulted read would cause a minor fault.
Still, this patch would make it worse. See a previous discussion of this
topic at [1].
Tested by mapping and reading a small subset of a large file, then using
the cachestat syscall to verify the number of cached pages didn't exceed
the mapping size.
In practical scenarios, the effect depends on the specific file and usage.
Sometimes there is no effect at all, but, for some ELF files in Android,
we see ~20% fewer pages pulled into the cache.
A comprehensive performance evaluation hasn't been done, but, in addition
to the anecdontal memory savings mentioned above, a benchmark was run with
fio 3.38, showing neutral looking results:
/data/local/tmp/fio --version
fio --name=mmap_test --ioengine=mmap --rw=read --bs=4k \
--offset=1G --size=1G --filesize=3G --numjobs=1 \
--filename=testfile.bin
Before: 4366.6 MiB/s (avg of 3459, 4592, 4613, 4697, 4472)
After: 4444.0 MiB/s (avg of 4633, 4655, 4511, 4571, 3850)
+1.7%
Same, with --ioengine=mmap --rw=randread
Before: 445.6 MiB/s (avg of 446, 447, 442, 452, 441)
After: 447.0 MiB/s (avg of 447, 446, 446, 451, 445)
+0.3%
Same, with --ioengine=psync --rw=read
Before: 3086.6 MiB/s (avg of 3122, 3094, 3066, 3094, 3057)
After: 3084.6 MiB/s (avg of 3039, 3103, 3103, 3084, 3094)
-0.06%
Same, with --ioengine=psync --rw=randread
Before: 2226.4 MiB/s (avg of 2256, 2183, 2207, 2265, 2221)
After: 2231.4 MiB/s (avg of 2236, 2241, 2236, 2193, 2251)
+0.2%
Link: https://lore.kernel.org/20260427030148.653228-1-fmayle@google.com
Link: https://lore.kernel.org/all/ivnv2crd3et76p2nx7oszuqhzzah756oecn5yuykzqfkqzoygw@yvnlkhjjssoz/ [1]
Signed-off-by: Frederick Mayle <fmayle@google.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Kalesh Singh <kaleshsingh@google.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
||
|
fujunjie
|
0b20c36c11 |
mm/madvise: reject invalid process_madvise() advice for zero-length vectors
process_madvise() used to validate the advice while walking each imported
iovec. If the vector has zero total length, vector_madvise() does not
enter the loop and can return success without checking whether the advice
value is valid.
For a local mm, such as process_madvise(PIDFD_SELF, ...), the remote-only
process_madvise_remote_valid() check is skipped. As a result, an invalid
advice can be reported as success when the vector has zero total length.
This differs from madvise(), which rejects an invalid advice before
returning success for a zero-length range.
Validate the generic madvise behavior at the syscall-facing entry points
before any vector walk. In process_madvise(), do this before the
remote-only advice restriction so unsupported advice is rejected with the
same priority for local and remote mm.
Use an errno-returning helper for address/length validation, and handle
zero-length ranges explicitly at the call sites. Requests with valid
advice and zero total length remain a noop and continue to return 0. Add
a selftest that covers invalid advice with a zero-length iovec and an
empty vector, while also checking that a request with valid advice and
zero length still succeeds.
Link: https://lore.kernel.org/tencent_C3AEB0E769C5F4F9370F9411B69B7F8B2907@qq.com
Fixes:
|
||
|
David Hildenbrand (Arm)
|
90f01f5d6b |
mm: remove page_mapped()
Let's replace the last user of page_mapped() by folio_mapped() so we can get rid of page_mapped(). Replace the remaining occurrences of page_mapped() in rmap documentation by folio_mapped(). Link: https://lore.kernel.org/20260427-page_mapped-v1-3-e89c3592c74c@kernel.org Signed-off-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Eduard Zingerman <eddyz87@gmail.com> Cc: Harry Yoo <harry@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Rich Felker <dalias@libc.org> Cc: Rik van Riel <riel@surriel.com> Cc: Song Liu <song@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Yonghong Song <yonghong.song@linux.dev> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
David Hildenbrand (Arm)
|
88692f0c33 |
bpf: arena: use page_ref_count() instead of page_mapped() in arena_free_pages()
Pages that BPF arena code maps are allocated through bpf_map_alloc_pages(), which does not allocate folios but pages. In the future, pages will not have a mapcount, only folios will. Converting the code to use folios and rely on folio_mapped() sounds like the wrong approach. Should BPF arena code allocate folios and use folio_mapped() here? But likely we would not want to use folios here longterm, as we don't really need folio information. Hard to tell. But in the meantime, we can simply use the page refcount instead, as a heuristic whether the page might be mapped to user space and we would want to try zapping it, so we can get rid of page_mapped(). Page allocation will give us a page with a refcount of 1. Any user space mapping adds a page reference. While there can be references from other subsystems (e.g., GUP), in the common case for this test here relying on the page count is good enough. Link: https://lore.kernel.org/20260427-page_mapped-v1-2-e89c3592c74c@kernel.org Signed-off-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Eduard Zingerman <eddyz87@gmail.com> Cc: Harry Yoo <harry@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Rich Felker <dalias@libc.org> Cc: Rik van Riel <riel@surriel.com> Cc: Song Liu <song@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Yonghong Song <yonghong.song@linux.dev> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
David Hildenbrand (Arm)
|
8803f88331 |
sh: use folio_mapped() instead of page_mapped() in sh4_flush_cache_page()
Patch series "mm: remove page_mapped()". While preparing my slides for an LSF/MM talk, I realized that I did not yet remove page_mapped(). So let's do that. In the BPF arena code it's unclear which memdesc we would want to allocate in the future: certainly something with a refcount, but likely none with a mapcount. So let's just rely on the page refcount instead to decide whether we want to try zapping the page from user page tables. This patch (of 3): We already have the folio in our hands, so let's just use folio_mapped(). Link: https://lore.kernel.org/20260427-page_mapped-v1-0-e89c3592c74c@kernel.org Link: https://lore.kernel.org/20260427-page_mapped-v1-1-e89c3592c74c@kernel.org Signed-off-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Eduard Zingerman <eddyz87@gmail.com> Cc: Harry Yoo <harry@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Martin KaFai Lau <martin.lau@linux.dev> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Rich Felker <dalias@libc.org> Cc: Rik van Riel <riel@surriel.com> Cc: Song Liu <song@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Yonghong Song <yonghong.song@linux.dev> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Asier Gutierrez
|
58996503b6 |
mm/damon: support MADV_COLLAPSE via DAMOS_COLLAPSE scheme action
This patch set introces a new action: DAMOS_COLLAPSE. For DAMOS_HUGEPAGE and DAMOS_NOHUGEPAGE to work, khugepaged should be working, since it relies on hugepage_madvise to add a new slot. This slot should be picked up by khugepaged and eventually collapse (or not, if we are using DAMOS_NOHUGEPAGE) the pages. If THP is not enabled, khugepaged will not be working, and therefore no collapse will happen. DAMOS_COLLAPSE eventually calls madvise_collapse, which will collapse the address range synchronously. In cases where there is a large VMA (databases, for example), DAMOS_COLLAPSE allows us to collapse only the hot region, and not the entire VMA. This new action may be required to support autotuning with hugepage as a goal[1]. ========= Benchmarks: ========= MySQL ===== Tests were performed in an ARM physical server with MariaDB 10.5 and sysbench. Read only benchmark was perform with gaussian row hitting, which follows a normal distribution. T n, D h: THP set to never, DAMON action set to hugepage T m, D h: THP set to madvise, DAMON action set to hugepage T n, D c: THP set to never, DAMON action set to collapse Memory consumption. Lower is better. +------------------+----------+----------+----------+ | | T n, D h | T m, D h | T n, D c | +------------------+----------+----------+----------+ | Total memory use | 2.13 | 2.20 | 2.20 | | Huge pages | 0 | 1.3 | 1.27 | +------------------+----------+----------+----------+ Performance in TPS (Transactions Per Second). Higher is better. T n, D h: 18225.58 T m, D h 18252.93 T n, D c: 18270.21 Performance counter I got the number of L1 D/I TLB accesses and the number a D/I TLB accesses that triggered a page walk. I divided the second by the first to get the percentage of page walkes per TLB access. The lower the better. +---------------+--------------+--------------+--------------+ | | T n, D h | T m, D h | T n, D c | +---------------+--------------+--------------+--------------+ | L1 DTLB | 127248242753 | 125431020479 | 125327001821 | | L1 ITLB | 80332558619 | 79346759071 | 79298139590 | | DTLB walk | 75011087 | 52800418 | 55895794 | | ITLB walk | 71577076 | 71505137 | 67262140 | | DTLB % misses | 0.058948623 | 0.042095183 | 0.044599961 | | ITLB % misses | 0.089100954 | 0.090117275 | 0.084821839 | +---------------+--------------+--------------+--------------+ Masim ===== I used masim with the "demo" configuration, but changing the times to 100 seconds for the initial phase and 50 seconds for the rest of the phases. Memory consumption: +------------------+----------+----------+----------+ | | T n, D h | T m, D h | T n, D c | +------------------+----------+----------+----------+ | Total memory use | 2.38 GB | 2.36 GB | 2.37 GB | | Huge pages | 0 | 190 MB | 188 MB | +------------------+----------+----------+----------+ Performance: THP never, DAMOS_HUGEPAGE initial phase: 40,491 accesses/msec, 100001 msecs run low phase 0: 39,658 accesses/msec, 50002 msecs run high phase 0: 41,678 accesses/msec, 50000 msecs run low phase 1: 39,625 accesses/msec, 50003 msecs run high phase 1: 41,658 accesses/msec, 50002 msecs run low phase 2: 39,642 accesses/msec, 50002 msecs run high phase 2: 41,640 accesses/msec, 50001 msecs run THP madvise, DAMOS_HUGEPAGE initial phase: 51,977 accesses/msec, 100000 msecs run low phase 0: 86,953 accesses/msec, 50000 msecs run high phase 0: 94,812 accesses/msec, 50000 msecs run low phase 1: 101,017 accesses/msec, 50000 msecs run high phase 1: 94,841 accesses/msec, 50000 msecs run low phase 2: 100,993 accesses/msec, 50000 msecs run high phase 2: 94,791 accesses/msec, 50001 msecs run THP never, DAMOS_COLLAPSE initial phase: 93,678 accesses/msec, 100001 msecs run low phase 0: 101,475 accesses/msec, 50000 msecs run high phase 0: 98,589 accesses/msec, 50000 msecs run low phase 1: 101,531 accesses/msec, 50001 msecs run high phase 1: 98,506 accesses/msec, 50001 msecs run low phase 2: 101,458 accesses/msec, 50001 msecs run high phase 2: 98,555 accesses/msec, 50000 msecs run Memory consumption dynamic (how quickly collapses occur): It shows in seconds how many huge pages are allocated. +----+----------+----------+ | | T m, D h | T n, D c | +----+----------+----------+ | 5 | 32 | 188 | | 10 | 48 | 188 | | 15 | 64 | 188 | | 20 | 96 | 188 | | 30 | 112 | 188 | | 35 | 144 | 188 | | 40 | 160 | 188 | | 45 | 190 | 188 | | 50 | 190 | 188 | | 55 | 190 | 188 | | 60 | 190 | 188 | +----+----------+----------+ ========= - We can see that DAMOS "hugepage" action works only when THP is set to madvise. "collapse" action works even when THP is set to never. - Performance for "collapse" action is slightly lower than "hugepage" action and THP madvise. This is due to the fact that collapases occur synchronously. With "hugepage" they may occur during page faults. - Memory consumption is slighly lower for "collapse" than "hugepage" with THP madvise. This is due to the khugepage collapses all VMAs, while "collapse" action only collapses the VMAs in the hot region. - There is an improvement in TLB utilization when collapse through "hugepage" or "collapse" actions are triggered. The amount of TLB misses is lower. - "collapse" action is performance synchronously, which means that page collapses happen earlier and more rapidly. This can be useful or not, depending on the scenario. - "hugepage" action may trigger a VMA split in some scenarios, since it needs to change the flag of the VMA to THP enabled. This may lead to additional overhead. Collapse action just adds a new option to chose the correct system balance. Link: https://lore.kernel.org/20260426231619.107231-5-sj@kernel.org Link: https://lore.kernel.org/damon/20260313000816.79933-1-sj@kernel.org/ [1] Signed-off-by: Asier Gutierrez <gutierrez.asier@huawei-partners.com> Signed-off-by: SeongJae Park <sj@kernel.org> Reviewed-by: SeongJae Park <sj@kernel.org> Cc: Cheng-Han Wu <hank20010209@gmail.com> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Liam R. Howlett <liam@infradead.org> Cc: Liew Rui Yan <aethernet65535@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Shuah Khan <skhan@linuxfoundation.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Liew Rui Yan
|
de3c60e1c8 |
mm/damon: add synchronous commit for commit_inputs
Problem ======= Writing invalid parameters to sysfs followed by 'commit_inputs=Y' fails silently (no error returned to shell), because the validation happens asynchronously in the kdamond. Solution ======== To fix this, the commit_inputs_store() callback now uses damon_call() to synchronously commit parameters in the kdamond thread's safe context. This ensures that validation errors are returned immediately to userspace, following the pattern used by DAMON_SYSFS. Changes ======= 1. Added commit_inputs_store() and commit_inputs_fn() to commit synchronously. 2. Removed handle_commit_inputs(). This change is motivated from another discussion [1]. Link: https://lore.kernel.org/20260426231619.107231-4-sj@kernel.org Link: https://lore.kernel.org/20260318153731.97470-1-aethernet65535@gmail.com [1] Signed-off-by: Liew Rui Yan <aethernet65535@gmail.com> Reviewed-by: SeongJae Park <sj@kernel.org> Signed-off-by: SeongJae Park <sj@kernel.org> Cc: Asier Gutierrez <gutierrez.asier@huawei-partners.com> Cc: Cheng-Han Wu <hank20010209@gmail.com> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Shuah Khan <skhan@linuxfoundation.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Cheng-Han Wu
|
abdca14655 |
Docs/admin-guide/mm/damon: fix 'parametrs' typo
Fix the misspelling of "parametrs" as "parameters" in reclaim.rst and lru_sort.rst. Link: https://lore.kernel.org/20260426231619.107231-3-sj@kernel.org Signed-off-by: Cheng-Han Wu <hank20010209@gmail.com> Signed-off-by: SeongJae Park <sj@kernel.org> Reviewed-by: SeongJae Park <sj@kernel.org> Cc: Asier Gutierrez <gutierrez.asier@huawei-partners.com> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Liam R. Howlett <liam@infradead.org> Cc: Liew Rui Yan <aethernet65535@gmail.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Shuah Khan <skhan@linuxfoundation.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Liew Rui Yan
|
1d258000b0 |
mm/damon/ops-common: optimize damon_hot_score() using ilog2()
Patch series "mm/damon: repost non-hotfix reviewed patches in damon/next
tree", v2.
The first patch from Liew Rui Yan add a minor performance optimization
using ilog2() instead of inefficient manual implementation of the
functionality.
The second patch from Cheng-Han Wu fixes a minor typo:
s/parametrs/parameters/.
The third patch from Liew Rui Yan make commit_inputs operation of
DAMON_RECLAIM and DAMON_LRU_SORT synchronous to improve the user
experience.
The fourth patch from Asier Gutierrez adds a new DAMOS action,
DAMOS_COLLAPSE for deterministic DAMOS-based access-aware THP system.
This patch (of 4):
The current implementation of damon_hot_score() uses a manual for-loop to
calculate the value of 'age_in_log'. This can be efficiently replaced by
ilog2(), which is semantically more appropriate for calculating the
logarithmic value of age.
In a simulated-kernel-module performance test with 10,000,000 iterations,
this optimization showed a significant reduction in latency (average
latency reduced from ~12ns to ~1ns).
Test results from the simulated-kernel-module:
- ilog2:
DAMON Perf Test: Starting 10000000 iterations
=============================================
Total Iterations : 10000000
Average Latency : 1 ns
P95 Latency : 41 ns
P99 Latency : 41 ns
---------------------------------------------
Range (ns) | Count | Percent
---------------------------------------------
0-19 | 0 | 0%
20-39 | 2625000 | 26%
40-59 | 7374000 | 73%
60-79 | 0 | 0%
80-99 | 0 | 0%
100+ | 1000 | 0%
=============================================
- for-loop:
DAMON Perf Test: Starting 10000000 iterations
=============================================
Total Iterations : 10000000
Average Latency : 12 ns
P95 Latency : 51 ns
P99 Latency : 60 ns
---------------------------------------------
Range (ns) | Count | Percent
---------------------------------------------
0-19 | 0 | 0%
20-39 | 0 | 0%
40-59 | 9862000 | 98%
60-79 | 135000 | 1%
80-99 | 1000 | 0%
100+ | 2000 | 0%
=============================================
Full raw benchmark results can be found at [1].
Link: https://lore.kernel.org/20260426231619.107231-1-sj@kernel.org
Link: https://lore.kernel.org/20260426231619.107231-2-sj@kernel.org
Link: https://github.com/aethernet65535/damon-hot-score-fls-optimize/tree/master/result-raw [1]
Signed-off-by: SeongJae Park <sj@kernel.org>
Signed-off-by: Liew Rui Yan <aethernet65535@gmail.com>
Reviewed-by: SeongJae Park <sj@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Asier Gutierrez <gutierrez.asier@huawei-partners.com>
Cc: Cheng-Han Wu <hank20010209@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
||
|
Muchun Song
|
cd681403a8 |
mm/mm_init: fix uninitialized struct pages for ZONE_DEVICE
If DAX memory is hotplugged into an unoccupied subsection of an early
section, section_activate() reuses the unoptimized boot memmap. However,
compound_nr_pages() still assumes that vmemmap optimization is in effect
and initializes only the reduced number of struct pages. As a result, the
remaining tail struct pages are left uninitialized, which can later lead
to unexpected behavior or crashes.
Fix this by treating early sections as unoptimized when calculating how
many struct pages to initialize.
Link: https://lore.kernel.org/20260428081855.1249045-7-songmuchun@bytedance.com
Fixes:
|
||
|
Muchun Song
|
94405c6136 |
mm/mm_init: fix pageblock migratetype for ZONE_DEVICE compound pages
The memmap_init_zone_device() function only initializes the migratetype of the first pageblock of a compound page. If the compound page size exceeds pageblock_nr_pages (e.g., 1GB hugepages with 2MB pageblocks), subsequent pageblocks in the compound page remain uninitialized. Move the migratetype initialization out of __init_zone_device_page() and into a separate pageblock_migratetype_init_range() function. This iterates over the entire PFN range of the memory, ensuring that all pageblocks are correctly initialized. Also remove the stale confusing comment about MEMINIT_HOTPLUG above the migratetype setting since it is an obsolete relic from commit |
||
|
Muchun Song
|
721a73e30c |
mm/sparse-vmemmap: fix DAX vmemmap accounting with optimization
When vmemmap optimization is enabled for DAX, the nr_memmap_pages counter
in /proc/vmstat is incorrect. The current code always accounts for the
full, non-optimized vmemmap size, but vmemmap optimization reduces the
actual number of vmemmap pages by reusing tail pages. This causes the
system to overcount vmemmap usage, leading to inaccurate page statistics
in /proc/vmstat.
Fix this by introducing section_nr_vmemmap_pages(), which returns the
exact vmemmap page count for a given pfn range based on whether
optimization is in effect.
Link: https://lore.kernel.org/20260428081855.1249045-5-songmuchun@bytedance.com
Fixes:
|
||
|
Muchun Song
|
3bbc54dd1b |
mm/sparse-vmemmap: pass @pgmap argument to memory deactivation paths
Currently, the memory hot-remove call chain -- arch_remove_memory(), __remove_pages(), sparse_remove_section() and section_deactivate() -- does not carry the struct dev_pagemap pointer. This prevents the lower levels from knowing whether the section was originally populated with vmemmap optimizations (e.g., DAX with vmemmap optimization enabled). Without this information, we cannot call vmemmap_can_optimize() to determine if the vmemmap pages were optimized. As a result, the vmemmap page accounting during teardown will mistakenly assume a non-optimized allocation, leading to incorrect memmap statistics. To lay the groundwork for fixing the vmemmap page accounting, we need to pass the @pgmap pointer down to the deactivation location. Plumb the @pgmap argument through the APIs of arch_remove_memory(), __remove_pages() and sparse_remove_section(), mirroring the corresponding *_activate() paths. Link: https://lore.kernel.org/20260428081855.1249045-4-songmuchun@bytedance.com Signed-off-by: Muchun Song <songmuchun@bytedance.com> Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org> Reviewed-by: Oscar Salvador <osalvador@suse.de> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Liam R. Howlett <liam@infradead.org> Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com> Cc: Joao Martins <joao.m.martins@oracle.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Hocko <mhocko@suse.com> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Muchun Song
|
2fac4afa0e |
mm/memory_hotplug: fix incorrect altmap passing in error path
In create_altmaps_and_memory_blocks(), when arch_add_memory() succeeds
with memmap_on_memory enabled, the vmemmap pages are allocated from
params.altmap. If create_memory_block_devices() subsequently fails, the
error path calls arch_remove_memory() with a NULL altmap instead of
params.altmap.
This is a bug that could lead to memory corruption. Since altmap is NULL,
vmemmap_free() falls back to freeing the vmemmap pages into the system
buddy allocator via free_pages() instead of the altmap.
arch_remove_memory() then immediately destroys the physical linear mapping
for this memory. This injects unowned pages into the buddy allocator,
causing machine checks or memory corruption if the system later attempts
to allocate and use those freed pages.
Fix this by passing params.altmap to arch_remove_memory() in the error
path.
Link: https://lore.kernel.org/20260428081855.1249045-3-songmuchun@bytedance.com
Fixes:
|
||
|
Muchun Song
|
c373f7f98e |
mm/sparse-vmemmap: fix vmemmap accounting underflow
Patch series "mm: Fix vmemmap optimization accounting and initialization", v8. The series fixes several bugs in vmemmap optimization, mainly around incorrect page accounting and memmap initialization in DAX and memory hotplug paths. It also fixes pageblock migratetype initialization and struct page initialization for ZONE_DEVICE compound pages. Patches 1-4 fix vmemmap accounting issues. Patch 1 fixes an accounting underflow in the section activation failure path by moving vmemmap page accounting into the lower-level allocation and freeing helpers. Patch 2 fixes incorrect altmap passing in the memory hotplug error path. Patch 3 passes pgmap through memory deactivation paths so the teardown side can determine whether vmemmap optimization was in effect. Patch 4 uses that information to account the optimized DAX vmemmap size correctly. Patches 5-6 fix initialization issues in mm/mm_init. One makes sure all pageblocks in ZONE_DEVICE compound pages get their migratetype initialized. The other fixes a case where DAX memory hotplug reuses an unoptimized early-section memmap while compound_nr_pages() still assumes vmemmap optimization, leaving tail struct pages uninitialized. This patch (of 6): In section_activate(), if populate_section_memmap() fails, the error handling path calls section_deactivate() to roll back the state. This causes a vmemmap accounting imbalance. Since commit |
||
|
Dev Jain
|
7e8983f317 |
selftests/mm: simplify byte pattern checking in mremap_test
The original version of mremap_test (7df666253f26: "kselftests: vm: add
mremap tests") validated remapped contents byte-by-byte and printed a
mismatch index in case the bytes streams didn't match. That was rather
inefficient, especially also if the test passed.
Later, commit
|
||
|
Davidlohr Bueso
|
8aa442cfce |
dax/kmem: account for partial discontiguous resource upon removal
When dev_dax_kmem_probe() partially succeeds (at least one range is
mapped) but a subsequent range fails request_mem_region() or
add_memory_driver_managed(), the probe silently continues, ultimately
returning success, but with the corresponding range resource NULL'ed out.
dev_dax_kmem_remove() iterates over all dax_device ranges regardless of if
the underlying resource exists. When remove_memory() is called later, it
returns 0 because the memory was never added which causes
dev_dax_kmem_remove() to incorrectly assume the (nonexistent) resource can
be removed and attempts cleanup on a NULL pointer.
Fix this by skipping these ranges altogether, noting that these cases are
considered success, such that the cleanup is still reached when all
actually-added ranges are successfully removed.
Link: https://lore.kernel.org/20260223201516.1517657-1-dave@stgolabs.net
Fixes:
|
||
|
Brendan Jackman
|
3f99414620 |
net/rds: use special gfp_t format specifier
%pGg produces nice readable output and decouples the format string from the size of gfp_t. Link: https://lore.kernel.org/20260326-gfp64-v2-4-d916021cecdf@google.com Signed-off-by: Brendan Jackman <jackmanb@google.com> Acked-by: Allison Henderson <achender@kernel.org> Cc: Alexander Potapenko <glider@google.com> Cc: Allison Collins <allison.henderson@oracle.com> Cc: Dave Airlie <airlied@gmail.com> Cc: David S. Miller <davem@davemloft.net> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Jakub Kacinski <kuba@kernel.org> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Marco Elver <elver@google.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Paolo Abeni <pabeni@redhat.com> Cc: Simon Horman <horms@kernel.org> Cc: Stanislaw Gruszka <stf_xl@wp.pl> Cc: Thomas Zimemrmann <tzimmermann@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Brendan Jackman
|
d36102a5f5 |
mm/kfence: use special gfp_t format specifier
%pGg produces nice readable output and decouples the format string from the size of gfp_t. Link: https://lore.kernel.org/20260326-gfp64-v2-3-d916021cecdf@google.com Signed-off-by: Brendan Jackman <jackmanb@google.com> Cc: Alexander Potapenko <glider@google.com> Cc: Allison Collins <allison.henderson@oracle.com> Cc: Allison Henderson <achender@kernel.org> Cc: Dave Airlie <airlied@gmail.com> Cc: David S. Miller <davem@davemloft.net> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Jakub Kacinski <kuba@kernel.org> Cc: Jakub Kicinski <kuba@kernel.org> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Marco Elver <elver@google.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Paolo Abeni <pabeni@redhat.com> Cc: Simon Horman <horms@kernel.org> Cc: Stanislaw Gruszka <stf_xl@wp.pl> Cc: Thomas Zimemrmann <tzimmermann@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Brendan Jackman
|
faae0ca362 |
drm/managed: use special gfp_t format specifier
Patch series "treewide: fixup gfp_t printks", v2. Use vprintf()'s special gfp_t conversion in a few places. This patch (of 3): %pGg produces nice readable output and decouples the format string from the size of gfp_t. Link: https://lore.kernel.org/20260326-gfp64-v2-0-d916021cecdf@google.com Link: https://lore.kernel.org/20260326-gfp64-v2-1-d916021cecdf@google.com Signed-off-by: Brendan Jackman <jackmanb@google.com> Cc: Alexander Potapenko <glider@google.com> Cc: Allison Collins <allison.henderson@oracle.com> Cc: Allison Henderson <achender@kernel.org> Cc: Dave Airlie <airlied@gmail.com> Cc: David S. Miller <davem@davemloft.net> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Jakub Kacinski <kuba@kernel.org> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Marco Elver <elver@google.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Paolo Abeni <pabeni@redhat.com> Cc: Simon Horman <horms@kernel.org> Cc: Stanislaw Gruszka <stf_xl@wp.pl> Cc: Thomas Zimemrmann <tzimmermann@suse.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Deepanshu Kartikey
|
15807d0ddd |
mm/hugetlb: fix hugetlb cgroup rsvd charge/uncharge mismatch
In alloc_hugetlb_folio(), a single h_cg pointer is used for both the rsvd
and non-rsvd hugetlb cgroup charges. When map_chg is set,
hugetlb_cgroup_charge_cgroup_rsvd() stores the charged cgroup in h_cg, but
the immediately following hugetlb_cgroup_charge_cgroup() overwrites h_cg
with the non-rsvd cgroup pointer.
As a result, hugetlb_cgroup_commit_charge_rsvd() stores the wrong
(non-rsvd) cgroup pointer into the folio's rsvd slot.
When the folio is later freed, free_huge_folio() unconditionally calls
both hugetlb_cgroup_uncharge_folio() and
hugetlb_cgroup_uncharge_folio_rsvd(). The rsvd uncharge reads back the
wrong cgroup from the folio and decrements a counter that was never
charged for that cgroup, causing a page_counter underflow:
page_counter underflow: -512 nr_pages=512
WARNING: mm/page_counter.c:61 at page_counter_cancel
Fix this by introducing a separate h_cg_rsvd pointer exclusively for the
rsvd charge path, keeping the rsvd and non-rsvd charges fully independent
through their charge, commit, and error uncharge paths.
Link: https://lore.kernel.org/20260328065534.346053-1-kartikey406@gmail.com
Fixes:
|
||
|
JP Kobryn (Meta)
|
9669b87065 |
mm/lruvec: preemptively free dead folios during lru_add drain
Of all observable lruvec lock contention in our fleet, we find that ~24%
occurs when dead folios are present in lru_add batches at drain time.
This is wasteful in the sense that the folio is added to the LRU just to
be immediately removed via folios_put_refs(), incurring two unnecessary
lock acquisitions.
Eliminate this overhead by preemptively cleaning up dead folios before
they make it into the LRU. Use folio_ref_freeze() to filter folios whose
only remaining refcount is the batch ref. When dead folios are found,
move them off the add batch and onto a temporary batch to be freed.
PG_active may be set on a batched folio as well as PG_unevictable (via
migration path). Since filtered folios bypass the normal lru_add()
cleanup, both flags must be cleared before freeing.
During A/B testing on one of our prod instagram workloads (high-frequency
short-lived requests), the patch intercepted almost all dead folios before
they entered the LRU. Data collected using the mm_lru_insertion
tracepoint shows the effectiveness of the patch:
Per-host LRU add averages at 95% CPU load
(60 hosts each side, 3 x 60s intervals)
dead folios/min total folios/min dead %
unpatched: 1,297,785 19,341,986 6.7097%
patched: 14 19,039,996 0.0001%
Within this workload, we save ~2.6M lock acquisitions per minute per host
as a result.
System-wide memory stats improved on the patched side also at 95% CPU load:
- direct reclaim scanning reduced 7%
- allocation stalls reduced 5.2%
- compaction stalls reduced 12.3%
- page frees reduced 4.9%
No regressions were observed in requests served per second or request tail
latency (p99). Both metrics showed directional improvement at higher CPU
utilization (comparing 85% to 95%).
Note that tests were performed using classic LRU.
Link: https://lore.kernel.org/20260425053417.351146-1-jp.kobryn@linux.dev
Signed-off-by: JP Kobryn (Meta) <jp.kobryn@linux.dev>
Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Baoquan He <bhe@redhat.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Chris Li <chrisl@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
||
|
David Rientjes
|
94e0bcde05 |
mm, page_alloc: reintroduce page allocation stall warning
Previously, we had warnings when a single page allocation took longer than reasonably expected. This was introduced in commit |
||
|
Julian Braha
|
b9fe373e7d |
mm/thp: dead code cleanup in Kconfig
There is already an 'if TRANSPARENT_HUGEPAGE' condition wrapping several config options e.g. 'READ_ONLY_THP_FOR_FS', making the 'depends on' statement for each of these a duplicate dependency (dead code). I propose leaving the outer 'if TRANSPARENT_HUGEPAGE...endif' and removing the individual 'depends on TRANSPARENT_HUGEPAGE' statement from each option. This dead code was found by kconfirm, a static analysis tool for Kconfig. Link: https://lore.kernel.org/20260331070730.33915-1-julianbraha@gmail.com Signed-off-by: Julian Braha <julianbraha@gmail.com> Reviewed-by: Lorenzo Stoakes <ljs@kernel.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Liam Howlett <liam@infradead.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Brendan Jackman
|
1d05e1f6ac |
mm/page_alloc: cleanup flag vars in alloc_pages_bulk_noprof()
These two variables are redundant, squash them to align alloc_pages_bulk_noprof() with the style used in alloc_frozen_pages_nolock_noprof(). Link: https://lore.kernel.org/20260331-b4-prepare_alloc_pages-flags-v1-1-ea2416def698@google.com Signed-off-by: Brendan Jackman <jackmanb@google.com> Reviewed-by: Suren Baghdasaryan <surenb@google.com> Reviewed-by: Vishal Moola <vishal.moola@gmail.com> Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Zi Yan <ziy@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
wangxuewen
|
c70a9f639b |
mm/memory-failure: replace magic number 3 with GET_PAGE_MAX_RETRY_NUM
Replace the hardcoded magic number 3 in get_any_page() with the existing GET_PAGE_MAX_RETRY_NUM macro for code consistency and maintainability. This change has no functional impact, only improves code readability and unifies the retry limit configuration. Link: https://lore.kernel.org/20260402064946.1124250-1-18810879172@163.com Signed-off-by: wangxuewen <wangxuewen@kylinos.cn> Acked-by: SeongJae Park <sj@kernel.org> Acked-by: Miaohe Lin <linmiaohe@huawei.com> Cc: Naoya Horiguchi <nao.horiguchi@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
David Carlier
|
d590df11be |
mm/page_io: rename swap_iocb fields for clarity
swap_iocb->pages tracks the number of bvec entries (folios), not base pages. Rename the array from bvec to bvecs and the counter from pages to nr_bvecs to accurately reflect their purpose. Link: https://lore.kernel.org/20260402072650.48811-1-devnexen@gmail.com Signed-off-by: David Carlier <devnexen@gmail.com> Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org> Suggested-by: David Hildenbrand <david@redhat.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Chris Li <chrisl@kernel.org> Cc: Baoquan He <bhe@redhat.com> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: NeilBrown <neil@brown.name> Cc: Nhat Pham <nphamcs@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
JP Kobryn (Meta)
|
f2a950170f |
mm/vmpressure: skip socket pressure for costly order reclaim
When reclaim is triggered by high order allocations on a fragmented system, vmpressure() can report poor reclaim efficiency even though the system has plenty of free memory. This is because many pages are scanned, but few are found to actually reclaim - the pages are actively in use and don't need to be freed. The resulting scan:reclaim ratio causes vmpressure() to assert socket pressure, throttling TCP throughput unnecessarily. Costly order allocations (above PAGE_ALLOC_COSTLY_ORDER) rely heavily on compaction to succeed, so poor reclaim efficiency at these orders does not necessarily indicate memory pressure. The kernel already treats this order as the boundary where reclaim is no longer expected to succeed and compaction may take over. Make vmpressure() order-aware through an additional parameter sourced from scan_control at existing call sites. Socket pressure is now only asserted when order <= PAGE_ALLOC_COSTLY_ORDER. Memcg reclaim is unaffected since try_to_free_mem_cgroup_pages() always uses order 0, which passes the filter unconditionally. Similarly, vmpressure_prio() now passes order 0 internally when calling vmpressure(), ensuring critical pressure from low reclaim priority is not suppressed by the order filter. The patch was motivated by a case of impacted net throughput in production. On one affected host, the memory state at the time showed ~15GB available, zero cgroup pressure, and the following buddyinfo state: Order FreePages 0: 133,970 1: 29,230 2: 17,351 3: 18,984 7+: 0 Using bpf, it was found that 94% of vmpressure calls on this host were from order-7 kswapd reclaim. TCP minimum recv window is rcv_ssthresh:19712. Before patch: 723 out of 3,843 (19%) TCP connections stuck at minimum recv window After live-patching and ~30min elapsed: 0 out of 3,470 TCP connections stuck at minimum recv window Link: https://lore.kernel.org/20260406195014.112521-1-jp.kobryn@linux.dev Signed-off-by: JP Kobryn (Meta) <jp.kobryn@linux.dev> Reviewed-by: Rik van Riel <riel@surriel.com> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Shakeel Butt <shakeel.butt@linux.dev> Acked-by: Jakub Kicinski <kuba@kernel.org> Reviewed-by: Barry Song <baohua@kernel.org> Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: David Hildenbrand <david@kernel.org> Cc: Eric Dumazet <edumazet@google.com> Cc: Kairui Song <kasong@tencent.com> Cc: Liam Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Qi Zheng <qi.zheng@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Wei Xu <weixugc@google.com> Cc: Yuanchu Xie <yuanchu@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Breno Leitao
|
1d8274b82c |
mm: huge_memory: refactor defrag_show() to use defrag_flags[]
Replace the hardcoded if/else chain of test_bit() calls and string literals in defrag_show() with a loop over defrag_flags[] and defrag_mode_strings[] arrays introduced in the previous commit. This makes defrag_show() consistent with defrag_store() and eliminates the duplicated mode name strings. Link: https://lore.kernel.org/20260408-thp_defrag-v2-2-bc544c1bde4e@debian.org Signed-off-by: Breno Leitao <leitao@debian.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Tested-by: Lance Yang <lance.yang@linux.dev> Reviewed-by: Lance Yang <lance.yang@linux.dev> Reviewed-by: Barry Song <baohua@kernel.org> Reviewed-by: Lorenzo Stoakes <ljs@kernel.org> Tested-by: Zi Yan <ziy@nvidia.com> Acked-by: Zi Yan <ziy@nvidia.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Liam Howlett <liam@infradead.org> Cc: Nico Pache <npache@redhat.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Breno Leitao
|
b0f3d00e15 |
mm: huge_memory: use sysfs_match_string() in defrag_store()
Patch series "mm: huge_memory: clean up defrag sysfs with shared", v2.
Refactor defrag_store() and defrag_show() to use shared data tables
instead of duplicated if/else chains.
Patch 1 introduces an enum defrag_mode, a defrag_mode_strings[] table, and
a defrag_flags[] mapping array, then rewrites defrag_store() to use
sysfs_match_string() with a loop over defrag_flags[].
Patch 2 refactors defrag_show() to use the same arrays, replacing its
hardcoded if/else chain of test_bit() calls and string literals.
This follows the same pattern applied to anon_enabled_store() in commit
522dfb4ba71f ("mm: huge_memory: refactor anon_enabled_store() with
change_anon_orders()").
This patch (of 2):
Replace the if/else chain of sysfs_streq() calls in defrag_store() with
sysfs_match_string() and a defrag_mode_strings[] table.
Introduce enum defrag_mode and defrag_flags[] array mapping each mode to
its corresponding transparent_hugepage_flag. The store function now loops
over defrag_flags[], setting the bit for the selected mode and clearing
the others. When mode is DEFRAG_NEVER (index 4), no index in the
4-element defrag_flags[] matches, so all flags are cleared.
Note that the enum ordering (always, defer, defer+madvise, madvise, never)
differs from the original if/else chain order in defrag_store() (always,
defer+madvise, defer, madvise, never). This is intentional to match the
display order used by defrag_show().
This is a follow-up cleanup to commit 522dfb4ba71f ("mm: huge_memory:
refactor anon_enabled_store() with change_anon_orders()") which applied
the same sysfs_match_string() pattern to anon_enabled_store().
Link: https://lore.kernel.org/20260408-thp_defrag-v2-0-bc544c1bde4e@debian.org
Link: https://lore.kernel.org/20260408-thp_defrag-v2-1-bc544c1bde4e@debian.org
Signed-off-by: Breno Leitao <leitao@debian.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Tested-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Barry Song <baohua@kernel.org>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Tested-by: Zi Yan <ziy@nvidia.com>
Acked-by: Zi Yan <ziy@nvidia.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Liam Howlett <liam@infradead.org>
Cc: Nico Pache <npache@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
||
|
Ye Liu
|
98e09ce7bb |
mm/khugepaged: use ALIGN helpers for PMD alignment
PMD alignment in khugepaged is currently implemented using a mix of rounding helpers and open-coded bitmask operations. Use ALIGN() and ALIGN_DOWN() consistently for PMD-sized address range alignment, matching the preferred style for address and size handling. No functional change intended. Link: https://lore.kernel.org/20260409014323.2385982-1-ye.liu@linux.dev Signed-off-by: Ye Liu <liuye@kylinos.cn> Reviewed-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Barry Song <baohua@kernel.org> Reviewed-by: Lorenzo Stoakes <ljs@kernel.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: SeongJae Park <sj@kernel.org> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Lance Yang <lance.yang@linux.dev> Cc: Liam Howlett <liam@infradead.org> Cc: Liu Ye <liuye@kylinos.cn> Cc: Nico Pache <npache@redhat.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Ye Liu
|
db5e2c01ca |
mm/memory-failure: use bool for forcekill state
'forcekill' is used as a boolean flag to control whether processes should be forcibly killed. It is only assigned from boolean expressions and never used in arithmetic or bitmask operations. Convert it from int to bool. No functional change intended. Link: https://lore.kernel.org/20260410074740.2524718-1-ye.liu@linux.dev Signed-off-by: Ye Liu <liuye@kylinos.cn> Reviewed-by: SeongJae Park <sj@kernel.org> Acked-by: Miaohe Lin <linmiaohe@huawei.com> Cc: Liu Ye <liuye@kylinos.cn> Cc: Naoya Horiguchi <nao.horiguchi@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Muchun Song
|
ffe5539313 |
mm/sparse: remove sparse buffer pre-allocation mechanism
Commit
|
||
|
SeongJae Park
|
8c2c7df58b |
Docs/mm/damon/maintainer-profile: add AI review usage guideline
DAMON is opted-in for DAMON patches scanning [1] and email delivery [2]. Clarify how that could be used on DAMON maintainer profile. Link: https://lore.kernel.org/20260412211932.89038-1-sj@kernel.org Link: https://github.com/sashiko-dev/sashiko/commit/ad9f4a98f958 [1] Link: https://github.com/sashiko-dev/sashiko/commit/b554c7b6e733 [2] Signed-off-by: SeongJae Park <sj@kernel.org> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Liam Howlett <liam.howlett@oracle.com> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Randy Dunlap <rdunlap@infradead.org> Cc: Suren Baghdasaryan <surenb@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Zhen Ni
|
781b0e7474 |
mm/page_owner: fix %pGp format specifier argument type
The %pGp format specifier expects an argument of type 'unsigned long *',
but page->flags is now of type 'memdesc_flags_t' (a struct containing an
unsigned long member 'f') after the introduction of memdesc_flags_t.
Fix the type mismatch by passing &page->flags.f instead of &page->flags,
which matches the expected type.
Link: https://lore.kernel.org/20260414075813.3425968-1-zhen.ni@easystack.cn
Fixes:
|
||
|
Anthony Yznaga
|
303c6bdfe7 |
selftests/mm: run the MAP_DROPPABLE selftest
The test was not being run by the selftest framework so it was never noticed that it would fail with an assertion failure on configs without support for MAP_DROPPABLE. Update the test so that it is skipped instead when MAP_DROPPABLE is not supported, and add it to the mmap category so that the test is run by the framework. Link: https://lore.kernel.org/20260416033939.49981-4-anthony.yznaga@oracle.com Signed-off-by: Anthony Yznaga <anthony.yznaga@oracle.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Jason A. Donenfeld <jason@zx2c4.com> Cc: Liam Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Shuah Khan <shuah@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Mark Brown <broonie@kernel.org> Cc: Vlastimil Babka (SUSE) <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Anthony Yznaga
|
c02dd57c57 |
selftests/mm: verify droppable mappings cannot be locked
For configs that support MAP_DROPPABLE verify that a mapping created with MAP_DROPPABLE cannot be locked via mlock(), and that it will not be locked if it's created after mlockall(MCL_FUTURE). Link: https://lore.kernel.org/20260416033939.49981-3-anthony.yznaga@oracle.com Signed-off-by: Anthony Yznaga <anthony.yznaga@oracle.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Jason A. Donenfeld <jason@zx2c4.com> Cc: Liam Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Shuah Khan <shuah@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka (SUSE) <vbabka@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Anthony Yznaga
|
d86c9e971a |
mm: fix mmap errno value when MAP_DROPPABLE is not supported
Patch series "fix MAP_DROPPABLE not supported errno", v4.
Mark Brown reported seeing a regression in -next on 32 bit arm with the
mlock selftests. Before exiting and marking the tests failed, the
following message was logged after an attempt to create a MAP_DROPPABLE
mapping:
Bail out! mmap error: Unknown error 524
It turns out error 524 is ENOTSUPP which is an error that userspace is not
supposed to see, but it indicates in this instance that MAP_DROPPABLE is
not supported.
The first patch changes the errno returned to EOPNOTSUPP. The second
patch is a second version of a prior patch to introduce selftests to
verify locking behavior with droppable mappings with the additional change
to skip the tests when MAP_DROPPABLE is not supported. The third patch
fixes the MAP_DROPPABLE selftest so that it is run by the framework and
skips if MAP_DROPPABLE is not supported.
This patch (of 3):
On configs where MAP_DROPPABLE is not supported (currently any 32-bit
config except for PPC32), mmap fails with errno set to ENOTSUPP. However,
ENOTSUPP is not a standard error value that userspace knows about. The
acceptable userspace-visible errno to use is EOPNOTSUPP. checkpatch.pl
has a warning to this effect.
Link: https://lore.kernel.org/20260416033939.49981-1-anthony.yznaga@oracle.com
Link: https://lore.kernel.org/20260416033939.49981-2-anthony.yznaga@oracle.com
Fixes:
|
||
|
Xiang Gao
|
eb8fc9d285 |
mm/vmscan: fix typos in comments
Fix three typos in comments: - Line 112: "zome_reclaim_mode" -> "zone_reclaim_mode" - Line 6208: "prioities" -> "priorities" - Line 7067: "that that high" -> "that the high" (duplicated word) Link: https://lore.kernel.org/20260416062302.727468-1-gxxa03070307@gmail.com Signed-off-by: Xiang Gao <gaoxiang17@xiaomi.com> Reviewed-by: Barry Song <baohua@kernel.org> Reviewed-by: Donet Tom <donettom@linux.ibm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Sang-Heon Jeon
|
e0974347f5 |
mm/sparse: remove unnecessary NULL check before allocating mem_section
Commit
|
||
|
Sunny Patel
|
13fe573656 |
mm/migrate_device: cleanup up PMD Checks and warnings
Remove the odd VM_WARN_ON_FOLIO(!folio, folio) usage and replace it with a simpler VM_WARN_ON_ONCE(!folio) check. Drop the redundant VM_WARN_ON_ONCE(!pmd_none(*pmdp) && !is_huge_zero_pmd(*pmdp)). Refactor the PMD checks, making the control flow clearer and avoiding duplicate condition checks. Link: https://lore.kernel.org/20260419174747.10701-1-nueralspacetech@gmail.com Signed-off-by: Sunny Patel <nueralspacetech@gmail.com> Acked-by: Zi Yan <ziy@nvidia.com> Reviewed-by: Huang Ying <ying.huang@linux.alibaba.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Alistair Popple <apopple@nvidia.com> Cc: Byungchul Park <byungchul@sk.com> Cc: Gregory Price <gourry@gourry.net> Cc: "Huang, Ying" <ying.huang@linux.alibaba.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Rakie Kim <rakie.kim@sk.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Li Wang
|
e5ab892d05 |
selftests/cgroup: test_zswap: wait for asynchronous writeback
zswap writeback is asynchronous, but test_zswap.c checks writeback counters immediately after reclaim/trigger paths. On some platforms (e.g. ppc64le), this can race with background writeback and cause spurious failures even when behavior is correct. Add wait_for_writeback() to poll get_cg_wb_count() with a bounded timeout, and use it in: test_zswap_writeback_one() when writeback is expected test_no_invasive_cgroup_shrink() for the wb_group check This keeps the original before/after assertion style while making the tests robust against writeback completion latency. No test behavior change, selftest stability improvement only. Link: https://lore.kernel.org/20260424040059.12940-9-li.wang@linux.dev Signed-off-by: Li Wang <li.wang@linux.dev> Acked-by: Nhat Pham <nphamcs@gmail.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Michal Koutný <mkoutny@suse.com> Cc: Muchun Song <muchun.song@linux.dev> Cc: Tejun Heo <tj@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Yosry Ahmed <yosryahmed@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Jiayuan Chen <jiayuan.chen@linux.dev> Cc: Waiman Long <longman@redhat.com> Cc: Yosry Ahmed <yosry@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Li Wang
|
883015a9c3 |
selftest/cgroup: fix zswap attempt_writeback() on 64K pagesize system
In attempt_writeback(), a memsize of 4M only covers 64 pages on 64K page size systems. When memory.reclaim is called, the kernel prefers reclaiming clean file pages (binary, libc, linker, etc.) over swapping anonymous pages. With only 64 pages of anonymous memory, the reclaim target can be largely or entirely satisfied by dropping file pages, resulting in very few or zero anonymous pages being pushed into zswap. This causes zswap_usage to be extremely small or zero, making zswap_usage/4 insufficient to create meaningful writeback pressure. The test then fails because no writeback is triggered. On 4K page size systems this is not an issue because 4M covers 1024 pages, and file pages are a small fraction of the reclaim target. Fix this by: - Always allocating 1024 pages regardless of page size. This ensures enough anonymous pages to reliably populate zswap and trigger writeback, while keeping the original 4M allocation on 4K systems. - Setting zswap.max to zswap_usage/4 instead of zswap_usage/2 to create stronger writeback pressure, ensuring reclaim reliably triggers writeback even on large page size systems. === Error Log === # uname -rm 6.12.0-211.el10.ppc64le ppc64le # getconf PAGESIZE 65536 # ./test_zswap TAP version 13 1..7 ok 1 test_zswap_usage ok 2 test_swapin_nozswap ok 3 test_zswapin not ok 4 test_zswap_writeback_enabled ... Link: https://lore.kernel.org/20260424040059.12940-8-li.wang@linux.dev Signed-off-by: Li Wang <li.wang@linux.dev> Acked-by: Yosry Ahmed <yosry@kernel.org> Acked-by: Nhat Pham <nphamcs@gmail.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Michal Koutný <mkoutny@suse.com> Cc: Muchun Song <muchun.song@linux.dev> Cc: Tejun Heo <tj@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Jiayuan Chen <jiayuan.chen@linux.dev> Cc: Waiman Long <longman@redhat.com> Cc: Yosry Ahmed <yosryahmed@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Li Wang
|
a19b474927 |
selftest/cgroup: fix zswap test_no_invasive_cgroup_shrink on large pagesize system
test_no_invasive_cgroup_shrink sets up two cgroups: wb_group, which is
expected to trigger zswap writeback, and a control group (renamed to
zw_group), which should only have pages sitting in zswap without any
writeback.
There are two problems with the current test:
1) The data patterns are reversed. wb_group uses allocate_bytes(), which
writes only a single byte per page — trivially compressible,
especially by zstd — so compressed pages fit within zswap.max and
writeback is never triggered. Meanwhile, the control group uses
getrandom() to produce hard-to-compress data, but it is the group
that does *not* need writeback.
2) The test uses fixed sizes (10K zswap.max, 10MB allocation) that are
too small on systems with large PAGE_SIZE (e.g. 64K), failing to
build enough memory pressure to trigger writeback reliably.
Fix both issues by:
- Swapping the data patterns: fill wb_group pages with partially
random data (getrandom for page_size/4 bytes) to resist compression
and trigger writeback, and fill zw_group pages with simple repeated
data to stay compressed in zswap.
- Making all size parameters PAGE_SIZE-aware: set allocation size to
PAGE_SIZE * 1024, memory.zswap.max to PAGE_SIZE, and memory.max to
allocation_size / 2 for both cgroups.
- Allocating memory inline instead of via cg_run() so the pages
remain resident throughout the test.
=== Error Log ===
# getconf PAGESIZE
65536
# ./test_zswap
TAP version 13
...
ok 5 test_zswap_writeback_disabled
ok 6 # SKIP test_no_kmem_bypass
not ok 7 test_no_invasive_cgroup_shrink
Link: https://lore.kernel.org/20260424040059.12940-7-li.wang@linux.dev
Signed-off-by: Li Wang <li.wang@linux.dev>
Acked-by: Nhat Pham <nphamcs@gmail.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Michal Koutný <mkoutny@suse.com>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Tejun Heo <tj@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Yosry Ahmed <yosryahmed@google.com>
Cc: Chengming Zhou <chengming.zhou@linux.dev>
Cc: Jiayuan Chen <jiayuan.chen@linux.dev>
Cc: Waiman Long <longman@redhat.com>
Cc: Yosry Ahmed <yosry@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
||
|
Li Wang
|
43743cc516 |
selftests/cgroup: replace hardcoded page size values in test_zswap
test_zswap uses hardcoded values of 4095 and 4096 throughout as page stride and page size, which are only correct on systems with a 4K page size. On architectures with larger pages (e.g., 64K on arm64 or ppc64), these constants cause memory to be touched at sub-page granularity, leading to inefficient access patterns and incorrect page count calculations, which can cause test failures. Replace all hardcoded 4095 and 4096 values with a global pagesize variable initialized from sysconf(_SC_PAGESIZE) at startup, and remove the redundant local sysconf() calls scattered across individual functions. No functional change on 4K page size systems. Link: https://lore.kernel.org/20260424040059.12940-6-li.wang@linux.dev Signed-off-by: Li Wang <li.wang@linux.dev> Acked-by: Yosry Ahmed <yosry@kernel.org> Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Michal Koutný <mkoutny@suse.com> Cc: Muchun Song <muchun.song@linux.dev> Cc: Tejun Heo <tj@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Waiman Long <longman@redhat.com> Cc: Yosry Ahmed <yosryahmed@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Li Wang
|
6e9f5c2eec |
selftests/cgroup: rename PAGE_SIZE to BUF_SIZE in cgroup_util
The cgroup utility code defines a local PAGE_SIZE macro hardcoded to 4096, which is used primarily as a generic buffer size for reading cgroup and proc files. This naming is misleading because the value has nothing to do with the actual page size of the system. On architectures with larger pages (e.g., 64K on arm64 or ppc64), the name suggests a relationship that does not exist. Additionally, the name can shadow or conflict with PAGE_SIZE definitions from system headers, leading to confusion or subtle bugs. To resolve this, rename the macro to BUF_SIZE to accurately reflect its purpose as a general I/O buffer size. Furthermore, test_memcontrol currently relies on this hardcoded 4K value to stride through memory and trigger page faults. Update this logic to use the actual system page size dynamically. This micro-optimizes the memory faulting process by ensuring it iterates correctly and efficiently based on the underlying architecture's true page size. (This part from Waiman) Link: https://lore.kernel.org/20260424040059.12940-5-li.wang@linux.dev Signed-off-by: Li Wang <li.wang@linux.dev> Signed-off-by: Waiman Long <longman@redhat.com> Acked-by: Nhat Pham <nphamcs@gmail.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Michal Koutný <mkoutny@suse.com> Cc: Muchun Song <muchun.song@linux.dev> Cc: Tejun Heo <tj@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Yosry Ahmed <yosryahmed@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Jiayuan Chen <jiayuan.chen@linux.dev> Cc: Yosry Ahmed <yosry@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Li Wang
|
b19ee588e1 |
selftests/cgroup: use runtime page size for zswpin check
test_zswapin compares memory.stat:zswpin (counted in pages) against a byte threshold converted with PAGE_SIZE. In cgroup selftests, PAGE_SIZE is hardcoded to 4096, which makes the conversion wrong on systems with non-4K base pages (e.g. 64K). As a result, the test requires too many pages to pass and fails spuriously even when zswap is working. Use sysconf(_SC_PAGESIZE) for the zswpin threshold conversion so the check matches the actual system page size. Link: https://lore.kernel.org/20260424040059.12940-4-li.wang@linux.dev Signed-off-by: Li Wang <li.wang@linux.dev> Reviewed-by: Yosry Ahmed <yosry@kernel.org> Acked-by: Nhat Pham <nphamcs@gmail.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Michal Hocko <mhocko@kernel.org> Cc: Michal Koutný <mkoutny@suse.com> Cc: Muchun Song <muchun.song@linux.dev> Cc: Tejun Heo <tj@kernel.org> Cc: Roman Gushchin <roman.gushchin@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Jiayuan Chen <jiayuan.chen@linux.dev> Cc: Waiman Long <longman@redhat.com> Cc: Yosry Ahmed <yosryahmed@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |