Commit Graph

3264 Commits

Author SHA1 Message Date
Linus Torvalds
dd384658c7 perf tools changes for v7.4
- Support instruction tracking on arm64 for data type profiling, resolving
   stack, global, per-CPU and 'current' variable accesses, so that 'perf
   annotate --data-type' resolves types on arm64 the way it does on x86.
 
 - Port the Python scripts to the perf module and standalone execution, and
   improve the module: lazy sample callchains and event copies, GIL
   handling, better session/thread lookup, new treport, flamegraph and
   sqlite/postgres exporters.
 
 - Reenable and fix the attribute expectation test suite: per-PMU bit
   masks, other core PMUs' events, dummy tracking events, group checks
   and spelled out skip reasons.
 
 - perf trace: filter the target's tasks in BPF, open the BPF output event
   on every CPU, add an option to disable syscall augmentation and raise
   TRACE_AUG_MAX_BUF.
 
 - perf jitdump: harden record parsing, validating unwinding sizes and
   symbol names, byte-swapping debug entries via safe accessors and fixing
   frees on error paths.
 
 - perf annotate-data fixes: bound the member nesting recursion, resolve
   type DIEs in the debug file they came from, adjust the offset for
   flex-arrays and show the sample count in the data-type browser.
 
 - perf symbol and debug info: try linked debug files for DSOs, avoid
   repeated failing searches, set binary/symtab types for split kallsyms,
   JIT maps and the build-id cache, fix libbfd symbol loading.
 
 - perf config: serialize config file access with a mutex, add the
   DEFINE_MUTEX() and DO_ONCE() primitives and a robust process shared
   mutex, add the core.hybrid-merge option.
 
 - perf report: add --progress and --no-progress options.
 
 - perf mem: add a false sharing workload to 'perf test', request
   PERF_SAMPLE_CPU by default, use the IBS swfilt filter when available
   and decode L0 in the memory level.
 
 - perf test: take turns using the perf mlock budget in the mmaping tests,
   add the 'needs_mlock' suite flag, scope the probe based tests to their
   PID and make more tests non-exclusive.
 
 - perf vendor events intel: update events and metrics to TMA 5.2, refresh
   the event tables and fix umask encodings.
 
 - perf build: run shellcheck, mypy and pylint in parallel, make pylint
   opt-out with NO_PYLINT=1, fix clean and install dependencies.
 
 - Add a script and a 'make sync-headers' target to sync the tools/ copies
   of kernel headers with the kernel sources, plus the resulting syncs of
   the cpufeatures, KVM, fs, mount, fcntl, socket, rtnetlink, if_addr and
   sound headers.
 
 - Fix cs-etm HW_ID mapping synthesis for raw trace, queue teardown leaks
   and duplicate v0 global mappings.
 
 - Add a 'perf bench atomic' benchmark for atomic operations.
 
 - Filter events before creating lock_stat in 'perf lock contention'.
 
 - Add a 'perf test' case for events re-armed with PERF_EVENT_IOC_REFRESH.
 
 - Fix memory leaks on 'perf c2c' output fields and add function view
   stdio coverage to its test.
 
 - Fix a NULL dereference resolving callchains in 'perf machine'.
 
 - Optimize the libdw fallback when DWARF is missing and try the symbol
   file first when checking for debuginfo.
 
 - Don't call eprintf() in pr_debugN() when not verbose.
 
 - Don't delete cumulative caller entries before the stat_acc decays in
   'perf report'.
 
 - Fix ring buffer cursor handling: keep the consumed cursor in forward
   mode and use the head selected by read_init.
 
 - Fix 'perf test mmap-basic' user rdpmc detection and only test userspace
   reading on supported platforms.
 
 - Build fixes: include sys/types.h for ssize_t, errno.h before kernel
   headers and fix jevents.py type annotations for mypy 2.x.
 
 Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQR2GiIUctdOfX2qHhGyPKLppCJ+JwUCauUL4QAKCRCyPKLppCJ+
 JzsyAQCNjqGOBYEUaDbQAbK1/1h9D9QER9s+4aU3Np2zT9XSPAD/RkY12uxT3YAO
 NcQlaj0On+I46SkgyGzGrV9mOvDZJwo=
 =b9um
 -----END PGP SIGNATURE-----

Merge tag 'perf-tools-for-v7.4-1-2026-10-30' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next

Pull perf tool updates from Arnaldo Carvalho de Melo:

 - Support instruction tracking on arm64 for data type profiling,
   resolving stack, global, per-CPU and 'current' variable accesses, so
   that 'perf annotate --data-type' resolves types on arm64 the way it
   does on x86.

 - Port the Python scripts to the perf module and standalone execution,
   and improve the module: lazy sample callchains and event copies, GIL
   handling, better session/thread lookup, new treport, flamegraph and
   sqlite/postgres exporters.

 - Reenable and fix the attribute expectation test suite: per-PMU bit
   masks, other core PMUs' events, dummy tracking events, group checks
   and spelled out skip reasons.

 - perf trace: filter the target's tasks in BPF, open the BPF output
   event on every CPU, add an option to disable syscall augmentation and
   raise TRACE_AUG_MAX_BUF.

 - perf jitdump: harden record parsing, validating unwinding sizes and
   symbol names, byte-swapping debug entries via safe accessors and
   fixing frees on error paths.

 - perf annotate-data fixes: bound the member nesting recursion, resolve
   type DIEs in the debug file they came from, adjust the offset for
   flex-arrays and show the sample count in the data-type browser.

 - perf symbol and debug info: try linked debug files for DSOs, avoid
   repeated failing searches, set binary/symtab types for split
   kallsyms, JIT maps and the build-id cache, fix libbfd symbol loading.

 - perf config: serialize config file access with a mutex, add the
   DEFINE_MUTEX() and DO_ONCE() primitives and a robust process shared
   mutex, add the core.hybrid-merge option.

 - perf report: add --progress and --no-progress options.

 - perf mem: add a false sharing workload to 'perf test', request
   PERF_SAMPLE_CPU by default, use the IBS swfilt filter when available
   and decode L0 in the memory level.

 - perf test: take turns using the perf mlock budget in the mmaping
   tests, add the 'needs_mlock' suite flag, scope the probe based tests
   to their PID and make more tests non-exclusive.

 - perf vendor events intel: update events and metrics to TMA 5.2,
   refresh the event tables and fix umask encodings.

 - perf build: run shellcheck, mypy and pylint in parallel, make pylint
   opt-out with NO_PYLINT=1, fix clean and install dependencies.

 - Add a script and a 'make sync-headers' target to sync the tools/
   copies of kernel headers with the kernel sources, plus the resulting
   syncs of the cpufeatures, KVM, fs, mount, fcntl, socket, rtnetlink,
   if_addr and sound headers.

 - Fix cs-etm HW_ID mapping synthesis for raw trace, queue teardown
   leaks and duplicate v0 global mappings.

 - Add a 'perf bench atomic' benchmark for atomic operations.

 - Filter events before creating lock_stat in 'perf lock contention'.

 - Add a 'perf test' case for events re-armed with
   PERF_EVENT_IOC_REFRESH.

 - Fix memory leaks on 'perf c2c' output fields and add function view
   stdio coverage to its test.

 - Fix a NULL dereference resolving callchains in 'perf machine'.

 - Optimize the libdw fallback when DWARF is missing and try the symbol
   file first when checking for debuginfo.

 - Don't call eprintf() in pr_debugN() when not verbose.

 - Don't delete cumulative caller entries before the stat_acc decays in
   'perf report'.

 - Fix ring buffer cursor handling: keep the consumed cursor in forward
   mode and use the head selected by read_init.

 - Fix 'perf test mmap-basic' user rdpmc detection and only test
   userspace reading on supported platforms.

 - Build fixes: include sys/types.h for ssize_t, errno.h before kernel
   headers and fix jevents.py type annotations for mypy 2.x.

* tag 'perf-tools-for-v7.4-1-2026-10-30' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next: (425 commits)
  perf machine: Fix NULL deref in thread__resolve_callchain_sample()
  perf pmu-events: Fix jevents.py type annotations for mypy 2.x
  perf parse-events: Include errno.h before kernel headers
  perf print_insn: Include sys/types.h for ssize_t
  perf tests mmap-basic: Fix user rdpmc detection logic
  perf tests mmap-basic: Only test userspace reading on supported platforms
  perf mmap: Preserve the consumed cursor in forward mode
  perf mmap: Use the head selected by read_init
  perf hist: Do not delete cumulative caller entries before stat_acc decays
  perf debug: Avoid calling eprintf() in pr_debugN() when verbose < n
  perf libdw: Optimize fallback logic for missing DWARF
  tools headers: Sync KVM headers with the kernel sources
  tools headers: Sync UAPI sound/asound.h with the kernel sources
  tools headers: Sync UAPI linux/mount.h with the kernel sources
  tools headers: Sync UAPI linux/fs.h with the kernel sources
  tools headers: Sync UAPI linux/fcntl.h with the kernel sources
  tools headers: Sync linux/socket.h with the kernel sources
  tools arch x86: Sync the cpufeatures.h copy with the kernel sources
  tools headers UAPI: Sync linux/rtnetlink.h with the kernel sources
  tools headers UAPI: Sync linux/if_addr.h with the kernel sources
  ...
2026-10-30 21:54:38 -07:00
Linus Torvalds
46806af187 bpf-fixes
-----BEGIN PGP SIGNATURE-----
 
 iQJRBAABCgA7FiEE+soXsSLHKoYyzcli6rmadz2vbToFAmrjbZEdHGFsZXhlaS5z
 dGFyb3ZvaXRvdkBnbWFpbC5jb20ACgkQ6rmadz2vbTqGEQ//bqWuSeYGypshirYQ
 4rxejupvJBJmrD+pI3MSDKVDaLgslm2vW8+vYB+rxpwzje4EsBtVqoHVxzk4yMoQ
 WyuJ7988W3RRZw5x5EEruS65C2pM5H2Ilzj2f8Z5gYUfvB72/TGJD2sDXxlTAtUb
 ESujBo/M9JRyYZBE5SMJT00r17clwf8NHoT3nr1FKAaDLQopE2zAnE/RwB1lEm3V
 MtUcnyDWEwu9ITOxMAwymzYI4FHadqxLJ5MCyCFUeqWAkJcPoVGHWN9Z/0cLTBYt
 46PZIotJUEio1PsF6/3y0ZAS2whCAzBuuQCcRG1qBoOdczaDN8fl7RAuTvFmSpPJ
 H5Z1GUAXvoFAPpfb9upwYwrEc5hH7+4w9LsGjEumKZ2kHzPccH8K3jtUJRRvYiL8
 W7LH4iiu/yPvYHKsgt89csGy6uoACUTn3Xbgfho4DzMoOO8Q8Ahw3SKMpPLIXPk2
 1q8Olv9d2pdaVRVJ4qXGPV4510LXW5KwDBHiztZKij8HvasPJS4goaTRKOSIqYOx
 GCWCuyM4F22N1suHyUZea507PPVFBKcNBRXWt3nNbKWZNFXpayyS2UI/3kusFFuD
 +EFcu0sNDENJ04/SGgujswvGGYrahT8AsHSkwM8jMpiajBiI0ozjWyLwVYk17mAY
 WrPIg+X/Uh5nxk5o5RIbniOJ8l8=
 =kHds
 -----END PGP SIGNATURE-----

Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf

Pull bpf fixes from Alexei Starovoitov:

 - Remove BPF_F_ARENA_SCALAR flag and .arena.data section in libbpf that
   were added for rust-bpf in this merge window. We found a cleaner
   approach that does it on compiler side instead of kernel/libbpf. No
   released kernel has them (Alexei Starovoitov)

 - Fix bpf_send_signal_task() returning -EPERM when it is called from a
   kernel thread (Aditya Sharma)

 - Various fixes (Alexei Starovoitov):
     - Use emit_rsb_call() to call bpf_unwind_target() in x86 JIT to fix
       objtool warning during the build
     - Fix scalar evolution: clear delta of a widened register, don't
       widen a pointer to a buffer, don't clamp a register whose type
       changed in the loop, and reset the range of a clamped packet
       pointer
     - Finish the insn in the state that maybe_fork_scalars() splits
       off, so that pruning doesn't drop it at the head of a counted
       loop
     - Preserve the registers that carry stack arguments across
       arch_bpf_timed_may_goto() on x86-64, arm64 and riscv64
     - Don't re-sort subprog_info[] for every new subprogram
     - Fix flaky map_kptr and rcu_reentry selftests

 - Use memmove() in bpf_skb_load_bytes(), bpf_skb_load_bytes_relative()
   and bpf_xdp_load_bytes(), since the destination can be a slice of the
   same packet (Ayushman Rout)

 - Fixes and follow-ups for scalar evolution: reject pointer operands in
   32-bit recurrences and latches, don't intersect with the current
   register range on clamp to avoid a false 'verifier bug', avoid false
   infinite loop reports, and recognize a 32-bit copy of the induction
   variable (Eduard Zingerman)

 - Fix out-of-bounds write to btf->types[] for split BTF and propagate
   btf_add_type() errors in btf_check_all_metas() (Feng Yang)

 - Fix use-after-free in bpf_timer callback that fires after its inner
   map was freed (Mani Tofigh)

 - Fix x86 JIT stopping before jumps converge: size backward jumps by
   the insns they jump over (Mykyta Yatsenko)

 - Fix use-after-free of struct_ops trampoline: dissociate struct_ops
   progs before the grace period of the map (Qiliang Yuan)

 - Validate stack argument reads against the caller state to fix
   out-of-bounds read in the verifier, and reject callbacks with stack
   arguments (Weiming Shi)

 - Tie borrowed iterator results and pointers loaded from trusted fields
   to the lifetime of their source, so that iterator next, destroy and
   reference release invalidate them (Xu Yunxiang)

* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (48 commits)
  bpf: Don't re-sort subprog_info[] for every new subprogram
  bpf: Fix out-of-bounds btf->types write for split BTF
  selftests/bpf: Test stack arguments across may_goto
  bpf: Keep stack arguments in arch_bpf_timed_may_goto()
  selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop
  bpf: Finish the insn in the state that maybe_fork_scalars() splits off
  selftests/bpf: Test the clamp of pointers in scalar evolution
  bpf: Reset the range of a packet pointer that scalar evolution clamps
  bpf: Don't clamp a register whose type changed in the loop
  selftests/bpf: Test a loop that moves a pointer to a buffer
  bpf: Don't widen a pointer to a buffer in scalar evolution
  selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed
  selftests/bpf: Wait for the last maps of map_kptr to be freed
  selftests/bpf: Test that a widened register is not linked
  bpf: Clear delta of a register that scalar evolution widens
  bpf, x86: Remove r0_on_stack from fetching atomics in arena
  bpf: Remove BPF_F_ARENA_SCALAR
  selftests/bpf: Remove tests for arena access through numbers
  libbpf: Keep format strings of bpf_printk() in .rodata
  libbpf: Remove .arena.data
  ...
2026-10-29 08:10:23 -10:00
Linus Torvalds
cb01c6914e A handful of late-arriving changes, most of which were ready ahead of the
merge window:
 
 - More Portuguese translation updates
 
 - Some docs-build and self-test fixes
 
 - Bring documentation for the user-space access primitives in
   include/linux/uaccess.h into the docs build.  This revealed a number of
   malformed kerneldoc comments, which are fixed in a subsequent tweak to
   uaccess.h.
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCgAtFiEEIw+MvkEiF49krdp9F0NaE2wMflgFAmriEykPHGNvcmJldEBs
 d24ubmV0AAoJEBdDWhNsDH5YC9IH/ikTCYIK+3Nz8odNLbEh505Ug0KDaAK6Cux5
 Wb6eKpMaLfwxMojV80V8/lZMMmKzkeGWRO60TgF4V6MNdziOuj2UoFNIx/M+9jkw
 5kPsc/vwGdtt2bR7FN1GKbchH9sh22Baw0aCTk+ymMMTFqDHDJ7qXrCfvm8MNAXg
 8+xPr9acR4hYy9Bd8HOKx+noym8qmRvLqeznM5hvOBlQtYk5ZJWwIfuiiH61pzyv
 89BrR9fMhObO/5ObSv9OpxFR+28SeB2986zXw1HTyskCrFes9s0Dd6EQfVvL6NxV
 ldCt1odfoyAa5KJurvJ3piILJANrcID5PBDlqN/T8jG9oO6gxRc=
 =xUko
 -----END PGP SIGNATURE-----

Merge tag 'docs-7.4-2' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux

Pull more documentation updates from Jonathan Corbet:
 "A handful of late-arriving changes, most of which were ready ahead of
  the merge window:

   - More Portuguese translation updates

   - Some docs-build and self-test fixes

   - Bring documentation for the user-space access primitives
     in include/linux/uaccess.h into the docs build.

     This revealed a number of malformed kerneldoc comments,
     which are fixed in a subsequent tweak to uaccess.h"

* tag 'docs-7.4-2' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux:
  uaccess: fix some erroneous kerneldoc comments
  docs/translations/pt_BR: admin-guide: add efi-stub.rst
  docs/translations/pt_BR: admin-guide: add kernel-parameters.rst
  docs/translations/pt_BR: admin-guide: add bootconfig.rst
  docs/translations/pt_BR: admin-guide: add perf-security.rst
  docs/translations/pt_BR: admin-guide: add LSM/index.rst
  docs/translations/pt_BR: admin-guide: add hw-vuln/index.rst
  docs/translations/pt_BR: admin-guide: add cputopology.rst
  docs/translations/pt_BR: admin-guide: add sysctl/index.rst
  docs/translations/pt_BR: admin-guide: add sysfs-rules.rst
  docs/translations/pt_BR: admin-guide: add features.rst
  docs: pt_BR: process: translate userspace debugging guide
  docs: kerneldoc.py: preserve headings in kernel-doc output
  docs: xforms_lists: Handle DECLARE_KHOSER_PTR() struct members
  docs: test_doc_build: don't select urllib3 yanked version
  docs: test_doc_build: address pylint warnings
  docs: test_doc_build: update it to support newer Sphinx versions
  docs: test_doc_build: better handle python version
  docs: admin-guide: Fix typo in kernel-parameters.txt
  docs: core-api: include documentation from linux/uaccess.h
2026-10-29 07:46:22 -10:00
Alexei Starovoitov
58cbd31c39
libbpf: Keep format strings of bpf_printk() in .rodata
The format strings of bpf_printk() moved from .rodata to .rodata.str for
one reason. .arena.data moved .rodata into arena, and helpers cannot read
strings from arena.

.arena.data is removed. Move the strings back to .rodata. As a result, an
object that has its own .rodata has one map less.

This reverts commit 373e1223ff
("libbpf: Keep format strings of bpf_printk() in .rodata.str"). libbpf
1.8.0 has the strings in .rodata.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-4-alexei.starovoitov@gmail.com
2026-10-28 04:07:02 +00:00
Alexei Starovoitov
5b0678b2f9
libbpf: Remove .arena.data
Rust has no address spaces. Thus a global variable cannot have the __arena
attribute. The .arena.data section was a workaround. It told libbpf to do
two things:
 - Move .data, .bss and .rodata into arena.
 - Load the progs with BPF_F_ARENA_SCALAR.

The compiler must select the location of a variable, not libbpf. libbpf
already puts the .addr_space.1 section in arena. The __arena variables of C
are in this section, and LLVM can use it for Rust too. Until LLVM does
that, the global data of Rust progs stays in array maps.

This reverts commit 8ab58ed42b ("libbpf: Keep global data in arena when
the object has .arena.data"). libbpf 1.8.0 does not have the section.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-3-alexei.starovoitov@gmail.com
2026-10-28 04:07:02 +00:00
Paul Moses
1e826983ce perf mmap: Preserve the consumed cursor in forward mode
perf_mmap__read_event() updates prev as it consumes forward-buffer
records. perf_mmap__read_done() then overwrites prev with a fresh
producer head, even when unread records remain.

perf_kvm__mmap_read_idx() stops after 25 records per mmap and calls
read_done(). If more records are pending, prev advances past them. The
next drain starts at that new position and skips the unread remainder.

An event arriving after the last empty read_event() and before
read_done() can also be skipped. read_done() does not itself update the
kernel's data_tail.

Only refresh prev for overwrite buffers, which need the current head as
the end of the next backward read. Leave the forward cursor at the
consumed position.

Fixes: 53172f9057 ("perf kvm: Switch to new perf_mmap__read_event() interface")
Signed-off-by: Paul Moses <p@1g4.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: James Clark <james.clark@linaro.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2026-10-27 17:01:14 -03:00
Linus Torvalds
55fbedcec6 bpf-next-7.4
-----BEGIN PGP SIGNATURE-----
 
 iQJRBAABCgA7FiEE+soXsSLHKoYyzcli6rmadz2vbToFAmrY/GYdHGFsZXhlaS5z
 dGFyb3ZvaXRvdkBnbWFpbC5jb20ACgkQ6rmadz2vbTrF3w//W1IXtb7BHM5DqyRY
 +IUpLlVdJETLJpdxHGq4SMGU7T+JpBJ7e58HIa9bNfyFaeOFuIvH4dy284RHhr8T
 3wPR61KmX4hZlmqz17lSKM4E4cnaYW/NMYfp4acA/VRNcvcM+IiGnuhzcFyIMs3w
 mPlw/r3rW7Cn7E2Ri2hRV/ZQoT7FyI7LtUEtHU8TUE9kh+T2LTut+FWMJrHe5w0J
 LDFwhVqZE0dWZ4695PK6cEIh8uxRJBnmIyMtN+zLBOtqAjiOSNNhFssdSCIkjPEe
 X9Xz1Zvqa8Z5QsyRsrwK4S62pnh0pVshQ8fDjrXKY7fkMPpzTvOfqOGS3qhOlw2y
 sz853Repug0ZerePFnPnfTIhRIcXagTT11BJUDhUzgy9/Cra3s0vY6c9/MrxTamh
 3Xpu/UNl8AIqC/OnbAo8kJ9aKAG6olIBHYIsVOUYk3tH8m2HaVqpKXKRKfqshSDT
 YyKoGQE8g9gemS44/RSHKi6QI2jUxNwcOI8FBzViDZaeE476Zvv/fHNC5Tv7lwmm
 5qGbcem9sakcd+n17aLy0xKYE2kUSQQLi/J80AY7gMR9VlGsbYPZ8KX8HeSaSLRy
 7SMn2dBvNmFaFNgWdLQn4dmBFhvuVgZP6A/ZTreCLuXjBcPo1TnYkVwi3eye4eY/
 CwgWBoUJTIqMMt+bRyNgBK0+5rQ=
 =kneb
 -----END PGP SIGNATURE-----

Merge tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next

Pull bpf updates from Alexei Starovoitov:
 "Major changes:

   - Introduce scalar evolution (SCEV) analysis in the verifier.

     It infers how registers and stack slots change in a loop and lets
     the verifier check bounded loops with induction variables as ranges
     instead of walking every iteration. The number of processed insns
     drops ten fold in programs with loops (Eduard Zingerman)

   - Add 'callx' instruction: indirect call of a static BPF subprogram.

     The address is taken with ld_imm64 or loaded from read-only data.
     Supported by x86-64, arm64 and riscv64 JITs (Alexei Starovoitov, Pu
     Lehui)

   - Add unwind_goto and unwind_exit instructions, so that Rust panic or
     C++ exception can run destructors of every frame it passes through.
     The normal path is free of unwind code. Supported by x86-64 and
     arm64 JITs (Alexei Starovoitov)

   - Support programs compiled by rust-bpf: treat load and store through
     a number as arena access with BPF_F_ARENA_SCALAR, accept rust names
     in BTF (Alexei Starovoitov)

   - Raise BPF program stack size from 512 bytes to 2 KiB on x86-64 and
     arm64 (Kumar Kartikeya Dwivedi)

   - Support struct, union and __int128 up to 16 bytes passed and
     returned by value in BPF subprograms and kfuncs, and fix x86-64 and
     arm64 trampolines for such arguments (Yonghong Song)

   - Introduce a common way to attach struct_ops to a cgroup via
     bpf_link. bpf_tcp_ops is the first user and mirrors TCP sock_ops
     callbacks (Martin KaFai Lau, Amery Hung)

   - Describe inlined functions in BTF. New kinds tell where the inline
     sites are and how to fetch the arguments there.

     CONFIG_DEBUG_INFO_BTF_INLINE keeps it apart from the main BTF in
     .BTF.inline section or in btf_vmlinux_inline.ko (Alan Maguire)

   - Fix many memory safety, lifetime and locking bugs in BPF core,
     verifier, iterators, local storage, dynptr, devmap, cpumap and
     seg6local (Josef Bacik, Alexei Starovoitov)

   - Diff stat is dominated by selftests and by removal of
     lib/test_bpf.c. Its 16k lines of instruction tests were converted
     to BPF selftests that BPF CI runs on every architecture it covers
     (Daniel Borkmann)

  Other features and fixes:

   - Add KASAN checks to memory accesses of JITed programs on x86-64
     (Alexis Lothoré)

   - Complete unification of helper, kfunc and subprog argument checks
     in the verifier (Amery Hung)

   - Add bpf_copy_from_user_mm[_str]() kfuncs (Anastasios Papagiannis)

   - Various features and fixes (Daniel Borkmann):
       - Add .bpf keyring for program signature verification, modelled
         after dm-verity keyring, and support ML-DSA in signing tools
       - Emit shorter and faster code in x86-64 JIT for patterns that
         LLVM generates
       - Fix various issues in gotox handling

   - Let BPF LSM set xattrs at inode creation with bpf_inode_init_xattr()
     (David Windsor, Daniel Borkmann)

   - Use sleepable page allocation in sleepable arena paths. Fix races
     between arena free, allocation and user page fault (Emil Tsalapatis)

   - Switch BPF crypto from lskcipher to AES-CBC and AES-ECB libraries
     (Eric Biggers)

   - Add bpf_proactive_reclaim() kfunc (Hui Zhu)

   - Render decl_tags in libbpf btf_dump and add tests for bpftool C
     dump (Ihor Solodrai)

   - Speed up tracing_multi attach on x86 (Jiri Olsa)

   - Various features and fixes (Kumar Kartikeya Dwivedi):
       - Add BPF_PROG_STREAM_OPEN command to read BPF streams through a
         file descriptor with poll support
       - Let kernel structs publish arena pointers with __arena type tag
       - Fix __uninit kfunc output buffers and fastcall rewrite with
         indirect stack accesses
       - Refuse CO-RE relocations for signed programs

   - Add enqueue_rcvq and dequeue_rcvq hooks to bpf_tcp_ops to adjust
     sk_rcvlowat and avoid unnecessary wakeups (Kuniyuki Iwashima)

   - Add bpf_ksock_sendto() kfunc and allow ksock connect and send from
     bpf_wq (Mahe Tardy)

   - Fix racy plt target update in arm64 bpf_arch_text_poke()
     (Matthew Wood)

   - Restructure BPF selftests Makefile as a layered build
     (Mykola Lysenko)

   - Speed up hash map lookups for u32 and u64 keys (Mykyta Yatsenko)

   - Add signed div/mod, movsx, bswap and gotol to MIPS JITs, signed
     div/mod to parisc JITs (Nicholas Dudar, Johan Almbladh)

   - Fix stack overflow in build ID parser on 32-bit (Nickolai
     Zeldovich)

   - Add stack arguments, signed arena loads and gotox to riscv64 JIT.
     Pu Lehui is now co-maintainer of riscv BPF JITs (Pu Lehui, Chen
     Pei, Feng Jiang)

   - Add bpf_call_rcu() and bpf_call_rcu_tasks_trace() kfuncs. Support
     fetching and/or/xor atomics in arena on x86-64 (Puranjay Mohan)

   - Fix flaky selftests, selftests with clang 23 and bpf-gcc
     (Qiliang Yuan, KaFai Wan, Vineet Gupta)

   - Fix builds broken by fatal resolve_btfids warnings, BTF generation
     on ARC, and define arch_cmpxchg_local() on arc, hexagon and
     openrisc (Sasha Levin)

   - Reduce memory use of hash maps by up to 50% per element
     (T.J. Mercier)

   - Mark pointer fields of sched_ext ops arguments as trusted (Tejun
     Heo)

   - Remove legacy bpf_asm, bpf_dbg and bpf_jit_disasm tools (Xia Yan)

   - Fix index overflow in queue and stack maps, arena range tree after
     allocation failure, tail call loop after freplace detach, lost
     signal in bpf_send_signal() (Yuan Chen)

   - Many more fixes and cleanups"

* tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (660 commits)
  selftests/bpf: Time tc_edt from its first packet with data
  selftests/bpf: Test unwind_exit of an exception callback
  selftests/bpf: Tests for 32-bit scalar evolution bounds computation
  bpf: Reject unwind_goto and unwind_exit in offloaded progs
  bpf: Support 32-bit linear scalar evolution
  bpf: Make unwind_exit of an exception callback exit in JITs
  selftests/bpf: Test forbidden bpf_ksock_sendto() LSM attach
  selftests/bpf: Test ksock sendto()
  bpf: Add bpf_ksock_sendto() kfunc
  selftests/bpf: Test unwinding from R0: R2 subprog
  selftests/bpf: Add a C++ prog with exceptions
  selftests/bpf: Add tests for unwind_goto and unwind_exit
  bpf, arm64: Support unwind_goto and unwind_exit
  bpf, x86: Support unwind_goto and unwind_exit
  bpf: Add unwind_goto and unwind_exit insns
  tools/bpf: Remove legacy BPF tools
  bpf: Annotate data races on bpf_lru_node type
  selftests/bpf: Add arena allocation race tests
  bpf: Fix arena race between page free and alloc leading to incoherency
  bpf: Revalidate faulted arena pages before installing user mappings
  ...
2026-10-23 13:08:49 -10:00
Linus Torvalds
c5fd5e67c7 mm.git review status for linus..mm-stable
Everything:
 
 Total patches:       539
 Fixes: tags:         101
 cc:stable patches:   67
 Reviews/patch:       1.62
 Reviewed rate:       76%
 
 Excluding DAMON:
 
 Total patches:       407
 Fixes: tags:         84
 cc:stable patches:   44
 Reviews/patch:       2.00
 Reviewed rate:       87%
 
 Excluding DAMON and selftests:
 
 Total patches:       360
 Fixes: tags:         78
 cc:stable patches:   44
 Reviews/patch:       2.02
 Reviewed rate:       89%
 
 Summary of patch series in this merge, ordered by descending size:
 
 - "selftests/mm: improve khugepaged coverage" (Kiryl Shutsemau (Meta)):
 
   Give khugepaged a proper workout.  Extend the tests to all supported mTHP
   orders, check exactly which windows collapsed, and make scan completion
   deterministic.  Race collapse against faults, pinning, fork and remap;
   verify that the contents survive.
 
   Add coverage for whole-page-table zaps and memory pressure too.
 
 - "selftests/mm: separate GUP microbenchmarking from functional testing"
   (Sarthak Sharma):
 
   Untangle GUP benchmarking from correctness testing.  Move the benchmark
   into tools/mm, add a standalone harness-based GUP selftest, and move shared
   helpers into tools/lib/mm.  Give each tool one job instead of mixing timing
   measurements with correctness checks.
 
   Cover the mapping, pinning and batch-size combinations explicitly.
 
 - "mm: Switch device DAX to section-based vmemmap optimization" (Muchun Song):
 
   Apply section-based vmemmap optimization to device DAX.  Rework the backing
   metadata layout to save memory while preserving the handling of device
   memory hotplug and its different alignment and lifetime requirements.
 
 - "mm: Introduce section-based vmemmap optimization for HugeTLB" (Muchun
   Song):
 
   Cut the memory consumed by HugeTLB's struct page array with section-based
   vmemmap optimization.  Arrange the metadata so large huge pages need fewer
   backing pages, while keeping initialization, teardown and the various
   hugepage configurations straight.
 
 - "mm/huge_memory: clean up and decouple the anon and file split helpers"
   (Kairui Song):
 
   Separate anonymous and file-backed folio splitting, which have rather
   different locking and bookkeeping requirements.  Move the work into
   separate helpers, drop dead checks and tidy swap-cache handling.  Fix error
   paths, including one that could leave a folio frozen.
 
   Keep the swap cache cases separate from file-cache splitting.
 
 - "mm: Unconditional per-VMA locks and cleanups" (Dave Hansen):
 
   Make per-VMA locking unconditional and clean up the old conditional code
   around it.  This removes configuration-dependent plumbing and lets callers
   depend on one locking model rather than carrying both cases along.
 
 - "memcg: remove the v1 soft limit" (Shakeel Butt):
 
   Remove the old cgroup v1 memory soft-limit machinery.  This has long since
   been superseded by the newer memory-control interfaces; hauling around a
   separate reclaim scheme for it no longer buys us much.
 
 - "read proc/pid/smaps_rollup under per-vma lock" (Suren Baghdasaryan):
 
   Read /proc/PID/smaps_rollup using per-VMA locks where possible, rather than
   holding mmap_lock across the entire walk.  Retain the contention fallback
   and add tearing tests.  This considerably reduces latency when another
   thread is busy changing its mappings.
 
 - "mm/damon: introduce data access-as-a-data attribute" (SJ Park):
 
   Teach DAMON to expose memory access observations as data attributes, making
   them available to the probe and filtering machinery.  Extend the
   interfaces, documentation and tests so the new information can be used
   rather than merely collected.
 
 - "Remove PG_private by using page/folio->private checks instead" (Zi Yan):
 
   Get rid of PG_private and use the folio's private pointer to say whether
   private data is attached.  Convert the filesystem users, including erofs,
   f2fs and md, and clean up the old accessors and flags.  There are a few
   special cases, notably swapcache and hugetlb, which need explicit care.
 
 - "mm: make userland page table freeing RCU-safe" (Lorenzo Stoakes (ARM)):
 
   Make freeing userspace page tables safe against RCU readers.  Sort out the
   page-table lifetime and locking rules so walkers cannot stumble over freed
   tables while mappings are being torn down.
 
 - "make unused huge shrinker memcg aware" (Qi Zheng):
 
   Make shmem's unused-huge-folio shrinker aware of memory cgroups, so reclaim
   can find those folios under the right memcg.  Wire up the per-memcg
   tracking and its lifetime rules, and give the shrinklist its own lockdep
   class to avoid spurious dependency reports.
 
 - "mm: Drop pxd_ERROR()" (Anshuman Khandual):
 
   Remove the old pxd_ERROR() page-table checks and their
   architecture-specific baggage.  Simplify the common page-table interfaces
   and callers now that these checks no longer serve a useful purpose.
 
 - "memcg: group struct fields by access pattern" (Shakeel Butt):
 
   Rearrange memcg's hot and cold fields according to how they're actually
   accessed.  Keep frequently updated counters and their neighboring state
   from fighting over cachelines and reduce needless cacheline traffic.
 
 - "mm/damon/core: fix the size charged for a filter-trimmed region" (Donggeun
   Yoo):
 
   Fix DAMOS quota accounting when a filter trims a monitoring region.  Charge
   only the part left after filtering, not the original region, and add KUnit
   coverage.  Previously the excluded memory could eat into the quota and make
   schemes run more slowly than requested.
 
 - "mm/damon: add kunit and selftests for probes and probe weights" (SJ Park):
 
   Add more tests for DAMON probes and their weights, including validation and
   the results reported to userspace.  This is mostly about catching
   regressions in the newer probe infrastructure before they become harder to
   diagnose.
 
 - "mm: memcontrol: constify the read side of the read side of the memcg API"
   (Tal Zussman):
 
   Mark the read-only parts of the memcg interface const, from folio and
   lruvec accessors to counters and swap helpers.  Most of this is mechanical,
   but it makes the contracts clearer and lets the compiler catch accidental
   writes.
 
 - "mm/damon/vaddr: support {prep,apply}_probes" (SJ Park):
 
   Support the new prepare/apply probe operations in DAMON's virtual-address
   backend.  This brings vaddr monitoring into line with the probe
   infrastructure and makes the added probe types useful there too.
 
 - "mm/mglru: clean up folio counters and flag usage" (Kairui Song):
 
   Simplify MGLRU's folio counters and flag handling.  Remove redundant
   accounting paths and make the state transitions easier to follow without
   changing the intended reclaim policy.
 
 - "arch, mm/execmem: resolve confusion about set_direct_map_valid_noflush()"
   (Mike Rapoport (Microsoft)):
 
   Clarify and straighten out the set_direct_map_valid_noflush() interface
   across architectures and execmem.  Make the requested mapping changes and
   the callers' expectations agree, rather than relying on subtly different
   architecture behavior.
 
 - "mm: zswap: free cold writeback folios promptly" (Alexandre Ghiti):
 
   Free zswap writeback folios as soon as their disk write completes.  They
   are cold temporary buffers, so putting them on the LRU only makes reclaim
   rediscover them later.  Extend swap-cache dropbehind and let zswap keep
   these folios off the LRU altogether.
 
 - "mm: gup: cleanup gup_fast call chain" (Aristeu Rozanski):
 
   Clean up the GUP-fast call chain and simplify how the fast-path helpers
   pass their arguments and results around.  Keep the page-pin behavior the
   same while making the call sequence easier to maintain.
 
 - "mm: optimize zone-device memmap initialization" (Li Zhe):
 
   Speed up initialization of zone-device struct pages.  Avoid redundant work
   when setting up large device-memory ranges, where touching every descriptor
   can get expensive.
 
 - "mm/zswap: shrink zswap_entry via a pool id" (Jianyue Wu):
 
   Reduce the size of zswap_entry by storing a compact pool identifier instead
   of a full pointer.  Keep the pool lookup and lifetime handling correct
   while saving per-entry metadata.
 
 - "mm/damon: support access monitoring of hugetlb-backed memory" (Krishna
   Iyer):
 
   Let DAMON monitor hugetlb-backed memory too.  Extend the relevant access
   checks and add coverage so those mappings no longer fall through a hole in
   the monitoring machinery.
 
 - "mm/execmem: fixes and cleanups for the ROX cache" (Mike Rapoport
   (Microsoft)):
 
   Tidy execmem's read-only-executable cache and fix its handling of the
   awkward allocation and release cases.  The aim is to make the cache's
   ownership and permission transitions more predictable.
 
 - "zsmallc: remove old object read API" (Sergey Senozhatsky):
 
   Drop zsmalloc's old object-read API and move its remaining users to the
   supported access mechanism.  This removes a duplicate interface which we no
   longer need.
 
 - "mm/mempolicy: stop copying state in the interleave paths" (Gregory Price):
 
   Stop copying mempolicy state in the interleave paths when a reference to
   the existing policy will do.  This avoids extra work and makes the
   ownership rules less obscure.
 
 - "mm/mglru: speed up inc_min_seq() and fix cold/hot inversions" (Barry Song
   (Xiaomi)):
 
   Make MGLRU's generation advancement cheaper and fix cases where cold and
   hot folios end up in the wrong order.  Adjust the counter and list handling
   so reclaim doesn't keep passing over the better eviction candidates.
 
 - "mm: Fix MF_DELAYED handling on memory failure" (Lisa Wang):
 
   Treat MF_DELAYED as a partially completed memory-failure operation, not an
   outright failure.  Propagate that result correctly through shmem and the
   generic handler, fix the extra-reference accounting, and add tests for the
   resulting SIGBUS behavior.
 
 - "tools/mm/page_owner_sort: fix --sort, add module filter, improve usage" (Ye
   Liu):
 
   Fix page_owner_sort's --sort handling, add module-based filtering and make
   the usage output more helpful.  This makes the tool less frustrating when
   tracking down who is holding on to pages.
 
 - "mm/damon: move damos filter range arguments validation to core" (SJ Park):
 
   Move DAMOS filter range validation into the core, where all callers can
   share it.  Also close the sysfs read-twice races which could pass
   inconsistent ranges, and add tests for invalid input.
 
 - "mm/damon: misc cleanups" (Zenghui Yu (Huawei)):
 
   Clean up a collection of small DAMON rough edges in the core, interfaces
   and tests.  No large behavior change here, just simpler code and fewer
   opportunities for confusion.
 
 - "mm/damon: cleanup code, add test cases, and update guidances in docs" (SJ
   Park):
 
   Clean up DAMON's core and interfaces, add missing test cases and bring the
   documentation into line with the code.  These changes should make the newer
   facilities easier to understand and maintain.
 
 - "mm/damon: introduce hugepage_size probe filter" (SJ Park):
 
   Add a DAMON probe filter for hugepage size.  Let users select a range of
   folio sizes through sysfs, validate the range and document the new
   interface, so monitoring can concentrate on memory backed by the desired
   hugepage sizes.
 
 - "selftests/mm: pagemap_ioctl test fixes and cleanups" (Zenghui Yu (Huawei)):
 
   Fix some pagemap ioctl test assumptions and clean up their setup and
   reporting.  Make failures point to the operation that went wrong rather
   than to test harness bookkeeping.
 
 - "mm: avoid large folio splits when swap is unavailable" (Johannes Weiner):
 
   Avoid splitting large folios merely to attempt swap when no swap space is
   available.  There is little point paying for an expensive split when
   reclaim cannot actually swap the resulting pages out.
 
 - "mm: workingset: fix the shadow node budget under MGLRU" (Hui Zhu):
 
   Fix the workingset shadow-node budget with MGLRU enabled.  Keep the shadow
   entries within the intended limits so refault tracking does not quietly
   consume more metadata than it should.
 
 - "selftests/mm: TAP output and global-state fixes" (Song Hu):
 
   Fix selftests which produce misleading TAP output or fail to restore global
   settings after a run.  A test ought to leave the system as it found it,
   particularly when a whole suite follows behind.
 
 - "mm: remove page_swap_entry()" (Tal Zussman):
 
   Remove page_swap_entry() and convert its remaining users to the folio and
   swap-entry helpers.  Lose an old page-level interface and make the swap
   paths a little more consistent.
 
 - "mm: refactor zonelist constructors and iterators" (Gregory Price):
 
   Refactor the zonelist construction and walk helpers to reduce duplicated
   logic.  Keep the allocation ordering rules intact while making it clearer
   how zones are selected.
 
 - "mm: kmemleak: batch the struct page scan" (Breno Leitao):
 
   Scan adjacent struct pages in batches instead of taking kmemleak_lock once
   for every single page.  This preserves scan coverage and the lock-hold
   bound, while cutting overhead substantially on debug kernels.
 
 - "mm/damon: misc improvements in tests and documents" (Zenghui Yu (Huawei)):
 
   Fix some DAMON selftests which leave kdamond running or fail under Python
   safe-path mode, and improve the KUnit coverage and documentation.  Nothing
   especially glamorous, but these are worth doing.
 
 - "mm/damon: introduce probe_hits_wsum DAMOS core filter" (SJ Park):
 
   Add a DAMOS core filter using weighted probe-hit counts.  This gives
   schemes another way to select regions based on what the monitoring probes
   observed.
 
 - "mm/mglru: fix ineffective memory protection for non-kswapd reclaim" (Ridong
   Chen):
 
   Fix MGLRU's memory protection handling for reclaim outside kswapd.  The
   protection settings weren't having the intended effect in these paths,
   leaving protected memory more exposed than requested.
 
 - "mm/damon: improve readability, clarity and test coverage" (SJ Park):
 
   Simplify a handful of DAMON core helpers, clarify their behavior and expand
   test coverage.  There are no intended userspace-visible changes in this
   batch.
 
 - "mm: replace PF_KCOMPACTD/PF_KSWAPD with kthread_func()" (Kefeng Wang):
 
   Stop identifying reclaim and compaction workers through the PF_KSWAPD and
   PF_KCOMPACTD task flags.  Use their kthread functions instead and remove
   the redundant flag plumbing.
 
 - "mm/hugetlb: fix surplus accounting and availability checks during demotion"
   (Longlong Xia):
 
   Fix HugeTLB surplus accounting and availability checks when demoting huge
   pages.  Keep the counters and reservation decisions consistent as one huge
   page turns into several smaller ones.
 
 - "mm/damon: add kunit tests for probe_hits handling and probe params
   validation" (Jason Angelov):
 
   Add KUnit tests for DAMON probe-hit handling and invalid probe parameters.
   These cover edge cases that were too easy to miss in ordinary monitoring
   tests.
 
 - "mm/damon: fix misc bugs in kunit, quota goals and sysfs refresh_ms" (SJ
   Park):
 
   Fix a few DAMON bugs in KUnit tests, quota-goal handling and the sysfs
   refresh interval.  Small fixes, but the incorrect results and control
   behavior were real.
 
 - "mm: zswap: optimize zswap invalidate and store" (Kefeng Wang):
 
   Simplify zswap's invalidate and store paths to avoid avoidable lookups and
   work.  Keep the entry lifetime rules intact while making those frequent
   operations cheaper.
 
 - "mm: replace custom ratelimiting logic" (Pedro Falcato):
 
   Replace hand-rolled MM rate limiting with the common kernel helpers.  There
   is no need to maintain another local version of the same mechanism.
 
 - "mm/vmalloc: minor cleanups" (Ye Liu):
 
   Tidy some vmalloc initialization and reporting code, and pull duplicated
   vmap-area insertion work into a helper.  No functional changes are
   intended.
 
 - "mm/mglru: clean up isolate_folios for readability and clarity" (Ridong
   Chen):
 
   Clean up MGLRU's isolate_folios() so the decisions are easier to read and
   verify.  Keep reclaim behavior unchanged while removing unnecessary twists
   in the control flow.
 
 - "Catch automatic storage in IDA and Maple Tree definitions" (Yury Norov):
 
   Catch IDA and maple-tree definitions placed in automatic storage where the
   lifetime rules are wrong or easy to get wrong.  Add checks so such mistakes
   are harder to slip through unnoticed.
 
 - "mm, swap: some random fixes and cleanups" (Kemeng Shi):
 
   Fix and simplify a few unrelated swap paths, removing redundant work and
   tidying up error handling.  A small collection of things which were better
   fixed than left lying around.
 
 - "mm/page_io: folio conversion cleanups" (Tal Zussman):
 
   Finish some page-to-folio conversions in swap I/O, removing obsolete
   wrappers and using folio operations consistently.  The I/O behavior is
   intended to remain the same.
 
 - "mm/swap: skip empty clusters in the swapoff scan" (Youngjun Park):
 
   Skip empty swap clusters during swapoff instead of walking slots which have
   nothing to release.  This avoids pointless work on sparsely used swap
   devices.
 
 - "mm: trace: decode architecture-specific VMA flags" (Meijing Zhao):
 
   Teach the VMA trace output to decode architecture-specific flags, rather
   than leaving them as unexplained bits.  That should make tracing a good
   deal more useful when debugging unusual mappings.
 
 - "mm, swap: don't spin on a bad swap entry" (Breno Leitao):
 
   Stop repeatedly retrying a swap entry which is already known to be bad.
   Make the error path terminate cleanly instead of wasting CPU chasing an
   entry that cannot be used.
 
 - "mm/page_isolation: fix UBSAN shift-out-of-bounds in
   isolate_single_pageblock" (Qi Xi):
 
   Fix an out-of-range shift in isolate_single_pageblock() and tighten its
   associated boundary handling.  UBSAN found the issue; the result should be
   more robust pageblock isolation.
 
 - "mm: khugepaged: fix tracepoint UAF" (Vernon Yang):
 
   Fix a use-after-free in khugepaged tracepoint handling.  Make sure the
   information recorded by tracing stays valid for as long as the trace event
   needs it.
 
 - "mm/damon: fix DAMOS bugs in core, paddr and vaddr" (SJ Park):
 
   Fix several DAMOS problems across the core and physical- and
   virtual-address backends.  The changes close error cases and bring the
   implementations back into agreement.
 
 - "mm/hugetlb_cgroup: move the per-node usage along with the folio" (Hongfu
   Li):
 
   Move HugeTLB per-node usage accounting when folios migrate between nodes or
   are reparented to another cgroup.  Otherwise numa_stat can retain stale
   charges and even underflow when the folio is freed.
 
 - "docs/core-api: memory-allocation: add k[mz]alloc_obj() and clarify kmalloc"
   (Mike Rapoport (Microsoft)):
 
   Document the typed k[mz]alloc_obj() allocation helpers and clear up some
   kmalloc guidance.  Make it easier for callers to choose the right
   interface.
 
 - "selftests/mm: Validate selections and scope memfd_secret setup" (Tianyi
   Chen):
 
   Check selftest selections before running and confine memfd_secret setup to
   the cases that actually need it.  Avoid confusing failures caused by the
   test harness itself.
 
 - "mm/damon: move zero damos quota target_value handling to the core layer"
   (SJ Park):
 
   Handle a zero DAMOS quota goal target in the core rather than making each
   user of the interface invent its own rule.  Keep quota decisions consistent
   across callers.
 
 - "mm/damon: Introduce a huge page collapsing mechanism using auto tuning"
   (Asier Gutierrez):
 
   Add an auto-tuned DAMON mechanism for collapsing huge pages.  Use
   monitoring feedback to decide when collapse is worthwhile rather than
   forcing one fixed policy onto every workload.
 
 - "kmemleak: fix stale documentation and raise the verbose default" (Breno
   Leitao):
 
   Bring the kmemleak documentation up to date and require a third confirming
   scan before verbose kernels print a suspected leak.  The extra scan helps
   suppress transient false positives in noisy debug builds.
 
 - "mm/memory: improve insert_pages() error handling" (Avi Weiss):
 
   Tighten insert_pages() error handling so partial failures are reported and
   cleaned up consistently.  The caller should not have to guess how far the
   operation got.
 
 - "mm/damon: allow NULL or unstarted damon_ctx parameter for damon_call()" (SJ
   Park):
 
   Make damon_call() handle NULL and not-yet-started contexts safely.
   Simplify the rules for callers and avoid an indefinite wait on an invalid
   state.
 
 - "mm/damon: introduce pgidle_set probe filter type" (SJ Park):
 
   Add a pgidle_set DAMON probe filter, giving monitoring users another way to
   select pages by their idle-tracking state.  Wire it through the filter
   interfaces and tests.
 
 - "mm/damon/core: remove unused helper functions" (Cheng-Han Wu):
 
   Remove DAMON core helpers which no longer have callers.  Keep the API
   surface smaller rather than retaining dead code just in case.
 
 - "mm: stop calling pmd_folio() on special PMDs" (Gregory Price):
 
   Avoid calling pmd_folio() on special PMD mappings that have no struct page.
   The old assumption could fault in procfs walkers; use the normal-folio
   checks and handle those mappings properly.
 
 - "mm: remove three unused helpers from mm.h" (Tal Zussman):
 
   Remove three unused helpers from mm.h.  Fewer obsolete interfaces for the
   next person to stumble over.
 
 - "mm/damon: improvements in efficiency, error handling, documents" (Liew Rui
   Yan):
 
   Skip needless DAMOS quota work once the quota is full, propagate a missing
   sysfs error and tidy a couple of comments.  The ignored error could make a
   monitoring-start request appear successful when it wasn't.
 
 - "docs/mm/damon/design: add explanation of nr_snapshots" (Liew Rui Yan):
 
   Explain DAMON's nr_snapshots accounting in the design documentation and
   straighten out the associated terminology.  Make it clearer what users
   should expect from that counter.
 
 - "selftests/cgroup: fixes for test_zswap on single core VM" (Wilson Felipe
   Pereira):
 
   Fix two test_zswap failures on single-CPU VMs.  Wait for the exiting
   child's cgroup to become unpopulated before reconfiguring it, and avoid
   another test assumption which breaks under constrained scheduling.
 
 - "docs/ksm: fix advisor documentation and comment" (Jinjiang Tu):
 
   Correct the KSM advisor documentation and a misleading code comment.  Keep
   the description aligned with how the advisor actually works.
 
 - "Fix bugs in HugeTLB allocation when mem_cgroup_charge_hugetlb() fails"
   (Ackerley Tng):
 
   Fix HugeTLB allocation cleanup when the memcg charge fails.  Make sure
   failure exits undo what was allocated rather than leaving stale accounting
   or resources behind.
 
 - "mm, memcg: fix memory.peak reset clobbering other fds' watermark" (Ridong
   Chen):
 
   Fix memory.peak reset handling so resetting one file descriptor doesn't
   overwrite the watermark kept for another.  Keep those per-open measurements
   independent as intended.
 
 - "mm: reject zone device folios in more folio walkers" (Gregory Price):
 
   Reject zone-device folios in page walkers that cannot safely handle them.
   Extend the existing checks rather than assuming every folio has ordinary
   system-memory backing.
 
 - "percpu: remove code with no effect" (Sang-Heon Jeon):
 
   Remove a few pieces of percpu code which no longer do anything.  There is
   no reason to keep maintaining dead paths.
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCatbvywAKCRDdBJ7gKXxA
 jrEfAP91yviLKCh8AEazeeBo4D65ntp0FmZB9qqCj46+YuEkowD9HPAyTpD+PjWS
 yp/lLvvS4RIg6chID4S2U/pLGzlypg4=
 =RIh/
 -----END PGP SIGNATURE-----

Merge tag 'mm-stable-2026-10-19-21-33' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull MM updates from Andrew Morton:

 - "selftests/mm: improve khugepaged coverage" (Kiryl Shutsemau):

   Give khugepaged a proper workout. Extend the tests to all supported
   mTHP orders, check exactly which windows collapsed, and make scan
   completion deterministic. Race collapse against faults, pinning, fork
   and remap; verify that the contents survive.

   Add coverage for whole-page-table zaps and memory pressure too.

 - "selftests/mm: separate GUP microbenchmarking from functional testing"
   (Sarthak Sharma):

   Untangle GUP benchmarking from correctness testing. Move the
   benchmark into tools/mm, add a standalone harness-based GUP selftest,
   and move shared helpers into tools/lib/mm. Give each tool one job
   instead of mixing timing measurements with correctness checks.

   Cover the mapping, pinning and batch-size combinations explicitly.

 - "mm: Switch device DAX to section-based vmemmap optimization" (Muchun
   Song):

   Apply section-based vmemmap optimization to device DAX. Rework the
   backing metadata layout to save memory while preserving the handling
   of device memory hotplug and its different alignment and lifetime
   requirements.

 - "mm: Introduce section-based vmemmap optimization for HugeTLB"
   (Muchun Song):

   Cut the memory consumed by HugeTLB's struct page array with
   section-based vmemmap optimization. Arrange the metadata so large
   huge pages need fewer backing pages, while keeping initialization,
   teardown and the various hugepage configurations straight.

 - "mm/huge_memory: clean up and decouple the anon and file split
   helpers" (Kairui Song):

   Separate anonymous and file-backed folio splitting, which have rather
   different locking and bookkeeping requirements. Move the work into
   separate helpers, drop dead checks and tidy swap-cache handling. Fix
   error paths, including one that could leave a folio frozen.

   Keep the swap cache cases separate from file-cache splitting.

 - "mm: Unconditional per-VMA locks and cleanups" (Dave Hansen):

   Make per-VMA locking unconditional and clean up the old conditional
   code around it. This removes configuration-dependent plumbing and
   lets callers depend on one locking model rather than carrying both
   cases along.

 - "memcg: remove the v1 soft limit" (Shakeel Butt):

   Remove the old cgroup v1 memory soft-limit machinery. This has long
   since been superseded by the newer memory-control interfaces; hauling
   around a separate reclaim scheme for it no longer buys us much.

 - "read proc/pid/smaps_rollup under per-vma lock" (Suren Baghdasaryan):

   Read /proc/PID/smaps_rollup using per-VMA locks where possible,
   rather than holding mmap_lock across the entire walk. Retain the
   contention fallback and add tearing tests. This considerably reduces
   latency when another thread is busy changing its mappings.

 - "mm/damon: introduce data access-as-a-data attribute" (SJ Park):

   Teach DAMON to expose memory access observations as data attributes,
   making them available to the probe and filtering machinery. Extend
   the interfaces, documentation and tests so the new information can be
   used rather than merely collected.

 - "Remove PG_private by using page/folio->private checks instead" (Zi
   Yan):

   Get rid of PG_private and use the folio's private pointer to say
   whether private data is attached. Convert the filesystem users,
   including erofs, f2fs and md, and clean up the old accessors and
   flags. There are a few special cases, notably swapcache and hugetlb,
   which need explicit care.

 - "mm: make userland page table freeing RCU-safe" (Lorenzo Stoakes):

   Make freeing userspace page tables safe against RCU readers. Sort out
   the page-table lifetime and locking rules so walkers cannot stumble
   over freed tables while mappings are being torn down.

 - "make unused huge shrinker memcg aware" (Qi Zheng):

   Make shmem's unused-huge-folio shrinker aware of memory cgroups, so
   reclaim can find those folios under the right memcg. Wire up the
   per-memcg tracking and its lifetime rules, and give the shrinklist
   its own lockdep class to avoid spurious dependency reports.

 - "mm: Drop pxd_ERROR()" (Anshuman Khandual):

   Remove the old pxd_ERROR() page-table checks and their
   architecture-specific baggage. Simplify the common page-table
   interfaces and callers now that these checks no longer serve a
   useful purpose.

 - "memcg: group struct fields by access pattern" (Shakeel Butt):

   Rearrange memcg's hot and cold fields according to how they're
   actually accessed. Keep frequently updated counters and their
   neighboring state from fighting over cachelines and reduce needless
   cacheline traffic.

 - "mm/damon/core: fix the size charged for a filter-trimmed region"
   (Donggeun Yoo):

   Fix DAMOS quota accounting when a filter trims a monitoring region.
   Charge only the part left after filtering, not the original region,
   and add KUnit coverage. Previously the excluded memory could eat into
   the quota and make schemes run more slowly than requested.

 - "mm/damon: add kunit and selftests for probes and probe weights" (SJ
   Park):

   Add more tests for DAMON probes and their weights, including
   validation and the results reported to userspace. This is mostly
   about catching regressions in the newer probe infrastructure before
   they become harder to diagnose.

 - "mm: memcontrol: constify the read side of the read side of the memcg
   API" (Tal Zussman):

   Mark the read-only parts of the memcg interface const, from folio and
   lruvec accessors to counters and swap helpers. Most of this is
   mechanical, but it makes the contracts clearer and lets the compiler
   catch accidental writes.

 - "mm/damon/vaddr: support {prep,apply}_probes" (SJ Park):

   Support the new prepare/apply probe operations in DAMON's
   virtual-address backend. This brings vaddr monitoring into line with
   the probe infrastructure and makes the added probe types useful there
   too.

 - "mm/mglru: clean up folio counters and flag usage" (Kairui Song):

   Simplify MGLRU's folio counters and flag handling. Remove redundant
   accounting paths and make the state transitions easier to follow
   without changing the intended reclaim policy.

 - "arch, mm/execmem: resolve confusion about set_direct_map_valid_noflush()"
   (Mike Rapoport (Microsoft)):

   Clarify and straighten out the set_direct_map_valid_noflush()
   interface across architectures and execmem. Make the requested
   mapping changes and the callers' expectations agree, rather than
   relying on subtly different architecture behavior.

 - "mm: zswap: free cold writeback folios promptly" (Alexandre Ghiti):

   Free zswap writeback folios as soon as their disk write completes.
   They are cold temporary buffers, so putting them on the LRU only
   makes reclaim rediscover them later. Extend swap-cache dropbehind and
   let zswap keep these folios off the LRU altogether.

 - "mm: gup: cleanup gup_fast call chain" (Aristeu Rozanski):

   Clean up the GUP-fast call chain and simplify how the fast-path
   helpers pass their arguments and results around. Keep the page-pin
   behavior the same while making the call sequence easier to maintain.

 - "mm: optimize zone-device memmap initialization" (Li Zhe):

   Speed up initialization of zone-device struct pages. Avoid redundant
   work when setting up large device-memory ranges, where touching every
   descriptor can get expensive.

 - "mm/zswap: shrink zswap_entry via a pool id" (Jianyue Wu):

   Reduce the size of zswap_entry by storing a compact pool identifier
   instead of a full pointer. Keep the pool lookup and lifetime handling
   correct while saving per-entry metadata.

 - "mm/damon: support access monitoring of hugetlb-backed memory"
   (Krishna Iyer):

   Let DAMON monitor hugetlb-backed memory too. Extend the relevant
   access checks and add coverage so those mappings no longer fall
   through a hole in the monitoring machinery.

 - "mm/execmem: fixes and cleanups for the ROX cache" (Mike Rapoport):

   Tidy execmem's read-only-executable cache and fix its handling of the
   awkward allocation and release cases. The aim is to make the cache's
   ownership and permission transitions more predictable.

 - "zsmallc: remove old object read API" (Sergey Senozhatsky):

   Drop zsmalloc's old object-read API and move its remaining users to
   the supported access mechanism. This removes a duplicate interface
   which we no longer need.

 - "mm/mempolicy: stop copying state in the interleave paths" (Gregory
   Price):

   Stop copying mempolicy state in the interleave paths when a reference
   to the existing policy will do. This avoids extra work and makes the
   ownership rules less obscure.

 - "mm/mglru: speed up inc_min_seq() and fix cold/hot inversions" (Barry
   Song):

   Make MGLRU's generation advancement cheaper and fix cases where cold
   and hot folios end up in the wrong order. Adjust the counter and list
   handling so reclaim doesn't keep passing over the better eviction
   candidates.

 - "mm: Fix MF_DELAYED handling on memory failure" (Lisa Wang):

   Treat MF_DELAYED as a partially completed memory-failure operation,
   not an outright failure. Propagate that result correctly through
   shmem and the generic handler, fix the extra-reference accounting,
   and add tests for the resulting SIGBUS behavior.

 - "tools/mm/page_owner_sort: fix --sort, add module filter, improve
   usage" (Ye Liu):

   Fix page_owner_sort's --sort handling, add module-based filtering and
   make the usage output more helpful. This makes the tool less
   frustrating when tracking down who is holding on to pages.

 - "mm/damon: move damos filter range arguments validation to core" (SJ
   Park):

   Move DAMOS filter range validation into the core, where all callers
   can share it. Also close the sysfs read-twice races which could pass
   inconsistent ranges, and add tests for invalid input.

 - "mm/damon: misc cleanups" (Zenghui Yu):

   Clean up a collection of small DAMON rough edges in the core,
   interfaces and tests. No large behavior change here, just simpler
   code and fewer opportunities for confusion.

 - "mm/damon: cleanup code, add test cases, and update guidances in
   docs" (SJ Park):

   Clean up DAMON's core and interfaces, add missing test cases and
   bring the documentation into line with the code. These changes should
   make the newer facilities easier to understand and maintain.

 - "mm/damon: introduce hugepage_size probe filter" (SJ Park):

   Add a DAMON probe filter for hugepage size. Let users select a range
   of folio sizes through sysfs, validate the range and document the new
   interface, so monitoring can concentrate on memory backed by the
   desired hugepage sizes.

 - "selftests/mm: pagemap_ioctl test fixes and cleanups" (Zenghui Yu):

   Fix some pagemap ioctl test assumptions and clean up their setup and
   reporting. Make failures point to the operation that went wrong
   rather than to test harness bookkeeping.

 - "mm: avoid large folio splits when swap is unavailable" (Johannes
   Weiner):

   Avoid splitting large folios merely to attempt swap when no swap
   space is available. There is little point paying for an expensive
   split when reclaim cannot actually swap the resulting pages out.

 - "mm: workingset: fix the shadow node budget under MGLRU" (Hui Zhu):

   Fix the workingset shadow-node budget with MGLRU enabled. Keep the
   shadow entries within the intended limits so refault tracking does
   not quietly consume more metadata than it should.

 - "selftests/mm: TAP output and global-state fixes" (Song Hu):

   Fix selftests which produce misleading TAP output or fail to restore
   global settings after a run. A test ought to leave the system as it
   found it, particularly when a whole suite follows behind.

 - "mm: remove page_swap_entry()" (Tal Zussman):

   Remove page_swap_entry() and convert its remaining users to the folio
   and swap-entry helpers. Lose an old page-level interface and make the
   swap paths a little more consistent.

 - "mm: refactor zonelist constructors and iterators" (Gregory Price):

   Refactor the zonelist construction and walk helpers to reduce
   duplicated logic. Keep the allocation ordering rules intact while
   making it clearer how zones are selected.

 - "mm: kmemleak: batch the struct page scan" (Breno Leitao):

   Scan adjacent struct pages in batches instead of taking kmemleak_lock
   once for every single page. This preserves scan coverage and the
   lock-hold bound, while cutting overhead substantially on debug
   kernels.

 - "mm/damon: misc improvements in tests and documents" (Zenghui Yu):

   Fix some DAMON selftests which leave kdamond running or fail under
   Python safe-path mode, and improve the KUnit coverage and
   documentation. Nothing especially glamorous, but these are worth
   doing.

 - "mm/damon: introduce probe_hits_wsum DAMOS core filter" (SJ Park):

   Add a DAMOS core filter using weighted probe-hit counts. This gives
   schemes another way to select regions based on what the monitoring
   probes observed.

 - "mm/mglru: fix ineffective memory protection for non-kswapd reclaim"
   (Ridong Chen):

   Fix MGLRU's memory protection handling for reclaim outside kswapd.
   The protection settings weren't having the intended effect in these
   paths, leaving protected memory more exposed than requested.

 - "mm/damon: improve readability, clarity and test coverage" (SJ Park):

   Simplify a handful of DAMON core helpers, clarify their behavior and
   expand test coverage. There are no intended userspace-visible changes
   in this batch.

 - "mm: replace PF_KCOMPACTD/PF_KSWAPD with kthread_func()" (Kefeng
   Wang):

   Stop identifying reclaim and compaction workers through the PF_KSWAPD
   and PF_KCOMPACTD task flags. Use their kthread functions instead and
   remove the redundant flag plumbing.

 - "mm/hugetlb: fix surplus accounting and availability checks during
   demotion" (Longlong Xia):

   Fix HugeTLB surplus accounting and availability checks when demoting
   huge pages. Keep the counters and reservation decisions consistent as
   one huge page turns into several smaller ones.

 - "mm/damon: add kunit tests for probe_hits handling and probe params
   validation" (Jason Angelov):

   Add KUnit tests for DAMON probe-hit handling and invalid probe
   parameters. These cover edge cases that were too easy to miss in
   ordinary monitoring tests.

 - "mm/damon: fix misc bugs in kunit, quota goals and sysfs refresh_ms"
   (SJ Park):

   Fix a few DAMON bugs in KUnit tests, quota-goal handling and the
   sysfs refresh interval. Small fixes, but the incorrect results and
   control behavior were real.

 - "mm: zswap: optimize zswap invalidate and store" (Kefeng Wang):

   Simplify zswap's invalidate and store paths to avoid avoidable
   lookups and work. Keep the entry lifetime rules intact while making
   those frequent operations cheaper.

 - "mm: replace custom ratelimiting logic" (Pedro Falcato):

   Replace hand-rolled MM rate limiting with the common kernel helpers.
   There is no need to maintain another local version of the same
   mechanism.

 - "mm/vmalloc: minor cleanups" (Ye Liu):

   Tidy some vmalloc initialization and reporting code, and pull
   duplicated vmap-area insertion work into a helper. No functional
   changes are intended.

 - "mm/mglru: clean up isolate_folios for readability and clarity"
   (Ridong Chen):

   Clean up MGLRU's isolate_folios() so the decisions are easier to read
   and verify. Keep reclaim behavior unchanged while removing
   unnecessary twists in the control flow.

 - "Catch automatic storage in IDA and Maple Tree definitions" (Yury
   Norov):

   Catch IDA and maple-tree definitions placed in automatic storage
   where the lifetime rules are wrong or easy to get wrong. Add checks
   so such mistakes are harder to slip through unnoticed.

 - "mm, swap: some random fixes and cleanups" (Kemeng Shi):

   Fix and simplify a few unrelated swap paths, removing redundant work
   and tidying up error handling. A small collection of things which
   were better fixed than left lying around.

 - "mm/page_io: folio conversion cleanups" (Tal Zussman):

   Finish some page-to-folio conversions in swap I/O, removing obsolete
   wrappers and using folio operations consistently. The I/O behavior is
   intended to remain the same.

 - "mm/swap: skip empty clusters in the swapoff scan" (Youngjun Park):

   Skip empty swap clusters during swapoff instead of walking slots
   which have nothing to release. This avoids pointless work on sparsely
   used swap devices.

 - "mm: trace: decode architecture-specific VMA flags" (Meijing Zhao):

   Teach the VMA trace output to decode architecture-specific flags,
   rather than leaving them as unexplained bits. That should make
   tracing a good deal more useful when debugging unusual mappings.

 - "mm, swap: don't spin on a bad swap entry" (Breno Leitao):

   Stop repeatedly retrying a swap entry which is already known to be
   bad. Make the error path terminate cleanly instead of wasting CPU
   chasing an entry that cannot be used.

 - "mm/page_isolation: fix UBSAN shift-out-of-bounds in
   isolate_single_pageblock" (Qi Xi):

   Fix an out-of-range shift in isolate_single_pageblock() and tighten
   its associated boundary handling. UBSAN found the issue; the result
   should be more robust pageblock isolation.

 - "mm: khugepaged: fix tracepoint UAF" (Vernon Yang):

   Fix a use-after-free in khugepaged tracepoint handling. Make sure the
   information recorded by tracing stays valid for as long as the trace
   event needs it.

 - "mm/damon: fix DAMOS bugs in core, paddr and vaddr" (SJ Park):

   Fix several DAMOS problems across the core and physical- and
   virtual-address backends. The changes close error cases and bring the
   implementations back into agreement.

 - "mm/hugetlb_cgroup: move the per-node usage along with the folio"
   (Hongfu Li):

   Move HugeTLB per-node usage accounting when folios migrate between
   nodes or are reparented to another cgroup. Otherwise numa_stat can
   retain stale charges and even underflow when the folio is freed.

 - "docs/core-api: memory-allocation: add k[mz]alloc_obj() and clarify
   kmalloc" (Mike Rapoport):

   Document the typed k[mz]alloc_obj() allocation helpers and clear up
   some kmalloc guidance. Make it easier for callers to choose the right
   interface.

 - "selftests/mm: Validate selections and scope memfd_secret setup"
   (Tianyi Chen):

   Check selftest selections before running and confine memfd_secret
   setup to the cases that actually need it. Avoid confusing failures
   caused by the test harness itself.

 - "mm/damon: move zero damos quota target_value handling to the core
   layer" (SJ Park):

   Handle a zero DAMOS quota goal target in the core rather than making
   each user of the interface invent its own rule. Keep quota decisions
   consistent across callers.

 - "mm/damon: Introduce a huge page collapsing mechanism using auto
   tuning" (Asier Gutierrez):

   Add an auto-tuned DAMON mechanism for collapsing huge pages. Use
   monitoring feedback to decide when collapse is worthwhile rather than
   forcing one fixed policy onto every workload.

 - "kmemleak: fix stale documentation and raise the verbose default"
   (Breno Leitao):

   Bring the kmemleak documentation up to date and require a third
   confirming scan before verbose kernels print a suspected leak. The
   extra scan helps suppress transient false positives in noisy debug
   builds.

 - "mm/memory: improve insert_pages() error handling" (Avi Weiss):

   Tighten insert_pages() error handling so partial failures are
   reported and cleaned up consistently. The caller should not have to
   guess how far the operation got.

 - "mm/damon: allow NULL or unstarted damon_ctx parameter for
   damon_call()" (SJ Park):

   Make damon_call() handle NULL and not-yet-started contexts safely.
   Simplify the rules for callers and avoid an indefinite wait on an
   invalid state.

 - "mm/damon: introduce pgidle_set probe filter type" (SJ Park):

   Add a pgidle_set DAMON probe filter, giving monitoring users another
   way to select pages by their idle-tracking state. Wire it through the
   filter interfaces and tests.

 - "mm/damon/core: remove unused helper functions" (Cheng-Han Wu):

   Remove DAMON core helpers which no longer have callers. Keep the API
   surface smaller rather than retaining dead code just in case.

 - "mm: stop calling pmd_folio() on special PMDs" (Gregory Price):

   Avoid calling pmd_folio() on special PMD mappings that have no struct
   page. The old assumption could fault in procfs walkers; use the
   normal-folio checks and handle those mappings properly.

 - "mm: remove three unused helpers from mm.h" (Tal Zussman):

   Remove three unused helpers from mm.h. Fewer obsolete interfaces for
   the next person to stumble over.

 - "mm/damon: improvements in efficiency, error handling, documents"
   (Liew Rui Yan):

   Skip needless DAMOS quota work once the quota is full, propagate a
   missing sysfs error and tidy a couple of comments. The ignored error
   could make a monitoring-start request appear successful when it
   wasn't.

 - "docs/mm/damon/design: add explanation of nr_snapshots" (Liew Rui
   Yan):

   Explain DAMON's nr_snapshots accounting in the design documentation
   and straighten out the associated terminology. Make it clearer what
   users should expect from that counter.

 - "selftests/cgroup: fixes for test_zswap on single core VM" (Wilson
   Felipe Pereira):

   Fix two test_zswap failures on single-CPU VMs. Wait for the exiting
   child's cgroup to become unpopulated before reconfiguring it, and
   avoid another test assumption which breaks under constrained
   scheduling.

 - "docs/ksm: fix advisor documentation and comment" (Jinjiang Tu):

   Correct the KSM advisor documentation and a misleading code comment.
   Keep the description aligned with how the advisor actually works.

 - "Fix bugs in HugeTLB allocation when mem_cgroup_charge_hugetlb()
   fails" (Ackerley Tng):

   Fix HugeTLB allocation cleanup when the memcg charge fails. Make sure
   failure exits undo what was allocated rather than leaving stale
   accounting or resources behind.

 - "mm, memcg: fix memory.peak reset clobbering other fds' watermark"
   (Ridong Chen):

   Fix memory.peak reset handling so resetting one file descriptor
   doesn't overwrite the watermark kept for another. Keep those per-open
   measurements independent as intended.

 - "mm: reject zone device folios in more folio walkers" (Gregory
   Price):

   Reject zone-device folios in page walkers that cannot safely handle
   them. Extend the existing checks rather than assuming every folio has
   ordinary system-memory backing.

 - "percpu: remove code with no effect" (Sang-Heon Jeon):

   Remove a few pieces of percpu code which no longer do anything. There
   is no reason to keep maintaining dead paths.

* tag 'mm-stable-2026-10-19-21-33' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (539 commits)
  mm: shmem: assign a lock class to the unused huge shrinklist
  mm: vmscan: put rotation-missed folios at the LRU tail
  mm: kmemleak: scan the struct page array in MAX_SCAN_SIZE batches
  mm: kmemleak: move the struct page scan into a helper
  mm/damon: document that a zero sample_interval is accepted
  mm/damon: fix typos in comments
  mm/damon/sysfs: propagate damon_call() error in turn_damon_on
  mm/damon/core: skip quota score setup when the quota is full
  mm/damon/tests/core-kunit: test the size charged for a filter-trimmed region
  mm/damon/core: charge only the part of a region the filter left
  mm: disallow raw PFN mappings of huge/shared zeropage
  selftests/mm: zap whole PTE tables in the khugepaged race harness
  selftests/mm: add memory-pressure threads to the khugepaged race harness
  selftests/mm: race the collapse of windows with holes
  selftests/mm: add khugepaged race harness
  selftests/mm: check that one khugepaged pass collapses one window
  selftests/mm: run every supported collapse order by default
  selftests/mm: cover a shared-source collapse write race
  selftests/mm: parameterize the mixed-source collapse case by source order
  selftests/mm: add order-parameterized khugepaged collapse cases
  ...
2026-10-23 07:42:11 -10:00
Pasha Tatashin
4180abdb5d docs: xforms_lists: Handle DECLARE_KHOSER_PTR() struct members
KHO ABI structures declare their preservation pointers with
DECLARE_KHOSER_PTR(name, type), which expands to a union of the pointer
and its physical address. kernel-doc does not know the macro and does
not see the member: it warns that an unnamed member is not described
and that the description of the member is excess.

Translate DECLARE_KHOSER_PTR(name, type) into "type name", as is done
for DECLARE_KFIFO_PTR().

Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Message-ID: <20261015020313.946940-1-pasha.tatashin@soleen.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
2026-10-22 09:45:11 -06:00
Linus Torvalds
bb29b957ea Kbuild / Kconfig updates for 7.4
Kbuild:
 
 - Suppress compiler flags in debug info for external module builds
 
 - Use '-fmacro-prefix-map' in more places in the build
 
 - Merge initial set of changes from Lorenzo's speed up series, along
   with some fixes found through testing
 
   - Avoid allocating .modinfo in vmlinux
 
   - Speed up kallsyms by indexing symbols by token and outputting binary
     data to disk directly then including in assembly via .incbin
 
   - Don't sort $(NM) output when unnecessary
 
   - Only emit relocations on the final vmlinx link
 
   - Reimplement mksysmap in C
 
   - Move many cc-option and ld-option calls into the Kconfig stage to
     cache their results
 
   - Use pigz for gzip compression when available
 
 - Move gen_init_cpio and gen_initramfs.sh from usr/ to scripts/
 
 - Add a TOML configuration to scripts/container and update documentation
   accordingly
 
 - Add a generic, opt-in mechanism for checking that headers can compile
   standalone
 
 - Update machine function splitting argument for LLVM 24 rename in
   AutoFDO build
 
 - Miscellaneous minor clean ups, fixes, and hardening, mostly
   found/fixed via LLMs
 
 Kconfig:
 
 - Turn recursive ranges into an error
 
 - Fix savedefconfig for defaults that were adjusted for being out of
   range and revert a previously applied workaround
 
 - Improvements and hardening around integer and hexadecimal values
 
 - Emit a warning when selecting or implying a choice symbol, which has no
   effect
 
 - Emit a warning when KCONFIG_PROBABILITY is malformed
 
 - Introduce def_string / def_int / def_hex to mirror the existing
   def_bool / def_tristate to succinctly set a value and type at the same
   time.
 
 - Add cc-option-str and ld-option-str macros to act like Kbuild's
   cc-option and ld-option, returning the string when the option is
   supported, allowing many checks to be moved into Kconfig
 
 - Minor usability improvements to Kconfig tests
 
 Signed-off-by: Nathan Chancellor <nathan@kernel.org>
 -----BEGIN PGP SIGNATURE-----
 
 iHUEABYKAB0WIQR74yXHMTGczQHYypIdayaRccAalgUCatRBoQAKCRAdayaRccAa
 lg+JAQCyjmh+xC2lLSHFGB9qTzqYpVoIT0Z9/6EGx09nYgMr5AEAmzg/+dor5u+6
 vjklzsGs95G01yetVE5ifNNsLvVxTAo=
 =HbPA
 -----END PGP SIGNATURE-----

Merge tag 'kbuild-7.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux

Pull Kbuild / Kconfig updates from Nathan Chancellor:
 "Kbuild:

   - Suppress compiler flags in debug info for external module builds

   - Use '-fmacro-prefix-map' in more places in the build

   - Merge initial set of changes from Lorenzo's speed up series, along
     with some fixes found through testing:

     - Avoid allocating .modinfo in vmlinux

     - Speed up kallsyms by indexing symbols by token and outputting
       binary data to disk directly then including in assembly via
       .incbin

     - Don't sort $(NM) output when unnecessary

     - Only emit relocations on the final vmlinx link

     - Reimplement mksysmap in C

     - Move many cc-option and ld-option calls into the Kconfig stage to
       cache their results

     - Use pigz for gzip compression when available

   - Move gen_init_cpio and gen_initramfs.sh from usr/ to scripts/

   - Add a TOML configuration to scripts/container and update
     documentation accordingly

   - Add a generic, opt-in mechanism for checking that headers can
     compile standalone

   - Update machine function splitting argument for LLVM 24 rename in
     AutoFDO build

   - Miscellaneous minor clean ups, fixes, and hardening, mostly
     found/fixed via LLMs

  Kconfig:

   - Turn recursive ranges into an error

   - Fix savedefconfig for defaults that were adjusted for being out of
     range and revert a previously applied workaround

   - Improvements and hardening around integer and hexadecimal values

   - Emit a warning when selecting or implying a choice symbol, which
     has no effect

   - Emit a warning when KCONFIG_PROBABILITY is malformed

   - Introduce def_string / def_int / def_hex to mirror the existing
     def_bool / def_tristate to succinctly set a value and type at the
     same time.

   - Add cc-option-str and ld-option-str macros to act like Kbuild's
     cc-option and ld-option, returning the string when the option is
     supported, allowing many checks to be moved into Kconfig

   - Minor usability improvements to Kconfig tests"

* tag 'kbuild-7.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux: (59 commits)
  kbuild: AutoFDO: Update split machine function argument for upstream rename
  kbuild: Mark the initcall ordering recipe as recursive
  kbuild: Mark the gzip recipes as recursive
  localmodconfig: Explicitly report boolean configs during integrity check
  scripts/make_fit: Ignore schema checks when decomposing DTBs
  clang-tools: Import os for broken pipe handling
  kconfig: use the value argument in sym_get_near_range_bound()
  kconfig: fix infinite loop in conf_choice() when no choice value is visible
  kconfig: cleanup stale comment about 'keys' array
  kconfig: Improve ergonomics around disabling warnings with cc-option
  scripts/Kconfig.toolchain: Drop compiler symbols from dependencies
  checkkconfigsymbols.py: Parse Git status output line by line
  gen_init_cpio: stop parsing on lines with a missing argument list
  .gitignore: drop Module.markers
  .gitignore: ignore pacman package signatures
  kbuild: add header check facility as a manually run static analyzer
  Documentation: dev-tools: refer to user ID rather than id
  Documentation: dev-tools: update container.rst with config file
  scripts: add TOML config to container tool
  kconfig: tests: {old,save}defconfig: Forward dynamic keyword arguments
  ...
2026-10-21 10:45:44 -10:00
Linus Torvalds
14c8977403 It was a reasonably busy cycle for docs; changes include:
- Add the AGENTS.md symbolic link into the top-level directory in the hopes
   that LLMs will read it and produce better work.
 
 - The removal of the Assisted-by requirement, as concluded at the
   Maintainer Summit.
 
 - A new "testdocs" make target to run the documentation unit tests.
 
 - Lots of work on the Portuguese, and Chinese translations; there is now
   maintenance for the traditional Chinese translations as well.
 
 ...and the usual pile of minor fixes, documentation updates, etc.
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCgAtFiEEIw+MvkEiF49krdp9F0NaE2wMflgFAmrXgRMPHGNvcmJldEBs
 d24ubmV0AAoJEBdDWhNsDH5YAScH/28hCTx+pChtG42h1qHi+4zMHvJG3L7he2pG
 NxCJnrSPdYOEtdF7QW31he//D5KS0utAf3lxka3kouG/wAyFnXPi4BvSZNlq63FS
 9SSrcVfp3QZiLo/5Q8vvKX4USKLaGo6JxgI9yQBoYejUTYpfejbP+8Xu+qP3qnd8
 pPFLZGGtc4QBJveJH9F7LKJo8VcgHD8S8YdwOPUnJ5N7m1A8+ZN3JPONO6UZYQip
 3STTENdk/fNrfONuyS6URN6Z3TA8+63IpPS5u3+jfCqohxDqeHpnj0Ibsh2pVd8V
 rpz3YaOYRiMlXVe5uIXxqywf6XzdMRW34NWwB+X/F87hxQFm0Lo=
 =Sk+o
 -----END PGP SIGNATURE-----

Merge tag 'docs-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux

Pull documentation updates from Jonathan Corbet:
 "It was a reasonably busy cycle for docs; changes include:

   - Add the AGENTS.md symbolic link into the top-level directory in the
     hopes that LLMs will read it and produce better work

   - The removal of the Assisted-by requirement, as concluded at the
     Maintainer Summit

   - A new "testdocs" make target to run the documentation unit tests

   - Lots of work on the Portuguese, and Chinese translations; there is
     now maintenance for the traditional Chinese translations as well

  ...and the usual pile of minor fixes, documentation updates, etc"

* tag 'docs-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux: (125 commits)
  docs: add AGENTS.md as a symlink to README
  docs: kernel-parameters: adjust reserve_mem build option
  docs: kernel-parameters: don't mix cmdline options & build options
  docs/ja_JP: translate submitting-patches.rst (review and report tags)
  docs: remove Assisted-by references
  Revert "docs: landlock: fix ENOMSG condition in create_ruleset kernel-doc"
  docs: kdoc: parse context_lock_struct() as struct declaration
  docs: timers: hpet: Document userspace API and ioctl commands
  docs: timers: hpet: Modernize file references
  Documentation: sched-preemption: Add a SPDX license identifier
  docs: ABI: sysfs-driver-xdata: use literal blocks for console commands
  docs: ABI: sysfs-class-firmware-attributes: use literal blocks for console commands
  ABI: sysfs-bus-usb: use literal block for console commands
  docs/ABI: testing: aspeed-uart-routing: use literal block for console commands
  docs: kernel-parameters: fix a truncated sentence for preempt=lazy
  sched/doc: add a preemption model overview
  docs: ABI: TMR manager: use literal blocks for console I/O
  Documentation/ABI: IFS: use literal blocks for commands
  ABI: sysfs-bus-pci: use literal blocks for commands
  docs: translations: pt_BR: add debugging index
  ...
2026-10-20 14:38:38 -10:00
Arnaldo Carvalho de Melo
9c0c6f47b7 Merge remote-tracking branch 'torvalds/master' into perf-tools-next
First merge in the window to do a first batch of header syncs and check
integration with libbpf, etc.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2026-10-20 11:47:06 -03:00
Kumar Kartikeya Dwivedi
7418d9291c
libbpf: Render the arena type tag with its address space
A kernel struct member tagged __arena reaches a program built against
vmlinux.h as a pointer in the kernel address space with the tag as an
attribute, struct scx_cmask __attribute__((btf_type_tag("arena"))) *cmask,
so the program casts it to the arena address space before dereferencing
it, although the verifier already hands it an arena pointer.

The tag denotes a pointer into the BPF arena, address_space(1) to the
compiler, so render it as such: __attribute__((address_space(1)))
followed by the tag attribute as before, which keeps the dump faithful
to the BTF. A program then reads the member as an arena pointer and
hands it to anything declared with its own __arena without a cast.

Nothing else changes for consumers of the C dump. Skeletons strip the
modifiers of variable types and never carried the tag. Clang without
casts between address spaces still accepts the declaration and only a
program touching such a member would need a cast there, as it did for
every arena pointer before. GCC has no address spaces for BPF and warns,
under -Wattributes, that it ignores the attribute, which the selftests
build already silences.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261019132203.3034698-15-memxor@gmail.com
2026-10-19 19:45:03 +00:00
Kumar Kartikeya Dwivedi
cd7a395838
libbpf: Bind the arena of an object to every program in it
A program has an arena only when one of its insns refers to the arena
map. The kernel can hand a program arena pointers all the same: as
__arena struct_ops arguments, through __arena struct members and as
kfunc return values. The verifier rejects those in a program without an
arena, since it has nowhere to place them, and so a struct_ops callback
that touches nothing but its arguments fails to load until it refers to
the arena artificially, e.g. with asm volatile("" :: "r"(&arena)).

Bind the arena of an object to every program loaded from it. The kernel
binds the maps of a continuous fd_array (fd_array_cnt) like maps an insn
refers to, so pass the arena in slot 0, which is free: a kfunc insn with
offset 0 names vmlinux BTF rather than a slot. The module BTF slots that
follow stay as they are. A feature probe keeps older kernels, which
reject a non-zero fd_array_cnt, on the previous behavior.

An object has at most one arena, libbpf refuses a second one, so there
is no ambiguity about which arena a program gets. Programs that never
touch the arena see it among their maps, and pay for the arena base
setup in their JIT prologue until a later patch spares them that; the
arena now lives as long as any program of the object. With the count,
the kernel also resolves every module BTF slot of the array for every
program of the object, so each of them holds the modules the object
calls into, not only the programs that call them. Where the JIT is
disabled or the loader lacks CAP_PERFMON, every program of such an
object now fails to load, since an arena needs both, where before only
the programs using the arena did. bpf_program__clone() binds the arena
the same way, so a program loads through veristat as it does through
bpf_object__load(). An arena exclusive to another program and the
light skeleton loader are left alone.

An arena exclusive to the program itself is bound only where the kernel
checks an exclusive map against the digest of the final program. Earlier
kernels compared a map bound through the fd_array against a digest they
had not computed yet and refused the load for the very program the map
is exclusive to, so a second probe loads a program with a map exclusive
to it through fd_array_cnt; where that fails, the program keeps binding
its arena through its insns, as before.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261019132203.3034698-12-memxor@gmail.com
2026-10-19 19:45:03 +00:00
Kumar Kartikeya Dwivedi
b1eaee8b9a
libbpf: Keep the fd_array fill index out of bpf_program__clone()
bpf_program__clone() hands the kernel the fd_array of the object along
with obj->fd_array_cnt when the caller supplies neither. That counter is
not a count the kernel can use: it is the index of the next free slot
of an array built for the sparse fd_array API, where module BTF fds
start at slot 1 and slot 0 stays empty, since a kfunc insn with offset
0 names vmlinux BTF rather than a slot. A non-zero fd_array_cnt turns
the array into a continuous one, which the kernel resolves slot by
slot, and slot 0 holds 0. Cloning a program from an object that calls a
module kfunc thus fails with "fd 0 is not pointing to valid bpf_map or
btf", for every program of the object, which is what veristat runs into
on such an object. bpf_object__load() passes the array without a count.

Do the same in bpf_program__clone(), leaving the count to callers who
pass an array of their own, and rename the counter to fd_array_used so
that it no longer reads like the uapi field.

Fixes: 970bd2dced ("libbpf: Introduce bpf_program__clone()")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://patch.msgid.link/20261019132203.3034698-4-memxor@gmail.com
2026-10-19 19:45:03 +00:00
Ihor Solodrai
3ead4ead07 libbpf: Start v1.9 development cycle
libbpf 1.8.0 has been released [1].

Update libbpf.map and libbpf_version.h to start v1.9 development cycle.

[1] https://github.com/libbpf/libbpf/releases/tag/v1.8.0

Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261005183511.201045-1-ihor.solodrai@linux.dev
2026-10-15 00:08:10 +02:00
Kiryl Shutsemau (Meta)
b79d51e92e selftests/mm: add khugepaged completion barrier helper
A khugepaged test has to tell "not collapsed" from "not scanned yet", and
nothing in the selftests can.  wait_for_scan() in khugepaged.c comes
closest: it polls full_scans until the counter has advanced by two, since
the pass in progress may already have passed the test's mm.  But it only
returns in time if scan_sleep_millisecs happens to be short, and it is
private to that one test.

Add khugepaged_full_pass() to hugepage_settings, built on the same
advance-by-two wait but driven through sysfs: a store to
scan_sleep_millisecs wakes the daemon, so the barrier completes whatever
the scan cadence.  A store made while the daemon is scanning rather than
sleeping is lost, so the helper keeps storing until the pass lands.

One wake completes one pass only if pages_to_scan covers every mm on the
list, so callers need it large.

Settings pushes must not start passes of their own.  A store to either
sleep knob wakes the daemon, so thp_write_settings() now writes a
khugepaged knob only when its value changes.

Link: https://lore.kernel.org/20260919002451.496763-11-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Assisted-by: LLM
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Barry Song <baohua@kernel.org>
Cc: "David Hildenbrand (arm)" <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Hugh Dickens <hughd@google.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: kernel-team@meta.com
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Nico Pache (Red Hat) <nico.pache@linux.dev>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: "Vlastimil Babka (SUSE)" <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
2026-10-12 21:26:35 -07:00
Kiryl Shutsemau (Meta)
ceffb8ff83 selftests/mm: skip khugepaged page cache cases without a PMD folio
The page cache caps folio order at MAX_PAGECACHE_ORDER, which is below the
PMD order on arm64 with 64K pages, where a PMD is 512M.  A PMD-sized page
cache folio is impossible there, so the kernel refuses these collapses:
MADV_COLLAPSE answers -EINVAL and khugepaged passes over the range.  Four
shmem cases ask for a PMD-sized folio anyway, fail, and the run bails out
in the middle.

Skip the shmem and file mem types where the cap is below the PMD order.

The cap is not shmem-specific: it applies to every file folio.  Add
thp_file_supported_orders() to read the orders the page cache allows.

Anonymous collapse is unaffected: its orders are not capped this way.

Link: https://lore.kernel.org/20260919002451.496763-5-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Assisted-by: LLM
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Barry Song <baohua@kernel.org>
Cc: "David Hildenbrand (arm)" <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Hugh Dickens <hughd@google.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: kernel-team@meta.com
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nico Pache (Red Hat) <nico.pache@linux.dev>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: "Vlastimil Babka (SUSE)" <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
2026-10-12 21:26:33 -07:00
Sarthak Sharma
32449ec2cc tools/lib/mm: move hugepage_settings out of selftests
Move hugepage_settings.[ch] from tools/testing/selftests/mm/ to
tools/lib/mm/ so the THP and HugeTLB helpers can be shared more easily
between selftests and other tools.

Keep the helpers exposed to mm selftests through vm_util.h where possible,
and use direct <mm/hugepage_settings.h> includes for files that do not
include vm_util.h.  Adjust the selftests/mm build to compile the moved
implementation from its new location.

Remove the remaining kselftest dependency by including file_utils.h
directly and using EXIT_FAILURE in the signal handler.

Link: https://lore.kernel.org/20260918112234.195857-4-sarthak.sharma@arm.com
Signed-off-by: Sarthak Sharma <sarthak.sharma@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Zi Yan <ziy@nvidia.com>
2026-10-12 21:26:30 -07:00
Sarthak Sharma
1d59dd1383 tools/lib/mm: add shared file helpers
Move read_file(), write_file(), read_num(), write_num() and
write_num_ignore_einval() out of tools/testing/selftests/mm/vm_util.c into
a new shared helper under tools/lib/mm/.

These helpers are used by mm selftests today and will also be needed by
shared hugepage helpers in subsequent patches.  Move them to a generic
location so they can be reused outside selftests as well.

Keep the helpers exposed to mm selftests through vm_util.h by including
the new shared header there, and link the new helper into the selftests/mm
build.

Update the explicit x86 protection_keys 32-bit and 64-bit build rules to
preserve prerequisite paths, now that file_utils.c is built from
tools/lib/mm.

Add tools/lib/mm/ to the MEMORY MANAGEMENT - MISC entry in MAINTAINERS.

Link: https://lore.kernel.org/20260918112234.195857-3-sarthak.sharma@arm.com
Signed-off-by: Sarthak Sharma <sarthak.sharma@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Zi Yan <ziy@nvidia.com>
2026-10-12 21:26:30 -07:00
Christian Brauner
fd0349aadb
pid: add helpers to operate on a struct pid array
We're about to extend af_unix sockets and coredump code with the ability
to hand out thread-specific pidfds. Add a few simple helpers that allow
to operate on multiple struct pids up to PIDTYPE_MAX with automatic
bounds checking.

Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Andrew Halaney <andrew@amutable.com>
Link: https://patch.msgid.link/20261005-work-unix-passpidfd-v4-1-350183b6e02a@amutable.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
2026-10-12 10:58:20 +02:00
Alan Maguire
5876080366
btf: Preserve string relocation maps for split BTF
Make btf_relocate() optionally return the string-offset map it builds
while relocating a BTF relative to a distilled base. A subsequent split
BTF object can use that map to rebase references to base strings.

Also account for the accumulated base string offset when assigning a
new BTF base. This is required when the base is itself split.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009112324.1442319-5-alan.maguire@oracle.com
2026-10-11 14:18:06 +00:00
Alexei Starovoitov
373e1223ff
libbpf: Keep format strings of bpf_printk() in .rodata.str
bpf_printk(), BPF_SNPRINTF(), BPF_SEQ_PRINTF() and bpf_stream_printk()
copy the format into a static const array, which the compiler puts into
.rodata. When global data of the object is in arena .rodata is there
too and the helper doesn't take the format:

  R1 type=scalar expected=fp, pkt, pkt_meta, map_key, map_value, mem,
  ringbuf_mem, buf, trusted_ptr_, ctx

String literals are in .rodata.str1.1, which libbpf keeps in a read-only
map. No compiler flag moves a named array there. Put the arrays into
.rodata.str with a section attribute.

An object that uses the macros gets one more map, .rodata.str.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-14-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-10-03 16:08:20 +02:00
Alexei Starovoitov
8ab58ed42b
libbpf: Keep global data in arena when the object has .arena.data
All memory of a Rust program is arena, global data included. There are
no address spaces in Rust to say so. LLVM also drops bounds checks of
indexes into constant tables of core that it proved, so the verifier
can't check the access as access to map value.

The object asks for it with a section named .arena.data. What is in
the section doesn't matter:

  #[used]
  #[link_section = ".arena.data"]
  static DATA_IN_ARENA: u8 = 0;

or in C:

  char data_in_arena SEC(".arena.data");

There is nothing to tell to libbpf, so bpftool, veristat and other
loaders work with such objects as they are. When the object has the
section:
- .data, .bss and .rodata sections are appended to arena data after
  __arena variables, at the alignment of the section. Like __arena
  variables the data is at the end of arena.
- relocations of insns against the sections become relocations against
  the arena map.
- array maps of the sections are not created.
- when the object has no arena map libbpf creates one that is as large
  as the data. bpf_object__find_map_by_name(obj, "arena") finds it and
  bpf_map__set_max_entries() makes room for allocations. __arena
  variables still need the arena map to be declared.

Read-only sections that stay frozen array maps:
- .data.rel.ro and sections that have relocations in them. The kernel
  recognizes pointers to functions in them by value for callx.
- .rodata.str*. They hold const strings for __str arguments of kfuncs.
  A copy of them is in arena too.

The program dereferences pointers that it loads from data, so pointers
to data that are stored in data become addresses of arena, wherever the
pointer is. The arena map is created ahead of the other maps then, since
its address goes into their data. A pointer to .rodata.str* points to
the copy. A pointer to a section that is not in arena fails the load:

  sec '.data': pointer to 'tbl' at offset 8 can't be resolved:
  sec '.data.rel.ro' is not in arena

So does an arena that is pinned or reused. libbpf doesn't mmap it, so
its address is not known.

Programs of the object are loaded with BPF_F_ARENA_SCALAR, since they
access the data through plain numbers.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-13-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-10-03 16:08:20 +02:00
Luis Vieira
acff58e305
libbpf: Fix cleanup on invalid CO-RE relocation offsets
bpf_object__relocate_core() allocates a CO-RE candidate cache before
processing relocation records. Two instruction offset validation
failures return -EINVAL directly instead of going through the common
cleanup path.

These checks were moved into bpf_object__relocate_core() when
bpf_core_apply_relo() was split, preserving direct returns that were
safe in the previous helper but now bypass cleanup.

Route both failures through the out path so the candidate cache and
target BTF override are released.

Signed-off-by: Luis Vieira <luisflavieira@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260929-libbpf-core-relo-cleanup-v1-1-c9cd0b5775a9@gmail.com
2026-09-30 09:45:19 +00:00
Ihor Solodrai
76ea312390
libbpf: Render decl_tags in btf_dump
BTF_KIND_DECL_TAG is an attribute on a declaration: with kind_flag=0
the tag name is the argument of a btf_decl_tag(), and with kind_flag=1
it encodes a complete __attribute__.

When doing btf_dump, render both forms at every declaration btf_dump
emits - a record, a record member and a typedef:

	struct foo { ... } __attribute__((bar));
	struct foo { int a __attribute__((bar)); };
	typedef struct { ... } __attribute__((bar)) foo_t;

A decl tag is a standalone type pointing at its target. Build an
index of decl tags in btf_dump_resize(). Keep it as a flat array
sorted by (target ID, tag ID). This allows for a stable emission order
in btf_dump_emit_decl_tags().

btf_dump_emit_decl_tags() binary searches for where the target's
entries would begin and walks tags while the target matches. A record
shares its target with its members, so the component_idx is matched
there.

A record attribute goes after the closing brace, where
__attribute__((packed)) already goes. A member attribute goes after
the bit-field width: clang rejects one between the declarator and the
':'. A typedef takes it after the declarator, so a typedef of an
anonymous record can carry two groups at once, one binding to the
record and one to the typedef.

This closes a generic gap in btf_dump, which silently dropped every
decl tag it was given.

Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260929234608.48503-3-ihor.solodrai@linux.dev
2026-09-30 09:04:25 +00:00
Ihor Solodrai
1419795b98
libbpf: Walk types in btf_dump_resize(), not in mark_referenced()
btf_dump_mark_referenced() both walks every type added since the last
resize and decides what each one references. Move the walk out to
btf_dump_resize() and hand the function one type at a time, so a second
per-type job can share the same pass instead of adding another one.

No functional change.

Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260929234608.48503-2-ihor.solodrai@linux.dev
2026-09-30 09:04:25 +00:00
Omokefe Emmanuel Onanaroghene
2d72a4c098 docs: kdoc: parse context_lock_struct() as struct declaration
kernel-doc fails to parse struct declarations wrapped in
context_lock_struct(), e.g. struct debugfs_cancellation:

  include/linux/debugfs.h:245 ... error: Cannot parse struct or union!

Add a struct transform that expands context_lock_struct() to a plain
struct declaration, and apply the struct transforms to the whole
declaration before split_struct_proto(), matching what dump_var() and
dump_function() already do for their prototypes. The member-only
transform in dump_struct() is then redundant.

Add a regression test for the debugfs_cancellation declaration.

Fixes: 6e530e2e31 ("debugfs: Make debugfs_cancellation a context lock struct")
Signed-off-by: Omokefe Emmanuel Onanaroghene <emmaonana18@gmail.com>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260925234054.128477-1-emmaonana18@gmail.com>
2026-09-29 16:59:29 -06:00
Andrii Nakryiko
d07d3efd01
libbpf: Fix static linking of externs placed in allocated sections
Clang puts an extern variable that ends up with a section, an extern
map in .maps or an arena global in .addr_space.<N>, into the BTF
DATASEC of that section, next to the object's own definitions and
with offset 0. Unlike .kconfig and .ksyms, that section is a real
allocated ELF section whenever the same object also defines a
variable in it. The static linker handles such externs incorrectly in
two ways.

The output symbol of an unresolved extern gets the destination
section's index and offset instead of staying SHN_UNDEF with a zero
value. For ephemeral sections both are 0, so this only shows up for
allocated sections, where the extern turns into a size-zero NOTYPE
definition. Linking the output again then fails with "conflicting
non-weak symbol" against a strong definition of that variable, or,
against a weak one, keeps the bogus symbol and resolves every
reference to offset 0 of the section. And if the section of the
object with the extern doesn't start at offset 0 of the output
section, the output symbol fails linker_sanity_check_elf_symtab() on
the next link with "invalid extern symbol" before any of that.

When the extern is resolved by a later object in the same link, the
DATASEC entry appended for the extern keeps its placeholder offset,
as linker_append_btf() only updates the size of an existing entry.
The ELF symbol and the BTF secinfo then disagree and skeleton
generation fails with:

  Error: Something is wrong for .addr_space.1's variable #1: need offset 0, already at 8.

Keep unresolved externs SHN_UNDEF with a zero value in the output
symbol table, and once an extern is resolved take the offset of its
DATASEC entry from the ELF symbol, which already accounts for weak
vs. strong resolution.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260926000243.2830819-1-andrii@kernel.org
2026-09-26 08:01:41 +00:00
Lorenzo Stoakes (ARM)
48ebe1cbbe
kbuild: compress the kernel with pigz if available
The gzip step of a kernel build is very lengthy, especially for larger
builds such as allmodconfig.

gzip itself cannot be run in parallel, however an alternative tool exists
that can, providing the same feature set as gzip itself - pigz - which
works as a drop-in replacement.

On a 128-core Threadripper, gzip -9 of a 36 MiB x86-64 vmlinux.bin takes
1.6s, and with pigz it takes 0.09s, so the performance increase is
significant.

Therefore introduce the KPGZIP build variable which specifies the
compressor to use for the kernel image and default it to pigz if available,
or KGZIP otherwise (which itself defaults to gzip).

Modules, packaging and the embedded kernel configuration (CONFIG_IKCONFIG)
continue to use KGZIP which defaults to a single-threaded gzip instance.

Module compression is run in parallel by make, so pigz offers no benefit
there, packaging sits outside the build and the embedded configuration is
too small to matter.

pigz is executed using scripts/jobserver-exec, which reserves the job slots
make has free - necessary, as otherwise pigz would just grab unbounded
CPUs.

The kernel image compression step is always performed at the end of the
build so should typically have all job slots available.

pigz always provides byte-for-byte identical output across runs regardless
of thread count, but pigz and gzip are not guaranteed to generate the same
binary output, so reproducible builds must use the same tools consistently.

Also update Documentation/kbuild/kbuild.rst to describe KPGZIP and
Documentation/process/changes.rst to list the optional pigz dependency.

While there, add entries for the other compressor programs.

Finally, update scripts/ver_linux to include the pigz version there.

Every x86 build ends with the compression of vmlinux.bin, 36MB for
defconfig and over 200MB for allmodconfig, no-op builds are unchanged.

Whole build, 128-thread Threadripper 9980X, best of N runs:

                                         before   after     delta
                                         -------------------------------
  x86 defconfig, touch mm/vma.c, gcc        7.7s     6.0s     -1.7s (-22%)
  x86 defconfig, touch mm/vma.c, clang      6.9s     5.4s     -1.5s (-21%)
  x86 defconfig, clean, gcc                28.2s    26.3s     -1.8s (-6%)
  x86 defconfig, clean, clang              28.7s    27.2s     -1.5s (-5%)
  x86 allmodconfig, touch mm/vma.c, gcc    23.7s    15.4s     -8.3s (-35%)
  x86 allmodconfig, touch mm/vma.c, clang  22.5s    15.6s     -7.0s (-31%)
  x86 allmodconfig, clean, gcc            294.1s   278.6s     -15.5s (-5%)
  x86 allmodconfig, clean, clang          283.9s   266.8s     -17.1s (-6%)

Suggested-by: Kees Cook <kees@kernel.org>
Link: https://zlib.net/pigz/
Assisted-by: LLM
Reviewed-by: Manuel Ebner <manuelebnerli@mailbox.org>
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260923-build-speedup-v4-22-73128809a4a4@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
2026-09-26 00:16:36 +01:00
Kumar Kartikeya Dwivedi
9054903ab5
libbpf: Add bpf_prog_stream_open()
Expose the BPF_PROG_STREAM_OPEN command through a low-level libbpf wrapper.
The options structure carries stream open flags, including non-blocking
mode, while leaving room for future extensions.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20260925045536.1480933-4-memxor@gmail.com
2026-09-25 21:04:35 +00:00
Nia Su
aa9265d507 perf evlist: Don't restrict uncore events to PMU CPUs
The PMU CPU map has different semantics for core and uncore PMUs.

For core PMUs, pmu_cpus contains the CPUs on which the event can be
opened. Intersect the user-requested CPU map with the PMU CPU map to
reject CPUs unsupported by the PMU.

For uncore PMUs, pmu_cpus is read from the PMU's cpumask sysfs attribute,
which lists the default CPU to open the event on per socket/die, not the
set of CPUs it can only be opened on. The user-requested CPU may override
these defaults, and the driver can redirect the event to the appropriate
control CPU.

Currently, __perf_evlist__propagate_maps() intersects the user-requested
CPU map with pmu_cpus for all PMUs. If the user-requested CPU is not in
pmu_cpus, the intersection becomes empty and the evsel is removed before
perf_event_open() is called.

Add evsel->is_pmu_uncore, set once from pmu->is_uncore at parse time,
to skip the intersection for uncore PMUs. Core PMUs, and other
non-core PMUs such as software events, breakpoints, or tracepoints,
continue to be restricted to their PMU CPU map.

Fixes: 811082e4b6 ("perf parse-events: Support user CPUs mixed with threads/processes")
Reviewed-by: Ian Rogers <irogers@google.com>
Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Nia Su <nia.su@sifive.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2026-09-25 15:29:56 +02:00
Alexei Starovoitov
0e4cf80d0d
Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf 7.3-rc4
Cross-merge BPF and other fixes after downstream PR.

Conflicts:
  kernel/bpf/helpers.c
  tools/testing/selftests/bpf/prog_tests/cb_refs.c
  tools/testing/selftests/bpf/prog_tests/verifier.c

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-24 16:15:14 +00:00
Alan Maguire
dad2fcc644
libbpf: Add support for BTF kinds LOC[_PARAM|_PROTO|SEC]
Add support for new kinds to libbpf.  BTF_KIND_LOC_PARAM and
BTF_KIND_LOC_PROTO are dedup-able so add support for their
deduplication, whereas since BTF_KIND_LOCSEC contains a unique
offset it is not.  LOC_PARAM is considered a primary type
since it contains no external references; LOC_PROTO is a
reference type consisting of LOC_PARAM references so they
are handled in the primary and reference dedup phases
respectively.

For BTF field iteration, BTF_KIND_LOCSEC needs 2 m_offs[] values
for the associated KIND_FUNC and KIND_LOC_PROTO type ids in
each LOCSEC entry.

Add APIs to add location param, location prototypes and location
sections and btf_is_* tests, data accessors for each.

For BTF distillation we add location info to split BTF.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260924111428.75957-3-alan.maguire@oracle.com
2026-09-24 15:59:45 +00:00
Alexei Starovoitov
9f99fc05fc
libbpf: Support pointers to functions in read-only data in light skeleton
Light skeleton doesn't create maps. The loader prog does at run time.
Teach gen_loader to create a copy of .rodata map with offsets of
functions for a prog, see create_func_ptr_map().

- Such maps are not maps of the object and are not in the loader ctx.
  Reserve slots in fd_array after the maps of the object for them and
  close their fds when the loader is done or fails. The progs hold them.
  Which progs have callx is not known when the loader is initialized,
  so reserve for every prog if .text has callx and for the progs that
  have callx otherwise.

- Add bpf_gen__func_ptr_map_create() that emits map create, update,
  freeze and returns the index in fd_array for ld_imm64
  BPF_PSEUDO_MAP_IDX_VALUE.

- User space can set .rodata before loading the skeleton, see
  initial_value in bpf_map_desc. Copy it into the copy of the map as
  well and store the offsets of functions on top, since user space
  doesn't have them.

- Light skeleton can be generated for a target of another endianness.
  Store the offsets in the byte order of the object.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-15-alexei.starovoitov@gmail.com
2026-09-24 05:39:49 +00:00
Alexei Starovoitov
6ec77f0a76
libbpf: Treat .data.rel.ro as read-only data
PIC code keeps constants with pointers, e.g. vtables, in .data.rel.ro
instead of .rodata. Nothing writes there after relocation.
libbpf treats every .data* section as writable, while the kernel looks
for pointers to functions in frozen read-only maps only.

Make .data.rel.ro and .data.rel.ro.* maps read-only and frozen like
.rodata and collect pointers to functions there.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-14-alexei.starovoitov@gmail.com
2026-09-24 05:39:49 +00:00
Alexei Starovoitov
b223044a68
libbpf: Resolve pointers to functions in read-only data
The kernel recognizes a pointer to a function in frozen read-only array
map by value: byte offset of a static function in the program.
Make it work for tables of functions, struct ops, vtables in .rodata:

  static const struct shape_ops square_ops = { 1, area, 10, perimeter };
  ...
  return ops->area(x) * ops->scale;

is compiled into

  r3 = *(u64 *)(r1 + 8)
  callx r3

where 'area' in .rodata is R_BPF_64_ABS64 relocation against .text.

- Collect relocations in .rodata* that are aligned pointers to
  functions. Relocations in data were ignored. The rest still are.

- Append all static functions that the section points to, since any
  of them can be called, if the prog refers to the section and has
  callx. callx cannot call global functions. Don't pull them in. Such
  pointer is NULL.

- Functions have different offsets in different progs, so create a copy
  of the map per prog and store the offsets there. The kernel replaces
  them with addresses at load and the prog must be the only user of
  the map. The map of the section itself is left as-is.

- Progs without callx cannot call them and the kernel doesn't look for
  pointers in their data. They keep using the map of the section and
  nothing is appended to them.

- The kernel treats any aligned u64 that matches an offset of a static
  function as a pointer. Warn if data has one that isn't.

Light skeleton is not supported yet.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-13-alexei.starovoitov@gmail.com
2026-09-24 05:39:49 +00:00
Alexei Starovoitov
296fd47827
libbpf: Support pointers to static functions in data when linking
A pointer to a static function in data, e.g.

  static int (* const handlers[])(void *ctx) = { foo, bar };

is R_BPF_64_ABS64 relocation against .text section symbol with the offset
of the function stored in place. The linker rejects it:

  relocation against STT_SECTION in non-exec section is not supported!

so an object with a table of static functions cannot go through
'bpftool gen object', which all selftests do.

Functions move by the offset of input .text in the output section.
Add it to the pointer. Swap bytes if the object is of foreign endianness,
since data sections are kept in the byte order of the object.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-12-alexei.starovoitov@gmail.com
2026-09-24 05:39:49 +00:00
Luis Vieira
6892723c5e libbpf: Validate ELF symbol table entry size
elf_sym_iter_new() calculates the number of symbols by dividing the
symbol table data size by sh_entsize. A malformed ELF file with a zero
sh_entsize causes a division by zero.

Reject symbol table sections with a zero entry size before calculating
the number of symbols.

Signed-off-by: Luis Vieira <luisflavieira@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260916-libbpf-elf-entsize-fix-v1-1-cf181da33eab@gmail.com
2026-09-21 14:21:36 -07:00
Leon Hwang
a885470df1 bpftool: Compute map size of light skeletons at runtime
bpftool rounds memory-mapped data map sizes to the host page size when
generating a light skeleton. The generated code therefore uses a 64K
mapping size when bpftool runs on a 64K-page host, even if the skeleton
runs on a 4K-page target. The target rejects the oversized map mmap(),
causing failure of loading the light skeleton.

When try to run 64K-page selftests on 4K-page VM, the error message does
not provide the reason about page size.

 test_atomics:PASS:atomics skeleton open 0 nsec
 test_atomics:FAIL:atomics skeleton load unexpected error: -12 (errno 22)
 #15      atomics:FAIL

Pass the original map value size and max entries to the generated code and
round mmap size to the runtime page size in the user-space light
skeleton helpers. This keeps generated light skeletons independent of the
build host page size.

Fixes: d510296d33 ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Quentin Monnet <qmo@kernel.org>
Link: https://lore.kernel.org/bpf/20260915161450.96249-1-leon.hwang@linux.dev
2026-09-21 14:16:07 -07:00
Fuyu Zhao
34b1847924 libbpf: Support selective kernel module BTF loading via bpf_object_open_opts
Add btf_module_allowlist and btf_module_allowlist_cnt fields to
bpf_object_open_opts to limit which kernel module BTFs libbpf is
allowed to load.

When the option is not specified, the existing behavior remains
unchanged. An explicitly specified empty allowlist prevents libbpf from
consulting any kernel module BTFs.

The allowlist limits which kernel module BTFs libbpf will consult
wherever module BTF might be needed.

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Fuyu Zhao <zhaofuyu@vivo.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260915124104.77287-2-zhaofuyu@vivo.com
2026-09-21 09:26:40 -07:00
Martin KaFai Lau
6bad848e3f
libbpf: Support attaching struct_ops to a cgroup
Add bpf_map__attach_cgroup_opts() to attach a struct_ops map to a cgroup
through a BPF link.

Also extend struct bpf_prog_query_opts with a type_id field so a
BPF_STRUCT_OPS query on a cgroup can select the struct_ops type to
enumerate.

Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20260917200542.3689605-14-ameryhung@gmail.com
2026-09-19 05:32:56 +00:00
Kumar Kartikeya Dwivedi
b4e875d397 libbpf: Reject truncated ldimm64 CO-RE relocations
CO-RE relocation of an ldimm64 instruction operates on two instruction
slots. A malformed BPF ELF can end a function after the first slot and
attach a CO-RE relocation to it. libbpf allocates the instruction array
according to the function symbol size, so the shared relocation code would
then access beyond the allocation.

Reject a terminal ldimm64 in libbpf's relocation loop, where the program
length is available, before resolving or applying the relocation. Both
resolved and unresolved relocations validate the absent second slot, and
unresolved relocation poisoning would additionally write past the array.

The in-kernel caller is protected by the verifier's early instruction-stream
check before it applies CO-RE relocations.

Fixes: eacaaed784 ("libbpf: Implement enum value-based CO-RE relocations")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/20260914140852.03DA21F0089B@smtp.kernel.org
Link: https://patch.msgid.link/20260917233222.2542500-11-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-17 18:01:43 -07:00
Kumar Kartikeya Dwivedi
394ae39833 bpf: Restrict CO-RE poisoning to relocatable instructions
CO-RE relocation records can name any instruction offset. When a
relocation cannot be resolved, bpf_core_patch_insn() currently poisons its
target before checking whether that instruction is a valid relocation
target. Malformed metadata can therefore replace jumps, calls, exits,
register-source arithmetic, or non-immediate loads instead of failing at
the relocation step.

Handle poisoning only after the instruction has passed the same class and
operand-form checks used for a resolved relocation. Route invalid forms
through the existing diagnostic and return a hard error. Keep poisoning
supported instructions, including both halves of a plain ldimm64, so an
unresolved relocation in dead code remains valid.

Extend bpf_core_poison_insn() to poison both halves of ldimm64, and return
its status directly from each validated instruction case. This avoids
routing the success path through a common label and leaves the helper free
to report errors.

The shared relocation code applies this restriction to both libbpf and
in-kernel CO-RE.

Fixes: d7a252708d ("libbpf: Improve handling of failed CO-RE relocations")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260917233222.2542500-7-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-17 18:01:42 -07:00
Mingpei CAO
961b8946ac libbpf: Reject local struct_ops bitfields before data access
A local struct_ops type can contain a bitfield absent from the
corresponding kernel BTF type. bpf_map__init_kern_struct_ops() checks
whether data for an absent local member is zero.

For a bitfield, member->offset contains the bit offset and field width.
bpf_map__init_kern_struct_ops() used the encoded value to calculate the
member data pointer before rejecting the bitfield. The zero-data check
could therefore read outside st_ops->data and crash libbpf.

Reject local bitfields before calculating the member data pointer. Keep
the existing rejection for bitfields in the kernel type.

Fixes: c911fc61a7 ("libbpf: Skip zeroed or null fields if not found in the kernel type.")
Signed-off-by: Mingpei CAO <caomingpei@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260915162252.473044-2-caomingpei@gmail.com
2026-09-17 15:25:13 -07:00
Luis Vieira
8cca9a6bb6 libbpf: Fix program log buffer size validation
bpf_program__set_log_buf() checks the existing prog->log_size instead
of the incoming log_size argument when enforcing the UINT_MAX limit.
As a result, an oversized value can be accepted.

The setter also accepts a non-NULL log_buf with a zero log_size, while
bpf_prog_load() rejects mismatched log buffer and size values.

Validate the log buffer and size pair consistently with bpf_prog_load(),
and check the supplied log_size against UINT_MAX.

Signed-off-by: Luis Vieira <luisflavieira@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://lore.kernel.org/bpf/20260915-libbpf-log-buf-fix-v2-1-2feca3fa4842@gmail.com
2026-09-17 15:21:30 -07:00
Ian Rogers
3cbb16ccd7 perf record: Fix unhandled POLLHUP on non_perf_event descriptors
When processing POLLHUP on a non_perf_event control descriptor,
evlist__finalize_ctlfd() is invoked to finalize the setup, correctly
setting the core evlist's poll array file descriptor to -1. However,
this finalized teardown state is never propagated back to the
individual thread's local replica of the pollfd array.

Consequently, on the next iteration of the main recording loop,
record__update_evlist_pollfd_from_thread() performs a strict
equivalence check between the core evlist's array and the thread's
localized poll array, detecting that the fd values no longer match.
This causes an immediate -EINVAL abort and a premature teardown.

Fix the underlying logic within
record__update_evlist_pollfd_from_thread() to sustainably propagate
the finalized teardown statuses (-1) originating from the core evlist
back to the thread's localized poll structure. This correctly
maintains synchronization and entirely prevents the unhandled
index mismatch crashes.

Additionally, add a unit test that explicitly validates that
fdarray__filter() preserves its invariants regarding
fdarray_flag__nonfilterable items to guard against regressions.

Fixes: fb4751e79c ("perf record: Fix teardown hang on system-wide multi-threaded sessions")
Signed-off-by: Ian Rogers <irogers@google.com>
Assisted-by: Gemini:gemini-3.1-pro
Cc: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2026-09-17 12:27:34 -03:00
Toke Høiland-Jørgensen
3041e498f9 libbpf: Add bpf_program__add_flags() and bpf_program__clear_flags()
When changing BPF program flags, applications often need to just add or
remove a single flag, without touching the existing ones. When using the
bpf_program__set_flags() function, this requires reading the existing
flags and doing bitwise manipulations before resetting the result, which
is slightly awkward to do, and easy to forget.

Add two new convenience helpers, bpf_program__add_flags() and
bpf_program__clear_flags(), which wraps bpf_program__set_flags() in the
bitwise operations required to modify flags without clobbering the
existing ones.

Suggested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/bpf/20260912084109.432834-1-toke@redhat.com
2026-09-14 10:26:49 -07:00
Mahe Tardy
e18e82ba9e libbpf: fix log level propagation for light skeleton loaders
bpftool's -d option is documented to enable bpf_trace_printk() messages
from the generated syscall loader when used with -L as specified in
commit d510296d33 ("bpftool: Use syscall/loader program in "prog load"
and "gen skeleton" command.")

However commit b59e4ce8bc ("bpftool: Switch bpf_object__load_xattr()
to bpf_object__load()") changed bpftool to use bpf_object__load() which
call the internal bpf_object_load with extra_log_level to 0 instead of
bpf_object__load_xattr with the user request log_level.

All the plumbing was still there to generate the bpf_trace_printk()
instructions from the generator but was now unreachable because
bpf_gen__init() was called with extra_log_level to 0, leaving
gen->log_level at 0.

This uses the obj->log_level field introduced in commit e0e3ea888c
("libbpf: Allow passing user log setting through bpf_object_open_opts")
set from reading verifier_logs in do_skeleton(). This preserves both
object-level and explicit load-time logging settings.

Fixes: b59e4ce8bc ("bpftool: Switch bpf_object__load_xattr() to bpf_object__load()")
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Link: https://lore.kernel.org/r/20260907170536.9996-1-mahe.tardy@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-13 21:48:26 -07:00
Mykyta Yatsenko
a4c6292256 libbpf: Defer arena map size check to load time
init_arena_map_data() used the build-host page size while it opened an
object. This could reject skeleton generation when the build host and
target use different page sizes.

Validate the size during load, before libbpf calculates the arena data
offset. This uses the running kernel page size and rejects an
undersized map before relocation.

This problem manifested when cross compiling BPF selftests for
arm64 (64K page) on x86 (4K page). In this setup skeleton generation
fails.

Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260908-libbpf_arena_thing-v1-1-0092c0e0d91b@meta.com
2026-09-11 10:27:20 -07:00