The trampoline could be corrupted by the blindly
'tr->flags = BPF_TRAMP_F_TAIL_CALL_CTX' in verifier.
1. A fexit attached to a tail_call_reachable prog. 'tr->flags' became
'BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_TAIL_CALL_CTX'. And, the
trampoline would poke the target prog's nop insn using jmp insn instead
of call insn.
2. Another fexit loaded with the same tail_call_reachable prog target.
'tr->flags' became 'BPF_TRAMP_F_TAIL_CALL_CTX'.
3. Close the first fexit link. Due to no BPF_TRAMP_F_CALL_ORIG in
'tr->flags', the trampoline will fail to restore the prog's nop insn
using call insn.
[ 3.410719] WARNING: kernel/bpf/syscall.c:3551 at bpf_tracing_link_release+0x53/0x60, CPU#1: test_progs/98
...
[ 3.428793] bpf_link_free+0x58/0x130
[ 3.429293] bpf_link_release+0x23/0x30
Fix the warning by updating 'tr->flags' with '|=' and lock.
Fixes: 2b5dcb31a1 ("bpf, x64: Fix tailcall infinite loop")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Reviewed-by: Pu Lehui <pulehui@huawei.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260722151909.69142-2-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
BPF subprograms use the bpf_callback_t ABI, but task work invokes the
callback through a three-argument function pointer. This trips kCFI.
Store and invoke the callback as bpf_callback_t.
Fixes: 38aa7003e3 ("bpf: task work scheduling kfuncs")
Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://lore.kernel.org/bpf/20260724-task_work_cfi-v1-1-2616691781ed@meta.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
arch_bpf_trampoline_size() allocates a buffer to get actual size required
for a trampoline.
This buffer must be in the module address space because
__arch_prepare_bpf_trampoline() calculates rel32 offsets relatively to
that buffer.
In preparation for enabling ROX mode for EXECMEM_BPF make sure that the
allocated memory is writable.
Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for
buffer allocation in arch_bpf_trampoline_size().
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260716-execmem-x86-rox-bpf-v0-v3-4-4e76158c01c5@kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
execmem_alloc() can return ROX memory that is already filled with
architecture defined trapping instructions.
In preparation for enabling this mode for BPF on x86, make sure that there
is no redundant management of the ROX memory.
There is no need to fill allocated memory with trapping instructions, to
request permissions reset on free and to set ROX permissions as this all
is handled by execmem_alloc().
Add bpf_jit_mem_is_rox() wrapper for execmem_is_rox(), use it to check if
execmem_alloc() returns ROX memory and skip the redundant steps in that
case.
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260716-execmem-x86-rox-bpf-v0-v3-3-4e76158c01c5@kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_jit_alloc_exec() and bpf_jit_free_exec() are wrappers for the
corresponding execmem APIs.
Architectures define the properties of the memory range needed by BPF in
their initialization of execmem and don't need to override neither of
them.
Drop the __weak qualifier from bpf_jit_alloc_exec() and
bpf_jit_free_exec().
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260716-execmem-x86-rox-bpf-v0-v3-2-4e76158c01c5@kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_dispatcher->rw_image is a temporary writable buffer that
arch_prepare_bpf_dispatcher() fills and then copies into
bpf_dispatcher->image using bpf_arch_text_copy().
The rel32 offsets emitted by emit_bpf_dispatcher() are calculated against
->image, so ->rw_image does not need to live in the module address range.
Allocate ->rw_image with vzalloc() to avoid permissions dance when
EXECMEM_BPF will be backed by ROX caches.
Using vzalloc() rather than vmalloc() ensures that the memory that
bpf_dispatcher_update() unconditionally copies into the executable buffer
is zeroed, which is not ideal but still better than random memory returned
by the existing bpf_jit_alloc_exec() or plain vmalloc().
Switching from bpf_jit_alloc_exec() to vzalloc() also saves a bit of
space in the more scarce module address space.
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260716-execmem-x86-rox-bpf-v0-v3-1-4e76158c01c5@kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
BPF_REFCOUNT is not marked as a unique field, while it should be. Fix
this oversight.
Fixes: d54730b50b ("bpf: Introduce opaque bpf_refcount struct and add btf_record plumbing")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260719153634.2908692-4-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
btf_find_struct_field() initializes a fresh seen mask for every recursive
descent. Unique special fields in different levels of the same aggregate
therefore do not see one another. The duplicate fields can reach
btf_parse_fields(), where they trigger an invariant WARN_ON_ONCE(). A
crafted user BTF can consequently trigger the warning before map creation
checks capabilities.
Initialize the seen mask once in btf_find_field() and pass the same pointer
through struct, datasec, and nested-struct walks. This gives the entire field
traversal one shared uniqueness state.
Fixes: 64e8ee8148 ("bpf: look into the types of the fields of a struct type recursively.")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260719153634.2908692-3-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Sashiko pointed out correctly that the case statement for
BPF_RES_SPIN_LOCK incorrectly checks offset for BPF_SPIN_LOCK.
Fix it by checking res_spin_lock_off instead.
Fixes: 0de2046137 ("bpf: Implement verifier support for rqspinlock")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260719153634.2908692-2-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Queue and stack pop/peek helpers accept an uninitialized output buffer
because the verifier expects the helper to initialize it. The empty-map
error path clears the buffer, but a failed lock acquisition returns
-EBUSY without writing it.
Clear the output before returning -EBUSY so BPF programs cannot observe
uninitialized stack contents after a failed helper call.
Fixes: a34a9f1a19 ("bpf: Avoid deadlock when using queue and stack maps from NMI")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260719125419.1782196-1-memxor@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The BPF_MOV64_PERCPU_REG insn requires JIT to emit native code to for
'dst_reg = src_reg + <percpu_base_off>'.
However, the interpreter ignores the 'off' at its ALU64_MOV_X label.
The 'off' indicates the insn is BPF_MOV64_PERCPU_REG insn. Then, when
the interpreter loads memory from the register, it will hit a page
fault.
[ 2.545572] BUG: unable to handle page fault for address: ffffffffacaaf034
[ 2.546485] #PF: supervisor read access in kernel mode
[ 2.547167] #PF: error_code(0x0000) - not-present page
[ 2.547850] PGD 134e63067 P4D 134e63067 PUD 134e64063 PMD 10021c063 PTE 800ffffeca550062
[ 2.548912] Oops: Oops: 0000 [#1] SMP PTI
Set jit_required as true in order to disallow interpreter fallback in
core.c::__bpf_prog_select_runtime(), if any BPF_ADDR_PERCPU insn is
patched to the prog.
BTW, rename the helper bpf_map_supports_cpu_flags() to
bpf_map_is_percpu_map().
Fixes: 7bdbf74463 ("bpf: add special internal-only MOV instruction to resolve per-CPU addrs")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-4-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The interpreter does not recognize the BPF_JMP|BPF_JA|BPF_X insn, which
is used for insn_array map. Thereafter, it would hit the BUG_ON() in
___bpf_prog_run() at run time.
[ 2.563726] BPF interpreter: unknown opcode 0d (imm: 0x0)
[ 2.564557] ------------[ cut here ]------------
[ 2.565206] kernel BUG at kernel/bpf/core.c:2349!
[ 2.565882] Oops: invalid opcode: 0000 [#1] SMP PTI
Set jit_required as true when insn_array map is used in the prog in
order to disallow interpreter fallback for gotox insn in
core.c::__bpf_prog_select_runtime().
Fixes: 493d9e0d60 ("bpf, x86: add support for indirect jumps")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-3-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Since the interpreter does not support the arena-related insns,
interpreter fallback should not be allowed for these insns in
core.c::__bpf_prog_select_runtime().
Currently, when the interpreter executes the arena ST/LDX/STX insns,
it would hit the BUG_ON() in ___bpf_prog_run() at run time.
[ 2.579196] BPF interpreter: unknown opcode a2 (imm: 0x0)
[ 2.579998] ------------[ cut here ]------------
[ 2.580652] kernel BUG at kernel/bpf/core.c:2349!
[ 2.581314] Oops: invalid opcode: 0000 [#1] SMP PTI
Set jit_required as true when arena map is used in the prog to disallow
interpreter fallback for arena-related insns.
Fixes: 6082b6c328 ("bpf: Recognize addr_space_cast instruction in the verifier.")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260715141122.15783-2-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_arena_free_pages() accepts scalar arena addresses. The runtime
masks the address to the low 32 bits and reconstructs a full user
address from the arena base before returning the range to the arena
free tree.
When the scalar value is below the low 32 bits of the arena base,
full_uaddr falls below user_vm_start. The existing upper-end clipping
then turns this into an out-of-range free-tree offset. A later
allocation can reuse that offset and return an address below the arena
mapping.
Reject such frees before computing the clipped range.
Fixes: 317460317a ("bpf: Introduce bpf_arena.")
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717-c10-031-public-bpf-next-v2-b4-v2-1-54b555443a7c@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
arena is the only map type whose map_mem_usage() still returns 0, so
"bpftool map show" and fdinfo always showed 0 memlock for an arena no
matter how many pages it had.
Count the pages that are actually mapped into the arena: bump a counter in
apply_range_set_cb() when a page goes in and drop it in
apply_range_clear_cb() when a page goes out, both under the arena spinlock.
map_mem_usage() then just returns nr_pages << PAGE_SHIFT.
Only real data pages are counted, not the scratch page.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717114117.350851-3-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Replace the scratch_page field in the pte-callback data with the arena
pointer; later patches use other arena fields from these callbacks. No
functional change.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717114117.350851-2-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
check_kfunc_call() reads meta.func_name when bpf_fetch_kfunc_arg_meta()
returns -EACCES, but that error can come from fetch_kfunc_meta() (e.g.
fd_array_get_btf() rejecting BTF binding for a signed program) before
meta is memset(), leaving it uninitialized and risking a garbage deref
in verbose().
Move the memset() to the start of bpf_fetch_kfunc_arg_meta() so meta is
zeroed on every error return. The intended "not allowed" -EACCES path
still sets func_name first, so its message is unchanged.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-3-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
For a bloom filter, the value argument of bpf_map_peek_elem() is always
an input. Therefore, the verifier should not allow passing uninitialized
stack memory to it to avoid information leak.
bpf_map_peek_elem() tags its value argument ARG_PTR_TO_MAP_VALUE |
MEM_UNINIT, telling the verifier the callee fills the buffer. This holds
for queue/stack maps, but not for a bloom filter, which reads the buffer
as an input to test set membership and never writes it.
As a result, a program can pass an uninitialized stack buffer to
bpf_map_peek_elem() on a bloom filter. The verifier accepts it and marks
the buffer initialized on return, letting the program read back leftover
kernel stack memory. Bloom maps require CAP_BPF to create, so this is a
CAP_BPF-gated stack infoleak that bypasses the boundary CAP_BPF is meant
to enforce (arbitrary kernel reads are gated behind CAP_PERFMON).
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260715172127.2416388-2-ameryhung@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
- Fix a UAF in socket clone early bailout paths (Matt Bobrowski)
- Reject unhashed UDP sockets on sockmap update to prevent refcount leaks
(Michal Luczaj)
- Account for receive queue data in FIONREAD on sockmap sockets without a
verdict program (Mattia Meleleo)
- Reject negative constant offsets for verifier buffer pointers (Sun Jian)
- Fix for tracing of kfuncs with implicit arguments (Ihor Solodrai)
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQR0/74CM8h5+1E9pgtHLTd7Y1QvgwUCalp3/AAKCRBHLTd7Y1Qv
g9J/AQCJzidAmStp1i074sZC7sC1uq+aYIvwycpMTp6hVgyP+AEAw7oKcnGkPGXs
jeEDctaLtxBnPMXGAR5UaeqdNqu9ywQ=
=zQgn
-----END PGP SIGNATURE-----
Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf
Pull bpf fixes from Kumar Kartikeya Dwivedi:
- Fix a UAF in socket clone early bailout paths (Matt Bobrowski)
- Reject unhashed UDP sockets on sockmap update to prevent refcount
leaks (Michal Luczaj)
- Account for receive queue data in FIONREAD on sockmap sockets without
a verdict program (Mattia Meleleo)
- Reject negative constant offsets for verifier buffer pointers (Sun
Jian)
- Fix for tracing of kfuncs with implicit arguments (Ihor Solodrai)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
selftests/bpf: Cover tracing implicit kfunc args
bpf: Fix tracing of kfuncs with implicit args
selftests/bpf: Cover negative buffer pointer offsets
bpf: Reject negative const offsets for buffer pointers
selftests/bpf: Test FIONREAD on a sockmap socket without a verdict program
bpf, sockmap: Account for receive queue in FIONREAD without a verdict program
selftests/bpf: Fail unbound UDP on sockmap update
selftests/bpf: Adapt sockmap update error handling
bpf, sockmap: Reject unhashed UDP sockets on sockmap update
selftests/bpf: Ensure UDP sockets are bound
bpf: Fix UAF in sock clone early bailouts
Some JIT compilers, such as x86_64, rely on a register to pass the TCC.
When subprograms of synchronous callback invoke tailcall, C helpers
invoking bpf callback clobber this register, and the corrupted TCC may
bypass the TCC limit, leading to infinite tailcall.
Fix this by rejecting tailcall inside all subprogs of sync callback.
This also cleanly consolidates the existing async and exception callback
checks into a single unified `is_cb` check.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Reported-by: Björn Töpel <bjorn@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260716120157.835937-3-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Currently in check_max_stack_depth_subprog, when the verifier enters a
new callee branch, the local tail_call_reachable is not properly
synchronized with the callee's state.
Consider a main prog branching into multiple subprogs:
subprog0 -> tailcall
main <
subprog1 -> subprog2
When the verifier finishes checking subprog0 and backtracks to main
prog, the local tail_call_reachable state is left as true. As it
proceeds to subprog1, this uncleared state leaks into the new branch,
falsely marking subprog1 and subprog2 as tailcall reachable.
Fix this by explicitly syncing tail_call_reachable with the callee's
has_tail_call state on entry. The caller's state is safely preserved and
restored via the existing backtracking logic.
Fixes: ebf7d1f508 ("bpf, x64: rework pro/epilogue and tailcall handling in JIT")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Link: https://patch.msgid.link/20260716120157.835937-2-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
A kfunc marked with KF_IMPLICIT_ARGS flag takes implicit arguments
(such as bpf_prog_aux) that the verifier injects at load time.
resolve_btfids strips those from the kfunc's BTF-visible prototype and
keeps the real kernel ABI in a counterpart _impl prototype [1].
fentry/fexit/fmod_ret/fsession programs may attach to the BPF kernel
functions, including those with implicit args. However
bpf_check_attach_target() and bpf_check_attach_btf_id_multi() extract
the struct btf_func_model from the wrong BTF prototype of the
kfunc. The btf_func_model is later read to construct the trampoline,
which then causes the injected implicit argument to be clobbered and
the kfunc dereferencing garbage.
Add btf_attach_func_proto() to resolve the real ABI prototype of the
kfunc the way the call site does: by looking up the _impl prototype
for a KF_IMPLICIT_ARGS kfunc. Use it at both attach-target model
construction sites.
To enable this, make two supporting changes:
* pass bpf_verifier_log instead of bpf_verifier_env to
find_kfunc_impl_proto(), so it can be reused from the attach path
* add btf_kfunc_check_flag() to test a flag across all of a kfunc's
hook sets, because a program attaching to a kfunc is not in the
kfunc's call-set
KF_IMPLICIT_ARGS must be consistent across the sets, so
btf_kfunc_check_flag() returns -EINVAL on inconsistency.
btf_kfunc_check_flag() reads the kfunc's flags from the target's
kfunc_set_tab. For a module BTF that table is stable only after the
module is live, so take a module reference around the read, mirroring
how the kfunc call path gates the same lookup with btf_try_get_module().
The remaining call sites of btf_distill_func_proto() are safe as
is. The BPF_TRACE_ITER case distills a registered iterator's
prototype, and bpf_struct_ops_desc_init() distills the
function-pointer members of a struct_ops type. Neither is a kfunc, and
so can't have implicit arguments.
[1] https://lore.kernel.org/all/20260120222638.3976562-1-ihor.solodrai@linux.dev/
Fixes: 64e1360524 ("bpf: Verifier support for KF_IMPLICIT_ARGS")
Reported-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://github.com/sched-ext/scx/issues/3687#issuecomment-4906694106
Link: https://patch.msgid.link/20260713235223.1639022-2-ihor.solodrai@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
The verifier rejects variable offsets for PTR_TO_TP_BUFFER and PTR_TO_BUF
accesses, but it currently accepts a constant negative offset produced by
pointer arithmetic.
Commit 022ac07508 ("bpf: use reg->var_off instead of reg->off for
pointers") moved constant pointer offsets from reg->off to reg->var_off.
However, __check_buffer_access() continued to check only the instruction
offset. An access with reg->var_off equal to -8 and an instruction offset
of zero therefore passes verification.
For writable raw tracepoints, the access end is also calculated from the
unsigned reg->var_off.value. An eight-byte access starting at -8 wraps
the calculated end to zero, allowing the program to load and attach
without increasing max_tp_access.
After ensuring that reg->var_off is constant, calculate the effective
access start using signed arithmetic and reject it when it is negative.
Use the validated start to calculate the access end for both
PTR_TO_TP_BUFFER and PTR_TO_BUF.
Fixes: 022ac07508 ("bpf: use reg->var_off instead of reg->off for pointers")
Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Cc: stable@vger.kernel.org # 5.2.0
Link: https://patch.msgid.link/20260714093846.18159-2-sun.jian.kdev@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Helper and kfunc argument checking carried two separate meta structs: the
verifier-local struct bpf_call_arg_meta and bpf_kfunc_call_arg_meta.
Merge them into a single struct bpf_call_arg_meta. This is groundwork for
sharing argument checking between helpers and kfuncs.
While merging, drop the btf_id field from the helper meta since it is
never used.
No functional change.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-8-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
meta->pkt_access is only ever a copy of fn->pkt_access, assigned once in
check_helper_call() and read back in may_access_direct_pkt_data(). Have
may_access_direct_pkt_data() take the bpf_func_proto and read
fn->pkt_access directly, and drop the meta field along with its
assignment.
The only non-NULL caller, check_func_arg(), already has fn in scope; the
remaining callers pass NULL and are unaffected.
No functional change intended.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-7-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The constant "size of the PTR_TO_MEM returned in R0" argument is handled by
both helpers (ARG_CONST_ALLOC_SIZE_OR_ZERO) and kfuncs (__rdonly_buf_size /
__rdwr_buf_size), each with its own meta field (meta->mem_size,
meta->r0_size) and duplicated validation. Add struct arg_alloc_mem_desc
and a shared process_const_alloc_mem_size(), and replace both fields with
meta->arg_alloc_mem.
The desc records presence with a 'found' flag instead of using a non-zero
size as the sentinel. This also fixes a pre-existing bug on the kfunc
return path: "no size argument" was tested as r0_size == 0, so an explicit
__rdonly_buf_size/__rdwr_buf_size of 0 was treated as absent and fell
through to btf_resolve_size(), giving R0 the size of the pointed-to return
type instead of 0. With 'found', an explicit zero size is honored and
btf_resolve_size() is used only when no size argument was passed.
The size is stored in a u32, matching regs[R0].mem_size. The U32_MAX
check now apply to both helper and kfunc through
process_const_alloc_mem_size().
Fold bpf_session_cookie return size assignment into current kfunc return
size resolution path.
Note that verifier saves kfunc return size through r0_size instead of
mem_size. The later has no active readers so remove it.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-5-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
check_func_arg() only ever handles register arguments (the caller
loops over the first MAX_BPF_FUNC_REG_ARGS arguments), so a single
argno_t built from the register number identifies the argument for
every callee. Remove the duplicated argno_from_reg() calls to
simplify check_func_arg().
'regno' is still kept for the few places that need the raw register
number directly (register reads, verbose R%d messages) and for
referring to the neighbouring size/memory argument in the
ARG_CONST_SIZE{,_OR_ZERO} cases.
No functional change intended.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-4-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
To prepare for unifying the helper and kfunc call_arg_meta, group the
scattered MEM_UNINIT ("raw") memory argument fields (raw_mode, regno and
access_size) into a new struct arg_raw_mem_desc. The intention is to
make it clear about when these are set and used instead of fields with
overly generic names.
Identify the raw argument once, up front, in check_raw_mode_ok() (like
check_proto_release_reg() does for release_regno), recording its regno.
check_stack_range_initialized() now recognizes the raw buffer by matching
that regno, so the separate raw_mode flag is no longer needed, and the
per-argument "meta->raw_mode = arg_type & MEM_UNINIT" assignments in
check_func_arg() go away with it.
A raw memory argument can be tagged either ARG_PTR_TO_MEM | MEM_UNINIT or
ARG_PTR_TO_MAP_VALUE | MEM_UNINIT (the output buffer of bpf_map_pop_elem()
and bpf_map_peek_elem()). Either may be passed as a PTR_TO_STACK, which
reaches check_stack_range_initialized() through check_helper_mem_access(),
so both must be treated as raw. Extend arg_type_is_raw_mem() to match the
map value case as well; otherwise check_raw_mode_ok() would not record the
regno for it and an uninitialized stack buffer passed to those helpers
would be wrongly rejected for programs without CAP_PERFMON.
No functional change intended. This patch does not enable raw_mode
memory access for kfunc (i.e., uninit stack will not be allowed to be
passed to kfunc for unprivileged programs). Existing kfuncs with arguments
tagged with __uninit are either priviledged or dynptr kfuncs, which take
another path to make sure the access is checked by check_mem_access().
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-3-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
For a MEM_UNINIT ("raw mode") helper argument,
check_stack_range_initialized() open-coded a scan that rejected any
STACK_DYNPTR slot in the range with "potential write to dynptr". This
duplicated, and was stricter than, the handling that runs when the
buffer is actually marked initialized. check_helper_call() later replays
the write byte by byte via check_mem_access(), which goes through
destroy_if_dynptr_stack_slot(), which rejects overwritting a referenced
dynptr. Therefore drop the redundant scan and rely on check_mem_access().
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-2-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
- A cpuset that never set its memory nodes could divide by zero when a
task's mempolicy rebinds on CPU hotplug. Rebind against the effective
nodes, which are always populated.
- Documentation fixes for memory.stat, io.stat, and the misc and v1 RDMA
controllers.
-----BEGIN PGP SIGNATURE-----
iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCalVWkg4cdGpAa2VybmVs
Lm9yZwAKCRCxYfJx3gVYGet9AQD3/qE93E+PctxXa+rmRfSSjIzqqv9NUZa4THjs
YOUqSAEApVSLPkhg3xHF61q9l+FuocLLY378Uf6LYGWet1SghAQ=
=1Gp+
-----END PGP SIGNATURE-----
Merge tag 'cgroup-for-7.2-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
- A cpuset that never set its memory nodes could divide by zero when a
task's mempolicy rebinds on CPU hotplug. Rebind against the effective
nodes, which are always populated
- Documentation fixes for memory.stat, io.stat, and the misc and v1
RDMA controllers
* tag 'cgroup-for-7.2-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
Docs/admin-guide/cgroup-v2: note blkcg_debug_stats gates io.latency stats
Docs/admin-guide/cgroup-v1: document rdma.peak, rdma.events and rdma.events.local
Docs/admin-guide/cgroup-v2: drop stale misc interface file count
cgroup/cpuset: rebind mm mempolicy to effective_mems, not mems_allowed
Docs/admin-guide/cgroup-v2: fix memory.stat doc details
- Lifecycle fixes for the new sub-scheduler support: two use-after-frees
and an enable-failure path that left a half-initialized sub-scheduler
linked.
- Two dispatch-path locking bugs: a spurious scheduler abort from a
migration race, and a lockdep splat from stale runqueue-lock tracking.
- Callback and task-state fixes: stale scheduler-owned state on a task
leaving SCX, a weight callback running after disable, and a bogus
warning on core-scheduling forced idle.
- On nohz_full, finite-slice tasks could miss the tick that expires their
slice. Enable it when such a task is picked, with a selftest.
- Smaller fixes: userspace CPU-mask helpers, ratelimited deprecation
warnings, docs and a sparse annotation.
-----BEGIN PGP SIGNATURE-----
iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCalVWgw4cdGpAa2VybmVs
Lm9yZwAKCRCxYfJx3gVYGTmUAP0QHX7Ae/g7yMPEB4slURpnSj/wE7hRCI5jTmay
2iIEJgD/RAhpwUUAp4Abozt0mjQMWh9UyEVjOxPCVNKaNh1XfwU=
=EBCm
-----END PGP SIGNATURE-----
Merge tag 'sched_ext-for-7.2-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
- Lifecycle fixes for the new sub-scheduler support: two
use-after-frees and an enable-failure path that left a
half-initialized sub-scheduler linked.
- Two dispatch-path locking bugs: a spurious scheduler abort from a
migration race, and a lockdep splat from stale runqueue-lock
tracking.
- Callback and task-state fixes: stale scheduler-owned state on a task
leaving SCX, a weight callback running after disable, and a bogus
warning on core-scheduling forced idle.
- On nohz_full, finite-slice tasks could miss the tick that expires
their slice. Enable it when such a task is picked, with a selftest.
- Smaller fixes: userspace CPU-mask helpers, ratelimited deprecation
warnings, docs and a sparse annotation.
* tag 'sched_ext-for-7.2-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Skip ops.set_weight() for disabled tasks
tools/sched_ext: scx - Fix cmask_subset(), cmask_equal() and cmask_weight()
sched_ext: Fix premature ops->priv publication in scx_alloc_and_add_sched()
sched_ext: Record an error on errno-only sub-enable failure
selftests/sched_ext: Verify nohz_full tick behavior
sched_ext: Enable tick for finite slices on nohz_full
sched_ext: Preserve rq tracking across local DSQ dispatch
sched_ext: Documentation: Fix ops table header reference
sched_ext: Don't warn on core-sched forced idle in put_prev_task_scx()
sched_ext: Pin parent scx_sched across a child sub-scheduler's lifetime
sched_ext: Annotate ksyncs with __rcu in alloc/free_kick_syncs()
sched_ext: Check remote rq eligibility under task's rq lock
sched_ext: Reset dsq_vtime and slice when a task leaves SCX
sched_ext: Avoid flooding the log with deprecation warnings
- Free field in error path of synthetic event parse
In __create_synth_event() the field was allocated but was not freed in the
error path.
- Fix ring_buffer_event_length() on 8 byte aligned architectures
On architectures with CONFIG_HAVE_64BIT_ALIGNED_ACCESS set to y,
the ring_buffer_event_length() may return the wrong size. This
is because archs with that config set will always use the "big
event meta header" as that is 8 bytes keeping the payload 8 bytes
aligned, even when a 4 byte header could hold the size of the
event.
But ring_buffer_event_length() doesn't take this into account and only
subtracts 4 bytes for the meta header in the length when it should have
subtracted 8 bytes.
- Have osnoise wait for a full rcu synchronization on unregister
osnoise_unregister_instance() used to call synchronize_rcu() before
freeing its copy of the instance but was switched to kfree_rcu().
The osniose tracer has code that traverses the instances that it
uses, and inst is just a pointer to that instance. By using kfree_rcu()
instead of synchronize_rcu(), the instance that the inst pointer is
pointing to can be freed while the osnoise code is still referencing it.
That is, a rmdir on an instance first unregisters the tracer. When the
unregister finishes, the rmdir expects that the tracer is finished with
the instance that it is using. By putting back the synchronize_rcu()
in osnoise_unregister_instance() the unregistering of osnoise will now
return when all the users of the instance have finished.
- Remove an unused setting of "ret" in tracing_set_tracer()
- Fix ring_buffer_read_page() copying events
The commit that changed ring_buffer_read_page() to show dropped events
from the buffer itself, split the "commit" variable between the commit
value (with flags) and "size" that holds the size of the sub-buffer.
A cut and paste error changed the test of the reading from checking the
size of the buffer to the size of the event causing reads to only read one
event at a time.
- Make tracepoint_printk a static variable
When the tracing sysctl knobs were move from sysctl.c to trace.c, the
variable tracepoint_printk no longer needed to be global. Make it static.
- Fix some typos
- Fix NULL pointer dereference in func_set_flag()
The flags update of the function tracer first checks if the value of the
flag is the same and exits if they are, and then it checks if the current
tracer is the function tracer and exits if it isn't. The problem is that
these checks need to be in a reversed order, as if the tracer isn't the
function tracer, then the flag being checked may not exist. Reverse the
order of these checks.
- Fix ufs core trace events to not dereference a pointer in TP_printk()
The TP_printk() part of the TRACE_EVENT() macro is called when the user
reads the "trace" file. This can be seconds, minutes, hours, days, weeks,
and even months after the data was recorded into the ring buffer. Thus,
saving a pointer to an object into the ring buffer and then dereferencing
it from TP_printk() can cause harm as the object the pointer is pointing
to may no longer exist.
Fix all the trace events in ufs core to save the device name in the ring
buffer instead of dereferencing the device descriptor from TP_printk().
- Prevent out-of-bound reads in glob matching of trace events
The filter logic of events allows simple glob logic to add wild cards to
filter on strings. But some events have fields that may not have a
terminating 'nul' character. This may cause the glob matching to go beyond
the string. Change the logic to always pass in the length of the field
that is being matched.
- Add no-rcu-check version of trace_##event##_enabled()
The trace_##event##_enabled() usually wraps trace events to do extra work
that is only needed when the trace event is enabled. But this can hide
events that are placed in locations where RCU is not watching, and can
make lockdep not see these bugs when the event is not enabled.
The trace_##event##_enabled() was updated to always test to make sure RCU
is watching to catch locations that may call events without RCU being
active.
This caused a false positive for the irq_disabled() and related events. As
that use trace_irq_disabled_enabled() to force RCU to be watching when the
event is enabled via the ct_irq_enter() function, calls the event, and
then calls ct_irq_exit() to put RCU back to its original state.
The trace_irq_disabled_enabled() should not trigger a warning when RCU is
not watching because the code within its block handles the case properly.
Make a __trace_##event##_enabled() version for this event to use that
doesn't check RCU is watching as it handles the case when it isn't.
- Fix use-after-free in user_event_mm_dup()
When the enabler is removed from the link list, it is freed immediately.
But it is protected via RCU and needs to be freed after an RCU
grace period. Use queue_rcu_work() so that the event_mutex can also
be taken as user_event_put() takes the mutex on the last reference
is released.
- Free type string in error path of parse_synth_field()
There's an error path in parse_synth_field() where the allocated type
string is not freed.
- Add selftest that tests deferred event teardown
- Fix leak in error path of trace_remote_alloc_buffer()
If page allocation fails, the desc->nr_cpus is not incremented for the
current CPU and the allocations done for it are not freed.
- Fix allocation length in trace_remote_alloc_buffer()
The logic to calculate the struct_len was doing a double count and setting
the value too large. Calculate the size upfront to fix the error and
simplify the logic.
- Fix sparse CPU masks in ring_buffer_desc()
If there are sparse CPUs (gaps in the numbering), the ring_buffer_desc()
will fail as it tests the CPU number against the number of CPUs that are
used.
-----BEGIN PGP SIGNATURE-----
iQJIBAABCgAyFiEEXtmkj8VMCiLR0IBM68Js21pW3nMFAmpTv/cUHHJvc3RlZHRA
Z29vZG1pcy5vcmcACgkQ68Js21pW3nN80w/9HEbliUUrJVNw8QCY8BmeBEzfCRhZ
2CeDzTg+UrLMb2BtqUhA9EjQnRPawe8mPxAfpZ750SdFnCkRhbfNwYmKar3M/8iW
y2Bhsvd3Hz6nRiHvKhG/1RkflgKOWLYf/TQSByKlakYLV8t4uffa0fe0N0zcnikK
XPGrhrTeRDJ7s94vb1u7p7mSSp2iy4ZhfXbagk966X3I4zvseIYNn5IIzyM3IIPT
M7mE60SxvgSRz0QqTJqihmqjXYSeNOQ2iU18wnvp4QfcT0IVuLKm027JnK/YpjWS
ZasMNGzO1kp5uo6hkXLrMlT3LjaNEMFTGogs1m9o9auhbMf443LK8GtZ1m1rwnTG
cXn1PocBevQIaXWPbnxMrSwIuN7YMUqjgX6SCxbXpDxSSuE5i/x1VmE08CjxnSm6
TdAdom9bE44FvMgaAz9+KhIUgm013c2rFDNiFroYtfkma5FqpfnfglA4/TkJvexY
e2L75cR3lrDEDiXXMQDndmvxNMhppwNlAH3mOZLiaCmGyFQFHhgAaG250K2Y6a5K
HtIfJkT6RQ3rWifV0E0+zlyRiOrvVCCA8WwNArWb1dTWrVU86Hr7q58jAq9p0Yrz
FnhpjdE3eWoUHZ5f28AMR2k5lMG3o6mOGT55JyUVCLw+rPSG5358UIg2xZiEo22k
RFlT26FFHbMW50Y=
=+//w
-----END PGP SIGNATURE-----
Merge tag 'trace-v7.2-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing fixes from Steven Rostedt:
- Free field in error path of synthetic event parse
In __create_synth_event() the field was allocated but was not freed
in the error path
- Fix ring_buffer_event_length() on 8 byte aligned architectures
On architectures with CONFIG_HAVE_64BIT_ALIGNED_ACCESS set to y, the
ring_buffer_event_length() may return the wrong size. This is because
archs with that config set will always use the "big event meta
header" as that is 8 bytes keeping the payload 8 bytes aligned, even
when a 4 byte header could hold the size of the event
But ring_buffer_event_length() doesn't take this into account and
only subtracts 4 bytes for the meta header in the length when it
should have subtracted 8 bytes
- Have osnoise wait for a full rcu synchronization on unregister
osnoise_unregister_instance() used to call synchronize_rcu() before
freeing its copy of the instance but was switched to kfree_rcu(). The
osniose tracer has code that traverses the instances that it uses,
and inst is just a pointer to that instance. By using kfree_rcu()
instead of synchronize_rcu(), the instance that the inst pointer is
pointing to can be freed while the osnoise code is still referencing
it
That is, a rmdir on an instance first unregisters the tracer. When
the unregister finishes, the rmdir expects that the tracer is
finished with the instance that it is using. By putting back the
synchronize_rcu() in osnoise_unregister_instance() the unregistering
of osnoise will now return when all the users of the instance have
finished
- Remove an unused setting of "ret" in tracing_set_tracer()
- Fix ring_buffer_read_page() copying events
The commit that changed ring_buffer_read_page() to show dropped
events from the buffer itself, split the "commit" variable between
the commit value (with flags) and "size" that holds the size of the
sub-buffer. A cut and paste error changed the test of the reading
from checking the size of the buffer to the size of the event causing
reads to only read one event at a time
- Make tracepoint_printk a static variable
When the tracing sysctl knobs were move from sysctl.c to trace.c, the
variable tracepoint_printk no longer needed to be global. Make it
static
- Fix some typos
- Fix NULL pointer dereference in func_set_flag()
The flags update of the function tracer first checks if the value of
the flag is the same and exits if they are, and then it checks if the
current tracer is the function tracer and exits if it isn't. The
problem is that these checks need to be in a reversed order, as if
the tracer isn't the function tracer, then the flag being checked may
not exist. Reverse the order of these checks
- Fix ufs core trace events to not dereference a pointer in TP_printk()
The TP_printk() part of the TRACE_EVENT() macro is called when the
user reads the "trace" file. This can be seconds, minutes, hours,
days, weeks, and even months after the data was recorded into the
ring buffer. Thus, saving a pointer to an object into the ring buffer
and then dereferencing it from TP_printk() can cause harm as the
object the pointer is pointing to may no longer exist
Fix all the trace events in ufs core to save the device name in the
ring buffer instead of dereferencing the device descriptor from
TP_printk()
- Prevent out-of-bound reads in glob matching of trace events
The filter logic of events allows simple glob logic to add wild cards
to filter on strings. But some events have fields that may not have a
terminating 'nul' character. This may cause the glob matching to go
beyond the string. Change the logic to always pass in the length of
the field that is being matched
- Add no-rcu-check version of trace_##event##_enabled()
The trace_##event##_enabled() usually wraps trace events to do extra
work that is only needed when the trace event is enabled. But this
can hide events that are placed in locations where RCU is not
watching, and can make lockdep not see these bugs when the event is
not enabled
The trace_##event##_enabled() was updated to always test to make sure
RCU is watching to catch locations that may call events without RCU
being active
This caused a false positive for the irq_disabled() and related
events. As that use trace_irq_disabled_enabled() to force RCU to be
watching when the event is enabled via the ct_irq_enter() function,
calls the event, and then calls ct_irq_exit() to put RCU back to its
original state
The trace_irq_disabled_enabled() should not trigger a warning when
RCU is not watching because the code within its block handles the
case properly. Make a __trace_##event##_enabled() version for this
event to use that doesn't check RCU is watching as it handles the
case when it isn't
- Fix use-after-free in user_event_mm_dup()
When the enabler is removed from the link list, it is freed
immediately. But it is protected via RCU and needs to be freed after
an RCU grace period. Use queue_rcu_work() so that the event_mutex can
also be taken as user_event_put() takes the mutex on the last
reference is released
- Free type string in error path of parse_synth_field()
There's an error path in parse_synth_field() where the allocated type
string is not freed
- Add selftest that tests deferred event teardown
- Fix leak in error path of trace_remote_alloc_buffer()
If page allocation fails, the desc->nr_cpus is not incremented for
the current CPU and the allocations done for it are not freed
- Fix allocation length in trace_remote_alloc_buffer()
The logic to calculate the struct_len was doing a double count and
setting the value too large. Calculate the size upfront to fix the
error and simplify the logic
- Fix sparse CPU masks in ring_buffer_desc()
If there are sparse CPUs (gaps in the numbering), the
ring_buffer_desc() will fail as it tests the CPU number against the
number of CPUs that are used
* tag 'trace-v7.2-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
ring-buffer: Allow sparse CPU masks in ring_buffer_desc()
tracing/remotes: Fix struct_len in trace_remote_alloc_buffer()
tracing/remotes: Fix leak in trace_remote_alloc_buffer() error path
selftests/user_events: Wait for deferred event teardown after unregister
tracing/synthetic: Free type string on error path
tracing/user_events: Fix use-after-free in user_event_mm_dup()
tracing: Add a no-rcu-check version of trace_##event##_enabled()
tracing: Prevent out-of-bounds read in glob matching
ufs: core: tracing: Do not dereference pointers in TP_printk()
tracing: Fix NULL pointer dereference in func_set_flag()
samples: ftrace: Fix typos in benchmark comment
tracing: Make tracepoint_printk static as not exported
ring-buffer: Fix ring_buffer_read_page() copying only one event per page
tracing: Remove unused ret assignment in tracing_set_tracer()
tracing/osnoise: Call synchronize_rcu() when unregistering
ring-buffer: Fix event length with forced 8-byte alignment
tracing/synthetic: Free pending field on error path
Since tracing_multi link does not set ftrace_managed, it would fail to
release the tracing_multi link when attaching tracing_multi link and
then attaching fentry link.
[ 3.714215] WARNING: kernel/bpf/trampoline.c:1727 at bpf_trampoline_multi_detach+0x20b/0x240, CPU#1: test_progs/97
...
[ 3.733170] bpf_tracing_multi_link_release+0x14/0x30
[ 3.733890] bpf_link_free+0x58/0x130
[ 3.734414] bpf_link_release+0x23/0x30
Fix it by setting 'ftrace_managed = true' in register_fentry_multi().
Fixes: aef4dfa790 ("bpf: Add bpf_trampoline_multi_attach/detach functions")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260711124822.29406-2-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_cpumask_populate() writes to its destination with bitmap_copy(), but
the destination is typed as struct cpumask *. That allows the verifier to
accept borrowed cpumask pointers returned by read-only kfuncs, such as
scx_bpf_get_online_cpumask(), as a writable destination.
Make the destination a struct bpf_cpumask * so populate follows the same
ownership rule as the other mutating cpumask kfuncs. Query kfuncs continue
to accept const struct cpumask * inputs.
Fixes: 950ad93df2 ("bpf: add kfunc for populating cpumask bits")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260709182800.2037938-2-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
unearthed other races in the area (Thomas Gleixner)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmpSJ0gRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hyKA//W5KqLrRgXjH9cS7tF6PLo9NhbZZn5FhH
jxThPvJosNyCj8UEVD+iCRjkqkNxiY9mzx3px2ZVlJg8K35ez2PeZ/zx0wvuSWlM
sD7k4S3JJQ7vAE4CkYvsf91TxCWqiOMw+NLyo7MgDhwsOdgPgUAAKlCdCyoFzRK7
rAV7FQWMQag24donwPiesk8bnU4pWrooq8BoY7XVCy6aHDwwPVxrs1yPyUEikIXq
AT2HiYFOu9ok6rSlJs+A2JQnmke6M6aIJ5dmUGR7gUDRAqlXesmM6jlwklzK0a4O
j6lpE05ouIhZpYzQVTDxvf8cytp0+rutTX8w0lPRlqxYkaeBVEpktxfBAoqFCsoe
GpK8hK0PM57W29zhnTygQib53QQAk9QBQetp0NwUjLSOu3ouhCAOC3wuoctJ4fKY
oNSKGPBhHwJN5932JuxlEll/xbo+bFyHfP87tqn1SA0bgKMSNet5q7+wYE+1gcTp
9LHQnoMpjyRBjuOupnvxqeOVbN3qlNdY6MINFwxBZuKG7UwmYpTH7l72fuUM/eNr
b4UHKgwv3vuQX/BfBoT1mLnzWQTltrfEywBCXHsEHUQ7KFhdRHEbnh3zVDZCT5S3
5N/RS8aFcryD/0/DhxWEhO2DBTgjDDvmOKplIPYFtikDI1dkjZ9dZ0BCI42u6wMH
by25mw2rF2s=
=2I60
-----END PGP SIGNATURE-----
Merge tag 'timers-urgent-2026-07-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fix from Ingo Molnar:
- Fix a subtle posix-cpu-timers vs. exec() race, which
unearthed other races in the area (Thomas Gleixner)
* tag 'timers-urgent-2026-07-11' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
posix-cpu-timers: Prevent UAF caused by non-leader exec() race
-----BEGIN PGP SIGNATURE-----
iQJIBAABCgAyFiEES0KozwfymdVUl37v6iDy2pc3iXMFAmpRNpEUHHBhdWxAcGF1
bC1tb29yZS5jb20ACgkQ6iDy2pc3iXN5+Q/+N1wsGx07LPmidKbPTx3R+yej3JCW
hoLhnmPt7AXxVL671W29vkazKZ8hyQ5hzRy3wITyue9hsiXTNIwNffE1iBKkqMVA
mchh38ziazJJ3h9PVWdiXdW/PoqOIAicFCUe0KJw8bV21Tdzjl+tCqVylU4j57k8
xpkGUifsK8FpkvDMzMlOfD5e4qShOqkPhacVz1OJDrW8gHV5Dh1AuTXdKz58dA1F
Plu7IrtNd+CM1AQwS8ucawDB5PXnxHa708evS5bId/9fG/x5L5ZPJw6+jCPVA+Ea
qUkWWalAJzOEp4jWDhvCyymtqiGhj9rFAydxCy32IAsF7TLTDCV8E+6Iq78wGtjg
Oep+Tn9wOqV5QMILqlZ0zQxeDWnQWA0MzVRqIOFAvMrpurPLUbYZLFbUyNWouGBD
cfU6O+AhADTmfkFfxxTFLsm0RQCEdnF1XeZEreXgwFuuF/A7SybmB2WTWE5wcmbu
Riks3O9NHK620vCLf+wNpDc7+BkNmfBBhLUsyk0NDzEsIpViWS24stumBAP7+Fge
0KnWaB+vOD8IVub1YGgrxTNf3DfK6p8CLgwGYZMBjYFczJF8gCYXTfKlqmRVpXzw
20WaBbriMeMKgC5azD3aPqCg+/vgfBdCcEHSXAIS2KZivMQqYaodlFtTUowoDKng
sn2hwA3SKfZ4aGw=
=wY2k
-----END PGP SIGNATURE-----
Merge tag 'audit-pr-20260710' of git://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/audit
Pull audit fixes from Paul Moore:
"Two relatively small audit patches to fix potential data races with
the main audit backlog queue as well as possible integer overflows
when logging data as hex strings"
* tag 'audit-pr-20260710' of git://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/audit:
audit: fix potential integer overflow in audit_log_n_hex()
audit: Fix data races of skb_queue_len() readers on audit_queue
No user currently relies on sparse CPU masks, but the descriptor logic already
supports them via linear fallback. Remove the arbitrary limitation.
Link: https://patch.msgid.link/20260709160017.1729517-4-vdonnefort@google.com
Fixes: 2e67fabd8b ("ring-buffer: Introduce ring-buffer remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Pre-calculate desc->struct_len up-front in trace_remote_alloc_buffer()
with trace_buffer_desc_size() to fix double-counting.
While at it, use the accessor __first_ring_buffer_desc().
Link: https://patch.msgid.link/20260709160017.1729517-3-vdonnefort@google.com
Fixes: 96e43537af ("tracing: Introduce trace remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
If page allocation fails in trace_remote_alloc_buffer(), desc->nr_cpus
is not yet incremented for the current CPU. As a consequence, on error,
half-allocated rb_desc will not be freed in trace_remote_free_buffer().
Increment desc->nr_cpus as soon as the first allocation for the current
CPU has succeeded.
Link: https://patch.msgid.link/20260709160017.1729517-2-vdonnefort@google.com
Fixes: 96e43537af ("tracing: Introduce trace remotes")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When switching a task's sched_class away from sched_ext, we get the
following sequence of events in __sched_setscheduler():
sched_change_begin()
switched_from_scx()
scx_disable_task(p)
ops.disable(p)
__setscheduler_params()
set_load_weight()
reweight_task_scx(p)
ops.set_weight(p)
p->sched_class = next_class;
sched_change_end()
...
Notably, ops.set_weight() is called _after_ ops.disable().
This violates the expected semantics of the callbacks, the expectation
being that ops.disable() can only be followed by ops.exit_task() or
ops.enable().
Skipping the weight adjustment for disabled tasks should be harmless
since the weight will be recalculated in scx_enable_task() if the task
ever rejoins SCX.
Fixes: 637b068282 ("sched: Fold sched_class::switch{ing,ed}_{to,from}() into the change pattern")
Cc: stable@vger.kernel.org # v6.19+
Signed-off-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
map_range() reads rb->aux_pages[], rb->aux_nr_pages and rb->aux_pgoff via
perf_mmap_to_page() while holding only event->mmap_mutex. Those fields are
serialized by rb->aux_mutex, and mmap_mutex is per event.
Thus, two events sharing one rb via PERF_EVENT_IOC_SET_OUTPUT can race
rb_alloc_aux() with map_range(), leading to a page-UAF scenario as follows:
CPU 0 CPU 1
===== =====
rb_alloc_aux() map_range()
[1]: allocate rb->aux_pages[0]
[2]: rb->aux_nr_pages++
[3]: perf_mmap_to_page()
returns rb->aux_pages[0]
[4]: map it as VM_PFNMAP
[5]: rb->aux_pgoff = 1
munmap the page
[6]: free rb->aux_pages[0]
Pages mapped as VM_PFNMAP have no refcount protection, so CPU 1 holds a
mapping to a freed physical frame.
Fix this by taking rb->aux_mutex across the page walk in map_range().
Fixes: b709eb872e ("perf: map pages in advance")
Signed-off-by: Lee Jia Jie <jiajie.lee@starlabs.sg>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: stable@vger.kernel.org
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
scx_alloc_and_add_sched() publishes @sch through ops->priv before allocating
the cgroup path. If that allocation fails, the unwind path clears ops->priv
and frees @sch immediately. scx_prog_sched() callers can dereference
ops->priv from RCU context the moment it is set, so freeing without a grace
period can use-after-free a concurrent kfunc caller.
Move the publication below the cgroup path allocation so that every failure
path after publication frees @sch through kobject_put(), whose release path
defers the freeing by a grace period.
Fixes: 105dcd005b ("sched_ext: Introduce scx_prog_sched()")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
scx_sub_enable_workfn() has several failure paths that only return an errno
(e.g. -ENOMEM from an allocation) and jump to err_disable without calling
scx_error(). scx_flush_disable_work() runs the disable, and thus ops.exit(),
only when an error has been recorded, so an errno-only failure leaves the
half-initialized sub-scheduler linked.
Record an error at the err_disable sink so every errno-only failure runs the
disable path.
Fixes: ebeca1f930 ("sched_ext: Introduce cgroup sub-sched support")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
check_kfunc_args() detects a kfunc argument named rdonly_buf_size or
rdwr_buf_size and stores reg->var_off.value into meta->r0_size, a u64,
and does not bound it. check_kfunc_call() later copies that value into
the returned register's mem_size field:
meta->r0_size = reg->var_off.value;
...
regs[BPF_REG_0].mem_size = meta.r0_size;
regs[BPF_REG_0].mem_size is u32. A constant whose upper 32 bits are set
gets truncated instead of causing a load-time rejection, so the verifier
records a PTR_TO_MEM register with an approximately 4 GiB mem_size for
whatever allocation the kfunc returned. A later access check against
that register uses the truncated, wrong bound.
Reject rdonly_buf_size/rdwr_buf_size values that exceed U32_MAX at the
point meta->r0_size is set.
Fixes: eb1f7f71c1 ("bpf/verifier: allow kfunc to return an allocated mem")
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260709155837.1879230-2-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
When a spilled scalar is filled back with a sign-extending narrowing load
(BPF_MEMSX), check_stack_read_fixed_off() copies the spilled register
including its scalar id, but coerce_reg_to_size_sx() then sign-extends the
filled register's value. If the same slot is also filled with a plain
zero-extending load (BPF_MEM), both destination registers share the id yet
hold different values. A later 'if <zext-reg> == const' then refines the
sign-extended register through sync_linked_regs() to a value it does not
have at runtime (e.g. the verifier believes 0x80000000 while the register
is 0xffffffff80000000), which can be turned into an out-of-bounds access.
Drop the shared scalar id at the sign-extension site in check_mem_access()
when sign extension actually changes the value, mirroring the BPF_MOVSX
handling in check_alu_op() (no_sext = reg_umax < 2^(size*8-1)).
Fixes: 3cd5c89065 ("bpf: Let the verifier assign ids on stack fills")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Commit 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
made the LSM hook wrappers for BPF object creation clean up the LSM
state internally upon denial, e.g. security_bpf_map_create() internally
calls security_bpf_map_free() when the bpf_map_create hook returns an
error. map_create() however still routes a denial to its free_map_sec
label, which invokes security_bpf_map_free() a second time, so the
bpf_map_free hook fires twice for a single denied map.
In-tree LSMs are unaffected in practice since the blob kfree() inside
security_bpf_map_free() is NULL-safe and idempotent and none of them
implement bpf_map_free, but a BPF LSM program attached to that hook
observes double invocations. Route the denial to free_map instead.
Fixes: 5816bf4273 ("lsm,selinux: Add LSM blob support for BPF objects")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260709073422.379247-1-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
When an architecture (such as LoongArch, ARM64, and RISC-V) implements
bpf_jit_inlines_helper_call(), the verifier skips rewriting the helper
call offset (insn->imm) in bpf_do_misc_fixups(). This is because the
helper is expected to be inlined by the JIT compiler later. Therefore,
insn->imm remains as the raw helper enum ID.
However, if JIT is disabled at runtime (net.core.bpf_jit_enable=0) or
if JIT compilation fails dynamically (e.g., due to OOM), the program
falls back to the BPF interpreter.
When the interpreter executes (__bpf_call_base + insn->imm) with the
unpatched raw ID, it jumps into an invalid address space, triggering
an instruction alignment fault or a kernel panic.
Although these helpers have valid C implementations in the kernel, the
omission of offset rewriting makes runtime interpreter fallback fatal.
Fix this by setting 'prog->jit_required = 1' when helper call rewriting
is skipped for JIT inlining. This ensures that such programs are safely
rejected if JIT is not available, preventing the runtime kernel panic.
Fixes: 2ddec2c80b ("riscv, bpf: inline bpf_get_smp_processor_id()")
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Introduce a 'jit_required' bitfield flag in struct bpf_prog to track
whether a BPF program strictly requires the JIT compiler to run. This
prevents a dangerous runtime fallback to the interpreter for features
that are only implemented in the JIT compiler.
Currently, bpf_prog_has_kfunc_call() is used only for kernel function
calls, replace the kfunc-specific helper with the new 'jit_required'
flag. This makes it easy to support other JIT-only BPF features, such
as inlined helpers.
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: KaFai Wan <kafai.wan@linux.dev>
Suggested-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
