Add stream_arena_load_acquire_fault, which performs a load-acquire from an
unmapped arena address, next to the existing read and write fault tests.
The test covers both halves of the JIT bug that treated a load-acquire as
a store when populating its exception table entry:
- the fault has to be reported as a READ, and at the address held by
the source register, which __stderr() and test_address() check, and
- the destination register has to be cleared by the fault handler,
which the program checks by poisoning it before the load-acquire
and returning it, so __retval(0) fails if it is left untouched
Note, load-acquire is open coded since linux/filter.h cannot be included
alongside vmlinux.h.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t stream_arena_fault_address
[...]
#462/1 stream_arena_fault_address/read_fault:OK
#462/2 stream_arena_fault_address/write_fault:OK
#462/3 stream_arena_fault_address/load_acquire_fault:OK
#462 stream_arena_fault_address:OK
Summary: 1/3 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260806201047.333389-5-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Extend test_resolve_btfids() to assert that resolve_btfids emits the
address_space(1) type attribute (a BTF_KIND_TYPE_TAG with kflag=1) on
the return type and/or arguments of kfuncs marked KF_ARENA_RET,
KF_ARENA_ARG1 or KF_ARENA_ARG2.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260807032029.78092-4-ihor.solodrai@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add task kptr tests that keep RCU protection active after a spin or RCU
unlock when preemption or IRQs remain disabled.
Also test the reverse order with explicit RCU. Verify that task kptrs are
rejected after leaving the final preemption-disabled or IRQ-disabled
region.
Signed-off-by: Ning Ding <dingning04@gmail.com>
Link: https://lore.kernel.org/bpf/20260805233940.3966981-3-dingning04@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Disabling preemption or local IRQs keeps the current CPU in an RCU
read-side critical section, but in_rcu_cs() does not account for either
state. The verifier therefore rejects safe kptr accesses and invalidates
pointers when another RCU source ends.
Include preemption-disabled and IRQ-disabled state in in_rcu_cs().
Invalidate RCU-protected pointers on RCU unlock, preempt enable, or IRQ
restore only after the final protection ends.
Signed-off-by: Ning Ding <dingning04@gmail.com>
Link: https://lore.kernel.org/bpf/20260805233940.3966981-2-dingning04@gmail.com
[ kkd: Simplify was_in_rcu_cs on spin unlock and adjust the selftest. ]
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The verifier uses kfunc registration flags to decide whether a kfunc may
be called while a BPF program holds a bpf_spin_lock.
Mark bpf_testmod_test_mod_kfunc() as KF_SPINLOCK_SAFE and verify that it
can be called while holding a bpf_spin_lock. Also attempt to call the
unmarked bpf_kfunc_trigger_ctx_check() under the lock and verify that the
program is rejected.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260805153340.34776-4-kaitao.cheng@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a test to verify that the sleepable tracing prog cannot attach to a
'__x64_sys' prefix prog target.
When CONFIG_FUNCTION_ERROR_INJECTION is disabled, without the fix, the
test would trigger the BUG:
[ 3.460756] BUG: sleeping function called from invalid context at kernel/bpf/trampoline.c:1324
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260805150810.34907-3-leon.hwang@linux.dev
Add a bpf_for() benchmark modelled on bench_bpf_loop so the per-iteration
iterator cost can be measured and compared against bpf_loop. It runs an
empty bpf_for(i, 0, nr_loops) loop 1000 times per trigger and accounts
nr_loops hits per outer iteration:
$ ./bench -p 1 --nr_loops 1000 bpf-for
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260804134601.2305303-7-puranjay@kernel.org
Add an __xlated test pinning the inlined bpf_iter_num_{new,next,destroy}()
shapes. The program is __naked, so there is no compiler glue and the whole
sequence is matched instruction for instruction.
Gate it to x86_64 and arm64 (bpf_jit_needs_zext() == false); elsewhere the
verifier interleaves "wN = wN" zero-extensions that would not match. The
inlining is arch independent, so these two are enough.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260804134601.2305303-6-puranjay@kernel.org
The verifier previously accepted a task kptr after the final spin unlock
ended its RCU protection in a sleepable program. The pointer could then be
used after the task was freed.
Add a negative test for that case. Add positive controls showing that the
pointer remains valid in a non-sleepable program and while an explicit RCU
read-side section is still active.
Assisted-by: Codex:gpt-5.6-sol
Assisted-by: ChatGPT:GPT-5.6-Pro
Signed-off-by: Ning Ding <dingning04@gmail.com>
Link: https://lore.kernel.org/bpf/20260803112615.3362122-3-dingning04@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add kfunc scalar argument types, classify them in get_kfunc_arg_type()
along side with pointer arguments and move scalar type verification
into the main switch in check_kfunc_args(). This keeps BTF-based
classification separate from register validation for every argument,
paving the way for generating the kfunc argument prototype at add-call
time. No functional change intended.
KF_ARG_MEM_SIZE and KF_ARG_CONST_MEM_SIZE now are reachable. Therefore,
remove the fallthrough from KF_ARG_PTR_TO_MEM case and adjust the
register indexing.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-18-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
get_kfunc_ptr_arg_type() decided part of a kfunc pointer argument's type
from the caller's register: a PTR_TO_BTF_ID (or reg2btf_ids) register made
the argument KF_ARG_PTR_TO_BTF_ID, otherwise it fell through to a memory
buffer. Folding register state into argument classification prevents
describing a kfunc's arguments from its BTF alone, which is a prerequisite
for generating a helper-like prototype and eventually sharing the argument
checking (check_func_arg()) between helpers and kfuncs.
Classify pointer arguments from BTF only, and resolve them against the
register in check_kfunc_args():
- A pointer to a struct that is not paired with a __sz/__szk size
argument is classified KF_ARG_PTR_TO_BTF_ID and then checked against
the register. A register carrying a BTF ID (PTR_TO_BTF_ID or a
reg2btf_ids type) must be referenced or trusted and is matched against
the expected type. The only relaxation is when the struct is composed
of scalars, the register may be verified as a fixed-size memory buffer
sized from the BTF type; anything else is rejected.
- A pointer paired with a size argument is always a memory buffer and is
never classified as BTF_ID, so the __sz/__szk case no longer detours
through BTF_ID.
The new design now accepts one previously rejected case: passing
PTR_TO_BTF_ID to a pointer to scalar w/o a following __sz/__szk. The
argument will be classified as KF_ARG_PTR_TO_MEM | MEM_FIXED_SIZE. The
PTR_TO_BTF_ID register will go through check_mem_reg() ->
check_helper_mem_access() -> check_ptr_to_btf_access(). For a pointer to
scalar arg, a kernel btf id will be rejected unless explicitly granted
by btf_struct_access(); a program allocated btf id will be allowed.
The referenced-or-trusted check thus moves into the KF_ARG_PTR_TO_BTF_ID
resolution, alongside the type match.
get_kfunc_ptr_arg_type() no longer needs the register, so drop its regs
and reg parameters; it is now a pure function of the kfunc's BTF.
When a register cannot satisfy a BTF_ID argument, report the register type
passed and, when the expected struct has a reg2btf_ids mapping, the
register type that would be accepted, instead of a confusing "socket".
Update the affected selftest messages accordingly.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-16-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
When a nullable buffer is passed as NULL, check_mem_size_reg() is skipped,
so the __szk memory size must be marked precise through the scalar argument
path instead. Exercise this with bpf_dynptr_slice() and a NULL buffer.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-12-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
ARG_CONST_SIZE does not require a constant: check_mem_size_reg() accepts
any bounded scalar and verifies the memory access against its maximum
(reg_umax). Rename ARG_CONST_SIZE and ARG_CONST_SIZE_OR_ZERO to
ARG_MEM_SIZE and ARG_MEM_SIZE_OR_ZERO to reflect that. ARG_CONST_ALLOC_
SIZE_OR_ZERO, which does require a constant, is left unchanged.
Pure rename, no functional change.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-10-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
A map-of-maps lookup value is refined to a map pointer (map_ptr_or_null)
at lookup time by refine_map_lookup_value(). Test that it is rejected
wherever a raw map value would be read as bytes, so the inner map
descriptor cannot leak.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-8-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_map_lookup_elem() is typed to return PTR_TO_MAP_VALUE for every
map, but for some map kinds the looked up value is actually a
different object: an inner map, a socket or an xsk socket.
Until now this reinterpretation happened once the pointer was
converted from its NULL-able form to a concrete value.
Such reinterpretation logic placement led to mark_ptr_not_null_reg()
being called for a temporary register copy in check_mem_reg() and
check_kfunc_mem_size_reg() (check_mem_size_reg() was buggy because of
not calling it). The temporary copy was necessary to pass
reinterpreted parameters as nullable helper and kfunc arguments.
Avoid this complication by refining map lookup result type right away.
The test case verifier_map_in_map/on_the_inner_map_pointer needs an
update because the verifier now prints a concrete NULL-able type for
the lookup.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260801074633.1595644-6-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The BPF trampoline preserves only 8 bytes of the target's return value
(R0), so attaching an fexit/fmod_ret/fsession program to a function that
returns a >8 byte value is now rejected by the verifier. Add a bpf_testmod
function returning __int128 and an fexit program that targets it. The
program is expected to fail to load with the "with a >8 byte return value
is not supported for this attach type" message.
A 128-bit __int128 argument is passed in a register pair and occupies two
trampoline context slots. Add a bpf_testmod function taking a leading
__int128 argument followed by an int and a long, and an fexit program that
reads those two trailing arguments and the return value, verifying that the
trampoline reserves enough stack for the 128-bit argument and places the
following arguments and the return value at the right context slots.
__int128 is only available on 64-bit targets (where the compiler defines
__SIZEOF_INT128__). The argument test additionally depends on the calling
convention: x86_64 and arm64 pass an __int128 in a register pair as the
trampoline expects, while other architectures pass it differently (e.g.
s390x passes larger arguments by reference), so that subtest runs only on
x86_64 and arm64 and is skipped elsewhere.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/bpf/20260729050209.2587581-1-yonghong.song@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add verifier coverage for the three cases affected by preserving the full
pointer state across scalar += pointer: stack frame number inheritance,
readonly-untrusted memory access, and dynptr data-slice invalidation.
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Tested-by: Daniel Wade <danjwade95@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260729-c3-035-public-bpf-v4-v4-4-8ee297e2346b@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
- Fix tcp_bpf_sendmsg() error path mistaking a concurrently-freed
sk_psock->cork for the local temporary message and freeing it
again. (Chengfeng Ye)
- Reject passing scalar NULL to nonnull arg of a global subprog.
Previously the verifier did not account for the cases directly
passing scalars to a global subprog, e.g.: 'global_func(0);'
would pass even if 'global_func' argument was marked nonnull.
(Amery Hung)
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRINyQBqoQUC24dy5htleuBPyPTXgUCamPsBwAKCRBtleuBPyPT
Xo/uAP0YR6/9em1W/sZnss6Wkwfwd9NdmmlrJqSlt7aq+A+W7AD/V5HbigM4pJ76
CZFkKvnzhhU3dSXm/dBT1/GGJbL4pwo=
=uzKM
-----END PGP SIGNATURE-----
Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf
Pull bpf fixes from Eduard Zingerman:
- Fix tcp_bpf_sendmsg() error path mistaking a concurrently-freed
sk_psock->cork for the local temporary message and freeing it again
(Chengfeng Ye)
- Reject passing scalar NULL to nonnull arg of a global subprog.
Previously the verifier did not account for the cases directly
passing scalars to a global subprog, e.g.: 'global_func(0);' would
pass even if 'global_func' argument was marked nonnull (Amery Hung)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
bpf, sockmap: Fix cork use-after-free in tcp_bpf_sendmsg()
selftests/bpf: Test passing scalar NULL to nonnull global subprog
bpf: Reject passing scalar NULL to nonnull arg of a global subprog
Enable verifier_may_goto_1 (raw instruction tests), stream_cond_break
(250ms timeout path), and the may_goto_interaction fastcall test on
riscv64 now that the JIT supports timed may_goto.
Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
Reviewed-by: Pu Lehui <pulehui@huawei.com>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>
Link: https://lore.kernel.org/bpf/20260723-riscv-bpf-timed-may-goto-v5-3-86acb54e5642@kylinos.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a test that checks R0-R5 are preserved across
arch_bpf_timed_may_goto() calls.
Use bpf_get_prandom_u32() to avoid the verifier removing the checks
via DCE.
Suggested-by: Björn Töpel <bjorn@kernel.org>
Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
Reviewed-by: Pu Lehui <pulehui@huawei.com>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>
Link: https://lore.kernel.org/bpf/20260723-riscv-bpf-timed-may-goto-v5-2-86acb54e5642@kylinos.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Largely due to my time off followed by a networking conference
which distracted most maintainers (less so the AI generators).
Including fixes from Bluetooth and WiFi.
Current release - regressions:
- wifi: mt76: fix MAC address for non OF pcie cards
Current release - new code bugs:
- mptcp: fix BUILD_BUG_ON on legacy ARM config
- wifi: cfg80211: guard optional PMSR nominal time
Previous releases - regressions:
- qrtr: ns: raise node count limit to 512, we arbitrarily picked
256 as a limit, turns out it was too low for real world deployments
- vhost-net: fix TX stall when vhost owns virtio-net header
- eth: amd-xgbe: fix MAC_AUTO_SW handling in CL37 AN
- wifi: ath12k: fix low MLO RX throughput on WCN7850
Previous releases - always broken:
- number of random AI fixes for SCTP, RDS and TIPC protocols
- more AI-looking fixes for WiFi drivers
- number of fixes for missing pointer reloading after skb pull
- reject BPF redirect use from qdisc qevent block
- tcp: initialize standalone TCP-AO response padding
- vsock/virtio: collapse receive queue under memory pressure to avoid
client OOMing the host with tiny messages
- ipv4: icmp: fill flow parameters in icmp_route_lookup decoy lookup,
make sure the ICMP response routing follows the routing policy
- gro: fix double aggregation of flush-marked skbs
- ovpn: fix various refcount bugs
- tls: device: push pending open record on splice EOF
- eth: mlx5:
- use sender devcom for MPV master-up
- fix MCIA register buffer overflow on 32 dword reads
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE6jPA+I1ugmIBA4hXMUZtbf5SIrsFAmpiX68ACgkQMUZtbf5S
IrvYsRAAkuMhUpz0Ss9aF7rBY8iTp4SofSvFeVe06ywraUfqPuflGlak07t1Lz/i
G4MuKXN0q8m+B0EZddfMeYw6rCGd0SCtFAkxUI3dd+pu4hssgioaCPL193drSsfC
/lYeacjVL45jNrQvAWwKsRaAs3xdwzxWf0ddIXWvVWbdDsVfIf/mYahSS3TvniWw
MQtEbWPnFwPvOrHzb+1ChLELCtig/yvK+3xS9JrwOkjUF4BczOUgqrYlG5MWerXP
f/JDLsegPcoZaTycW5F5fshY05umeRQza/zCFqMKQNcQux49fjREnYxBuyTacVCo
0cxhsNbKOhvBpBFNsHA6TjUbDxuiyL8L/g3e7VOlQFxI4hX3IMsnsP+UrSdE2zyG
lgFAQ6HIcelgFnzFcwp9YEGsiZ5nDoJKe5aBcgftzTFPx3Plh1UeCrNjYtJawcjk
1POovopI+G6eszwluVOoucUdDD3wf0jPgDqvdOcI9P9FVTsFmvRESsfen7NbdjG0
v5mk9+sasWL1dns6mre6nt5is4QWSg7PDjufQUhuPKSSEnld+csEgmyxmUm0/FgL
krUZLHdx0Yj9yIOAIYAvz8QoW9jHIyK05Mr7CoL4a/9RJ4rtxjb+3CT9qebeyd49
jK5uzYX6tPHvILFK4CgZwcE/z9S+DoxCAuDEp6LhfstKsJW4KIM=
=DIF8
-----END PGP SIGNATURE-----
Merge tag 'net-7.2-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
Pull networking fixes from Jakub Kicinski:
"Lots of fixes, double the count even for the 'new normal'. Largely due
to my time off followed by a networking conference which distracted
most maintainers (less so the AI generators).
Including fixes from Bluetooth and WiFi.
Current release - regressions:
- wifi: mt76: fix MAC address for non OF pcie cards
Current release - new code bugs:
- mptcp: fix BUILD_BUG_ON on legacy ARM config
- wifi: cfg80211: guard optional PMSR nominal time
Previous releases - regressions:
- qrtr: ns: raise node count limit to 512, we arbitrarily picked
256 as a limit, turns out it was too low for real world deployments
- vhost-net: fix TX stall when vhost owns virtio-net header
- eth: amd-xgbe: fix MAC_AUTO_SW handling in CL37 AN
- wifi: ath12k: fix low MLO RX throughput on WCN7850
Previous releases - always broken:
- number of random AI fixes for SCTP, RDS and TIPC protocols
- more AI-looking fixes for WiFi drivers
- number of fixes for missing pointer reloading after skb pull
- reject BPF redirect use from qdisc qevent block
- tcp: initialize standalone TCP-AO response padding
- vsock/virtio: collapse receive queue under memory pressure to avoid
client OOMing the host with tiny messages
- ipv4: icmp: fill flow parameters in icmp_route_lookup decoy lookup,
make sure the ICMP response routing follows the routing policy
- gro: fix double aggregation of flush-marked skbs
- ovpn: fix various refcount bugs
- tls: device: push pending open record on splice EOF
- eth: mlx5:
- use sender devcom for MPV master-up
- fix MCIA register buffer overflow on 32 dword reads"
* tag 'net-7.2-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (234 commits)
drop_monitor: perform u64_stats updates under IRQ-disabled section
drop_monitor: fix size calculations for 64-bit attributes
net: drop_monitor: fix info leak in NET_DM_ATTR_PAYLOAD
mptcp: fix BUILD_BUG_ON on legacy ARM config
selftests: mptcp: userspace_pm: fix undefined variable port
mptcp: fix stale skb->sk reference on subflow close
mptcp: pm: userspace: fix use-after-free in get_local_id
mptcp: decrement subflows counter on failed passive join
mac802154: hold an interface reference across the scan worker
sctp: don't free the ASCONF's own transport in DEL-IP processing
phonet: check register_netdevice_notifier() error in phonet_device_init()
phonet: pep: fix use-after-free in pep_get_sb()
bnge/bng_re: fix ring ID widths
tipc: fix integer overflow in tipc_recvmsg() and tipc_recvstream()
net: airoha: fix ETS channel derivation in airoha_tc_setup_qdisc_ets()
mctp: check register_netdevice_notifier() error in mctp_device_init()
ptp: netc: explicitly clear TMR_OFF during initialization
rds: tcp: unregister sysctl before tearing down listen socket
ipv6: Change allocation flags to match rcu_read_lock section requirements
net: slip: serialize receive against buffer reallocation
...
Add a regression test for the NULL current->bpf_net_context deref hit
when a BPF classifier attached to a qdisc qevent block asks for a
redirect. The classifier runs from tcf_qevent_handle() on the qdisc
enqueue path, outside any bpf_net_context.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t qevent
[...]
+ /etc/rcS.d/S50-startup
./test_progs -t qevent
#496/1 tc_qevent/redirect_verdict:OK
#496/2 tc_qevent/redirect_helper:OK
#496 tc_qevent:OK
Summary: 1/2 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260706185609.330006-4-daniel@iogearbox.net
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a verifier_arena case that fills a two-page arena, calls
bpf_arena_free_pages() with a scalar address one page below the arena
base, and then verifies that another allocation is still rejected.
Before the runtime guard, the invalid free can repopulate the free
tree with an out-of-domain offset and the final allocation succeeds.
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717-c10-031-public-bpf-next-v2-b4-v2-2-54b555443a7c@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Allocate and free arena pages, both from BPF and via user-space fault-in,
and check that the map's memlock in fdinfo tracks the number of pages that
are actually populated.
Like the other arena tests it runs serially.
test:
./test_progs -a arena_mem_usage
#5 arena_mem_usage:OK
Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260717114117.350851-5-jiayuan.chen@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add 2 testcases for callback with tailcall:
1. failure case: callback->subprog->tailcall.
2. success case: subprog with tailcall do not affect
no-tailcall callback.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Link: https://patch.msgid.link/20260716120157.835937-4-pulehui@huaweicloud.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
KF_IMPLICIT_ARGS kfuncs have a BPF-call prototype and a real kernel
target prototype. Add a tracing selftest that attaches fentry and fexit
programs to bpf_kfunc_implicit_arg(), runs a syscall BPF program that
calls it, and checks that the tracing context exposes both the explicit
argument and the implicit prog aux pointer.
Co-developed-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://patch.msgid.link/20260713235223.1639022-3-ihor.solodrai@linux.dev
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add verifier coverage for constant negative offsets on PTR_TO_TP_BUFFER
and PTR_TO_BUF pointers. Both programs adjust the buffer pointer by -8
and access it at offset zero, so the negative effective start must be
rejected at load time.
Switch the raw tracepoint writable attach checks from nbd_send_request
to bpf_testmod_test_writable_bare_tp, avoiding a dependency on the NBD
tracepoint. Keep the existing past-end case and add a case with a
negative var_off compensated by a positive instruction offset. The
effective start remains non-negative, so the program loads, but its
access end exceeds the writable context size and
bpf_raw_tracepoint_open() must return -EINVAL.
Cc: stable@vger.kernel.org # 5.2.0
Signed-off-by: Sun Jian <sun.jian.kdev@gmail.com>
Acked-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Link: https://patch.msgid.link/20260714093846.18159-3-sun.jian.kdev@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add a test passing an explicit rdwr_buf_size of 0 to
bpf_kfunc_call_test_get_rdwr_mem() and then reading the returned R0.
R0 should be a zero-sized PTR_TO_MEM, so the access must be rejected
with "min value is outside of the allowed memory range".
This covers the pre-existing bug where a zero size argument was
treated as "no size argument": the verifier fell through to
btf_resolve_size() and sized R0 after the pointed-to return type,
wrongly allowing the read.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-6-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
For a MEM_UNINIT ("raw mode") helper argument,
check_stack_range_initialized() open-coded a scan that rejected any
STACK_DYNPTR slot in the range with "potential write to dynptr". This
duplicated, and was stricter than, the handling that runs when the
buffer is actually marked initialized. check_helper_call() later replays
the write byte by byte via check_mem_access(), which goes through
destroy_if_dynptr_stack_slot(), which rejects overwritting a referenced
dynptr. Therefore drop the redundant scan and rely on check_mem_access().
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260715064047.1793790-2-ameryhung@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Cover both new VLAN flags in the fib_lookup test. BPF_FIB_LOOKUP_VLAN
reduces a VLAN egress to its physical parent plus the tag, and
BPF_FIB_LOOKUP_VLAN_INPUT scopes the lookup to a VLAN subinterface.
BPF_FIB_LOOKUP_VLAN is XDP-only, since VLAN devices have no XDP xmit; the
tc helper rejects it with -EINVAL, which the table runner asserts for
every flag arm, and the egress result is checked through
bpf_xdp_fib_lookup(). Non-VLAN cases run through both helpers and assert
the path-independent results match; the XDP loop also checks dmac and,
for the tot_len cases, the route mtu_result, so the VLAN-egress dmac and
frag-needed coverage stays even though the tc path no longer reaches it.
The egress arms pin the reduction (parent ifindex plus tag, including
via a neighbour on the VLAN device, in OUTPUT mode, over a bond, and
through a DIRECT|TBID table) and the failure contract: a stacked-VLAN
(QinQ) egress returns BPF_FIB_LKUP_RET_VLAN_FAILURE with params->ifindex
left at the input. That is distinct from a no-neighbour return, which
reports the egress ifindex; only VLAN_FAILURE rewinds params->ifindex,
and a guard arm whose input and egress devices differ pins the
distinction. The VLAN_FAILURE arms are IPv4; the IPv6 path reaches it
through the same shared code, so an IPv6 arm would only re-test that.
The input arms use an iif rule that routes one destination to two
gateways, so the asserted gateway reveals which device the lookup used
as ingress, including VRF table selection through the l3mdev rule and
l3mdev_fib_table_rcu(). The VRF arms are IPv4-only: the l3mdev match
and table resolution are family-independent core shared by both rule
paths, and the IPv6 iif feed is pinned by the IPv6 VLAN input arm. A
cross-netns subtest moves a VLAN device into a second netns while it
stays registered on its parent and checks both directions fail closed
at the boundary.
A live-frames subtest (test_fib_lookup_vlan_redirect, with
BPF_F_TEST_XDP_LIVE_FRAMES) drives real frames through the native
xdp_do_redirect() / xdp_do_flush() path: a reducible egress is
redirected to the parent and delivered to its peer, while a QinQ egress
is passed to the stack, since redirecting to the VLAN device would drop
the frame at flush (no ndo_xdp_xmit).
The remaining per-case assertions are in the test table: resolution
semantics, the -EINVAL and NOT_FWDED error arms, and the SRC/SKIP_NEIGH
combinations.
Signed-off-by: Avinash Duduskar <avinash.duduskar@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260713162305.1237211-4-avinash.duduskar@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Verify that there's no any issue to attach tracing_multi link, then attach
fentry link.
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260711124822.29406-3-leon.hwang@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_cpumask_populate() now takes a struct bpf_cpumask *, so update the
kfunc declaration and drop the struct cpumask * casts in the existing
populate tests. Add test_populate_borrowed_destination, which passes a
borrowed task->cpus_ptr and asserts the verifier rejects it as a writable
destination.
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260709182800.2037938-3-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
This test is similar to test_icmp_send_unreach_cgroup but checks that,
in case of recursion, meaning that the BPF program calling the kfunc was
re-triggered by the icmp_send done by the kfunc, the kfunc will stop
early and return -EBUSY.
The test attaches to the root cgroup to ensure the ICMP packet generated
by the kfunc re-triggers the BPF program.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-5-mahe.tardy@gmail.com
This test extends the existing cgroup_skb tests with IPv6 support.
Note that we need to set IPV6_RECVERR on the socket for IPv6 in
connect_to_fd_nonblock otherwise the error will be ignored even if
we are in the middle of the TCP handshake. See in
net/ipv6/datagram.c:ipv6_icmp_error for more details.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-4-mahe.tardy@gmail.com
This test opens a server and client, enters a new cgroup, attach a
cgroup_skb program on egress and calls the bpf_icmp_send function from
the client egress so that an ICMP unreach control message is sent back
to the client. It then fetches the message from the error queue to
confirm the correct ICMP unreach code has been sent.
Note that, for the client, we have to connect in non-blocking mode to
let the test execute faster. Otherwise, we need to wait for the TCP
three-way handshake to timeout in the kernel before reading the errno.
Also note that we don't set IP_RECVERR on the socket in
connect_to_fd_nonblock since the error will be transferred anyway in our
test because the connection is rejected at the beginning of the TCP
handshake. See in net/ipv4/tcp_ipv4.c:tcp_v4_err for more details.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jordan Rife <jordan@jrife.io>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260709144900.245904-3-mahe.tardy@gmail.com
Add a load-failure test to the kfunc_call suite using the existing
bpf_kfunc_call_test_get_rdwr_mem() test kfunc. Its rdwr_buf_size
argument is a const int, so the test uses a 64-bit immediate load in
inline asm to place 2^64 - 192 (0xffffffffffffff40) in the argument
register. The verifier records r0_size from the full 64-bit register
value, and the test asserts that BPF_PROG_LOAD rejects it with
"rdonly/rdwr_buf_size exceeds u32 max".
Signed-off-by: Nicholas Dudar <main.kalliope@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260709155837.1879230-3-main.kalliope@gmail.com
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
With write-only excl member removed from struct bpf_map, ops moves
to offset 32 and inner_map_meta to offset 40. Update the expected
verifier message for the former and retarget the latter at the sha
byte array, so the beyond-member-end rejection path stays covered:
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_map_ptr
[...]
#619/5 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected:OK
#619/6 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected @unpriv:OK
#619/7 verifier_map_ptr/bpf_map_ptr: read beyond sha field rejected:OK
#619/8 verifier_map_ptr/bpf_map_ptr: read beyond sha field rejected @unpriv:OK
#619/9 verifier_map_ptr/bpf_map_ptr: read ops field accepted:OK
#619/10 verifier_map_ptr/bpf_map_ptr: read ops field accepted @unpriv:OK
[...]
#620 verifier_map_ptr_mixing:OK
Summary: 2/20 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260708075343.358712-7-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a TCP congestion-control struct_ops load test for a write through a
BTF pointer produced by bpf_rdonly_cast().
The test expects the verifier to reject the program before the TCP CA
btf_struct_access callback can whitelist the tcp_sock field write.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
A few selftests checking that the verifier represents spills for the
following pointer types w/o losing precision:
- PTR_TO_INSN
- PTR_TO_TP_BUFFER
- CONST_PTR_TO_DYNPTR
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260707-missing-spillable-types-v1-2-44a92121dc41@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
tc, xdp, socket_filter and flow_dissector programs can no longer update
or delete a sockmap. Adjust the tests:
- verifier_sockmap_mutate: the tc, xdp, socket_filter and
flow_dissector cases now expect __failure with "cannot update sockmap
in this context".
- sockmap_basic: drop "sockmap update" / "sockhash update", which load
a SEC("tc") program that copies a sock between maps.
- fexit_bpf2bpf: drop "func_sockmap_update", whose freplace program
updates a sockmap in the tc cls_redirect context.
Remove the now-unused test_sockmap_update.c and freplace_cls_redirect.c.
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: John Fastabend <john.fastabend@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260630145410.3648099-3-rhkrqnwk98@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a test which checks on each CPU that the bpf_get_smp_processor_id
BPF helper is returning the correct CPU number.
Signed-off-by: Maxim Khmelevskii <max@linux.ibm.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Link: https://lore.kernel.org/bpf/20260703125648.919196-6-max@linux.ibm.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a linked_list negative loader case for a program-BTF type whose last
member is a zero-length flexible array. The program writes through the
first flexible-array element of an object allocated by bpf_obj_new().
The verifier should reject the access when the BTF walk reaches beyond the
static size of the allocated object.
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/e36fd5d2f4047809f0e5da46a7077083297e64db.1782807039.git.chenyy23@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
The uprobe nop5 optimization used to replace a 5-byte NOP with a 5-byte
CALL to a trampoline. The CALL pushes a return address onto the stack at
[rsp-8], clobbering whatever was stored there.
On x86-64, the red zone is the 128 bytes below rsp that user code may use
for temporary storage without adjusting rsp. Compilers can place USDT
argument operands there, generating specs like "8@-8(%rbp)" when rbp ==
rsp. With the CALL-based optimization, the return address overwrites that
argument before the BPF-side USDT argument fetch runs.
Add two tests for this case. The uprobe_syscall subtest stores known values
at -8(%rsp), -16(%rsp), and -24(%rsp), executes an optimized nop10 uprobe,
and verifies the red-zone data is still intact. The USDT subtest triggers a
probe in a function where the compiler places three USDT operands in the
red zone and verifies that all 10 optimized invocations deliver the expected
argument values to BPF.
On an unfixed kernel, the first hit goes through the INT3 path and later
hits use the optimized CALL path, so the red-zone checks fail after
optimization.
[ updates to use nop10 ]
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://patch.msgid.link/20260703114917.238144-13-jolsa@kernel.org
- Initialize task local storage before fork bails out to
free the task (Jann Horn)
- Fix insn_aux_data leak on verifier error path (KaFai Wan)
- Reject BPF inode storage map creation when BPF LSM is
uninitialized (Matt Bobrowski)
- Mask pseudo pointer values in verifier logs when pointer
leaks are not allowed (Nuoqi Gui)
- Harden BPF JIT against spraying via IBPB flush (Pawan Gupta)
- Reject a skb-modifying SK_SKB stream parser since the latter
is only meant to measure the next message (Sechang Lim)
- Fix bpf_refcount_acquire to reject refcounted allocation
arguments with a non-zero fixed offset (Yiyang Chen)
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
-----BEGIN PGP SIGNATURE-----
iIsEABYKADMWIQTFp0I1jqZrAX+hPRXbK58LschIgwUCaka0NhUcZGFuaWVsQGlv
Z2VhcmJveC5uZXQACgkQ2yufC7HISIMPywD/ePh+eOZjFwXy88kOdNa2qmKLuTQm
nv6jPDOEafVDfbUA/R0qT9t+144K5Ig1NU58uR9vMIevqWRjLrnQtWU7KI0D
=4aKP
-----END PGP SIGNATURE-----
Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf
Pull BPF fixes from Daniel Borkmann:
- Initialize task local storage before fork bails out to free the task
(Jann Horn)
- Fix insn_aux_data leak on verifier error path (KaFai Wan)
- Reject BPF inode storage map creation when BPF LSM is uninitialized
(Matt Bobrowski)
- Mask pseudo pointer values in verifier logs when pointer leaks are
not allowed (Nuoqi Gui)
- Harden BPF JIT against spraying via IBPB flush (Pawan Gupta)
- Reject a skb-modifying SK_SKB stream parser since the latter is only
meant to measure the next message (Sechang Lim)
- Fix bpf_refcount_acquire to reject refcounted allocation arguments
with a non-zero fixed offset (Yiyang Chen)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
bpf: Prefer dirty packs for eBPF allocations
bpf: Prefer packs that won't trigger an IBPB flush on allocation
bpf: Skip redundant IBPB in pack allocator
bpf: Restrict JIT predictor flush to cBPF
x86/bugs: Enable IBPB flush on BPF JIT allocation
bpf: Support for hardening against JIT spraying
bpf: Reject BPF_MAP_TYPE_INODE_STORAGE creation if BPF LSM is uninitialized
bpf,fork: wipe ->bpf_storage before bailouts that access it
bpf: Fix insn_aux_data leak on verifier err_free_env path
selftests/bpf: Cover pseudo-BTF ksym log masking
bpf: Mask pseudo pointer values in verifier logs
selftests/bpf: Cover refcount acquire node offsets
bpf: Reject offset refcount acquire arguments
selftests/bpf: test rejection of a packet-modifying SK_SKB stream parser
bpf, sockmap: reject a packet-modifying SK_SKB stream parser
selftests/bpf: don't modify the skb in the strparser parser prog
test_maps fails in the sockmap test because sockmap_verdict_prog.c
drops the packet when the first 8 bytes are not directly accessible:
if (data + 8 > data_end)
return SK_DROP;
The blamed commit removed bpf_skb_pull_data() from the stream parser
program so that the parser no longer modifies the skb. That was needed,
but it also removed an implicit side effect: bpf_skb_pull_data()
linearized enough of the skb for later direct packet access.
In this test, the send side goes through the sockmap SK_MSG path. The
skb can have skb->len == 20 while its linear area is empty, so the
verdict program sees data == data_end and drops the packet even though
the payload length is sufficient.
Keep the parser read-only, and pull the first 8 bytes in the verdict
program before reading or writing them. Reload data/data_end after
bpf_skb_pull_data() as required.
Fixes: 22a0cc10da ("selftests/bpf: don't modify the skb in the strparser parser prog")
Reported-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260701071501.39628-1-jiayuan.chen@linux.dev
Closes: https://lore.kernel.org/bpf/e3a91acd-2b4d-4e93-a3bb-a0e9ee5ede0f@linux.dev/
The mptcpify BPF prog upgrades eligible TCP sockets to MPTCP, but only
when the socket type is exactly SOCK_STREAM. Its update_socket_protocol()
hook runs on the raw type from userspace, before the socket core masks
it with SOCK_TYPE_MASK, so the type may still carry SOCK_CLOEXEC or
SOCK_NONBLOCK in its upper bits and the equality check fails.
As a result, a socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0) -- what
common libraries do by default -- is silently left as plain TCP. This
was hit in practice with curl. Since mptcpify.c is referenced as example
code for enabling MPTCP transparently, the same mistake is likely to be
copied into real deployments where it fails the same way and is hard to
diagnose.
Mask the type before comparing, mirroring the socket core. Extend the
test to also create the server with SOCK_CLOEXEC set; the same masking
is applied to start_server_addr() so a flagged type still listens.
Fixes: ddba122428 ("selftests/bpf: Add mptcpify test")
Signed-off-by: Guillaume Maudoux <layus.on@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://lore.kernel.org/bpf/20260630095723.564392-1-layus.on@gmail.com
Adding tracing_multi link info tests that follow the kprobe_multi
and uprobe_multi tests logic.
Assisted-by: Codex:GPT-5
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260629212208.895962-3-jolsa@kernel.org
When lookup element from those per-CPU maps, which have special field
in their values and their value size is not equal to roundup(value_sz, 8),
the padding size of temporary non-zeroed kernel heap memory allocated by
kvmalloc should not be exposed to user space.
Without the fix:
test_map_uninit_mem_exposure:FAIL:zeroed tail bytes unexpected memory mismatch
actual:
2B 2B 2B 2B
expected:
00 00 00 00
Assisted-by: Codex:gpt-5.5
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260624155115.85196-3-leon.hwang@linux.dev
Add a selftest that loads the kfunc in sleepable and non-sleepable
lsm/socket_connect programs and checks that a value set via fsetxattr()
on a socket is read back.
Link: https://patch.msgid.link/20260617-work-bpf-sock-xattr-v1-2-a1276f7c9da3@kernel.org
Reviewed-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
1, Add THREAD_INFO_IN_TASK implementation;
2, Add build salt to the vDSO;
3, Add some BPF JIT inline helpers;
4, Update DTS for I2C clocks and clock-frequency;
5, Some bug fixes and other small changes.
-----BEGIN PGP SIGNATURE-----
iQJKBAABCAA0FiEEzOlt8mkP+tbeiYy5AoYrw/LiJnoFAmo+MBYWHGNoZW5odWFj
YWlAa2VybmVsLm9yZwAKCRAChivD8uImesGWD/9HygPT6/JFvFR2rijLaym9TNYi
+X8m0GTUJDrBGasDQ1rXSgUAkWKIQHEvx0Q947HYWhceiIrwbiZrgMStVapk40hd
/WsiuTHFaiVzjiiXf2bdTSr9uWHIon3P1EZ6zkmdRO4l2VsvAq2elMJXazIP7NUa
fTnTGAMPXvA/TinNXZqGq/Ny9c6gp63pVUgATcBpSUmYCNYGz6xx7OwSWyGV0ncq
Kfxm87nVq6wdyCgmrzfYSGze9vM0FxfFpzvZbX9vLOfGZBGaxzDcBjH9LRU09vyp
/9fNpacKT49yNOfLO3R3MHSPFSa5DCg8ph8CDfMnqPmP01SfZpC+zmlFWv23ojx5
4v47fyaRQu6ucKL2vxkIJfKO6l8bXgcuzVgZPGVt5q5yJ5O+7yhuamncrxUKk6Ol
KGpIEJM3iKbcHKr9GF0EUcgJkbA2J/pU/094snBU3HlkoHDVcp28XxlJmNtW+jY4
49dCTbX/5v+680uf/KTUyXJ4j3m9bxMOB6feE1uVaSJF1nx6f7WYHRg7A/1pXnWy
f2chgwR4eUeqZRpFlUhcLx4R+OOD2ORvvdM0qcNv4T7E8FD31JtFVD873R2iFEG8
AlKG4Ln3DUeJdhskXXmJRIDD5nZb9XZNYlnM4ghQLBas5dtSxsSPnzkSjJHirTjR
naFFT9xxRBQOBBTrjQ==
=EvQG
-----END PGP SIGNATURE-----
Merge tag 'loongarch-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson
Pull LoongArch updates from Huacai Chen:
- Add THREAD_INFO_IN_TASK implementation
- Add build salt to the vDSO
- Add some BPF JIT inline helpers
- Update DTS for I2C clocks and clock-frequency
- Some bug fixes and other small changes
* tag 'loongarch-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson:
selftests/bpf: Test jited inline of bpf_get_smp_processor_id() for LoongArch
selftests/bpf: Test jited inline of bpf_get_current_task() for LoongArch
selftests/bpf: Add __arch_loongarch to limit test cases for LoongArch
selftests/bpf: Add get_preempt_count() support for LoongArch
LoongArch: dts: Add i2c clocks and clock-frequency properties to LS2K2000
LoongArch: dts: Add i2c clocks and clock-frequency properties to LS2K1000
LoongArch: dts: Add i2c clocks and clock-frequency properties to LS2K0500
LoongArch: BPF: Inline bpf_get_smp_processor_id() helper
LoongArch: BPF: Inline bpf_get_current_task/_btf() helpers
LoongArch: BPF: Fix off-by-one error in tail call
LoongArch: BPF: Fix outdated tail call comments
LoongArch: Add build salt to the vDSO
LoongArch: Fix nr passing in set_direct_map_valid_noflush()
LoongArch: Fix missing dirty page tracking in {pte,pmd}_wrprotect()
LoongArch: Move struct kimage forward declaration before use
LoongArch: Report dying CPU to RCU in stop_this_cpu()
LoongArch: Add PIO for early access before ACPI PCI root register
LoongArch: Add THREAD_INFO_IN_TASK implementation
Extend the resolve_btfids selftest to cover kfunc sets defined with
BTF_KFUNCS_START/BTF_KFUNCS_END.
The test verifies that resolve_btfids correctly processes BTF_ID_FLAGS,
resolves function IDs, and checks the kfunc set is sorted.
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260624005546.1818483-5-ihor.solodrai@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add verifier_unpriv coverage for a raw socket-filter load of the
bpf_prog_active typed ksym. The test verifies that the unprivileged load
remains accepted and that the verbose verifier log prints the ldimm64
immediate as 0x0 instead of exposing a nonzero kernel address.
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Link: https://lore.kernel.org/r/20260623-f01-13-pseudo-btf-id-cap-bpf-v2-2-a190ebb8f3e2@mails.tsinghua.edu.cn
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Add verifier_var_off coverage for variable-offset stack reads from spilled
scalar constant zero values.
Cover single-slot and cross-slot spilled zero reads, a sub-8-byte spill
with neighbouring STACK_ZERO bytes, and a sub-8-byte spill with
neighbouring STACK_MISC bytes that must not be treated as zero.
Add verifier_spill_fill coverage for a fixed-offset stack read spanning
both STACK_ZERO bytes and scalar const-zero STACK_SPILL bytes.
Use verifier log assertions to check both the zero result and the precision
backtracking trail.
Assisted-by: opencode:gpt-5.5
Signed-off-by: Woojin Ji <random6.xyz@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260625-bpf-stack-var-off-zero-v1-v3-2-a068210a761b@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add regression coverage for bpf_refcount_acquire() on graph-node-derived
pointers.
The rejected case passes a popped list node pointer directly to
bpf_refcount_acquire(), which must fail because the pointer carries a
non-zero fixed offset.
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/bf2a2033ced272106292de4465b8ef3fb991c912.1782192383.git.chenyy23@mails.tsinghua.edu.cn
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Verify that attaching an SK_SKB stream parser that can modify the packet
is rejected, while a read-only parser still attaches.
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Link: https://lore.kernel.org/r/20260620024423.4141004-4-rhkrqnwk98@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
sockmap_parse_prog.c is attached as an SK_SKB stream parser and modifies
the skb: it calls bpf_skb_pull_data() and writes a byte into the packet.
A stream parser runs on strparser's message head and must not modify it.
A resize frees the frag_list segments strparser still tracks, leading to
a use-after-free.
Make the parser read-only. It only needs to return the message length,
which keeps it attaching once packet-modifying parsers are rejected.
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Link: https://lore.kernel.org/r/20260620024423.4141004-2-rhkrqnwk98@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Extend redirect tests to cover bpf_redirect_peer(BPF_F_EGRESS). SRC
redirects to DST using bpf_redirect_peer(BPF_F_EGRESS) then traffic is
hairpinned into DST using bpf_redirect.
Signed-off-by: Jordan Rife <jordan@jrife.io>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Paul Chaignon <paul.chaignon@gmail.com>
Link: https://lore.kernel.org/r/20260618182035.43811-3-jordan@jrife.io
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Current release - regressions:
- net: do not acquire dev->tx_global_lock in netdev_watchdog_up()
- net: ethtool: keep rtnl_lock for ops using ethtool_op_get_link()
- net: fix deadlock in nested UP notifier events
Current release - new code bugs:
- eth: cn20k: fix subbank free list indexing for search order
- eth: airoha: fix BQL underflow in shared QDMA TX ring
Previous releases - regressions:
- netfilter:
- flowtable: fix offloaded ct timeout never being extended
- nf_conncount: prevent connlimit drops for early confirmed ct
Previous releases - always broken:
- require CAP_NET_ADMIN in the originating netns when modifying
cross-netns devices
- report NAPI thread PID in the caller's pid namespace
- mac802154: fix dirty frag in in-place crypto for IOT radios
- sctp: hold socket lock when dumping endpoints in sctp_diag,
avoid an overflow
- eth: gve: fix header buffer corruption with header-split and HW-GRO
- af_key: initialize alg_key_len for IPComp states, prevent OOB read
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE6jPA+I1ugmIBA4hXMUZtbf5SIrsFAmo9ZaUACgkQMUZtbf5S
IrtO5w/+JgRT+RjKiCCLsEUM1R9hCA2Hxhos99YFpmMZitU2v3IZIdD/sZL6Ppxa
fq7hlsfMicF/isez+sharJ8H8NrX5MZ9YtvcbbRG2+aUPlAJDVSfxbqCcsjEMqdP
s4wtAhUoPwIVpw/rwcSSubQrGFTJlshVi1Yp48xxNHzNAk1wNoitSeKS+lAToaJf
6ZUl9uamxrbMG8j9FBOSWeOcQ2ywmrQiSGbRy74X0qolRfS+1r0pNchwGsv8s2RP
Sa3an4qfD3vBV0YNh8KglycBuvU6LMAt90/XIZ6zY8TBzISyM9PgrQbcNqKMlxQ0
LNPy5qosWvIEFVBjrUlw1IyyfSWOBnybwXViIE83/x/sUTyhRX4vnn20hdNlMyB2
FkWfU1C/X+D/idSoynCXGY4VXq7ikkfvTOXlTGcd9Q8sse98sQV+TpeVdu5gkqX8
KvjhIM896Kt4eVHkmVQ0DCu+GN8srbzmMCoPqBv04zfQX4rk9tWH4iyMIaNgniZe
8SrgFg3IA/WoaIU+Ac3oc2LNCf0XbeNQXMSfTu5t3H1bXgteDzdar/ncNXaPx/S2
72kP35DooMpgApP4TbpxiPZ1bVSfFK7mtoxxdHQVZVGwo9NY1Fz2eqpG28y/pk18
RXn2GSfeEmzsPGRYte+09QkwoaDnRj5GZ2ETbVxibPmJ71t6Iz0=
=dIVQ
-----END PGP SIGNATURE-----
Merge tag 'net-7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
Pull networking fixes from Jakub Kicinski:
"Including fixes from netfilter and IPsec.
Current release - regressions:
- do not acquire dev->tx_global_lock in netdev_watchdog_up()
- ethtool: keep rtnl_lock for ops using ethtool_op_get_link()
- fix deadlock in nested UP notifier events
Current release - new code bugs:
- eth:
- cn20k: fix subbank free list indexing for search order
- airoha: fix BQL underflow in shared QDMA TX ring
Previous releases - regressions:
- netfilter:
- flowtable: fix offloaded ct timeout never being extended
- nf_conncount: prevent connlimit drops for early confirmed ct
Previous releases - always broken:
- require CAP_NET_ADMIN in the originating netns when modifying
cross-netns devices
- report NAPI thread PID in the caller's pid namespace
- mac802154: fix dirty frag in in-place crypto for IOT radios
- sctp: hold socket lock when dumping endpoints in sctp_diag, avoid
an overflow
- eth: gve: fix header buffer corruption with header-split and HW-GRO
- af_key: initialize alg_key_len for IPComp states, prevent OOB read"
* tag 'net-7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (213 commits)
selftests: bonding: add a test for VLAN propagation over a bonded real device
vlan: defer real device state propagation to netdev_work
net: add the driver-facing netdev_work scheduling API
net: turn the rx_mode work into a generic netdev_work facility
net: ethtool: keep rtnl_lock for ops using ethtool_op_get_link()
rxrpc: Fix rxrpc_rotate_tx_rotate() to check there's something to rotate
rxrpc: Fix leak of released call in recvmsg(MSG_PEEK)
rxrpc: Fix socket notification race
rxrpc: Fix potential infinite loop in rxrpc_recvmsg()
rxrpc: Fix oob challenge leak in cleanup after notification failure
rxrpc: Fix the reception of a reply packet before data transmission
afs: Fix uncancelled rxrpc OOB message handler
afs: Fix further netns teardown to cancel the preallocation charger
rxrpc: Fix double unlock in rxrpc_recvmsg()
rxrpc: Fix leak of connection from OOB challenge
rxrpc: Fix ACKALL packet handling
net: hns3: differentiate autoneg default values between copper and fiber
net: hns3: fix permanent link down deadlock after reset
net: hns3: refactor MAC autoneg and speed configuration
net: hns3: unify copper port ksettings configuration path
...
Add the testcase for the jited inline of bpf_get_smp_processor_id(),
only for LoongArch currently.
Here is the test result on LoongArch:
$ sudo ./test_progs -t verifier_jit_inline
#604/1 verifier_jit_inline/inline_bpf_get_current_task:OK
#604/2 verifier_jit_inline/inline_bpf_get_smp_processor_id:OK
#604 verifier_jit_inline:OK
Summary: 1/2 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Make it possible to limit certain tests to LoongArch, just like it is
already done for x86_64, arm64, riscv64, and s390x.
This is a follow up patch of:
commit ee7fe84468 ("selftests/bpf: __arch_* macro to limit test cases to specific archs")
commit 1e4e6b9e26 ("selftests/bpf: Add __arch_s390x macro")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Test that an LWT encapsulation does not silently corrupt XDP metadata
sitting in the skb headroom. Exercise all three LWT dispatch paths:
- BPF LWT xmit prog reserves headroom on the LWT .xmit redirect,
- mpls pushes an MPLS label on the LWT .xmit redirect,
- seg6 in encap mode runs on the LWT .input redirect,
- ioam6 encap inserts an IOAM Hop-by-Hop option on LWT .output redirect.
Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://patch.msgid.link/20260619-bpf-lwt-drop-skb-metadata-v3-2-71d6a33ab76b@cloudflare.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a verifier test that catches the stale-bounds issue fixed in the
previous patch. The test sets r6 = 0 to create known bounds, then loads
the LSM hook return value into r6 from the context. Without the fix,
the verifier intersects the retval range with the stale bounds and
incorrectly narrows r6 to a single value, pruning the fall-through
branch as dead code and missing the div-by-zero.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Tristan Madani <tristan@talencesecurity.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260622230123.3695446-3-tristmd@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a conntrack kfunc regression check for opts__sz values that do not
cover opts->error. The BPF program initializes opts->error with a guard
value, calls the lookup and allocation kfuncs with opts__sz set to
sizeof(opts->netns_id), and verifies that the guard is still intact
after the kfunc returns NULL.
Without the conntrack wrapper guard, the kfunc error path overwrites
that guard with -EINVAL even though the verifier checked only the first
four bytes of the options object.
Fixes: b4c2b9593a ("net/netfilter: Add unstable CT lookup helpers for XDP and TC-BPF")
Fixes: d7e79c97c0 ("net: netfilter: Add kfuncs to allocate and insert CT")
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Link: https://lore.kernel.org/r/007dfd0341cd84560e4795a2a951cc56d4adff1d.1781765747.git.chenyy23@mails.tsinghua.edu.cn
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a verifier regression test for a pointer spill whose high half is
cleaned dead while the low half remains live. Force checkpoint creation
with BPF_F_TEST_STATE_FREQ and assert the verifier log reaches the
checkpoint and the subsequent 32-bit fill before rejecting the partial fill
from a non-scalar spill.
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Link: https://lore.kernel.org/r/20260617-f01-06-half-slot-pointer-spill-v2-2-42b9cdc3cf64@mails.tsinghua.edu.cn
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a verifier test for the fixed-offset stack write case where two 4-byte
stores initialize opposite halves of the same stack slot.
The test runs through the unprivileged loader lane and expects both
half-slot writes to emit nospec in the translated program.
Acked-by: Luis Gerhorst <luis.gerhorst@fau.de>
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Link: https://lore.kernel.org/r/20260618-f01-11-stack-nospec-slot-index-v3-2-780297041721@mails.tsinghua.edu.cn
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE+soXsSLHKoYyzcli6rmadz2vbToFAmowIHwACgkQ6rmadz2v
bTqRog/+KIO6wpvuizt8N+vzgKeEYR/KlEPEyV5dGAopIk1Ct5Ahzti1/V6zg7hk
oddWhPewF7QncVLdnL6zRCRahXCHOi30148acG8pj3Fwj/iYw/zVihVkppVt9qRU
REAvFj0/hYJOGzttSt9UfQ78ZftrZNhXFCRIK/zUWj72IVMJD04Vbawo8iIixVtu
I+fCB3A1kv+RMlZ/3TWk11z3Fko4lK/jKpujegIzv6s+n+0lAraF0aw0cWMwNpUj
mtJx2PktPHYbjA5CwE1Fyxf6EhV4OhV5gwyET7gzqITG5zjUo45pMNcA7EAxDyiC
NvTGIZMsAC+9iSSLNSfdjBWExXFYHHuCkeoxkw9ei05Qrye/7lYzwwtgImuyhBZG
xNXrnSNCG5BPl+DCLy09LzzwNdRoP7zYZV3QH/ZU0uMTS6foegYoSM8jSa2skmM+
mR8a4qrMeZTv+XWvScduOqtFMBOv3mSPkZMkoyqReLaMIPse5NuUOTxOGn5REu4X
tQLW0GLa6ad/7udv7DYrSfmwK1dihyOPmZl/GMuGroXm4rW4jNojljNBdTFEWjCQ
Rd9qArq1Z1z/g8mWjTdiRRGa/QzF5rmJRLgjjUKIzUlQ5/77t96ls0jLexsRZQx5
hCdu/HoK+pIRIUy20NtdqA8ykowsfUkyEBjYR28GV4RHvtmgW5k=
=rtQD
-----END PGP SIGNATURE-----
Merge tag 'bpf-next-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Alexei Starovoitov:
"Major changes:
- Recover from BPF arena page faults using a scratch page and add
ptep_try_set() for lockless empty-slot installs on x86 and arm64.
This allows BPF kfuncs to access arena pointers directly.
The 'arena_direct_access' stable branch was created for this work
and was pulled into sched-ext and bpf-next trees (Tejun Heo, Kumar
Kartikeya Dwivedi)
- Lift old restriction and support 6+ arguments in BPF programs and
kfuncs on x86 and arm64 (Yonghong Song, Puranjay Mohan)
Other features and fixes:
- Add 24-bit BTF vlen and reclaim unused bits in the BTF UAPI to ease
addition of new BTF kinds (Alan Maguire)
- Raise the maximum BPF call chain depth from 8 to 16 frames (Alexei
Starovoitov)
- Refactor object relationship tracking in the verifier and fix a
dynptr use-after-free bug (Amery Hung)
- Harden the signed program loader and reject exclusive maps as inner
maps (Daniel Borkmann)
- Replace the verifier min/max bounds fields with a circular number
(cnum) representation and improve 32->64 bit range refinements
(Eduard Zingerman)
- Introduce the arena library and runtime (libarena) with a buddy
allocator, rbtree and SPMC queue data structures, ASAN support and
a parallel test harness. Allow subprograms to return arena pointers
and switch to a BTF type-tag based __arena annotation (Emil
Tsalapatis)
- Cache build IDs in the sleepable stackmap path and avoid faultable
build ID reads under mm locks (Ihor Solodrai)
- Introduce the tracing_multi link to attach a single BPF program to
many kernel functions at once. Allow specifying the uprobe_multi
target via FD (Jiri Olsa)
- Extend the bpf_list family of kfuncs with bpf_list_add/del(), and
bpf_list_is_first/is_last/empty() (Kaitao Cheng)
- Extend the BPF syscall with common attributes support for
prog_load, btf_load and map_create (Leon Hwang)
- Wrap rhashtable as BPF map (Mykyta Yatsenko, Herbert Xu)
- Add sleepable support for tracepoint programs and fix deadlocks in
LRU map due to NMI reentry (Mykyta Yatsenko)
- Fix OOB access in bpf_flow_keys, fix nullness analysis of inner
arrays, enforce write checks for global subprograms (Nuoqi Gui)
- Report the maximum combined stack depth and print a breakdown of
instructions processed per subprogram (Paul Chaignon)
- Add an XDP load-balancer benchmark and arm64 JIT support for stack
arguments (Puranjay Mohan)
- Add kfuncs to traverse over wakeup_sources (Samuel Wu)
- Allow sleepable BPF programs to use LPM trie maps directly (Vlad
Poenaru)
- Many more fixes and cleanups across the verifier, BTF, sockmap,
devmap, bpffs, security hooks, s390/riscv/loongarch JITs,
rqspinlock, libbpf, bpftool, selftests"
* tag 'bpf-next-7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (336 commits)
selftests/bpf: Work around llvm stack overflow in crypto progs
selftests/bpf: add test for bpf_msg_pop_data() overflow
bpf, sockmap: fix integer overflow in bpf_msg_pop_data() bounds check
sockmap: Fix use-after-free in udp_bpf_recvmsg()
bpf, sockmap: keep sk_msg copy state in sync
bpf, sockmap: Fix wrong rsge offset in bpf_msg_push_data()
bpf, sockmap: reject overflowing copy + len in bpf_msg_push_data()
selftsets/bpf: Retry map update on helper_fill_hashmap()
selftests/bpf: Add test for sleepable lsm_cgroup rejection
selftests/bpf: Add test to verify the fix for bpf_setsockopt() helper
bpf: Fix bpf_get/setsockopt to tos for ipv4-mapped ipv6 socket
selftests/bpf: Avoid static LLVM linking for cross builds
selftests/bpf: Use common CFLAGS for urandom_read
selftests/bpf: Initialize operation name before use
tools/bpf: build: Append extra cflags
libbpf: Initialize CFLAGS before including Makefile.include
bpftool: Append extra host flags
bpftool: Avoid adding EXTRA_CFLAGS to HOST_CFLAGS
bpftool: Pass host flags to bootstrap libbpf
selftests/bpf: correct CONFIG_PPC64 macro name in comment
...
With the sockmap + kTLS tests gone, the BPF-side support in test_sockmap
is dead: the tls_sock_map map and bpf_prog3 (which redirected skbs into
it) are no longer referenced. Remove them, along with the now-unused
bpf_write_pass() helper.
bpf_prog3 was progs[2], so renumber the progs[] users in test_sockmap.c:
the sockops program drops to progs[2] and the sk_msg tx programs to
progs[3..7]. Shrink the map/prog arrays from 9 to 8 and drop the
tls_sock_map entry (the last one) from map_names[] to match.
Link: https://patch.msgid.link/20260614014102.461064-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The combination of sockmap and TLS is no longer supported - installing
the TLS ULP on a sockmap socket (and vice versa) is now rejected. Remove
the tests that exercise the combination along with their BPF program;
the file covered nothing but sockmap sockets holding kTLS contexts.
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://patch.msgid.link/20260614014102.461064-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
clang 23 fails to build crypto_bench.c and crypto_sanity.c with
"BPF stack limit exceeded". The progs fill a 408-byte
bpf_crypto_params on the stack and pass it to bpf_crypto_ctx_create().
clang 23 copies the byte-aligned cipher/key globals into it one byte at
a time through the stack, and keeps more than one copy of the struct
around. Together that blows the 512-byte limit.
Align the source arrays to 8 bytes so the copy is word-wise, and move
params off the stack into a static .bss var. static keeps it out of the
skeleton, where bpf_crypto_params is an incomplete type. Either change
alone is not enough.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a test in sockmap_basic.c that calls bpf_msg_pop_data() with a length
close to U32_MAX, which overflows the start + len bounds check. The sk_msg
program records the return value over a sendmsg and the test checks that
the call is rejected with -EINVAL.
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Cc: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260615021959.140010-7-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Verify the fix by:
1. Attach cgroup sockops prog.
2. Build a tcp connection using ipv4 addr in ipv6 socket.
3. Verify the return value of bpf_setsockopt() helper.
Assisted-by: Codex:gpt-5.5-xhigh
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/r/20260613162443.60515-3-leon.hwang@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
A comment in tools/testing/selftests/bpf/progs/test_fill_link_info.c
incorrectly refers to CONFIG_PPC6 instead of CONFIG_PPC64. Correct it.
Discovered while searching for CONFIG_* symbols referenced in code but
not defined in any Kconfig file.
Signed-off-by: Ethan Nelson-Moore <enelsonmoore@gmail.com>
Link: https://lore.kernel.org/r/20260610044023.225820-1-enelsonmoore@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add test to check the return value when a BPF program exits with 0 for
a boolean and an errno LSM hook.
For each hook, two BPF programs are attached. The first program returns
0 without calling bpf_set_retval() to exercise the return value translation
logic, while the second program reads the retval via bpf_get_retval().
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
Link: https://lore.kernel.org/r/20260610201724.733943-3-xukuohai@huaweicloud.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add tail-call selftests for prog-array ownership when cgroup storage
is in use. Verify that loading succeeds when callers and callees reuse
the owner's cgroup storage map, and that loading fails for a different
storage map and for the A(storage) -> B(no storage) -> C(storage)
bridge case addressed in the previous commit.
Also verify that a storage-less leaf program which cannot perform tail
calls itself is still allowed to join a storage-owned prog array, while
a storage-less tail-caller is rejected also at map update time.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t tailcalls
[...]
#475/25 tailcalls/tailcall_freplace:OK
#475/26 tailcalls/tailcall_bpf2bpf_freplace:OK
#475/27 tailcalls/tailcall_failure:OK
#475/28 tailcalls/reject_tail_call_spin_lock:OK
#475/29 tailcalls/reject_tail_call_rcu_lock:OK
#475/30 tailcalls/reject_tail_call_preempt_lock:OK
#475/31 tailcalls/reject_tail_call_ref:OK
#475/32 tailcalls/tailcall_sleepable:OK
#475/33 tailcalls/tailcall_cgrp_storage:OK
#475/34 tailcalls/tailcall_cgrp_storage_diff_storage:OK
#475/35 tailcalls/tailcall_cgrp_storage_no_storage:OK
#475/36 tailcalls/tailcall_cgrp_storage_no_storage_leaf:OK
#475/37 tailcalls/tailcall_cgrp_storage_no_storage_bridge:OK
#475 tailcalls:OK
Summary: 1/37 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Lin Ma <malin89@huawei.com>
Signed-off-by: Rongzhen Cui <cuirongzhen@huawei.com>
Signed-off-by: Jingguo Tan <tanjingguo@huawei.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260610105539.705887-2-daniel@iogearbox.net
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
For selftest bpf_iter/task_vma, I got a failure like below on my qemu run:
test_task_vma_common:FAIL:compare_output unexpected compare_output:
actual
'561593546000-561593585000r--p0000000000:241256579534/root/devshare/bpf-next/tools/testing/selftests/bpf/test_progs'
!= expected
'561593546000-561593585000r--p0000000000:245551546830/root/devshare/bpf-next/tools/testing/selftests/bpf/test_progs'
Further debugging found out file->f_inode->i_ino value may exceed 32bit,
e.g., i_ino = 0x14c2eae35, but the format string is '%u'. This caused
inode mismatch between bpf iter and proc result.
Fix the issue by using format string '%llu' to accommodate 64bit i_ino.
Fixes: e8168840e1 ("selftests/bpf: Add test for bpf_iter_task_vma")
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/r/20260610051831.1346659-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add focused map_kptr coverage for BPF-side map updates that touch values
containing referenced kptrs.
The new syscall programs stash the testmod refcounted object in an array
map, a preallocated hash map, and a no-prealloc hash map, then update the
same map from BPF. The refcount must remain elevated after the update,
while the userspace runner destroys the skeleton and reuses the existing
refcount wait to confirm map teardown releases the kptr.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260609202548.3571690-5-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add task_kfunc failure cases for bpf_obj_drop() on local objects with
referenced kptr fields from tracing and NMI tracing programs. These programs
must be rejected because dropping the object would run full special-field
destruction synchronously in an unsafe context.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260609202548.3571690-4-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Map update and delete paths currently call bpf_obj_free_fields() when a
value is being replaced or recycled. That makes field destruction depend
on the context of the update/delete operation. For tracing programs this
can include NMI context, where referenced kptr destructors, uptr
unpinning, and graph root destruction are not generally safe.
Introduce bpf_obj_cancel_fields() for the reusable-value path. It only
performs NMI-safe cleanup for timer, workqueue, and task_work fields.
Fields that need full destruction are left attached to the recycled value
and are destroyed by the final cleanup path instead.
Switch array and hashtab update/delete/recycle paths to this cancel
helper. Keep bpf_obj_free_fields() for final map destruction and for
bpf_mem_alloc destructors. Preallocated hashtabs do not have allocator
destructors, so teardown continues to walk the normal and extra elements
and fully destroy their fields.
This deliberately relaxes the eager-free semantics of map update/delete
for special fields. Programs that relied on a recycled map slot becoming
empty immediately after update/delete were relying on behavior that
cannot be implemented safely from every BPF execution context without
offloading arbitrary destructors.
There is a chance this change breaks programs making assumptions
regarding the eager freeing of fields. If so, we can relax semantics to
cancellation only when irqs_disabled() is true in the future. However,
theoretically, map values that get reused eagerly already have weaker
guarantees as parallel users can recreate freed fields before the new
element becomes visible again.
Fixes: 14a324f6a6 ("bpf: Wire up freeing of referenced kptr")
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Co-developed-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260609202548.3571690-3-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_obj_drop() runs bpf_obj_free_fields() synchronously for
program-allocated objects. When such an object contains NMI unsafe
fields, tracing programs that can run from arbitrary instrumented
context can reach that destruction from unsafe contexts, including NMI.
NMI is likely one instance of this problem, and other instances would
include possible unsafe reentrancy. Deferring bpf_obj_drop() is not
appealing either: it would add delayed-free machinery to a release
operation that otherwise has straightforward synchronous ownership
semantics.
Reject bpf_obj_drop() and bpf_percpu_obj_drop() from tracing programs
that may run from unsafe contexts unless every field in the object's BTF
record is explicitly NMI safe. Do not reject sleepable
BPF_PROG_TYPE_TRACING programs, since they are not the arbitrary/NMI
contexts that motivate the restriction.
Note that while bpf_rb_root and bpf_list_head would be NMI safe on their
own to free, the objects recursively held by them may not be; be
conservative and just mark them as not NMI safe for now.
Use a whitelist for the NMI-safe field set instead of listing only known
NMI unsafe fields. Locks, async fields, unreferenced kptrs, and
refcounts are known to be NMI safe because their destruction is either a
no-op, simple state reset, or async cancellation. Referenced kptrs,
percpu referenced kptrs, uptrs, graph roots, graph nodes, and any future
field type are rejected until audited for arbitrary tracing and NMI
contexts. This is less susceptible to future changes in fields that were
previously safe by exclusion, and to new fields being added without
updating this check.
Convert the existing recursive local-object drop success case to a
syscall program in the same commit, since this verifier change makes the
old tracing program form invalid. The test still exercises
bpf_obj_drop() releasing a referenced task kptr from a safe program
type.
Fixes: ac9f06050a ("bpf: Introduce bpf_obj_drop")
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Co-developed-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260609202548.3571690-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
test_pkt_access_subprog2() is defined in C as
int test_pkt_access_subprog2(int val, volatile struct __sk_buff *skb)
but llvm optimizes away the unused 'int val' argument. Before llvm23 the
BTF signature did not match the optimized assembly, so the verifier set
attach_func_proto to NULL and fell back to MAX_BPF_FUNC_REG_ARGS (5) u64
arguments (see btf_ctx_access()). The fexit ctx struct therefore placed
the return value after args[5].
With llvm23 the 'true' signature
int test_pkt_access_subprog2(volatile struct __sk_buff *skb)
is recorded in BTF, so nr_args becomes 1 and the return value moves to
the slot right after args[1]. Select the matching args_subprog2 layout
based on __clang_major__ so the test works with both old and new llvm.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260609233412.2712178-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
LLVM23 ([1]) supports 'true' function signature in BTF. The return type
of the caller of a tailcall must be an 'int'. Otherwise, verification will
fail (see check_btf_func() in check_btf.c). So with llvm23, it is possible
that the compiler may change the caller's return type from 'int' to 'void'.
To prevent this, barrier_var() and __sink() are used to avoid returning
a constant prone to be optimized.
[1] https://github.com/llvm/llvm-project/pull/198426
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260609233407.2711577-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a verifier test for passing a BTF-backed task_struct field pointer to a
global subprogram argument typed as writable memory.
The direct field store is already rejected.
The global subprogram path should be rejected too.
The callee must not lose the BTF pointer's read-only provenance.
It must not validate the argument as ordinary writable memory.
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Link: https://lore.kernel.org/bpf/20260609-f01-04-btf-writable-arg-v1-2-f449cd970669@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Add a verifier regression test that looks up a constant key through a
dynamic inner array template and dereferences the result without a NULL
check.
The verifier must reject the program because BPF_F_INNER_MAP allows the
concrete runtime array to have fewer entries than the template.
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260607-f01-v2-v2-2-da48453146e8@mails.tsinghua.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Introduces stress test for bpf_lru_list that exercises
lock-failures and orphan-recovery, added by the LRU rqspinlock
conversion.
Runs three subtests: common LRU, per-CPU LRU lists (BPF_F_NO_COMMON_LRU),
and per-CPU LRU map. Each pins one userspace hammer per CPU and attaches
the perf_event NMI BPF prog (update+delete mix) on every online CPU.
Pre-fix, lockdep fires the "INITIAL USE -> IN-NMI" splat during stress.
After stress test, drain_then_verify_capacity() drains every key
and refills the lru map.
A stranded node on any CPU's pool would have forced eviction of
a just-inserted key on that CPU, surfacing here as a missing lookup.
Marked serial_ because per-CPU pinning and high-rate HW perf events
would perturb parallel tests.
Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://lore.kernel.org/r/20260607-lru_map_spin-v3-3-bcd9332e911b@meta.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Adding tests for the rollback code when the tracing_multi
link won't get attached, covering 2 reasons:
- wrong btf id passed by user, where all previously allocated
trampolines will be released
- trampoline for requested function is fully attached (has already
maximum programs attached) and the link fails, the rollback code
needs to release all previously link-ed trampolines and release
them
We need the bpf_fentry_test* unattached for the tests to pass,
so the rollback tests are serial.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260606123955.345967-30-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Adding benchmark test that attaches to (almost) all allowed tracing
functions and display attach/detach times.
# ./test_progs -t tracing_multi_bench_attach -v
bpf_testmod.ko is already unloaded.
Loading bpf_testmod.ko...
Successfully loaded bpf_testmod.ko.
serial_test_tracing_multi_bench_attach:PASS:btf__load_vmlinux_btf 0 nsec
serial_test_tracing_multi_bench_attach:PASS:tracing_multi_bench__open_and_load 0 nsec
serial_test_tracing_multi_bench_attach:PASS:get_syms 0 nsec
serial_test_tracing_multi_bench_attach:PASS:bpf_program__attach_tracing_multi 0 nsec
serial_test_tracing_multi_bench_attach: found 51186 functions
serial_test_tracing_multi_bench_attach: attached in 1.295s
serial_test_tracing_multi_bench_attach: detached in 0.243s
#507 tracing_multi_bench_attach:OK
Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED
Successfully unloaded bpf_testmod.ko.
Exporting skip_entry as is_unsafe_function and using it in the test.
Also updating trace_blacklist with ___migrate_enable to be in sync
with kernel functions deny list.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260606123955.345967-29-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Adding tracing multi tests for intersecting attached functions.
Using bits from (from 1 to 16 values) to specify (up to 4) attached
programs, and randomly choosing bpf_fentry_test* functions they are
attached to.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260606123955.345967-24-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Adding tests for tracing_multi link attachment via all possible
libbpf apis - skeleton, function pattern and btf ids on top of
bpf_testmod kernel module.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260606123955.345967-23-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Adding tests for tracing_multi link attachment via all possible
libbpf apis - skeleton, function pattern and btf ids.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260606123955.345967-22-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add verifier tests covering pointer arithmetic on a PTR_TO_FLOW_KEYS
register. This covers the bpf-next regression where an out-of-bounds
constant offset introduced as flow_keys += K and then dereferenced at
insn->off 0 was accepted, while the equivalent flow_keys + K direct offset
was rejected.
The tests check that in-bounds constant arithmetic on the keys pointer is
still accepted, out-of-bounds constant arithmetic is rejected for both read
and write, and a truly varying offset from bpf_get_prandom_u32() remains
rejected by the existing PTR_TO_FLOW_KEYS pointer arithmetic rules.
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260606-c3-01-v3-v3-2-97c51f592f15@mails.tsinghua.edu.cn
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a minimal BPF LSM program on lsm/bpf_prog_load that, for loads on
the monitored thread, reads back prog->aux->sig.{verdict,keyring_type,
keyring_serial}, and a signed_loader subtest that drives the same
gen_loader loader through the hook twice: i) /unsigned/ where the LSM
must observe UNSIGNED, no keyring and serial 0; ii) /signed/ where the
very same insns signed against the session keyring must be observed as
VERIFIED with a user keyring, and the recorded keyring_serial must be
equal to the resolved session keyring serial. Loading (not running) the
loader is sufficient since the verdict is attached at load time.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t signed_loader
[ 1.970530] clocksource: Switched to clocksource tsc
#405/1 signed_loader/metadata_check_shape:OK
#405/2 signed_loader/metadata_match:OK
#405/3 signed_loader/metadata_sha_mismatch:OK
#405/4 signed_loader/metadata_not_exclusive:OK
#405/5 signed_loader/metadata_hash_not_computed:OK
#405/6 signed_loader/signature_enforced:OK
#405/7 signed_loader/signature_too_large:OK
#405/8 signed_loader/signature_bad_keyring:OK
#405/9 signed_loader/metadata_ctx_max_entries_ignored:OK
#405/10 signed_loader/metadata_ctx_initial_value_ignored:OK
#405/11 signed_loader/signature_authenticates_insns:OK
#405/12 signed_loader/hash_requires_frozen:OK
#405/13 signed_loader/no_update_after_freeze:OK
#405/14 signed_loader/freeze_writable_mmap:OK
#405/15 signed_loader/no_writable_mmap_frozen:OK
#405/16 signed_loader/map_hash_matches_libbpf:OK
#405/17 signed_loader/map_hash_multi_element:OK
#405/18 signed_loader/map_hash_bad_size:OK
#405/19 signed_loader/map_hash_unsupported_type:OK
#405/20 signed_loader/lsm_signature_verdict:OK
#405 signed_loader:OK
Summary: 1/20 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/r/20260605213518.544262-2-daniel@iogearbox.net
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Since commit 68a99f6a0e ("media: lirc: report ir receiver overflow"),
the rc-loopback driver does not accept edges over 50ms, as these are
never seen in real life ir protocols. Fix this.
Signed-off-by: Sean Young <sean@mess.org>
Link: https://lore.kernel.org/r/20260605151417.777614-1-sean@mess.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Test sk_bypass_prot_mem passes an unchecked value as argument to helper
bpf_set_retval(). The argument can be outside the valid range enforced
by the strict retval validation added in the next patch.
Restrict the argument to -EFAULT when it is outside the valid range, so
the test will not be rejected by the verifier when retval validation
is enforced.
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
Link: https://lore.kernel.org/r/20260605140243.664590-2-xukuohai@huaweicloud.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add two tests to verify the transport header of skb has been set when
encapsulate VxLAN using bpf_lwt_push_encap() helper.
1. VxLAN over IPv4.
2. VxLAN over IPv6.
Without the fix, the tests would fail:
lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 70 != expected 20
#208 lwt_ip_encap_vxlan_ipv4:FAIL
lwt_ip_encap_vxlan:FAIL:transport_hdr offset unexpected transport_hdr offset: actual 110 != expected 40
#209 lwt_ip_encap_vxlan_ipv6:FAIL
The unexpected offsets are: outer encap headers
(IPv4: iphdr+udp+vxlan+eth = 50 bytes, IPv6: ipv6hdr+udp+vxlan+eth = 70 bytes)
plus the inner IP header (20 or 40 bytes), because without the fix
transport_header still points at the inner transport layer instead of the
outer UDP header.
Assisted-by: Claude:claude-sonnet-4-6
Cc: Leon Hwang <leon.huangfu@shopee.com>
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/r/20260602150931.49629-3-leon.hwang@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
On RISC-V, the current task pointer is stored in the thread pointer
register (tp). Emit a single `mv a5, tp` instead of a full helper
call for BPF_FUNC_get_current_task and BPF_FUNC_get_current_task_btf.
Register bpf_jit_inlines_helper_call() entries for both helpers so the
verifier treats them as inlined, and add the expected `mv a5, tp`
annotation to the riscv64 selftests.
The following show changes before and after this patch.
Before patch:
auipc t1,0x817a # load upper PC-relative address
jalr -2004(t1) # call bpf_get_current_task helper
mv a5,a0 # move return value to BPF_REG_0
After patch:
mv a5,tp # directly: a5 = current (tp = thread pointer)
Benchmark (bpf_prog_test_run wrapping bpf_get_current_task in loop,
batch=100, 10s, QEMU RISC-V):
| runs/sec | helper-calls/sec | ns/call
-------------+-----------+------------------+---------
Before patch | 173,490 | 17,349,090 | 57
After patch | 320,497 | 32,049,780 | 31
-------------+-----------+------------------+---------
Improvement | +84.7% | +84.7% | -45.6%
Signed-off-by: Varun R Mallya <varunrmallya@gmail.com>
Acked-by: Björn Töpel <bjorn@kernel.org>
Link: https://lore.kernel.org/r/20260602205847.102825-3-varunrmallya@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
use_file_dynptr_slice_after_put_file() reads the dynptr via
bpf_dynptr_data(), which always returns NULL for a read-only file
dynptr, making the example confusing. Switch to bpf_dynptr_slice(), the
correct read API for file dynptrs, and read (rather than write) the slice
since it is read-only. The test still fails as expected.
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/r/20260605202056.1780352-6-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
file_reader/on_open_expect_fault test expects page fault
when reading pages from the test harness executable.
It is not guaranteed that those are paged out, even
after madvise(MADV_PAGEOUT).
Relax the condition in the test to succeed with both
0 and -EFAULT returned.
Fixes: 784cdf9315 ("selftests/bpf: add file dynptr tests")
Reported-by: Shung-Hsi Yu <shung-hsi.yu@suse.com>
Closes: https://lore.kernel.org/all/ah6g7JSYOWGp2oAG@u94a/
Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Tested-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260603-file_reader_flake-v1-1-7f3f52d1e388@meta.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Test basic BPF iterator functionality for BPF_MAP_TYPE_RHASH,
verifying all elements are visited.
Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://lore.kernel.org/r/20260605-rhash-v7-10-5b8e05f8630d@meta.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When running the selftests on a retbleed-affected platform (eg:
Skylake), with call depth accounting enabled
(CONFIG_CALL_DEPTH_TRACKING=y) _and_ with retbleed=stuff, some verifier
selftests fail to validate the jited instructions. For example:
MATCHED SUBSTR: ' endbr64'
MATCHED SUBSTR: ' nopl (%rax,%rax)'
MATCHED SUBSTR: ' xorq %rax, %rax'
MATCHED SUBSTR: ' pushq %rbp'
MATCHED SUBSTR: ' movq %rsp, %rbp'
MATCHED SUBSTR: ' endbr64'
MATCHED SUBSTR: ' cmpq $0x21, %rax'
MATCHED SUBSTR: ' ja L0'
MATCHED SUBSTR: ' pushq %rax'
MATCHED SUBSTR: ' movq %rsp, %rax'
MATCHED SUBSTR: ' jmp L1'
MATCHED SUBSTR: 'L0: pushq %rax'
MATCHED SUBSTR: 'L1: pushq %rax'
MATCHED SUBSTR: ' movq -0x10(%rbp), %rax'
WRONG LINE REGEX: ' callq 0x{{.*}}'
Those affected selftests allways fail on some call instruction: this
failure is due to the JIT compiler emitting call depth accounting for
retbleed mitigation (see x86_call_depth_emit_accounting calls in
bpf_jit_comp.c), resulting in an additional instruction being inserted
in front of every call instruction, similar to this one:
sarq $0x5, %gs:-0x39882741(%rip)
Fix those selftests by allowing them to ignore this possibly present
call depth accounting instruction.
Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/r/20260528-fix_tests_for_retbleed_stuff-v1-1-c2022a1f3bee@bootlin.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The positive path for signed BPF loaders is covered today by the
signed lskels (fentry_test, fexit_test, atomics).
But the runtime metadata check the generated loader performs (libbpf
gen_loader's emit_signature_match), the map content hash it relies
on, the load-time signature, and the immutability invariants of its
metadata map are not yet covered.
Thus, add a new, extensive test suite which drives libbpf's gen_loader
(bpf_object__gen_loader, gen_hash=true), the same machinery which
bpftool uses for signed light skeletons, and exercise corner cases
so that we can assert this in BPF CI:
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t signed_loader
[...]
[ 1.840842] clocksource: Switched to clocksource tsc
#405/1 signed_loader/metadata_check_shape:OK
#405/2 signed_loader/metadata_match:OK
#405/3 signed_loader/metadata_sha_mismatch:OK
#405/4 signed_loader/metadata_not_exclusive:OK
#405/5 signed_loader/metadata_hash_not_computed:OK
#405/6 signed_loader/signature_enforced:OK
#405/7 signed_loader/signature_too_large:OK
#405/8 signed_loader/signature_bad_keyring:OK
#405/9 signed_loader/metadata_ctx_max_entries_ignored:OK
#405/10 signed_loader/metadata_ctx_initial_value_ignored:OK
#405/11 signed_loader/signature_authenticates_insns:OK
#405/12 signed_loader/hash_requires_frozen:OK
#405/13 signed_loader/no_update_after_freeze:OK
#405/14 signed_loader/freeze_writable_mmap:OK
#405/15 signed_loader/no_writable_mmap_frozen:OK
#405/16 signed_loader/map_hash_matches_libbpf:OK
#405/17 signed_loader/map_hash_multi_element:OK
#405/18 signed_loader/map_hash_bad_size:OK
#405/19 signed_loader/map_hash_unsupported_type:OK
#405 signed_loader:OK
Summary: 1/19 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/r/20260603211658.471212-2-daniel@iogearbox.net
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
sashiko complained that 38498c0eba ("selftests/bpf: Adjust verifier_map_ptr
for the map's excl field") would slightly decrease the test coverage given
before the test was against the verifier rejecting the ops pointer. Recover
the old test with the right offsets and add the existing one as an additional
test case.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_map_ptr
[ 1.672932] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel
#637/1 verifier_map_ptr/bpf_map_ptr: read with negative offset rejected:OK
#637/2 verifier_map_ptr/bpf_map_ptr: read with negative offset rejected @unpriv:OK
#637/3 verifier_map_ptr/bpf_map_ptr: write rejected:OK
#637/4 verifier_map_ptr/bpf_map_ptr: write rejected @unpriv:OK
#637/5 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected:OK
#637/6 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected @unpriv:OK
#637/7 verifier_map_ptr/bpf_map_ptr: read beyond excl field rejected:OK
#637/8 verifier_map_ptr/bpf_map_ptr: read beyond excl field rejected @unpriv:OK
#637/9 verifier_map_ptr/bpf_map_ptr: read ops field accepted:OK
#637/10 verifier_map_ptr/bpf_map_ptr: read ops field accepted @unpriv:OK
#637/11 verifier_map_ptr/bpf_map_ptr: r = 0, map_ptr = map_ptr + r:OK
#637/12 verifier_map_ptr/bpf_map_ptr: r = 0, map_ptr = map_ptr + r @unpriv:OK
#637/13 verifier_map_ptr/bpf_map_ptr: r = 0, r = r + map_ptr:OK
#637/14 verifier_map_ptr/bpf_map_ptr: r = 0, r = r + map_ptr @unpriv:OK
#637 verifier_map_ptr:OK
[...]
Summary: 2/20 PASSED, 0 SKIPPED, 0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/r/20260602133052.423725-4-daniel@iogearbox.net
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add selftests that combine the new type-based __arena identifier with
the volatile qualifier both in functions' arguments and return values.
This way we test both that they are recognized as arena arguments and
that they are not sensitive to the position they are placed in the type
compared to other qualifiers.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/r/20260602004120.17087-7-emil@etsalapatis.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Now that BPF __arg_arena has been subsumed by __arena, remove
__arg_arena from the codebase. This way the user has one fewer
annotation to worry about.
To remove __arg_arena we remove the typedefs we were previously
using to minimize __arena annotations. This is because __arena
now also includes a BTF type tag, which is ignored for non-pointer
types. As a result, we cannot capture the whole __arena annotation
inside a typedef and need to directly annotate the pointer type when
declaring the variable.
The extra verbosity is worth it because the use of the __arena tag
is intuitive to the programmer and removes the __arg_arena tag that
has been a consistent source of confusion for users. The typedefs
can be reintroduced later (without __arg_arena) once compilers start
supporting BTF type tags for non-pointer types.
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/r/20260602004120.17087-5-emil@etsalapatis.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
File dynptr and slice should be invalidated when the parent file's
reference is dropped in the program. Without the verifier tracking
dyntpr's parent referenced object, the dynptr would continute to be
incorrectly used even if the underlying file is being tear down or gone.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/r/20260529014936.2811085-13-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The parent object of a cloned dynptr is skb not the original dynptr.
Invalidate the original dynptr should not prevent the program from
using the slice derived from the cloned dynptr.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/r/20260529014936.2811085-12-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The verifier currently does not allow creating dynptr from dynptr data
or slice. Add a selftest to test this explicitly.
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/r/20260529014936.2811085-11-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Introduce release_reg() to consolidate the release logic shared by both
helpers and kfuncs: dynptr release, kptr_xchg percpu-to-RCU conversion,
regular reference release, and NULL pass-through. NULL pass-through is
only allowed if the prototype indicates the argument may be null.
Determine release_regno from the function prototype/metadata before
argument checking, rather than discovering it dynamically during
argument processing. For helpers, scan the arg_type array in
check_func_proto() via check_proto_release_reg(). For kfuncs, set
release_regno to BPF_REG_1 in bpf_fetch_kfunc_arg_meta() when
KF_RELEASE is set. In the future when we start adding decl_tag to
kfunc arguments, we can just look at the function prototype instead
of a release_regno.
Extract ref_convert_alloc_rcu_protected() and
invalidate_rcu_protected_refs() to make it more clear what the code is
doing. For ref_convert_alloc_rcu_protected(), it pre-converts
MEM_ALLOC | MEM_PERCPU registers to MEM_RCU (clearing id so they
survive), then calls release_reference() to invalidate the remaining
registers and release the reference state.
Add KF_RELEASE to bpf_dynptr_file_discard() so its release_regno is set
via fetch_kfunc_meta rather than being assigned manually in the dynptr
argument processing. Set arg_type to ARG_PTR_TO_DYNPTR for
KF_ARG_PTR_TO_DYNPTR so that check_func_arg_reg_off() correctly allows
non-zero stack offsets for dynptr release arguments same as helper.
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/r/20260529014936.2811085-9-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Helpers and kfuncs independently tracked referenced object metadata
using standalone id fields in their respective arg_meta structs.
This led to duplicated logic and inconsistent error handling between the
two paths.
Introduce struct ref_obj_desc to consolidate id and parent_id along with
a count of how many arguments carry a reference. Add update_ref_obj() to
populate it from a bpf_reg_state, replacing open-coded assignments in
check_func_arg(), check_kfunc_args(), and process_iter_arg(). Add
validate_ref_obj() to check for ambiguous ref_obj before using it.
For ref_obj releasing helpers and kfuncs, keep checking it before
calling update_ref_obj() for now. A later patch will make these
functions not depending on ref_obj. For other users of ref_obj, move the
checks to the use locations. For helper, this means moving the checks
inside helper_multiple_ref_obj_use() to use locations.
is_acquire_function() is dropped as ref_obj is never used.
Pass ref_obj_desc into process_dynptr_func()/mark_stack_slots_dynptr()
instead of a bare parent_id to make it less confusing.
Drop the selftest introduced in 7ec899ac90 ("selftests/bpf: Negative
test case for ref_obj_id in args") since the verifier no longer
complains about ambiguous ref_obj if it is not used.
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/r/20260529014936.2811085-8-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
unmark_stack_slots_dynptr() already makes sure that CONST_PTR_TO_DYNPTR
cannot be released. process_dynptr_func() also prevents passing
uninitialized dynptr to helpers expecting initialized dynptr. Now that
unmark_stack_slots_dynptr() also reports error returned from
release_reference(), there should be no reason to keep these redundant
checks.
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/r/20260529014936.2811085-7-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Refactor object relationship tracking in the verifier and fix a dynptr
use-after-free bug where file/skb dynptrs are not invalidated when the
parent referenced object is freed.
Add parent_id to bpf_reg_state to precisely track child-parent
relationships. A child object's parent_id points to the parent object's
id. This replaces the PTR_TO_MEM-specific dynptr_id.
Remove ref_obj_id from bpf_reg_state by folding its role into the
existing id field. Previously, id tracked pointer identity for null
checking while ref_obj_id tracked the owning reference for lifetime
management. These are now unified: acquire helpers and kfuncs set id
to the acquired reference id, and release paths use id directly.
Add reg_is_referenced() which checks if a register is referenced by
looking up its id in the reference array. This replaces all former
ref_obj_id checks.
For release_reference(), invalidating an object now also invalidates
all descendants by traversing the object tree. This is done using
stack-based DFS to avoid recursive call chains of release_reference() ->
unmark_stack_slots_dynptr() -> release_reference(). Referenced objects
encountered during tree traversal are reported as leaked references.
Add parent_id to bpf_reference_state to enable hierarchical reference
tracking. When acquiring a reference, a parent_id can be specified to
link the new reference to an existing one (e.g., referenced dynptrs
acquire a reference with parent_id linking to the parent object's
reference).
Pointer casting:
For pointer casting helpers (bpf_sk_fullsock, bpf_tcp_sock), instead of
propagating ref_obj_id, the cast result reuses the same reference id as
the source pointer. Since the cast may return NULL for a non-NULL input,
the NULL case is explored as a separate verifier branch. This allows
releasing any of the original or cast pointers to invalidate all others.
Referenced dynptrs:
When constructing a referenced dynptr, acquire a intermediate reference
with parent_id linking to the parent referenced object. The dynptr and
all clones share the same parent_id (pointing to the intermediate ref)
but get unique ids for independent slice tracking. Releasing a
referenced dynptr releases the parent reference, which in turn
invalidates all clones and their derived slices.
Owning to non-owning reference conversion:
After converting owning to non-owning by clearing id (e.g.,
object(id=1) -> object(id=0)), the verifier releases the reference
state via release_reference_nomark().
Note that the error message "reference has not been acquired before" in
the helper and kfunc release paths is removed. This message was already
unreachable. The verifier only calls release_reference() after
confirming the reference is valid, so the condition could never trigger
in practice.
Fixes: 870c28588a ("bpf: net_sched: Add basic bpf qdisc kfuncs")
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260529014936.2811085-6-ameryhung@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Global subprogram argument checking derives generic pointer sizes from BTF
and passes the resolved size to check_mem_reg() as a u32. The access-size
validation path then uses a signed int, and stack pointers negate the value
before calling check_helper_mem_access().
This creates a wrap when BTF describes a pointee size larger than S32_MAX.
For example, a global subprogram argument of type:
int (*p)[0x3fffffff]
has a BTF-resolved pointee size of 0xfffffffc bytes. At a call site the
caller can pass a pointer to a 4-byte stack slot at fp-4. The current
PTR_TO_STACK path computes:
size = -(int)mem_size
so 0xfffffffc becomes -4 as a signed int and the negation validates only
a 4-byte stack range. That range is covered by the caller's stack slot,
so the call is accepted.
The callee is then verified independently with R1 as PTR_TO_MEM and
mem_size 0xfffffffc. A small instruction such as:
r0 = *(u32 *)(r1 + 4)
is accepted as being inside that BTF-described memory region. At run time,
however, the actual argument value is still fp-4, so r1 + 4 addresses fp+0,
outside the 4-byte object that the caller provided.
Reject sizes that cannot be represented by the verifier's signed
access-size API before the stack-specific negation. Add a verifier
regression test for the oversized BTF argument.
Fixes: 2cb27158ad ("bpf: poison dead stack slots")
Signed-off-by: Taegu Ha <hataegu0826@gmail.com>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260528062155.3988156-1-hataegu0826@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
On arm64, the first 8 arguments are passed in registers (x0-x7), so
tests with 8 or fewer arguments never exercise the native stack argument
path in the JIT. Increase argument counts to at least 10 across all
BPF-to-BPF subprog and kfunc stack argument tests so that at least 2
arguments land on the arm64 stack.
For the two-callees test, bump foo1 from 8 to 10 and foo2 from 10 to 12
args to preserve the different-stack-depth flavor of the test.
The bpf_kfunc_call_stack_arg_mem kfunc is left unchanged at 7 args to
avoid breaking the precision backtracking test which relies on hardcoded
verifier log instruction indices.
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Acked-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260528161750.1900674-3-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Simplify fq_flows_is_empty() by replacing the pop/push based emptiness
check with a direct call to bpf_list_empty().
This avoids unnecessary list mutation and simplifies the code while
preserving correctness.
Signed-off-by: Suchit Karunakaran <suchitkarunakaran@gmail.com>
Changes since v1:
- Removed unused variable node
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/r/20260524025853.13786-1-suchitkarunakaran@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Extend refcounted_kptr with tests for bpf_list_add (including prev from
bpf_list_front and bpf_refcount_acquire), bpf_list_del (including node
from bpf_list_front, bpf_rbtree_remove and bpf_refcount_acquire),
bpf_list_empty, bpf_list_is_first/last, and push_back on uninit head.
To verify the validity of bpf_list_del/add, the test also expects the
verifier to reject calls to bpf_list_del/add made without holding the
spin_lock.
Signed-off-by: Kaitao Cheng <chengkaitao@kylinos.cn>
Link: https://lore.kernel.org/r/20260521032306.97118-9-kaitao.cheng@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Test the scenario where a socket is inserted into a sockmap with a
BPF_SK_SKB_VERDICT program before TLS RX is configured. Previously
sk_psock_verdict_data_ready() would call tcp_read_skb() and drain the
receive queue without advancing copied_seq, causing tls_decrypt_sg()
to walk a dangling frag_list pointer (use-after-free).
The test drives the full vulnerable sequence and verifies that after
the fix recv() returns the correct decrypted data.
Signed-off-by: Xingwang Xiang <v3rdant.xiang@gmail.com>
Link: https://patch.msgid.link/20260517145630.20521-3-v3rdant.xiang@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
batch_hash = (batch_gen ^ cpu_id) * KNUTH_HASH_MULT;
When batch_gen == cpu_id the XOR produces zero, batch_hash is zero,
and *saddr ^= 0 is a no-op. Every iteration hits the warm LRU entry.
During validation batch_gen is 2, so running on CPU 2 triggers:
[udp-v4-lru-miss] COUNTER FAIL: LRU misses=0, expected 1
Replace XOR with addition so the multiplier input is always >= 1.
This also preserves the per-CPU salt for multi-producer runs.
Fixes: 4b4f222910 ("selftests/bpf: Add XDP load-balancer BPF program")
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Link: https://lore.kernel.org/r/20260520133338.3392667-2-puranjay@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This patch fixes the "bounds refinement with single-value tnum on umin"
verifier selftest. This selftest was introduced in commit e6ad477d1b
("selftests/bpf: Test refinement of single-value tnum") to cover the
logic from __update_reg64_bounds(), introduced in commit efc11a6678
("bpf: Improve bounds when tnum has a single possible value"). However,
the test still passes if that last commit is reverted.
The test is supposed to cover the case when the tnum and u64 range (or
cnum64 now) overlap in a single value. __update_reg64_bounds() detects
that case and refines the bounds to a known constant. However, the
constants for the test were poorly chosen and the bounds get refined to
a known constant even without __update_reg64_bounds(). The code is as
follows:
0: call bpf_get_prandom_u32#7 ; R0=scalar()
1: r0 |= 224 ; R0=scalar(umin=umin32=224,var_off=(0xe0; 0xffffffffffffff1f))
2: r0 &= 240 ; R0=scalar(smin=umin=smin32=umin32=224,smax=umax=smax32=umax32=240,var_off=(0xe0; 0x10))
3: if r0 == 0xf0 goto pc+2 ; R0=224
After instruction 3, we have u64=[0xe0; 0xef] and tnum=(0xe0; 0x10).
__reg_bound_offset() is able to deduce a new tnum from the u64,
tnum=(0xe0; 0x0f), which combined with the existing tnum gives us a
constant: 0xe0 or 224.
We can easily fix this by choosing different starting bounds. If we make
it u64=[0xe1; 0xf0], then __reg_bound_offset() doesn't have any impact.
Fixes: e6ad477d1b ("selftests/bpf: Test refinement of single-value tnum")
Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com>
Link: https://lore.kernel.org/r/be2dc2c3d85120286e60b3029b3338fff339f942.1779121582.git.paul.chaignon@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add tests to verify that bpf_throw() correctly unwinds the stack
when the program uses outgoing stack arguments (functions with >5
args). Without the preceding x86 fix, these tests crash the kernel
on x86 due to corrupted callee-saved register restore. There is
no change for arm64 to support exception with stack arguments.
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260517150707.289273-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a verifier failure case where the caller holds a reference across a
global subprog call that may throw. The program must be rejected because
the exceptional path would skip the caller's reference release.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260517075530.3461166-3-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
arg_track_join() logs state transitions at CFG merge points. For
stack arg slots (r >= MAX_BPF_REG), it printed "r11:", "r12:", etc.,
which is misleading since r11 is a special register (BPF_REG_PARAMS)
not meaningful to the user.
Fix it to print "sa0:", "sa1:", etc., matching the per-instruction
transition log in arg_track_log() which already uses the "sa" prefix.
Update the existing stack_arg_pruning_type_mismatch selftest to expect
the corrected format.
Fixes: 2af4e79277 ("bpf: Extend liveness analysis to track stack argument slots")
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260515225056.823086-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Commit 2af4e79277 ("bpf: Extend liveness analysis to track stack argument slots")
added stack arg supports. For selftest
verifier_stack_arg/stack_arg: pruning with different stack arg types
the following are two arg JOIN messages:
arg JOIN insn 9 -> 10 r1: fp0-8 + _ => fp0-8|fp0+0
arg JOIN insn 9 -> 10 r11: fp0-8 + _ => fp0-8|fp0+0
Here the "r11:" label for stack arg slot 0 is misleading since r11
is a special register (BPF_REG_PARAMS). The next patch corrects
this to "sa0:", properly representing the 'stack arg slot 0'.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260515225051.822739-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add negative tests for the outgoing stack arg validation.
A static subprog with a 'long *' arg causes
btf_prepare_func_args() to fail after setting arg_cnt. The
validation ensures check_outgoing_stack_args() still runs.
Also update two existing tests (release_ref, stale_pkt_ptr) whose
expected error messages changed: invalidated stack arg slots are now
caught by check_outgoing_stack_args() at the call site instead of
at the callee's dereference.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260515225045.822104-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Introduce a set of BPF selftests to verify the safety and functionality
of wakeup_source kfuncs.
The suite includes:
1. A functional test (test_wakeup_source.c) that iterates over the
global wakeup_sources list. It uses CO-RE to read timing statistics
and validates them in user-space via the BPF ring buffer.
2. A negative test suite (wakeup_source_fail.c) ensuring the BPF
verifier correctly enforces reference tracking and type safety.
3. Enable CONFIG_PM_WAKELOCKS in the test config, allowing creation of
wakeup sources via /sys/power/wake_lock.
A shared header (wakeup_source.h) is introduced to ensure consistent
memory layout for the Ring Buffer data between BPF and user-space.
Signed-off-by: Samuel Wu <wusamuel@google.com>
Link: https://lore.kernel.org/r/20260511174559.659782-3-wusamuel@google.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This patch tests the maximum stack depth reporting in verifier logs,
with a couple special cases covered: fastcall, private stacks (main
subprog & callee), and rounding up to 16 bytes. For that last one, we
need to skip the test when JIT compilation is disabled as the rounding
is then to 32 bytes.
Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/075d22efd4338385a92f13b7817025cc3f04ec60.1778700777.git.paul.chaignon@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Now that arm64 supports stack arguments, enable the existing stack_arg,
stack_arg_kfunc and verifier_stack_arg tests for __TARGET_ARCH_arm64.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260513045204.2403441-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Move the BPF return value register from x7 to x8, freeing x7 for use
as an argument register. AAPCS64 designates x8 as the indirect result
location register; it is caller-saved and not used for argument
passing, making it a suitable home for BPF_REG_0.
This is a prerequisite for stack argument support, which needs x5-x7
to pass arguments 6-8 to native kfuncs following the AAPCS64 calling
convention.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260513045153.2402197-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
