Cover the two loads which used to lose the BPF_PROBE_MEM rewrite, both reached
from an RCU read-side critical section. The purpose of this patch is to assert
load success in order to make sure to not trigger verifier_bug_if() on
bpf_may_fault_on_deref() due to forgotten rewrite of a probed pointer.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t rcu_read_lock
[...]
#332/1 rcu_read_lock/success:OK
#332/2 rcu_read_lock/rcuptr_acquire:OK
#332/3 rcu_read_lock/negative_tests_inproper_region:OK
#332/4 rcu_read_lock/negative_tests_rcuptr_misuse:OK
#332 rcu_read_lock:OK
Summary: 1/4 PASSED, 0 SKIPPED, 0/0 FAILED
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260817141015.878071-2-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
bpf_arena_spin_lock.h defines its 64KB qnodes array in the header, so
every translation unit including it emits a copy. __weak makes them all
resolve to one instance, but bpftool gen object merges only the symbols
and concatenates each input's .addr_space.1 bytes, leaving the surplus
copies unreferenced in the linked object.
libarena links ten such units, so nine copies were dead weight (bytes):
object before after
-----------------------------------------------------
.addr_space.1 in libarena.bpf.o 676200 86376
libarena.skel.h 2100123 892371
libarena_asan.skel.h 2641124 1466477
Declare qnodes in the header and let each program define it once:
libarena in src/common.bpf.c, and the arena_spin_lock test beside the
lock it guards.
Tested with test_progs -t arena_spin_lock and -t libarena.
Signed-off-by: Changwoo Min <changwoo@igalia.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260817160249.655916-1-changwoo@igalia.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Each cgroup_iter_memcg subtest touches 1024 pages and expects the matching
memcg counter to be non-zero. On a host with many CPUs it reads zero
instead:
test_anon:FAIL:final anon mapped val: actual 0 <= expected 0
memcg stats are cached per-cpu and only become visible once the periodic
flusher runs (FLUSH_TIME, 2s), or once pending updates cross
MEMCG_CHARGE_BATCH * num_online_cpus(). That threshold is 512 pages at 8
CPUs but 8192 at 128, so a single pass no longer reaches it and
bpf_mem_cgroup_flush_stats() returns without flushing anything.
Retry the stat generation, sleeping in between, so that a flusher cycle is
always covered. Sleep before dropping the mapping, so that a flusher cycle
landing in the sleep observes the mapped state. nr_anon_mapped and
nr_file_mapped are rmap gauges, and unmapping first would post a matching
negative delta for the flusher to aggregate to a net zero.
test_file asserts on both nr_file_pages and nr_file_mapped, which have
different lifetimes, as page cache pages outlive the mapping. Retry
while either one is still zero.
Fixes: 6bce6ddbe6 ("bpf: selftests: selftests for memcg stat kfuncs")
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260814232017.2839959-1-andrii@kernel.org
add_subprogs() rejects both BPF-to-BPF calls and BPF_PSEUDO_FUNC loads for
unprivileged programs. The latter loads a subprogram address for use as a
callback, but its Policy report currently describes it as a function call and
suggests avoiding calls that the program does not contain.
Select the operation and suggestion from the instruction kind. Preserve the
existing call wording for BPF_PSEUDO_CALL, and describe BPF_PSEUDO_FUNC as a
BPF function reference.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/bpf/d02e6a6d3b2dc43a207b8ba836ce62497b250dede9252e7409c5212201c794b7@mail.kernel.org
Link: https://lore.kernel.org/bpf/20260816015746.2632990-14-memxor@gmail.com
GCC emits BTF line records with a file name and line number, but leaves the
source line string empty. bpf_diag_source() currently treats that empty string
as if the complete line record were unavailable, so diagnostics fall back to
an instruction number and discard the function, file, and line attribution.
Print the available source location before deciding whether source context can
be rendered. When source text is absent, omit only the source context and retain
the diagnostic annotation and instruction context.
Fixes: b9c5d822f6 ("bpf: Add source and instruction diagnostic context")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260816015746.2632990-12-memxor@gmail.com
The Call Type Safety diagnostics mishandle three kfunc argument classes.
BTF type ID 0 represents void, but btf_show_name() also uses zero to end
type traversal. A pointer that resolves to void therefore loses its pointee
name and is rendered as "()". End traversal directly for concrete terminal
types, but resolve referenced types before testing for ID zero, and name the
void terminal type explicitly. Format the complete parameter pointer type
for nullable kfunc arguments, so void pointers are reported as (void *).
Also add the missing structured report when an __szk memory-size argument is
not a verifier-known constant. Describe the generic bpf_refcount_acquire()
contract without deriving an object type from its void pointer prototype.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/668871823f90f69896d3db27b56db2f53e481162.camel@gmail.com
Link: https://lore.kernel.org/bpf/20260816015746.2632990-7-memxor@gmail.com
The main diagnostic identifies the first outgoing stack slot as stack
argument 1 and the sixth function argument. The causal history instead
labels the same value as stack arg6, making it look like a different slot.
Render causal-history targets in the verifier's canonical stack-argument
location form. The first outgoing slot is now shown as *(R11-8), matching
reg_arg_name(), while the main diagnostic retains its fuller slot and
ordinal description.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/eb1be5327d136b7e5bd6d68e76fef6de20c40790.camel@gmail.com
Link: https://lore.kernel.org/bpf/20260816015746.2632990-6-memxor@gmail.com
Add a cgroup getsockopt selftest that sets ctx->optlen to -1. Use
TCP_ZEROCOPY_RECEIVE to exercise the kernel-buffer getsockopt hook.
The userspace-visible result is -EFAULT on both patched and unpatched
kernels, so the return value alone cannot distinguish the bug. The test
still exercises the kernel-buffer getsockopt path with a negative
ctx->optlen, which reproduces the hardened usercopy warning on unpatched
kernels.
Signed-off-by: Junseo Lim <zirajs7@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/4dbdcda23b2f2be06c5659f8102cd6bd036825b3.1786448307.git.zirajs7@gmail.com
A cgroup getsockopt BPF program can shrink ctx->optlen after the
kernel getsockopt handler has run. The kernel-buffer variant, used by
TCP_ZEROCOPY_RECEIVE, only rejects values larger than the original
length.
If BPF writes a negative optlen, that value is accepted and propagated
back to the TCP getsockopt code. It can then be passed to
copy_to_sockptr() as a size_t and trigger the hardened usercopy
bytes > INT_MAX warning.
Reject negative ctx.optlen in __cgroup_bpf_run_filter_getsockopt_kern(),
matching the lower-bound validation already present in the sockptr-based
getsockopt hook.
Fixes: 9cacf81f81 ("bpf: Remove extra lock_sock for TCP_ZEROCOPY_RECEIVE")
Reported-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Junseo Lim <zirajs7@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/187a4d756275aaaee5d65eecb63c1477b3b66554.1786448307.git.zirajs7@gmail.com
tc_tunnel only partially validated decap state and missed some tunnel
cases. In particular, IPXIP decap checks were not exercised for
IPIP/SIT paths, and non-GSO decap encapsulation state was not
verified.
Tighten the test by:
- setting DECAP_IPXIP4/6 flags for IPIP/SIT/IP6 decap paths based on
the outer tunnel header family;
- requiring needed DECAP enum values via CO-RE enum existence checks
so missing kernel support fails fast;
- validating post-decap tunnel state for both GSO and non-GSO packets:
expected gso_type bits must be cleared and skb->encapsulation must
match remaining tunnel flags;
- removing forced TSO disable in the test harness so GSO validation is
exercised.
This improves coverage for decap tunnel-state regressions and ensures
sit_none/ipip-style paths are checked correctly.
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260812083115.73100-7-nhudson@akamai.com
On shrink in bpf_skb_adjust_room(), apply decapsulation state updates
according to BPF_F_ADJ_ROOM_DECAP_* flags.
For GSO skbs, clear only the tunnel gso_type bits that correspond to
the requested decap layer:
- DECAP_L4_UDP: SKB_GSO_UDP_TUNNEL{,_CSUM}
- DECAP_L4_GRE: SKB_GSO_GRE{,_CSUM}
- DECAP_IPXIP4: SKB_GSO_IPXIP4
- DECAP_IPXIP6: SKB_GSO_IPXIP6
Then clear skb->encapsulation only if no tunnel GSO bits remain, keeping
encapsulation set for cases such as ESP-in-UDP where tunnel state remains.
For non-GSO skbs, there are no tunnel GSO bits to consult, so clear
skb->encapsulation directly when DECAP_L4_* or DECAP_IPXIP_* flags are set.
This keeps decap state handling consistent between GSO and non-GSO packets.
Co-developed-by: Max Tottenham <mtottenh@akamai.com>
Co-developed-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Max Tottenham <mtottenh@akamai.com>
Signed-off-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/bpf/20260812083115.73100-6-nhudson@akamai.com
Add checks to require shrink-only decap, reject conflicting decap flag
combinations, and verify removed length is sufficient for claimed header
decapsulation.
Co-developed-by: Max Tottenham <mtottenh@akamai.com>
Co-developed-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Max Tottenham <mtottenh@akamai.com>
Signed-off-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/bpf/20260812083115.73100-5-nhudson@akamai.com
Add new bpf_skb_adjust_room() decapsulation flags:
- BPF_F_ADJ_ROOM_DECAP_L4_GRE
- BPF_F_ADJ_ROOM_DECAP_L4_UDP
- BPF_F_ADJ_ROOM_DECAP_IPXIP4
- BPF_F_ADJ_ROOM_DECAP_IPXIP6
These flags let BPF programs describe which tunnel layer is being
removed, so later changes can update tunnel-related GSO state
accordingly during decapsulation.
This patch only introduces the UAPI flag definitions and helper
documentation.
Co-developed-by: Max Tottenham <mtottenh@akamai.com>
Co-developed-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Max Tottenham <mtottenh@akamai.com>
Signed-off-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/bpf/20260812083115.73100-4-nhudson@akamai.com
Refactor the helper masks for bpf_skb_adjust_room() flags to simplify
validation logic and introduce:
- BPF_F_ADJ_ROOM_ENCAP_MASK
- BPF_F_ADJ_ROOM_DECAP_MASK
Refactor existing validation checks in bpf_skb_net_shrink() and
bpf_skb_adjust_room() to use the new masks (no behavior change).
This is in preparation for supporting the new decap flags.
Co-developed-by: Max Tottenham <mtottenh@akamai.com>
Co-developed-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Max Tottenham <mtottenh@akamai.com>
Signed-off-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/bpf/20260812083115.73100-3-nhudson@akamai.com
The existing anonymous enum for BPF_FUNC_skb_adjust_room flags is
named to enum bpf_adj_room_flags to enable CO-RE (Compile Once -
Run Everywhere) lookups in BPF programs.
Co-developed-by: Max Tottenham <mtottenh@akamai.com>
Co-developed-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Max Tottenham <mtottenh@akamai.com>
Signed-off-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/bpf/20260812083115.73100-2-nhudson@akamai.com
Enable stream_cond_break, may_goto_interaction, and verifier_may_goto_1
tests for LoongArch, aligning with recent architectural infrastructure
support (timed may_goto and arch_bpf_stack_walk JIT).
With this patch, the following tests passed on LoongArch:
sudo ./test_progs -a stream_success/stream_cond_break
sudo ./test_progs -a verifier_bpf_fastcall/may_goto_interaction
sudo ./test_progs -a verifier_may_goto_1
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260813070906.5164-1-yangtiezhu@loongson.cn
_bpf_setsockopt() and _bpf_getsockopt() call sock_owned_by_me() for
full sockets, so these helpers expect the socket lock to be held.
BPF_CGROUP_UNIX_GETPEERNAME and BPF_CGROUP_UNIX_GETSOCKNAME run BPF
programs without acquiring the socket lock. A program attached to
either hook can therefore trigger the sock_owned_by_me() warning by
calling bpf_setsockopt() or bpf_getsockopt().
Disallow bpf_setsockopt() and bpf_getsockopt() for CGROUP_UNIX_GETPEERNAME
and CGROUP_UNIX_GETSOCKNAME.
Fixes: 859051dd16 ("bpf: Implement cgroup sockaddr hooks for unix sockets")
Reported-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Junseo Lim <zirajs7@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://lore.kernel.org/bpf/20260812091654.244752-1-zirajs7@gmail.com
ip_finish_output2() expands an skb to LL_RESERVED_SPACE(dev) before LWT
xmit. An LWT_XMIT BPF program can then modify the skb head and still
return BPF_OK, so bpf_xmit() rechecks the remaining headroom before the
skb continues to neighbour output.
That recheck uses dst->dev->hard_header_len. This is not enough for the
neighbour cached-header path: neigh_hh_output() copies the cached hardware
header using the aligned hh_cache size, HH_DATA_MOD for short headers or
HH_DATA_ALIGN(hh_len) otherwise.
On Ethernet, hard_header_len is 14 but the cached copy needs 16 bytes. If
an LWT_XMIT BPF program calls bpf_skb_change_head(skb, 1, 0), the skb can
still have 15 bytes of headroom after the program. The existing check
accepts that, after which neigh_hh_output() hits its headroom warning and
drops the skb.
Use LL_RESERVED_SPACE(dst->dev) in the post-BPF headroom check to match
the reservation made before LWT xmit.
Fixes: 3a0af8fd61 ("bpf: BPF for lightweight tunnel infrastructure")
Reported-by: Sechang Lim <rhkrqnwk98@gmail.com>
Suggested-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Junseo Lim <zirajs7@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260811044149.118235-1-zirajs7@gmail.com
BPF subsystem requires multi-line comments to have the opening /* start
on its own line. Update multi-line comments in network_helpers.c to follow
this requirement.
Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260814-tc_tunnel_flaky-v5-1-5b93d030c42c@bootlin.com
psock's hold on the looked up socket isn't dropped until sk_psock_drop() ->
queue_rcu_work() -> sk_psock_destroy() runs, which happens only after the
entry is unlinked and an RCU grace period elapses. Since the lookup runs
under RCU, a non-NULL result guarantees sk_refcnt >= 1:
refcount_inc_not_zero() can never fail here. Use sock_hold() instead.
Signed-off-by: Michal Luczaj <mhal@rbox.co>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://lore.kernel.org/bpf/20260813-sockmap-lookup-get-ref-v1-2-31f5d55f44ac@rbox.co
__bpf_sk_lookup() and bpf_sk_lookup() duplicate the same sk_to_full_sk()
reqsk-to-listener upgrade. Extract it into a helper. Leave the currently
unreachable WARN_ONCE as a defensive assert. No functional change.
Signed-off-by: Michal Luczaj <mhal@rbox.co>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
Link: https://lore.kernel.org/bpf/20260813-sockmap-lookup-get-ref-v1-1-31f5d55f44ac@rbox.co
bpf_convert_ctx_accesses() turns a BPF_LDX into a BPF_PROBE_MEM one by
matching the type recorded for the insn against a list of exact pointer
types. The list cannot keep up with the flag combinations the verifier
produces, and a type which is missing from it ends up as a plain load
without an exception table entry, so a bad address panics the kernel
instead of being handled.
Two such types exist today and are reachable:
- PTR_TO_BTF_ID | PTR_UNTRUSTED | MEM_ALLOC | NON_OWN_REF
- PTR_TO_BTF_ID | PTR_UNTRUSTED | MEM_RCU
Rather than adding the two, just drop the list and state the property
itself in the default case of the switch. This is a superset of what
the list matched, the untrusted PTR_TO_MEM does not have to carry
MEM_RDONLY for it anymore, and it stays in sync with the verifier side
which uses the same match in save_aux_ptr_type() and reg_type_mismatch_ok().
Assert that a fault prone type which does not get the rewrite for whatever
reason is rejected at load time rather than left to fault at runtime to
catch any future cases.
Fixes: 1b12171533 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref")
Fixes: 6fcd486b3a ("bpf: Refactor RCU enforcement in the verifier.")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260814215301.709827-4-daniel@iogearbox.net
check_ptr_to_btf_access() allows the program to store before the default
BTF access path gets to reject a non read access. ac65c710cc ("bpf:
Reject writes through untrusted BTF pointers") closed that for a
PTR_UNTRUSTED pointer, but a bare PTR_TO_BTF_ID may fault on a dereference
just the same and is let through.
A BPF_LDX gets the BPF_PROBE_MEM rewrite in bpf_convert_ctx_accesses()
and a bad address is handled, but a BPF_STX does not and cannot, there
is no probed store to rewrite. The store is emitted as a plain one without
an exception table entry and a bad address panics the kernel.
A bpf_qdisc program can reach this, bpf_qdisc_btf_struct_access() permits a
write to Qdisc::limit and Qdisc::next_sched is a plain struct Qdisc pointer
which the walk turns into the compat type:
struct Qdisc *next = sch->next_sched;
next->limit = 1000;
BUG: kernel NULL pointer dereference, address: 0000000000000014
RIP: 0010:bpf_prog_c6e14e7f32c8e325_bpf_fifo_enqueue+0x3a/0x12b
Code: [...] bf e8 03 00 00 <89> 7e 14 41 8b 7f 14 [...]
Kernel panic - not syncing: Fatal exception in interrupt
Fix by widen the check to bpf_may_fault_on_deref() so that it covers both.
Fixes: 27ae7997a6 ("bpf: Introduce BPF_PROG_TYPE_STRUCT_OPS")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260814215301.709827-3-daniel@iogearbox.net
reg_type_mismatch_ok() enumerates the pointer types which must not
silently share a BPF_LDX with a different one, since the type recorded
for the insn drives a rewrite in bpf_convert_ctx_accesses().
f2362a57ae ("bpf: allow void* cast using bpf_rdonly_cast()") added
PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED as another type in need of one,
namely the BPF_PROBE_MEM rewrite, but did not add it there. Fix it by
adding the missing case to reg_type_mismatch_ok(), so that a PTR_TO_MEM
which may fault on deref is not mismatch ok anymore. The triage in
save_aux_ptr_type() then merges them.
Fixes: f2362a57ae ("bpf: allow void* cast using bpf_rdonly_cast()")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260814215301.709827-2-daniel@iogearbox.net
When the same BPF_LDX instruction is reached through paths that yield
different pointer types, save_aux_ptr_type() merges them into a single
type which is later used by bpf_convert_ctx_accesses() to decide whether
the load has to be rewritten into a BPF_PROBE_MEM one.
Before f2362a57ae ("bpf: allow void* cast using bpf_rdonly_cast()")
the merge only accepted two PTR_TO_BTF_ID pointers and unconditionally
fell back to PTR_TO_BTF_ID | PTR_UNTRUSTED, so the merged type was always
one that gets the BPF_PROBE_MEM rewrite. However, the mentioned commit
widened the merge to also cover a PTR_TO_MEM base and replaced the
fallback by a union of the PTR_UNTRUSTED and MEM_RDONLY flags.
A union of flags though cannot express the property the later rewrite
is built upon, some examples:
- PTR_TO_MEM merged with PTR_TO_BTF_ID | PTR_UNTRUSTED gets
PTR_TO_MEM | PTR_UNTRUSTED but only the MEM_RDONLY variant is valid
- PTR_TO_MEM merged with a plain PTR_TO_BTF_ID gets PTR_TO_MEM
dropping the rewrite the latter type would have gotten
- PTR_TO_MEM | MEM_RDONLY merged with a plain PTR_TO_BTF_ID gets
PTR_TO_MEM | MEM_RDONLY which is not rewritten either since only
its PTR_UNTRUSTED variant is
In all three cases a program can take the unsafe path at runtime with a
NULL or otherwise bad pointer and panic the kernel on the faulting load:
BUG: kernel NULL pointer dereference, address: 0000000000000038
RIP: 0010:bpf_prog_77531a87032eeaf1_mixed_mem_btf_id_type+0x4b/0x65
Call Trace:
<TASK>
bpf_test_run+0x20b/0x460
bpf_prog_test_run_skb+0x650/0xbe0
__sys_bpf+0xb96/0x3140
__x64_sys_bpf+0x2c/0x40
do_syscall_64+0xba/0x590
Kernel panic - not syncing: Fatal exception in interrupt
Note that the last two shapes have to be fixed right here, otherwise
the merged type retains nothing which marks the load as fault prone,
thus no rule in bpf_convert_ctx_accesses() can recover it. Fix it by
normalizing the merged type instead.
Reuse it in is_load_acq_unsafe() to avoid open coding, and trim the
overly verbose comment which is more of an implementation detail of
bpf_convert_ctx_accesses() anyway.
Fixes: f2362a57ae ("bpf: allow void* cast using bpf_rdonly_cast()")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260814215301.709827-1-daniel@iogearbox.net
Yiyang Chen says:
====================
bpf: Reject mixed arena and ordinary atomic paths
Atomic RMW instructions use a single aux pointer type to select their final
instruction encoding. The verifier currently records that type only for
PTR_TO_ARENA, allowing a second path with an ordinary pointer to reach the
same instruction before fixups rewrite it to BPF_PROBE_ATOMIC.
Patch 1 records the destination type for every atomic RMW path so the existing
pointer mismatch check rejects incompatible uses of one instruction.
Patch 2 adds a verifier regression test with PTR_TO_ARENA and
PTR_TO_STACK paths converging on one atomic add.
====================
Link: https://patch.msgid.link/20260816-bpf-next-038-mixed-atomic-v1-v2-0-4644c1886dbc@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add a verifier test with one atomic RMW instruction reached through
PTR_TO_ARENA and PTR_TO_STACK paths. The verifier must reject the
shared instruction with the existing incompatible-pointer diagnostic.
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Link: https://patch.msgid.link/20260816-bpf-next-038-mixed-atomic-v1-v2-2-4644c1886dbc@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Atomic RMW verification records an instruction pointer type only when the
current destination is PTR_TO_ARENA. A second path can therefore reach the
same instruction with an ordinary pointer without comparing it against the
saved arena type.
The post-verification fixup uses the saved type to rewrite the instruction
to BPF_PROBE_ATOMIC for every path. Record the actual destination type for
all atomic RMW paths so the existing mismatch check rejects incompatible
uses of one instruction.
Fixes: d503a04f8b ("bpf: Add support for certain atomics in bpf_arena to x86 JIT")
Signed-off-by: Yiyang Chen <chenyy23@mails.tsinghua.edu.cn>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260816-bpf-next-038-mixed-atomic-v1-v2-1-4644c1886dbc@mails.tsinghua.edu.cn
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Because the kfuncs are going through LSM hooks, allowing their use via
workqueue callbacks would expose the wrong credentials. This test
ensures the kfunc are preventing any use from these contexts.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260813110540.103550-6-mahe.tardy@gmail.com
The bpf_ksock_send() kfunc eventually calls security_socket_sendmsg(),
thus creating a possible recursion if a program calling the kfunc is
attached on that specific hook. A filter is added on the kfunc
registration to prevent that at load time from the verifier. This test
exercises that the verifier will reject such program on that attach
point.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260813110540.103550-5-mahe.tardy@gmail.com
Add a selftest that exercises the ksock kfuncs end-to-end. One syscall
BPF setup program creates a ksock context and connects the socket.
Another LSM sleepable BPF program looks up the context and send test
data. The userspace harness creates a network namespace and a new socket
on loopback, run the setup syscall prog and send LSM BPF prog then check
that the userspace socket received the data from BPF.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260813110540.103550-4-mahe.tardy@gmail.com
Add BPF kfuncs that allow BPF LSM programs to create and use sockets for
sending data. This provides a mechanism for BPF programs to emit
telemetry. For this first patch set, it's restricted to SOCK_DGRAM
socket types with IPPROTO_UDP protocol but could be easily extended to
SOCK_STREAM and IPPROTO_TCP in the future.
The API consists of five kfuncs:
bpf_ksock_create() - Create a socket (sleepable)
bpf_ksock_connect() - Connect socket to remote address (sleepable)
bpf_ksock_send() - Send data through the socket (sleepable)
bpf_ksock_acquire() - Acquire a reference to a socket context
bpf_ksock_release() - Release a reference (cleanup via
queue_rcu_work since sock_release sleeps)
The setup kfuncs bpf_ksock_create, bpf_ksock_connect, can be called from
SYSCALL programs only. While bpf_ksock_acquire, bpf_ksock_release and
bpf_ksock_send can be called from SYSCALL and LSM programs.
The implementation follows the established kfunc lifecycle pattern
(create/acquire/release with refcounting, kptr map storage, dtor
registration). The kernel socket is wrapped in a refcounted bpf_ksock
struct. Cleanup is deferred via queue_rcu_work() because sock_release()
may sleep.
The kfuncs are only compiled when CONFIG_INET is enabled, as they
specifically support AF_INET and AF_INET6 sockets.
The socket operations go through the expected LSM hooks instead of
by-passing them like many kernel sockets since those are created by BPF
programs and thus system users. Thus, the bpf_ksock_send() kfunc, which
is exposed to LSM progs has a verifier filter protection to avoid
recursion so that the whole bpf_kfunc_set kfunc set cannot be called in
a program attached to security_socket_sendmsg(). Also, because of the
LSM checks, we prevent the use of the kfuncs from asynchronous workqueue
as the current value would then be invalid.
In bpf_ksock_create(), we copy the arg values to avoid TOCTOU races
since the kfunc can sleep and the arg values could be stored in a map
that could be re-written by BPF progs or even userspace programs if the
map is mmaped.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260813110540.103550-3-mahe.tardy@gmail.com
Add a helper that connects an existing socket while invoking the LSM
hook. Reuse it in __sys_connect_file() to avoid duplicating the connect
logic. Other socket operations have equivalent helpers that trigger the
appropriate LSM hooks that can be reused, this one was the only one
missing. This will be used in the next commit for a new BPF kfunc that
needs to connect a socket and trigger the LSM hook.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Acked-by: Song Liu <song@kernel.org>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260813110540.103550-2-mahe.tardy@gmail.com
Kumar Kartikeya Dwivedi says:
====================
Redesign Verification Errors
TL;DR: This set reworks verifier error messages to include source and
instruction annotations, together with more causal context, making
failures easier to understand and more actionable when debugging and
repairing BPF programs.
Changelog:
----------
v4 -> v5
v4: https://lore.kernel.org/bpf/20260812233326.3575958-1-memxor@gmail.com
* Defer Verifier Limit reports and the dependent call-chain allocation
guards to follow-up work, reducing the series from 16 to 14 patches.
(Eduard)
* Make kfunc-name disassembly read-only before module-kfunc metadata is
resolved, retain instruction context without usable source metadata,
consolidate its fallback, and restrict source discovery to the containing
subprogram. (Eduard, Sashiko)
* Retain the newest diagnostic history in a bounded 64 MiB rotating buffer,
use absolute logical positions across verifier path switches, report
evicted shared history, and grow storage geometrically. (Eduard)
* Complete active-path history for BPF_LD_IMM64 and atomic fetches, call
clobbers and returns, outgoing stack arguments, legacy packet loads, and
RCU pointer transitions. (Eduard, Sashiko)
* Preserve causal lineage across equal snapshots, nullable pointer-cast
branches, and repeated same-depth function invocations using unique
diagnostic frame identities. Bound each rendered causal path to the oldest
and newest 32 matching events with an omission summary. (Eduard)
* Harden diagnostics for malformed release-kfunc signatures, fixed-size
argument ranges, and dynptr, iterator, memory-size, and required-RCU
failures by reporting the actual offending type or invariant. (Eduard,
Sashiko)
* Remove unrelated formatting and cross-patch churn, dead or single-use
helpers and filter paths, and align helper placement, includes, and commit
descriptions with the patches that first need them. (Eduard)
v3 -> v4
v3: https://lore.kernel.org/bpf/20260713153910.2556007-1-memxor@gmail.com
* Introduce helpers with their first callers and add printf annotations.
(Eduard, Sashiko)
* Remove "report" from diagnostic function names. (Sashiko)
* Reuse bpf_linfo_source and seq_buf, simplify internal names, and use shared
formatting storage. (Eduard)
* Use compact common event fields and record branches at successor entry.
(Eduard)
* Bound event storage at 1 MiB, use kvrealloc(), and drop events non-fatally.
(Eduard, Sashiko)
* Restore diagnostic history only for activated queued states, preserving the
active failure trace during cleanup. (Eduard, Sashiko)
* Record register changes through begin/end and scrub helpers, deriving targets
and origins without caller-saved snapshots. (Eduard)
* Store lineage marks on events and rewind shared formatting storage after
rendering each event. (Eduard)
* Record iterator return values before snapshotting alternate paths. (Sashiko)
* Use the current verifier instruction for global-subprogram dynptr errors.
(Sashiko)
* Use the supplied call name for nullable global-subprogram arguments.
(Sashiko)
* Describe global calls under locks as a verifier restriction rather than a
sleepability failure. (Sashiko)
* Keep diagnostic strings unsplit and put long call openings on their own
line. (Eduard)
* Keep kfunc metadata zeroed before early fetch and allowability failures.
(Sashiko)
* Drop the Verifier Internal Error report patch. (Eduard)
* Distinguish never-initialized registers from invalidated registers.
(local review)
* Preserve the legacy different-lock verifier message. (local review)
* Preserve nullable type qualifiers and stable mismatch formatting.
(local review)
* Mark truncated call chains with an ellipsis. (local review)
v2 -> v3
v2: https://lore.kernel.org/bpf/20260619205934.1312876-1-memxor@gmail.com
* Address various comments from Eduard and Sashiko.
* Move instruction context from a separate gutter into a new section
following source context, since surrounding source lines and BPF
instructions do not map one-to-one.
* Fix active-path branch reconstruction when switching to queued states,
and expand register histories to follow value lineage across spills,
fills, stack reads, helper/kfunc clobbers, and dynptr invalidation.
* Misc improvements and refinements.
v1 -> v2
v1: https://lore.kernel.org/bpf/20260605063412.974640-1-memxor@gmail.com
* Reworked diagnostic history from per-verifier-state log to active
path log with positions saved and reset when verifier search
backtracks. (Eduard)
* Moved reusable diagnostic formatting storage into struct bpf_diag
under struct bpf_verifier_env, and removed large per-report scratch
buffers from verifier stack frames. (Eduard)
* Added stack-slot events so diagnostics follow ordinary stack
spill/fill value flow and invalidations in register-scoped
histories. (Eduard)
* Reused existing source and BTF formatting helpers for diagnostics,
including bpf_get_linfo_file_line() and
btf_type_snprintf_show_name(). (Eduard)
* Fixed diagnostic edge cases around signed offset text,
BPF_MAX_VAR_OFF reporting, negative-offset clamping, poisoned
stack reads, and borrowed-reference invalidations. (Eduard)
* Fixed various miscellaneous diagnostic bugs. (Sashiko)
* Misc improvements and refinements.
---
Motivation
~~~~~~~~~~
The verifier log is the primary interface through which the verifier
communicates to the user its verdict on whether a program was accepted
or rejected.
To aid the debugging of rejection decisions, the verifier also reports
the symbolic state of the program at each instruction, across every explored
path of the BPF program. Such detailed information is critical to
introspect the correctness of verification decisions, and provide
insight into why a given program may have failed to load in the kernel.
A constant pain point in the BPF ecosystem throughout the years has
been the difficulty of debugging verification errors. The human-readable error
messages produced in response to a failure in satisfying safety-related
constraints are often terse, context-dependent, or insufficient for
understanding why a given error may have happened. Users must fall back
to the verbose instruction-by-instruction breakdown of how the symbolic
state evolved to surface the root cause. For programs with a huge log
volume due to high verification complexity, such logs quickly become
inscrutable.
All of this has made life difficult for users lacking an understanding
of how the verifier works, and the various heuristics and idiosyncrasies
used by it. In some cases, even seasoned BPF experts spend significant
time reverse engineering why a program may have failed, and have to
reach into the verifier's source code to form a complete picture of the
verification process.
Such a steep learning curve and cognitive burden also hurts the speed of
BPF development, as the verifier sits right in the middle of the user's
iteration loop while they make use of BPF to solve any given problem.
Expertise in debugging verifier errors does not scale in terms of teams
deploying these programs in production across a diverse set of kernels.
Overall, this leads to a poorer developer experience, causes visible
user dissatisfaction, and remains a drag on wider BPF adoption. With
some of the more recent developments where users increasingly leverage
AI tooling [0] to author their code, this bottleneck becomes even more
critical to address, since it throttles the much faster iteration loop
of AI agents.
[0]: https://lwn.net/Articles/1075067
Approach
~~~~~~~~
This series starts moving selected failures from terse terminal messages
toward diagnostics that carry the relevant context for a verification
failure. The existing verbose log remains the low-level trace. For selected
failures, the new report is emitted after this trace and answers the
immediate debugging questions:
- what verifier rule failed,
- why the current state does not satisfy it,
- where the failing instruction maps to source,
- which earlier branch or state event made this path fail,
- what kind of source change would satisfy the verifier.
The series adds a text-only diagnostics framework under kernel/bpf and
uses it to augment selected verifier errors. Existing verbose(env, ...)
messages are kept, so current selftest expectations and existing log
consumers continue to see the legacy text. The new report has a uniform
outer shape:
Verification failed: <category>: <problem>
Reason:
exact reason for the verification failure, with details
At:
source and instruction annotation
Causal path:
compressed branch and verifier-state events relevant for debugging
Suggestion:
speculation on potential fixes to repair the program
The outer shape is shared, but report construction is category-specific.
The categories are intentionally broad and reviewable. This revision
covers representative cases in Register Type Safety, Memory Safety,
Resource Lifetime Safety, Call Type Safety, Execution Context Safety,
Program Structure and Policy.
It does not attempt to convert every verbose(env, ...) site for now.
Additional verbose-only errors can be moved into the same framework
incrementally.
The following excerpts are copied from this current run on this branch:
./test_progs -j1 \
-a cpumask/test_populate_invalid_destination,\
cpumask/test_alloc_no_release,\
verifier_helper_value_access/via_variable_no_max_check_1,\
verifier_sock/invalidate_pkt_pointers_from_global_func \
-vv
They show the old terminal error and the exact new diagnostic report,
including the source and instruction annotations.
Call Type Safety, cpumask/test_populate_invalid_destination:
Legacy:
R1 type=scalar expected=fp
Diagnostic:
Verification failed: Call Type Safety: Invalid call argument
Reason:
The first argument (R1) to bpf_cpumask_populate does not satisfy the verifier contract: the kfunc
expects 24 bytes of memory for (struct bpf_cpumask), but it is an integer scalar and not
verifier-known memory.
At:
test_populate_invalid_destination @ cpumask_failure.c:234:8
Source context:
232 | ...
233 | ...
>>> 234 | ret = bpf_cpumask_populate(invalid, &bits, sizeof(bits));
| ^-- error: invalid first argument (R1) for bpf_cpumask_populate
235 | if (!ret)
236 | err = 2;
Instruction context:
2 | (b7) r1 = 1193046
3 | (b7) r3 = 8
>>> 4 | (85) call bpf_cpumask_populate#62860
5 | (56) if w0 != 0x0 goto pc+4
6 | (18) r1 = 0xffffc9000028e000
Causal path:
test_populate_invalid_destination @ cpumask_failure.c:234:8
Source context:
232 | ...
233 | ...
>>> 234 | ret = bpf_cpumask_populate(invalid, &bits, sizeof(bits));
| ^-- update: R1 changed from context pointer at offset 0 to integer scalar value
| 1193046
235 | if (!ret)
236 | err = 2;
Instruction context:
0 | (bf) r2 = r10
1 | (07) r2 += -8
>>> 2 | (b7) r1 = 1193046
3 | (b7) r3 = 8
4 | (85) call bpf_cpumask_populate#62860
Suggestion:
Pass stack, map, context, or other verifier-known memory of the expected type and size, not an
integer cast to a pointer.
Register Type Safety, verifier_sock/invalidate_pkt_pointers_from_global_func:
Legacy:
R7 invalid mem access 'scalar'
Diagnostic:
Verification failed: Register Type Safety: Invalid dereference
Reason:
R7 is an integer scalar here, not a pointer to memory.
At:
invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1067:5
Source context:
1065 | ...
1066 | skb_pull_data1(sk, 0);
>>> 1067 | *p = 42; /* this is unsafe */
| ^-- error: invalid dereference of R7 (an integer scalar)
1068 | ...
1069 | }
Instruction context:
8 | (85) call pc+4
9 | (b4) w1 = 42
>>> 10 | (63) *(u32 *)(r7 +0) = r1
11 | (bc) w0 = w6
12 | (95) exit
Causal path:
invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1062:29
Source context:
1060 | int invalidate_pkt_pointers_from_global_func(struct __sk_buff *sk)
1061 | ...
>>> 1062 | int *p = (void *)(long)sk->data;
| ^-- update: R7 changed from uninitialized value to pkt at offset 0
1063 | ...
1064 | if ((void *)(p + 1) > (void *)(long)sk->data_end)
Instruction context:
0 | (b4) w6 = 2
1 | (61) r2 = *(u32 *)(r1 +80)
>>> 2 | (61) r7 = *(u32 *)(r1 +76)
3 | (bf) r3 = r7
4 | (07) r3 += 4
invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1064:22
Source context:
1062 | int *p = (void *)(long)sk->data;
1063 | ...
>>> 1064 | if ((void *)(p + 1) > (void *)(long)sk->data_end)
| ^-- branch: took the false branch of this conditional, goto not followed
1065 | ...
1066 | skb_pull_data1(sk, 0);
Instruction context:
3 | (bf) r3 = r7
4 | (07) r3 += 4
>>> 5 | (2d) if r3 > r2 goto pc+5
6 | (b4) w6 = 0
7 | (b4) w2 = 0
invalidate_pkt_pointers_from_global_func @ verifier_sock.c:1066:2
Source context:
1064 | if ((void *)(p + 1) > (void *)(long)sk->data_end)
1065 | ...
>>> 1066 | skb_pull_data1(sk, 0);
| ^-- invalidated: R7: packet data may have moved; previous value was pkt at
| offset 0
1067 | *p = 42; /* this is unsafe */
1068 | ...
Instruction context:
6 | (b4) w6 = 0
7 | (b4) w2 = 0
>>> 8 | (85) call pc+4
9 | (b4) w1 = 42
10 | (63) *(u32 *)(r7 +0) = r1
Suggestion:
Preserve a pointer-valued register where needed, or reload and revalidate the pointer after scalar
arithmetic, helper calls, or other operations that can invalidate it.
Memory Safety, verifier_helper_value_access/via_variable_no_max_check_1:
Legacy:
R1 unbounded memory access, make sure to bounds check any such access
Diagnostic:
Verification failed: Memory Safety: Access outside bounds
Reason:
The verifier cannot prove offset + access_size <= object_size. Here, the maximal bound for a
memory access is 4294967295 and exceeds maximum allowed offset of 536870912. R1 is map_value;
offset is variable: known bits 0x0, unknown mask 0xffffffff; signed range [0, 4294967295],
unsigned range [0, 4294967295]; access_size is 1; object_size is 48.
At:
via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2
Source context:
625 | ...
626 | ...
>>> 627 | asm volatile (" \
| ^-- error: access may be outside object bounds
628 | ...
629 | ...
Instruction context:
11 | (b7) r2 = 1
12 | (b7) r3 = 0
>>> 13 | (85) call bpf_probe_read_kernel#113
14 | (95) exit
Causal path:
via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2
Source context:
625 | ...
626 | ...
>>> 627 | asm volatile (" \
| ^-- update: R0 changed from uninitialized value to nullable map value from
| map_hash_48b at offset 0
628 | ...
629 | ...
Instruction context:
4 | (18) r1 = 0xffff88810a3ea000
>>> 6 | (85) call bpf_map_lookup_elem#1
7 | (15) if r0 == 0x0 goto pc+6
8 | (bf) r1 = r0
via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2
Source context:
625 | ...
626 | ...
>>> 627 | asm volatile (" \
| ^-- branch: took the false branch of this conditional, goto not followed
628 | ...
629 | ...
Instruction context:
6 | (85) call bpf_map_lookup_elem#1
>>> 7 | (15) if r0 == 0x0 goto pc+6
8 | (bf) r1 = r0
9 | (61) r3 = *(u32 *)(r0 +0)
via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2
Source context:
625 | ...
626 | ...
>>> 627 | asm volatile (" \
| ^-- update: R1 changed from uninitialized value to map value from map_hash_48b
| at offset 0
628 | ...
629 | ...
Instruction context:
6 | (85) call bpf_map_lookup_elem#1
7 | (15) if r0 == 0x0 goto pc+6
>>> 8 | (bf) r1 = r0
9 | (61) r3 = *(u32 *)(r0 +0)
10 | (0f) r1 += r3
via_variable_no_max_check_1 @ verifier_helper_value_access.c:627:2
Source context:
625 | ...
626 | ...
>>> 627 | asm volatile (" \
| ^-- update: R1 changed from map value from map_hash_48b at offset 0 to map value
| from map_hash_48b with variable offset: known bits 0x0, unknown mask
| 0xffffffff, signed range [0, 4294967295], unsigned range [0, 4294967295]
628 | ...
629 | ...
Instruction context:
8 | (bf) r1 = r0
9 | (61) r3 = *(u32 *)(r0 +0)
>>> 10 | (0f) r1 += r3
11 | (b7) r2 = 1
12 | (b7) r3 = 0
Suggestion:
Add or adjust a bounds check that proves offset + access_size stays within the object.
Resource Lifetime Safety, cpumask/test_alloc_no_release:
Legacy:
Unreleased reference id=2 alloc_insn=0
BPF_EXIT instruction in main prog would lead to reference leak
Diagnostic:
Verification failed: Resource Lifetime Safety: Unreleased resource
Reason:
Owned resource (id=2) was acquired at instruction 0 and still needs to be released before this
exit path.
At:
test_alloc_no_release @ cpumask_failure.c:36:5
Source context:
34 | ...
35 | ...
>>> 36 | int BPF_PROG(test_alloc_no_release, struct task_struct *task, u64 clone_flags)
| ^-- error: owned resource (id=2) still needs release
37 | ...
38 | ...
Instruction context:
19 | (7b) *(u64 *)(r10 -8) = r6
20 | (b4) w0 = 0
>>> 21 | (95) exit
Causal path:
test_alloc_no_release @ cpumask_common.h:78:12
Source context:
76 | ...
77 | ...
>>> 78 | cpumask = bpf_cpumask_create();
| ^-- acquired: owned resource (id=2)
79 | if (!cpumask) {
80 | err = 1;
Instruction context:
>>> 0 | (85) call bpf_cpumask_create#62851
1 | (bf) r6 = r0
2 | (55) if r6 != 0x0 goto pc+5
test_alloc_no_release @ cpumask_common.h:79:6
Source context:
77 | ...
78 | cpumask = bpf_cpumask_create();
>>> 79 | if (!cpumask) {
| ^-- branch: took the true branch of this conditional, goto followed
80 | err = 1;
81 | ...
Instruction context:
0 | (85) call bpf_cpumask_create#62851
1 | (bf) r6 = r0
>>> 2 | (55) if r6 != 0x0 goto pc+5
3 | (18) r1 = 0xffffc90000252000
test_alloc_no_release @ cpumask_common.h:84:6
Source context:
82 | ...
83 | ...
>>> 84 | if (!bpf_cpumask_empty(cast(cpumask))) {
| ^-- branch: took the true branch of this conditional, goto followed
85 | err = 2;
86 | bpf_cpumask_release(cpumask);
Instruction context:
9 | (85) call bpf_cpumask_empty#62852
10 | (54) w0 &= 1
>>> 11 | (56) if w0 != 0x0 goto pc+7
12 | (18) r1 = 0xffffc90000252000
Suggestion:
Release or transfer ownership of the acquired resource on every path before the program exits.
Patch layout:
- Patches 1-2 add the initial renderer, source-line lookup, and separate
source and instruction context blocks. Reusable report sections arrive with their first
category-specific consumers.
- Patches 3-7 add bounded, growable environment-owned diagnostic
history. It grows to 64 MiB and then retains the newest events in a
rotating buffer. The history follows the active verifier path and is
pruned when backtracking; it records branch outcomes, material register
changes, reference lifetime events, and execution-context events so
reports can explain the path and causal state transitions that led to
the failure.
- Patches 8-14 add the first category-specific reports. These patches
hook selected verifier failure sites and choose the evidence that is
useful for that error class.
Evaluation
~~~~~~~~~~
The evaluation below is retained from v4 while v5 changes are in progress.
It includes two Verifier Limit cases removed from v5 and must be refreshed
before posting.
To quantitatively assess diagnostic quality beyond subjective human
feedback, we use AI models (called over APIs) and veristat metrics to
compare results.
Models are used as a way to measure repair utility of the extra
diagnostics over a fixed test set. Each prompt contains only a sanitized
source snippet and either the legacy verifier log or the new diagnostic
log. To avoid leaking the answer through the test itself, comments,
annotations, and other source hints that describe the intended failure
were removed. The model is not given internet access, repository access,
test execution, verifier access, or the expected fix. The expected
causes and intended repairs are kept outside the prompt. Under those
constraints, correctness, exact repair rate, output size, reasoning
tokens, cost, and wall time provide a proxy for whether the additional
verifier context makes the failure easier to understand and turn into a
source-level fix.
Verifier cost is assessed by forcing the collection of diagnostics
information during normal verification. By default, this information is
collected and processed only when verbose logs are enabled, but forcing
it even without a verbose log helps us measure the CPU time and memory
cost of the extra data.
Both evaluations are covered in the sections below.
Repair Quality
--------------
Repair quality is measured by asking API-only models to propose source
fixes from a sanitized source snippet and verifier log. The criterion is
score >= 3 on a 0-4 local grading scale, where 3 means a likely fix with
incomplete detail and 4 means an actionable source-level fix. Score 4 is
reported separately as the exact repair rate. The reported model set
contains 596 completed API responses: 298 diagnostic and 298 legacy.
Main results (details available in Appendix):
Metric Diagnostic Legacy Delta
---------------------------------- ----------- ----------- --------
Answers 298 298
Success rate 97.0% 97.3% -0.3 pp
Exact repair rate 82.2% 72.1% +10.1 pp
Mean score 3.79 3.69 +0.10
Solver cost $8.93 $10.37 -13.8%
Mean output tokens per answer 1662 1975 -15.8%
Mean reasoning tokens per answer 951 1080 -11.9%
Mean wall time per answer 37.3s 44.1s -15.4%
Diagnostic prompts carry more input context. The resulting answers are
still shorter and cheaper. In this run, diagnostics do not materially
change the coarse success rate, but they increase exact repairs by 10.1
percentage points while reducing cost, output tokens, reasoning tokens,
and wall time.
Verifier cost
-------------
Verifier cost is measured with veristat over the BPF selftest programs
selected by tools/testing/selftests/bpf/veristat.cfg, with five
repetitions per configuration. With diagnostics gated by log level, wall
time and verifier duration stay close to baseline. Forcing diagnostics
on for every verifier run adds modest overhead on this workload.
memory.peak is measured with cgroup v2 memory accounting for each
program load. The table reports the mean wall time, the mean summed
verifier duration, and the mean of the per-repetition maximum
memory.peak values.
Configuration Wall time mean Verifier duration memory.peak
---------------------------- -------------- ----------------- -----------
bpf-next baseline 25.78s 9.86s 142 MiB
diagnostics, gated 26.64s 10.16s 144 MiB
diagnostics, forced on 28.01s 11.00s 148 MiB
TODO
~~~~
Known follow-up work:
- Convert more verbose-only verifier errors into category-specific
reports.
- Integrate loop-convergence failure summarization from Eduard.
- Report candidate kfuncs/helpers for releasing owned resources.
- Explore association of source variables with verifier registers
where debug info permits it.
- Refine suggestions per category and, where useful, link diagnostics
to maintained documentation.
- Bring verifier warnings into the same reporting framework.
Appendix: AI repair details
~~~~~~~~~~~~~~~~~~~~~~~~~~~
The 20 verifier-failing selftest cases are:
Case Diff Category Selftest selector
------- ------ -------------------------- ---------------------------------------------
case-001 easy Call Type Safety cpumask/test_populate_invalid_destination
case-002 easy Resource Lifetime Safety cpumask/test_alloc_no_release
case-003 easy Register Type Safety verifier_spill_fill/check_corrupted_spill_fill
case-004 easy Register Type Safety test_global_funcs/global_func12
case-005 easy Execution Context Safety preempt_lock/preempt_sleepable_helper
case-006 easy Policy verifier_helper_restricted/in_bpf_prog_type_kprobe_1
case-007 medium Memory Safety dynptr/dynptr_slice_var_len1
case-008 medium Call Type Safety dynptr/test_dynptr_skb_small_buff
case-009 medium Call Type Safety task_kfunc/task_kfunc_acquire_untrusted
case-010 medium Register Type Safety test_global_funcs/global_func6
case-011 medium Resource Lifetime Safety dynptr/ringbuf_missing_release2
case-012 medium Execution Context Safety irq/irq_sleepable_helper_global_subprog
case-013 medium Verifier Limit test_global_funcs/global_func1
case-014 hard Memory Safety verifier_helper_value_access/via_variable_no_max_check_1
case-015 hard Register Type Safety verifier_sock/invalidate_pkt_pointers_from_global_func
case-016 hard Resource Lifetime Safety verifier_ref_tracking/check_free_in_one_subbranch
case-017 hard Resource Lifetime Safety irq/irq_restore_ooo
case-018 hard Resource Lifetime Safety res_spin_lock_failure/res_spin_lock_ooo_unlock
case-019 hard Program Structure verifier_loops1/bounded_recursion
case-020 hard Verifier Limit verifier_liveness_exp/liveness_exponential_complexity
The grading scale is:
- 4: identifies the verifier cause and gives an actionable source-level fix.
- 3: gives a likely fix, but with incomplete explanation or detail.
- 2: identifies part of the issue, but not enough to fix confidently.
- 1: gives only a broad verifier-area answer, or a wrong/insufficient fix.
- 0: does not identify the intended verifier failure.
Detailed effort metrics for the model set:
Metric Variant Mean Median P99
----------------------- ---------- -------- -------- --------
Cost per answer diagnostic $0.030 $0.019 $0.203
Cost per answer legacy $0.035 $0.018 $0.223
Input tokens diagnostic 1391 1220 4048
Input tokens legacy 1052 805 3655
Output tokens diagnostic 1662 954 8680
Output tokens legacy 1975 1034 9912
Reasoning tokens diagnostic 951 208 8108
Reasoning tokens legacy 1080 228 6322
Wall time diagnostic 37.3s 18.3s 222.7s
Wall time legacy 44.1s 19.8s 255.5s
Per-model results for diagnostic prompts:
Model profile Ans Succ Exact Mean Cost OutK ReasK Wall
----------------------------------------- --- ----- ----- ---- ------- ---- ----- -----
anthropic-haiku-4.5-default 20 90.0 80.0 3.70 $0.087 11.4 0.0 5.0s
anthropic-opus-4.8-high 20 100.0 90.0 3.90 $0.819 25.5 0.0 15.5s
anthropic-opus-4.8-medium 20 95.0 90.0 3.85 $0.870 27.5 0.0 12.7s
anthropic-sonnet-4.6-high 20 95.0 80.0 3.75 $0.824 48.9 0.0 21.6s
anthropic-sonnet-4.6-medium 20 100.0 65.0 3.65 $0.278 12.4 0.0 6.6s
openai-gpt-5.3-codex-high 20 100.0 80.0 3.80 $0.601 39.8 33.9 25.0s
openai-gpt-5.3-codex-medium 20 95.0 85.0 3.80 $0.287 17.5 11.4 13.5s
openai-gpt-5.5-high 20 100.0 90.0 3.90 $2.356 74.4 65.2 56.8s
openai-gpt-5.5-low 20 100.0 90.0 3.90 $0.686 18.7 8.5 21.3s
openai-gpt-5.5-medium 19 100.0 84.2 3.84 $1.353 41.1 31.8 37.4s
openai-gpt-5.5-none 20 95.0 90.0 3.85 $0.457 11.1 0.0 10.4s
openrouter-deepseek-r1-0528 20 100.0 75.0 3.75 $0.145 61.5 53.8 98.3s
openrouter-deepseek-v3.2 19 100.0 78.9 3.79 $0.028 64.2 58.1 87.3s
openrouter-glm-5.1-high 20 95.0 80.0 3.75 $0.113 28.8 20.7 19.3s
openrouter-qwen3-coder 20 90.0 75.0 3.65 $0.028 12.4 0.0 7.1s
Per-model results for legacy prompts:
Model profile Ans Succ Exact Mean Cost OutK ReasK Wall
----------------------------------------- --- ----- ----- ---- ------- ---- ----- -----
anthropic-haiku-4.5-default 20 90.0 45.0 3.35 $0.081 11.6 0.0 5.0s
anthropic-opus-4.8-high 20 90.0 70.0 3.60 $1.192 42.2 0.0 17.5s
anthropic-opus-4.8-medium 20 95.0 85.0 3.80 $1.001 34.5 0.0 13.4s
anthropic-sonnet-4.6-high 20 100.0 75.0 3.75 $1.181 74.1 0.0 24.4s
anthropic-sonnet-4.6-medium 20 95.0 65.0 3.60 $0.420 23.4 0.0 12.3s
openai-gpt-5.3-codex-high 20 100.0 85.0 3.85 $0.562 37.8 31.6 27.1s
openai-gpt-5.3-codex-medium 20 100.0 75.0 3.75 $0.318 20.3 13.7 13.6s
openai-gpt-5.5-high 19 100.0 78.9 3.79 $2.613 84.0 75.4 98.1s
openai-gpt-5.5-low 20 100.0 75.0 3.75 $0.664 19.0 9.7 21.7s
openai-gpt-5.5-medium 20 100.0 75.0 3.75 $1.602 50.2 41.0 56.1s
openai-gpt-5.5-none 20 95.0 85.0 3.80 $0.416 10.7 0.0 10.9s
openrouter-deepseek-r1-0528 20 95.0 70.0 3.65 $0.149 64.6 57.5 92.5s
openrouter-deepseek-v3.2 20 100.0 60.0 3.60 $0.030 74.3 67.8 98.3s
openrouter-glm-5.1-high 19 100.0 63.2 3.63 $0.115 32.1 24.9 30.4s
openrouter-qwen3-coder 20 100.0 75.0 3.75 $0.022 9.5 0.0 5.4s
====================
Link: https://patch.msgid.link/20260815064612.378577-1-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Augment selected helper and kfunc allowability failures with Policy reports.
These reports explain which requested operation is forbidden and why, without
adding path history for non-path-dependent policy checks.
Cover unprivileged bpf2bpf and kfunc use, helper program-type restrictions,
GPL-only helpers, helper-specific allow callbacks, kfunc allowability, and
destructive kfunc capability checks.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-15-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Augment selected whole-program and subprogram CFG validation failures with
Program Structure reports. These errors are structural rather than
path-dependent, so the reports focus on source and instruction context
instead of causal history.
Cover direct and indirect jumps outside the program or current subprogram,
unprivileged backedges, missing and out-of-range jump tables, targets in the
second half of an ldimm64, unreachable instructions, subprogram fallthrough,
and recursive bpf2bpf call graph edges.
Format long jump-range reasons directly in diagnostics.c, and keep the
fallthrough suggestion aligned with the verifier check by suggesting exit or
explicit jumps.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-14-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Augment selected sleepability and critical-section failures with Execution
Context Safety reports. Keep the existing verifier messages and add source
context, path history, and suggestions tied to the active context.
Use the context history recorded earlier to anchor causal paths to lock, IRQ,
RCU, and preempt regions instead of unrelated register updates.
Cover global calls while holding a lock, sleepable global function calls,
sleepable helpers, sleepable kfunc calls from disallowed contexts, operations
that exit while a context is still active, and unmatched context exits.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-13-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Augment selected Resource Lifetime Safety failures with structured diagnostics
while preserving the existing verifier messages.
Report unreleased references from check_reference_leak() using
reference-scoped diagnostic history, and add state reports for dynptr,
iterator, lock, and IRQ-flag lifetime misuse.
IRQ restore mismatch and out-of-order diagnostics use IRQ context-scoped
history when an IRQ-disabled region is active, so retained save/restore context
is still visible after per-state history removal.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-11-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Augment selected register-state verifier failures with Register Type Safety
reports. The existing verbose verifier messages remain in place; the new
reports add reason, source context, causal path, and suggestions.
Cover invalid pointer dereferences, unreadable registers, missing outgoing
stack arguments for bpf2bpf and kfunc calls, and rejected pointer arithmetic.
Use scoped diagnostic history so reports start from the latest relevant value
change and then show later branch outcomes.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-9-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Record verifier context transitions in the diagnostic history so later reports
can anchor causal paths to the critical section that made an operation invalid.
This covers lock, IRQ, RCU, and preempt regions without adding any new
verifier error reports. Category-specific commits decide where those recorded
events should be rendered.
Use context depth when selecting scoped history so nested regions anchor at the
outer active region, and fall back to the earliest retained event when the
matching entry was pruned.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-8-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add reference acquire and release events to diagnostic history so Resource
Lifetime Safety reports can show the lifetime of a specific reference id along
the path.
Record acquisitions after the verifier assigns the reference id. Record
releases only after release_reference_nomark() succeeds, including the
kptr_xchg RCU conversion path and owning-to-non-owning conversion path that
consume an owning reference.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-7-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Record material register and outgoing stack argument changes so diagnostics can
explain how a value reached its current type, bounds, or unreadable state.
Store old and new register types, scalar ranges, tnum value and mask, map and
BTF type identity, and basic operand metadata in the environment-owned
diagnostic event stream.
Record invalidations when packet data moves, references are released, or
borrowed references leave their protected region. Register-scoped history
starts at the latest matching modification and then shows later branch
outcomes.
Also record fixed stack spills and overwrites, and tag register fills from
stack so register-scoped history can follow value flow through spilled stack
slots.
The type_is_map_ptr() helper previously lived as a static function in
kernel/bpf/log.c since commit 0c95c9fdb6 ("bpf: emit map name in register
state if applicable and available"). Move it verbatim to
include/linux/bpf_verifier.h as a static inline, next to the other type
classifiers, so diagnostics.c can reuse it without duplicating the case list.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-6-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Save the diagnostic event-log position with each verifier stack entry and
reset the environment-owned stream together with the normal verifier log
when a queued state is popped. Also reset the diagnostic stream after
successful subprogram verification even when level-2 logging preserves the
normal verifier log.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-5-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Add an environment-owned diagnostic history for verifier reports. Event
payloads keep the user-facing branch history shape, while storage lives
in bpf_verifier_env and follows the active verifier path.
Grow the event array geometrically up to a 64 MiB limit. Once storage
reaches the limit, or an allocation fails, overwrite the oldest event so
diagnostics retain the newest useful suffix without adding per-event
metadata.
Represent saved positions as absolute logical sequence numbers. A restore
truncates to a retained position. If its prefix has already been evicted,
clear the abandoned suffix and preserve the missing-history position. This
keeps marks stable across rotation without increasing their size.
Add the branch event renderer and branch recording.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-4-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
