github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Merge branch 'fix-generic-__uninit-kfunc-output-buffers'

Browse Source 
Kumar Kartikeya Dwivedi says:

====================
Fix generic __uninit kfunc output buffers

Generic __uninit kfunc arguments are output buffers. Stack liveness treats
them as writes, but argument checking still requires readable contents and
does not record definite initialization after the call. Check these
arguments as write-only and record privileged output initialization after
validating all inputs, including inputs that alias an output.

Following Eduard's rework, helpers and kfuncs record generic outputs in the
same argument-checking path after type resolution. Generated kfunc
prototypes now mark generic buffers with MEM_WRITE, so __uninit buffers are
checked as write-only ahead of the fix while ordinary buffers stay
read/write.

The final two patches derive memory access from MEM_WRITE and MEM_UNINIT
alone, so input/output helpers such as bpf_check_mtu() require read as
well as write permission without a new prototype field, and add the
permission tests.

Changelog:
----------
v4 -> v5
v4: https://lore.kernel.org/bpf/20260918052906.12226-1-memxor@gmail.com

 * Check __uninit kfunc outputs as write-only and mark generated kfunc
   buffers MEM_WRITE, replacing the kfunc-specific access override. (Amery)
 * Derive memory access from MEM_WRITE and MEM_UNINIT after relaxing the
   MEM_UNINIT contract, so partial-output helpers keep write-only map
   destinations while input/output helpers gain read checks. (Eduard, Amery,
   Sashiko)
 * Move the helper read-access fix and its permission tests after the
   MEM_UNINIT relaxation, and drop its Fixes tag: it only closes a
   write-only map permission gap for input/output helpers.
 * Reuse the raw-memory predicate after type resolution and clarify how
   argument ordering affects output tracking. (Eduard, BPF CI)
 * Keep prospective struct-output counting separate from raw-memory checks.
 * Clarify the __uninit buffer contract and privilege rules. (BPF CI, Eduard)
 * Add read-only FIB rejection and partial-output write-only map coverage.
   (BPF CI)
 * Carry Eduard's Acked-by on the generic __uninit fix.
 * Allow partial initialization for all generic MEM_UNINIT buffers and
   annotate partial-output helpers with MEM_UNINIT. (Eduard)
 * Preserve per-byte initialization and prior stack liveness for generic
   outputs when uninitialized stack reads are not allowed. (Eduard)
 * Retain output classification when variable sizes disable raw mode.
 * Cover stricter readback rules for full-writing helpers and kfuncs.
 * Retain helper-argument fallback coverage using map_update_elem inputs.

v3 -> v4
v3: https://lore.kernel.org/bpf/20260916192805.3991983-1-memxor@gmail.com

 * Record helper and kfunc outputs after type resolution. (Eduard, Amery)
 * Derive generic memory access from flags for helpers and kfuncs. (Eduard)
 * Fix MEM_WRITE read checks separately and add permission tests. (Eduard)
 * Remove the output-count validator in the multiple-output extension.
   (Amery)
 * Clarify argument-slot naming and move its accessor into the fix. (BPF CI)
 * Add the unaligned header and order the new test registrations. (BPF CI)
 * Shorten the kfunc fix description while retaining its rationale. (BPF CI)

v2 -> v3
v2: https://lore.kernel.org/bpf/20260916160821.3157543-1-memxor@gmail.com

 * Reuse check_raw_mode_ok() after kfunc prototype generation, including
   struct outputs resolved to generic memory later. (Amery)
 * Remove the now-redundant kfunc output-count check in the multiple-output
   extension and simplify the helper validator.
 * Leave the stack-passed output uninitialized so the reduced-capability
   test detects missing __uninit handling. (Sashiko)

v1 -> v2
v1: https://lore.kernel.org/bpf/20260915141004.1196460-1-memxor@gmail.com

 * Separate the single-output fix and tests from multiple-output support
   and its tests; reduce coverage to focused cases. (Eduard)
 * Skip inactive output slots before looking up argument register state.
   (Sashiko, Amery)
 * Separate sysctl restrictions from mitigation-related test skips.
   (BPF CI)
 * Use an int-width initialization store in the alias test for big-endian
   targets. (BPF CI)
 * Centralize conversion from argument numbers to slots. (Eduard)
 * Share clear access-mode selection between fixed-size and sized arguments.
   (Amery)
 * Clarify the opt-in prepare/load capability boundary and retain the
   reduced-capability alias rejection test. (Eduard)
====================

Link: https://patch.msgid.link/20260921023843.411943-1-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This commit is contained in:
Alexei Starovoitov 2026-09-21 17:12:40 +00:00
commit 15972a29ac
No known key found for this signature in database
20 changed files with 1046 additions and 121 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
Documentation/bpf/kfuncs.rst
View File

@ -164,19 +164,32 @@ suffix should be used.
2.3.3 __uninit Annotation
-------------------------
This annotation is used to indicate that the argument will be treated as
uninitialized.
Use ``__uninit`` on a pointer parameter for an output buffer whose incoming
contents the kfunc does not read.
An example is given below::
For generic memory buffers, the kfunc may leave bytes untouched, including
on error returns. The writable range is determined by the pointed-to type
or the associated ``__sz`` or ``__szk`` size argument.
__bpf_kfunc int bpf_dynptr_from_skb(..., struct bpf_dynptr_kern *ptr__uninit)
The annotation does not change the accepted pointer types. A stack-backed
struct passed as a generic memory buffer must still be scalar-only.
Generic stack buffers may be uninitialized before the call. The call does
not make previously uninitialized bytes readable unless the program is
allowed to read uninitialized stack memory (normally requiring
``CAP_PERFMON``). Other callers must initialize those bytes themselves
before reading them. Stack bounds and variable-offset restrictions still
apply.
For dynptr parameters, ``__uninit`` indicates that the kfunc constructs a
dynptr in the supplied storage. For example::
__bpf_kfunc int bpf_dynptr_from_skb(..., struct bpf_dynptr *ptr__uninit)
{
...
}
Here, the dynptr will be treated as an uninitialized dynptr. Without this
annotation, the verifier will reject the program if the dynptr passed in is
not initialized.
Without this annotation, a dynptr argument must already be initialized.
2.3.4 __nullable Annotation
---------------------------

5
include/linux/bpf.h
View File

@ -781,7 +781,10 @@ enum bpf_type_flag {
*/
PTR_UNTRUSTED = BIT(6 + BPF_BASE_TYPE_BITS),
/* MEM can be uninitialized. */
/*
* MEM can be uninitialized. Generic memory outputs need not be fully
* initialized by the callee.
*/
MEM_UNINIT = BIT(7 + BPF_BASE_TYPE_BITS),
/* DYNPTR points to memory local to the bpf program. */

13
include/linux/bpf_verifier.h
View File

@ -1547,12 +1547,15 @@ struct ref_obj_desc {
};
/*
* A memory argument a call fills in. The verifier allows the stack to be uninitialized if
* the range is a known constant. Stack slots are marked as STACK_MISC by check_mem_access().
* Generic MEM_UNINIT arguments, indexed by ABI slot. var_size_mask excludes
* variable-sized buffers from raw mode without losing the output annotation.
* size records constant ranges to mark initialized after checking all arguments,
* only when the caller is allowed to read uninitialized stack memory.
*/
struct arg_raw_mem_desc {
u8 regno;
int size;
u16 mask;
u16 var_size_mask;
int size[MAX_BPF_FUNC_ARGS];
};
/* Size of PTR_TO_MEM returned, taken from a constant allocation-size argument */
@ -1579,6 +1582,7 @@ struct bpf_call_arg_meta {
struct bpf_dynptr_desc dynptr;
struct ref_obj_desc ref_obj;
struct ret_mem_desc ret_mem;
struct arg_raw_mem_desc arg_raw_mem;
/* Only set by kfunc */
bool r0_rdonly;
@ -1617,7 +1621,6 @@ struct bpf_call_arg_meta {
s64 const_map_key;
struct btf *ret_btf;
struct btf_field *kptr_field;
struct arg_raw_mem_desc arg_raw_mem;
};
int bpf_get_helper_proto(struct bpf_verifier_env *env, int func_id,

2
kernel/bpf/cgroup.c
View File

@ -2491,7 +2491,7 @@ static const struct bpf_func_proto bpf_sysctl_get_name_proto = {
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_PTR_TO_MEM | MEM_WRITE,
.arg2_type = ARG_PTR_TO_MEM | MEM_UNINIT | MEM_WRITE,
.arg3_type = ARG_MEM_SIZE,
.arg4_type = ARG_ANYTHING,
};

2
kernel/bpf/helpers.c
View File

@ -1130,7 +1130,7 @@ const struct bpf_func_proto bpf_snprintf_proto = {
.func = bpf_snprintf,
.gpl_only = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_MEM_OR_NULL | MEM_WRITE,
.arg1_type = ARG_PTR_TO_MEM_OR_NULL | MEM_UNINIT | MEM_WRITE,
.arg2_type = ARG_MEM_SIZE_OR_ZERO,
.arg3_type = ARG_PTR_TO_CONST_STR,
.arg4_type = ARG_PTR_TO_MEM | PTR_MAYBE_NULL | MEM_RDONLY,

167
kernel/bpf/verifier.c
View File

@ -302,6 +302,12 @@ static int arg_idx_from_argno(argno_t a)
return arg_from_argno(a) - 1;
}
/* Normalize helper register numbers and kfunc argument numbers to ABI slots. */
static u32 arg_slot_from_argno(argno_t a)
{
return abs(a.argno) - 1;
}
static const char *btf_type_name(const struct btf *btf, u32 id)
{
return btf_name_by_offset(btf, btf_type_by_id(btf, id)->name_off);
@ -6987,8 +6993,11 @@ static int check_stack_range_initialized(
* but BTF based global subprog validation isn't accurate enough.
*/
bool allow_poison = access_size < 0 || clobber;
/* The call will initialize the memory; uninitialized stack allowed */
bool raw_mode = meta && meta->arg_raw_mem.regno == reg_from_argno(argno);
u32 arg_slot = arg_slot_from_argno(argno);
bool uninit = clobber && meta && arg_slot < MAX_BPF_FUNC_ARGS &&
(meta->arg_raw_mem.mask & BIT(arg_slot));
bool raw_mode = uninit && env->allow_uninit_stack &&
!(meta->arg_raw_mem.var_size_mask & BIT(arg_slot));
access_size = abs(access_size);
@ -7017,11 +7026,9 @@ static int check_stack_range_initialized(
reg_arg_name(env, argno), tn_buf);
return -EACCES;
}
/* Only initialized buffer on stack is allowed to be accessed
* with variable offset. With uninitialized buffer it's hard to
* guarantee that whole memory is marked as initialized on
* helper return since specific bounds are unknown what may
* cause uninitialized stack leaking.
/*
* The call may touch any byte in the possible range, but does not
* definitely initialize all of it. Fall back to ordinary stack checks.
*/
raw_mode = false;
@ -7029,8 +7036,9 @@ static int check_stack_range_initialized(
max_off = reg_smax(reg) + off;
}
/* Unprivileged outputs retain each byte's initialization state. */
if (raw_mode) {
meta->arg_raw_mem.size = access_size;
meta->arg_raw_mem.size[arg_slot] = access_size;
return 0;
}
@ -7048,8 +7056,8 @@ static int check_stack_range_initialized(
if (*stype == STACK_MISC)
goto mark;
if ((*stype == STACK_ZERO) ||
(*stype == STACK_INVALID && env->allow_uninit_stack)) {
if (clobber) {
(*stype == STACK_INVALID && (uninit || env->allow_uninit_stack))) {
if (clobber && (*stype != STACK_INVALID || env->allow_uninit_stack)) {
/* helper can write anything into the stack */
*stype = STACK_MISC;
}
@ -7068,8 +7076,11 @@ static int check_stack_range_initialized(
}
if (*stype == STACK_POISON) {
if (allow_poison)
if (allow_poison) {
if (uninit && env->allow_uninit_stack)
*stype = STACK_MISC;
goto mark;
}
verbose(env, "reading from stack %s off %d+%d size %d, slot poisoned by dead code elimination\n",
reg_arg_name(env, argno), min_off, i - min_off, access_size);
} else if (tnum_is_const(reg->var_off)) {
@ -7217,13 +7228,13 @@ static int check_mem_size_reg(struct bpf_verifier_env *env,
*/
meta->msize_max_value = reg_umax(size_reg);
/* The register is SCALAR_VALUE; the access check happens using
* its boundaries. For unprivileged variable accesses, disable
* raw mode so that the program is required to initialize all
* the memory that the helper could just partially fill up.
/*
* Check variable ranges byte by byte instead of using raw mode. Keep the
* MEM_UNINIT annotation so invalid bytes are accepted without marking them
* initialized when the caller cannot read uninitialized stack memory.
*/
if (!tnum_is_const(size_reg->var_off))
meta->arg_raw_mem.regno = 0;
meta->arg_raw_mem.var_size_mask |= BIT(arg_slot_from_argno(mem_argno));
if (reg_smin(size_reg) < 0) {
verbose(env, "%s min value is negative, either use unsigned or 'var &= const'\n",
@ -8896,6 +8907,19 @@ static int process_map_ptr_arg(struct bpf_verifier_env *env, struct bpf_reg_stat
return 0;
}
/*
* MEM_WRITE alone denotes an input/output buffer. MEM_UNINIT marks an output
* whose incoming contents are not read.
*/
static enum bpf_access_type func_arg_access_type(enum bpf_arg_type arg_type)
{
if (!(arg_type & MEM_WRITE))
return BPF_READ;
if (arg_type & MEM_UNINIT)
return BPF_WRITE;
return BPF_READ | BPF_WRITE;
}
static int check_func_arg(struct bpf_verifier_env *env, u32 arg, u32 slot, u32 prev_slot,
struct bpf_call_arg_meta *meta,
int insn_idx)
@ -8940,6 +8964,9 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, u32 slot, u32 p
if (err)
return err;
if (arg_type_is_raw_mem(arg_type))
meta->arg_raw_mem.mask |= BIT(slot);
if (bpf_register_is_null(reg) && type_may_be_null(arg_type)) {
err = mark_arg_precision(env, argno);
if (err)
@ -9029,14 +9056,6 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, u32 slot, u32 p
return -EFAULT;
}
/*
* Disable raw mode for bpf_map_peek_elem() on a bloom filter. The helper reads
* the value buffer as an input rather than filling it.
*/
if (is_helper_call(meta, BPF_FUNC_map_peek_elem) &&
meta->map.ptr->map_type == BPF_MAP_TYPE_BLOOM_FILTER)
meta->arg_raw_mem.regno = 0;
err = check_helper_mem_access(env, reg, argno, meta->map.ptr->value_size,
arg_type & MEM_WRITE ? BPF_WRITE : BPF_READ,
false, meta, NULL);
@ -9193,9 +9212,7 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, u32 slot, u32 p
if (!(arg_type & MEM_FIXED_SIZE))
break;
access_type = arg_type & MEM_WRITE ? BPF_WRITE : BPF_READ;
if (meta->btf)
access_type = BPF_READ | BPF_WRITE;
access_type = func_arg_access_type(arg_type);
err = check_mem_reg(env, reg, argno, arg_size, access_type, meta, &known_memory);
if (err < 0) {
@ -9241,9 +9258,7 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, u32 slot, u32 p
if (meta->btf && bpf_register_is_null(buff_reg))
break;
access_type = fn->arg_type[arg - 1] & MEM_WRITE ? BPF_WRITE : BPF_READ;
if (meta->btf)
access_type = BPF_READ | BPF_WRITE;
access_type = func_arg_access_type(fn->arg_type[arg - 1]);
zero_size_allowed = meta->btf || base_type(arg_type) == ARG_MEM_SIZE_OR_ZERO;
@ -9568,6 +9583,37 @@ static int check_func_args(struct bpf_verifier_env *env, struct bpf_call_arg_met
return 0;
}
static int mark_raw_stack(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
int insn_idx)
{
struct bpf_func_state *caller = cur_func(env);
u32 slot;
int i, err;
/*
* Validate every argument before initializing outputs: an input argument
* may alias an output buffer. Use the normal stack-write checks to discard
* stale spills and preserve the rules for special stack objects.
*/
for (slot = 0; slot < MAX_BPF_FUNC_ARGS; slot++) {
struct bpf_reg_state *reg;
argno_t argno = argno_from_arg(slot + 1);
if (!meta->arg_raw_mem.size[slot])
continue;
reg = get_func_arg_reg(caller, cur_regs(env), slot);
for (i = 0; i < meta->arg_raw_mem.size[slot]; i++) {
err = check_mem_access(env, insn_idx, reg, argno, i, BPF_B,
BPF_WRITE, -1, false, false);
if (err)
return err;
}
}
return 0;
}
static bool may_update_sockmap(struct bpf_verifier_env *env, int func_id)
{
enum bpf_attach_type eatype = env->prog->expected_attach_type;
@ -9860,23 +9906,6 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,
return -EINVAL;
}
static bool check_raw_mode_ok(const struct bpf_func_proto *fn, struct bpf_call_arg_meta *meta)
{
int i;
for (i = 0; i < ARRAY_SIZE(fn->arg_type); i++) {
if (fn->arg_type[i] == ARG_UNUSED)
break;
if (!arg_type_is_raw_mem(fn->arg_type[i]))
continue;
if (meta->arg_raw_mem.regno)
return false;
meta->arg_raw_mem.regno = i + 1;
}
return true;
}
static bool check_args_pair_invalid(const struct bpf_func_proto *fn, int arg)
{
bool is_fixed = fn->arg_type[arg] & MEM_FIXED_SIZE;
@ -10003,7 +10032,6 @@ static int check_func_proto(struct bpf_verifier_env *env, const struct bpf_func_
struct bpf_call_arg_meta *meta)
{
return check_arg_prog_aux(env, fn) &&
check_raw_mode_ok(fn, meta) &&
check_arg_pair_ok(fn) &&
check_mem_arg_rw_flag_ok(fn) &&
check_proto_release_reg(fn, meta) &&
@ -11590,16 +11618,9 @@ static int check_helper_call(struct bpf_verifier_env *env, struct bpf_insn *insn
regs = cur_regs(env);
/* Mark slots with STACK_MISC in case of raw mode, stack offset
* is inferred from register state.
*/
for (i = 0; i < meta.arg_raw_mem.size; i++) {
err = check_mem_access(env, insn_idx, regs + meta.arg_raw_mem.regno,
argno_from_reg(meta.arg_raw_mem.regno), i, BPF_B,
BPF_WRITE, -1, false, false);
if (err)
return err;
}
err = mark_raw_stack(env, &meta, insn_idx);
if (err)
return err;
if (meta.release_regno) {
struct bpf_reg_state *reg = &regs[meta.release_regno];
@ -12460,7 +12481,8 @@ static int resolve_func_arg_type(struct bpf_verifier_env *env,
PTR_ERR(resolve_ret));
return -EINVAL;
}
*arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | (*arg_type & PTR_MAYBE_NULL);
*arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | MEM_WRITE |
(*arg_type & (PTR_MAYBE_NULL | MEM_UNINIT));
return 0;
}
@ -12973,7 +12995,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
reg_arg_name(env, argno), btf_type_str(ref_t), ref_tname);
return -EINVAL;
}
arg_type = ARG_PTR_TO_MEM;
arg_type = ARG_PTR_TO_MEM | MEM_WRITE;
} else if (btf_type_is_struct(ref_t))
/* A pointer to a struct without a size argument is classified as ARG_PTR_TO_BTF_ID */
arg_type = ARG_PTR_TO_BTF_ID;
@ -12998,7 +13020,7 @@ get_kfunc_arg_type(struct bpf_verifier_env *env, struct bpf_call_arg_meta *meta,
return -EINVAL;
}
proto->arg_size[arg] = type_size;
arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE;
arg_type = ARG_PTR_TO_MEM | MEM_FIXED_SIZE | MEM_WRITE;
}
if (is_kfunc_arg_uninit(meta->btf, &args[arg]))
@ -13721,12 +13743,20 @@ s64 bpf_helper_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn
struct bpf_insn_aux_data *aux = &env->insn_aux_data[insn_idx];
const struct bpf_func_proto *fn;
enum bpf_arg_type at;
bool full_write;
s64 size;
if (bpf_get_helper_proto(env, insn->imm, &fn) < 0)
return S64_MIN;
at = fn->arg_type[arg];
/*
* Generic outputs may leave bytes untouched. Keep prior initialization
* live when the caller cannot read uninitialized bytes. Constructors of
* special objects, such as dynptrs, still define their storage.
*/
full_write = (at & MEM_UNINIT) &&
(!arg_type_is_raw_mem(at) || env->allow_uninit_stack);
switch (base_type(at)) {
case ARG_PTR_TO_MAP_KEY:
@ -13795,7 +13825,7 @@ s64 bpf_helper_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn
* Size arg is const on each path but differs across merged
* paths. MAX_BPF_STACK is a safe upper bound for reads.
*/
if (at & MEM_UNINIT)
if (full_write)
return 0;
return MAX_BPF_STACK;
}
@ -13815,10 +13845,10 @@ s64 bpf_helper_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn
}
out:
/*
* MEM_UNINIT args are write-only: the helper initializes the
* buffer without reading it.
* Other accesses keep the previous state live, including untouched bytes
* of an unprivileged generic output.
*/
if (at & MEM_UNINIT)
if (full_write)
return -size;
return size;
}
@ -13898,7 +13928,8 @@ s64 bpf_kfunc_stack_access_bytes(struct bpf_verifier_env *env, struct bpf_insn *
/* KF_ITER_NEW kfuncs initialize the iterator state at arg 0 */
if (arg == 0 && meta.kfunc_flags & KF_ITER_NEW)
return -size;
if (is_kfunc_arg_uninit(btf, &args[arg]))
if (is_kfunc_arg_uninit(btf, &args[i]) &&
(is_kfunc_arg_dynptr(btf, &args[i]) || env->allow_uninit_stack))
return -size;
return size;
}
@ -14179,6 +14210,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
if (err < 0)
return err;
err = mark_raw_stack(env, &meta, insn_idx);
if (err)
return err;
if ((is_bpf_obj_drop_kfunc(meta.func_id) ||
is_bpf_percpu_obj_drop_kfunc(meta.func_id)) && (is_tracing_prog_type(prog_type) ||
/* is_tracing_prog_type() for now doesn't cover non-iterator tracing progs. */

6
kernel/trace/bpf_trace.c
View File

@ -995,7 +995,7 @@ static const struct bpf_func_proto bpf_d_path_proto = {
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_BTF_ID,
.arg1_btf_id = &bpf_d_path_btf_ids[0],
.arg2_type = ARG_PTR_TO_MEM | MEM_WRITE,
.arg2_type = ARG_PTR_TO_MEM | MEM_UNINIT | MEM_WRITE,
.arg3_type = ARG_MEM_SIZE_OR_ZERO,
.allowed = bpf_d_path_allowed,
};
@ -1052,7 +1052,7 @@ const struct bpf_func_proto bpf_snprintf_btf_proto = {
.func = bpf_snprintf_btf,
.gpl_only = false,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_MEM | MEM_WRITE,
.arg1_type = ARG_PTR_TO_MEM | MEM_UNINIT | MEM_WRITE,
.arg2_type = ARG_MEM_SIZE,
.arg3_type = ARG_PTR_TO_MEM | MEM_RDONLY,
.arg4_type = ARG_MEM_SIZE,
@ -1565,7 +1565,7 @@ static const struct bpf_func_proto bpf_read_branch_records_proto = {
.gpl_only = true,
.ret_type = RET_INTEGER,
.arg1_type = ARG_PTR_TO_CTX,
.arg2_type = ARG_PTR_TO_MEM_OR_NULL | MEM_WRITE,
.arg2_type = ARG_PTR_TO_MEM_OR_NULL | MEM_UNINIT | MEM_WRITE,
.arg3_type = ARG_MEM_SIZE_OR_ZERO,
.arg4_type = ARG_ANYTHING,
};

4
tools/testing/selftests/bpf/prog_tests/verifier.c
View File

@ -56,6 +56,8 @@
#include "verifier_jeq_infer_not_null.skel.h"
#include "verifier_jit_convergence.skel.h"
#include "verifier_kfunc_packet_access.skel.h"
#include "verifier_kfunc_uninit.skel.h"
#include "verifier_kfunc_uninit_multi.skel.h"
#include "verifier_ld_ind.skel.h"
#include "verifier_ldsx.skel.h"
#include "verifier_leak_ptr.skel.h"
@ -223,6 +225,8 @@ void test_verifier_iterating_callbacks(void) { RUN(verifier_iterating_callbacks
void test_verifier_jeq_infer_not_null(void) { RUN(verifier_jeq_infer_not_null); }
void test_verifier_jit_convergence(void) { RUN(verifier_jit_convergence); }
void test_verifier_kfunc_packet_access(void) { RUN_TESTS(verifier_kfunc_packet_access); }
void test_verifier_kfunc_uninit(void) { RUN_TESTS(verifier_kfunc_uninit); }
void test_verifier_kfunc_uninit_multi(void) { RUN_TESTS(verifier_kfunc_uninit_multi); }
void test_verifier_load_acquire(void) { RUN(verifier_load_acquire); }
void test_verifier_ld_ind(void) { RUN(verifier_ld_ind); }
void test_verifier_ldsx(void) { RUN(verifier_ldsx); }

9
tools/testing/selftests/bpf/progs/bpf_misc.h
View File

@ -9,7 +9,8 @@
#define QUOTE(str) #str
#define EXPAND_QUOTE(str) QUOTE(str)
/* This set of attributes controls behavior of the
/*
* This set of attributes controls behavior of the
* test_loader.c:test_loader__run_subtests().
*
* The test_loader sequentially loads each program in a skeleton.
@ -131,6 +132,11 @@
* Several __arch_* annotations could be specified at once.
* When test case is not run on current arch it is marked as skipped.
* __caps_unpriv Specify the capabilities that should be set when running the test.
* __prepare_priv In unprivileged mode, prepare the object with the fixture's
* initial capabilities before dropping them for program loading.
* Preparation includes map creation and BTF/kfunc resolution;
* these operations are not tested at the reduced capabilities.
* Program loading uses the normal __caps_unpriv selection.
*
* __linear_size Specify the size of the linear area of non-linear skbs, or
* 0 for linear skbs.
@ -166,6 +172,7 @@
#define __arch_s390x __arch("s390x")
#define __arch_loongarch __arch("LOONGARCH")
#define __caps_unpriv(caps) __test_tag("test_caps_unpriv=" EXPAND_QUOTE(caps))
#define __prepare_priv __test_tag("test_prepare_priv")
#define __load_if_JITed() __test_tag("load_mode=jited")
#define __load_if_no_JITed() __test_tag("load_mode=no_jited")
#define __stderr(msg) __test_tag("test_expect_stderr=" msg)

332
tools/testing/selftests/bpf/progs/verifier_helper_access_var_len.c
View File

@ -1,4 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
/* Converted from tools/testing/selftests/bpf/verifier/helper_access_var_len.c */
#include <linux/bpf.h>
@ -822,4 +823,335 @@ __naked void bytes_no_leak_init_memory(void)
: __clobber_all);
}
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__uint(map_flags, BPF_F_WRONLY_PROG);
__type(key, __u32);
__type(value, struct bpf_fib_lookup);
} map_fib_wo SEC(".maps");
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__type(key, __u32);
__type(value, struct bpf_fib_lookup);
} map_fib_rw SEC(".maps");
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__uint(map_flags, BPF_F_RDONLY_PROG);
__type(key, __u32);
__type(value, struct bpf_fib_lookup);
} map_fib_ro SEC(".maps");
SEC("tc")
__failure __msg("read from map forbidden")
int writeonly_sized_input(struct __sk_buff *ctx)
{
struct bpf_fib_lookup *params;
__u32 key = 0;
params = bpf_map_lookup_elem(&map_fib_wo, &key);
if (params)
bpf_fib_lookup(ctx, params, sizeof(*params), 0);
return 0;
}
SEC("tc")
__success
int readwrite_sized_input(struct __sk_buff *ctx)
{
struct bpf_fib_lookup *params;
__u32 key = 0;
params = bpf_map_lookup_elem(&map_fib_rw, &key);
if (params)
bpf_fib_lookup(ctx, params, sizeof(*params), 0);
return 0;
}
SEC("tc")
__failure __msg("write into map forbidden")
int readonly_sized_output(struct __sk_buff *ctx)
{
struct bpf_fib_lookup *params;
__u32 key = 0;
params = bpf_map_lookup_elem(&map_fib_ro, &key);
if (params)
bpf_fib_lookup(ctx, params, sizeof(*params), 0);
return 0;
}
SEC("xdp")
__failure __msg("read from map forbidden")
int xdp_writeonly_sized_input(struct xdp_md *ctx)
{
struct bpf_fib_lookup *params;
__u32 key = 0;
params = bpf_map_lookup_elem(&map_fib_wo, &key);
if (params)
bpf_fib_lookup(ctx, params, sizeof(*params), 0);
return XDP_PASS;
}
SEC("sockops")
__failure __msg("read from map forbidden")
int writeonly_header_option(struct bpf_sock_ops *ctx)
{
struct bpf_fib_lookup *buf;
__u32 key = 0;
buf = bpf_map_lookup_elem(&map_fib_wo, &key);
if (buf)
bpf_load_hdr_opt(ctx, buf, sizeof(*buf), 0);
return 0;
}
SEC("sockops")
__success
int readwrite_header_option(struct bpf_sock_ops *ctx)
{
struct bpf_fib_lookup *buf;
__u32 key = 0;
buf = bpf_map_lookup_elem(&map_fib_rw, &key);
if (buf)
bpf_load_hdr_opt(ctx, buf, sizeof(*buf), 0);
return 0;
}
SEC("tc")
__success
int snprintf_writeonly_output(struct __sk_buff *ctx)
{
void *buf;
__u32 key = 0;
buf = bpf_map_lookup_elem(&map_fib_wo, &key);
if (buf)
bpf_snprintf(buf, 16, "ok", NULL, 0);
return 0;
}
SEC("tc")
__failure __msg("write into map forbidden")
int snprintf_readonly_output(struct __sk_buff *ctx)
{
void *buf;
__u32 key = 0;
buf = bpf_map_lookup_elem(&map_fib_ro, &key);
if (buf)
bpf_snprintf(buf, 16, "ok", NULL, 0);
return 0;
}
SEC("cgroup/sysctl")
__success
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__success_unpriv
int sysctl_writeonly_output(struct bpf_sysctl *ctx)
{
void *buf;
__u32 key = 0;
buf = bpf_map_lookup_elem(&map_fib_wo, &key);
if (buf)
bpf_sysctl_get_name(ctx, buf, 16, 0);
return 0;
}
SEC("cgroup/sysctl")
__success
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__success_unpriv
__naked void sysctl_initialized_stack(void)
{
asm volatile (
"*(u64 *)(r10 - 16) = 0;"
"*(u64 *)(r10 - 8) = 0;"
"r2 = r10;"
"r2 += -16;"
"r3 = 16;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = *(u8 *)(r10 - 1);"
"r0 &= 1;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("cgroup/sysctl")
__success
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__success_unpriv
__naked void sysctl_uninitialized_stack(void)
{
asm volatile (
"r2 = r10;"
"r2 += -16;"
"r3 = 16;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = 0;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("cgroup/sysctl")
__success
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__success_unpriv
__naked void sysctl_partial_initialized_bytes(void)
{
asm volatile (
"*(u32 *)(r10 - 16) = 0;"
"*(u8 *)(r10 - 1) = 1;"
"goto +0;"
"r2 = r10;"
"r2 += -16;"
"r3 = 16;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = *(u32 *)(r10 - 16);"
"r1 = *(u8 *)(r10 - 1);"
"r0 += r1;"
"r0 &= 1;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("cgroup/sysctl")
__success
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__failure_unpriv __msg_unpriv("invalid read from stack off -1+0 size 1")
__naked void sysctl_partial_invalid_bytes(void)
{
asm volatile (
"*(u32 *)(r10 - 16) = 0;"
"r2 = r10;"
"r2 += -16;"
"r3 = 16;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = *(u8 *)(r10 - 1);"
"r0 &= 1;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("cgroup/sysctl")
__success
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__success_unpriv
__naked void sysctl_partial_variable_size(void)
{
asm volatile (
"r3 = *(u32 *)(r1 + 0);"
"r3 &= 15;"
"r3 += 1;"
"*(u8 *)(r10 - 1) = 1;"
"goto +0;"
"r2 = r10;"
"r2 += -16;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = *(u8 *)(r10 - 1);"
"r0 &= 1;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("cgroup/sysctl")
__success
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__failure_unpriv __msg_unpriv("invalid read from stack off -1+0 size 1")
__naked void sysctl_partial_variable_invalid(void)
{
asm volatile (
"r3 = *(u32 *)(r1 + 0);"
"r3 &= 15;"
"r3 += 1;"
"r2 = r10;"
"r2 += -16;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = *(u8 *)(r10 - 1);"
"r0 &= 1;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("cgroup/sysctl")
__success
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__failure_unpriv __msg_unpriv("invalid read from stack R2")
__naked void sysctl_partial_spilled_pointer(void)
{
asm volatile (
"*(u64 *)(r10 - 8) = r1;"
"goto +0;"
"r2 = r10;"
"r2 += -8;"
"r3 = 8;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = 0;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("cgroup/sysctl")
__success
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__failure_unpriv __msg_unpriv("invalid read from stack R2")
int sysctl_partial_dynptr(struct bpf_sysctl *ctx)
{
struct bpf_dynptr ptr;
long long key = 0, *data;
data = bpf_map_lookup_elem(&map_hash_8b, &key);
if (!data)
return 0;
bpf_dynptr_from_mem(data, sizeof(*data), 0, &ptr);
bpf_sysctl_get_name(ctx, (char *)&ptr, sizeof(ptr), 0);
return 0;
}
SEC("cgroup/sysctl")
__success
__naked void sysctl_partial_variable_offset(void)
{
asm volatile (
"r2 = *(u32 *)(r1 + 0);"
"r2 &= 8;"
"r2 += r10;"
"r2 += -24;"
"r3 = 16;"
"r4 = 0;"
"call %[bpf_sysctl_get_name];"
"r0 = *(u8 *)(r10 - 16);"
"r0 &= 1;"
"exit;"
: : __imm(bpf_sysctl_get_name) : __clobber_all);
}
SEC("tc")
__success __retval(42)
int snprintf_partial_output_runtime(struct __sk_buff *ctx)
{
char buf[16];
buf[15] = 42;
bpf_snprintf(buf, sizeof(buf), "ok", NULL, 0);
return buf[15];
}
char _license[] SEC("license") = "GPL";

236
tools/testing/selftests/bpf/progs/verifier_kfunc_uninit.c Normal file
View File

@ -0,0 +1,236 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include "bpf_misc.h"
#include "../test_kmods/bpf_testmod_kfunc.h"
/* Keep the kfunc BTF records used by the inline assembly. */
void __kfunc_btf_root(void)
{
asm volatile ("" :
: "r"(&bpf_kfunc_test_uninit_struct),
"r"(&bpf_kfunc_test_uninit_mem),
"r"(&bpf_kfunc_test_uninit_partial),
"r"(&bpf_kfunc_test_uninit_alias));
}
SEC("tc")
__success __retval(10)
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void struct_poisoned_at_checkpoint(void)
{
asm volatile (
"*(u64 *)(r10 - 16) = 0;"
"*(u64 *)(r10 - 8) = 0;"
"goto +0;"
"r1 = r10;"
"r1 += -16;"
"call %[bpf_kfunc_test_uninit_struct];"
"r0 = *(u32 *)(r10 - 16);"
"r1 = *(u32 *)(r10 - 12);"
"r0 += r1;"
"r1 = *(u32 *)(r10 - 8);"
"r0 += r1;"
"r1 = *(u32 *)(r10 - 4);"
"r0 += r1;"
"exit;"
: : __imm(bpf_kfunc_test_uninit_struct) : __clobber_all);
}
SEC("tc")
__success __retval(0x2a2a2a2a)
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void sized_buffer_poisoned_at_checkpoint(void)
{
asm volatile (
"*(u64 *)(r10 - 8) = 0;"
"goto +0;"
"r1 = r10;"
"r1 += -8;"
"r2 = 8;"
"call %[bpf_kfunc_test_uninit_mem];"
"r0 = *(u32 *)(r10 - 8);"
"r1 = *(u32 *)(r10 - 4);"
"if r0 == r1 goto +1;"
"r0 = 0;"
"exit;"
: : __imm(bpf_kfunc_test_uninit_mem) : __clobber_all);
}
SEC("tc")
__success __retval(7)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void initialized_input_alias(void)
{
asm volatile (
"*(u32 *)(r10 - 8) = 7;"
"r1 = r10;"
"r1 += -8;"
"r2 = r1;"
"call %[bpf_kfunc_test_uninit_alias];"
"exit;"
: : __imm(bpf_kfunc_test_uninit_alias) : __clobber_all);
}
SEC("tc")
__success
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__failure_unpriv __msg_unpriv("invalid read from stack")
__naked void uninitialized_input_alias(void)
{
asm volatile (
"r1 = r10;"
"r1 += -8;"
"r2 = r1;"
"call %[bpf_kfunc_test_uninit_alias];"
"exit;"
: : __imm(bpf_kfunc_test_uninit_alias) : __clobber_all);
}
SEC("tc")
__success __retval(0)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void uninitialized_struct_output_ignored(void)
{
asm volatile (
"r1 = r10;"
"r1 += -16;"
"call %[bpf_kfunc_test_uninit_struct];"
"r0 = 0;"
"exit;"
: : __imm(bpf_kfunc_test_uninit_struct) : __clobber_all);
}
SEC("tc")
__success __retval(0)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void variable_size_uninitialized_output_ignored(void)
{
asm volatile (
"r2 = *(u32 *)(r1 + 0);"
"r2 &= 7;"
"r2 += 1;"
"r1 = r10;"
"r1 += -8;"
"call %[bpf_kfunc_test_uninit_mem];"
"r0 = 0;"
"exit;"
: : __imm(bpf_kfunc_test_uninit_mem) : __clobber_all);
}
SEC("tc")
__success __retval(49)
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void partial_output_preserves_initialized_bytes(void)
{
asm volatile (
"*(u8 *)(r10 - 8) = 0;"
"*(u8 *)(r10 - 1) = 7;"
"goto +0;"
"r1 = r10;"
"r1 += -8;"
"r2 = 8;"
"call %[bpf_kfunc_test_uninit_partial];"
"r0 = *(u8 *)(r10 - 8);"
"r1 = *(u8 *)(r10 - 1);"
"r0 += r1;"
"exit;"
: : __imm(bpf_kfunc_test_uninit_partial) : __clobber_all);
}
SEC("tc")
__success __retval(42)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__failure_unpriv __msg_unpriv("invalid read from stack off -8+0 size 1")
__naked void uninitialized_sized_output_read(void)
{
asm volatile (
"r1 = r10;"
"r1 += -8;"
"r2 = 8;"
"call %[bpf_kfunc_test_uninit_mem];"
"r0 = *(u8 *)(r10 - 8);"
"exit;"
: : __imm(bpf_kfunc_test_uninit_mem) : __clobber_all);
}
SEC("tc")
__success __retval(42)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__failure_unpriv __msg_unpriv("invalid read from stack off -8+0 size 1")
__naked void variable_size_uninitialized_output_read(void)
{
asm volatile (
"r2 = *(u32 *)(r1 + 0);"
"r2 &= 7;"
"r2 += 1;"
"r1 = r10;"
"r1 += -8;"
"call %[bpf_kfunc_test_uninit_mem];"
"r0 = *(u8 *)(r10 - 8);"
"exit;"
: : __imm(bpf_kfunc_test_uninit_mem) : __clobber_all);
}
SEC("tc")
__success __retval(1)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__failure_unpriv __msg_unpriv("invalid read from stack off -16+0 size 4")
__naked void fixed_struct_uninitialized_output_read(void)
{
asm volatile (
"r1 = r10;"
"r1 += -16;"
"call %[bpf_kfunc_test_uninit_struct];"
"r0 = *(u32 *)(r10 - 16);"
"exit;"
: : __imm(bpf_kfunc_test_uninit_struct) : __clobber_all);
}
SEC("tc")
__success __retval(49)
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void partial_output_variable_size(void)
{
asm volatile (
"r2 = *(u32 *)(r1 + 0);"
"r2 &= 7;"
"r2 += 1;"
"*(u8 *)(r10 - 8) = 0;"
"*(u8 *)(r10 - 1) = 7;"
"goto +0;"
"r1 = r10;"
"r1 += -8;"
"call %[bpf_kfunc_test_uninit_partial];"
"r0 = *(u8 *)(r10 - 8);"
"r1 = *(u8 *)(r10 - 1);"
"r0 += r1;"
"exit;"
: : __imm(bpf_kfunc_test_uninit_partial) : __clobber_all);
}
char _license[] SEC("license") = "GPL";

113
tools/testing/selftests/bpf/progs/verifier_kfunc_uninit_multi.c Normal file
View File

@ -0,0 +1,113 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2026 Meta Platforms, Inc. and affiliates. */
#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include "bpf_misc.h"
#include "../test_kmods/bpf_testmod_kfunc.h"
/* Keep the kfunc BTF records used by the inline assembly. */
void __kfunc_btf_root(void)
{
asm volatile ("" :
: "r"(&bpf_kfunc_test_uninit_multi),
"r"(&bpf_kfunc_test_uninit_pair),
"r"(&bpf_kfunc_test_uninit_stack));
}
SEC("tc")
__success __retval(42)
__flag(BPF_F_TEST_STATE_FREQ)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__success_unpriv
__naked void multiple_outputs(void)
{
asm volatile (
"*(u64 *)(r10 - 16) = 0;"
"*(u64 *)(r10 - 8) = 0;"
"goto +0;"
"r1 = r10;"
"r1 += -16;"
"r2 = r10;"
"r2 += -8;"
"r3 = 8;"
"call %[bpf_kfunc_test_uninit_multi];"
"r0 = *(u32 *)(r10 - 16);"
"r1 = *(u32 *)(r10 - 8);"
"if r1 != 0x2a2a2a2a goto 1f;"
"r1 = *(u32 *)(r10 - 4);"
"if r1 == 0x2a2a2a2a goto 2f;"
"1:;"
"r0 = 0;"
"2:;"
"exit;"
: : __imm(bpf_kfunc_test_uninit_multi) : __clobber_all);
}
SEC("tc")
__success __retval(42)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__failure_unpriv __msg_unpriv("invalid read from stack off -16+0 size 4")
__naked void variable_size_preserves_other_output(void)
{
asm volatile (
"r3 = *(u32 *)(r1 + 0);"
"r3 &= 7;"
"*(u64 *)(r10 - 8) = 0;"
"r1 = r10;"
"r1 += -16;"
"r2 = r10;"
"r2 += -8;"
"call %[bpf_kfunc_test_uninit_multi];"
"r0 = *(u32 *)(r10 - 16);"
"exit;"
: : __imm(bpf_kfunc_test_uninit_multi) : __clobber_all);
}
SEC("tc")
__arch_x86_64 __arch_arm64
__success __retval(42)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__failure_unpriv __msg_unpriv("invalid read from stack off -8+0 size 4")
__naked void output_after_by_value_argument(void)
{
asm volatile (
"r1 = 20;"
"r2 = 22;"
"r3 = r10;"
"r3 += -8;"
"call %[bpf_kfunc_test_uninit_pair];"
"r0 = *(u32 *)(r10 - 8);"
"exit;"
: : __imm(bpf_kfunc_test_uninit_pair) : __clobber_all);
}
#if defined(__BPF_FEATURE_STACK_ARGUMENT)
SEC("tc")
__arch_x86_64 __arch_arm64 __arch_riscv64
__success __retval(15)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__prepare_priv
__failure_unpriv __msg_unpriv("invalid read from stack off -8+0 size 4")
__naked void output_passed_on_stack(void)
{
asm volatile (
"r1 = 1;"
"r2 = 2;"
"r3 = 3;"
"r4 = 4;"
"r5 = 5;"
"r6 = r10;"
"r6 += -8;"
"*(u64 *)(r11 - 8) = r6;"
"call %[bpf_kfunc_test_uninit_stack];"
"r0 = *(u32 *)(r10 - 8);"
"exit;"
: : __imm(bpf_kfunc_test_uninit_stack) : __clobber_all);
}
#endif
char _license[] SEC("license") = "GPL";

33
tools/testing/selftests/bpf/progs/verifier_live_stack.c
View File

@ -21,8 +21,6 @@ struct {
__type(value, __u64);
} array_map_8b SEC(".maps");
const char snprintf_u64_fmt[] = "%llu";
SEC("socket")
__log_level(2)
__msg("0: (79) r1 = *(u64 *)(r10 -8) ; use: fp0-8")
@ -1947,15 +1945,15 @@ static __used __naked void fwd_parent_key_to_helper(void)
/*
* Regression for keeping later helper args after a whole-stack fallback
* on an earlier local arg. The first bpf_snprintf() arg is a local
* on an earlier local arg. The bpf_map_update_elem() key is a local
* frame-derived pointer with offset-imprecise tracking (`fp1 ?`), which
* conservatively marks the whole local stack live. The fourth arg still
* conservatively marks the whole local stack live. The value arg still
* forwards &parent_fp-8 and must contribute nonlocal_use[0]=0:3.
*/
SEC("socket")
__log_level(2)
__success
__msg("call bpf_snprintf{{.*}} ; use: fp1-8..-512 fp0-8")
__msg("call bpf_map_update_elem{{.*}}; use: fp1-8..-512 fp0-8")
__naked void helper_arg_fallback_keeps_scanning(void)
{
asm volatile (
@ -1963,32 +1961,33 @@ __naked void helper_arg_fallback_keeps_scanning(void)
"*(u64 *)(r10 - 8) = r1;"
"r1 = r10;"
"r1 += -8;"
"call helper_snprintf_parent_after_local_fallback;"
"call helper_update_parent_after_local_fallback;"
"r0 = 0;"
"exit;"
::: __clobber_all);
}
static __used __naked void helper_snprintf_parent_after_local_fallback(void)
static __used __naked void helper_update_parent_after_local_fallback(void)
{
asm volatile (
"r6 = r1;" /* save &parent_fp-8 */
"call %[bpf_get_prandom_u32];"
"r0 &= 8;"
"r1 = r10;"
"r1 += -16;"
"r1 += r0;" /* local fp, offset-imprecise */
"r2 = 8;"
"r3 = %[snprintf_u64_fmt] ll;"
"r4 = r6;" /* later arg: parent fp-8 */
"r5 = 8;"
"call %[bpf_snprintf];"
"*(u64 *)(r10 - 16) = 0;"
"*(u64 *)(r10 - 8) = 0;"
"r2 = r10;"
"r2 += -16;"
"r2 += r0;" /* local fp, offset-imprecise */
"r1 = %[array_map_8b] ll;"
"r3 = r6;" /* later arg: parent fp-8 */
"r4 = 0;"
"call %[bpf_map_update_elem];"
"r0 = 0;"
"exit;"
:
: __imm(bpf_get_prandom_u32),
__imm(bpf_snprintf),
__imm_addr(snprintf_u64_fmt)
__imm(bpf_map_update_elem),
__imm_addr(array_map_8b)
: __clobber_all);
}

88
tools/testing/selftests/bpf/progs/verifier_mtu.c
View File

@ -17,4 +17,92 @@ int tc_uninit_mtu(struct __sk_buff *ctx)
return TCX_PASS;
}
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__uint(map_flags, BPF_F_WRONLY_PROG);
__type(key, __u32);
__type(value, __u32);
} map_mtu_wo SEC(".maps");
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__type(key, __u32);
__type(value, __u32);
} map_mtu_rw SEC(".maps");
struct {
__uint(type, BPF_MAP_TYPE_ARRAY);
__uint(max_entries, 1);
__uint(map_flags, BPF_F_RDONLY_PROG);
__type(key, __u32);
__type(value, __u32);
} map_mtu_ro SEC(".maps");
SEC("tc/ingress")
__failure __msg("read from map forbidden")
int tc_writeonly_mtu(struct __sk_buff *ctx)
{
__u32 key = 0;
__u32 *mtu;
mtu = bpf_map_lookup_elem(&map_mtu_wo, &key);
if (mtu)
bpf_check_mtu(ctx, 0, mtu, 0, 0);
return TCX_PASS;
}
SEC("tc/ingress")
__success
int tc_readwrite_mtu(struct __sk_buff *ctx)
{
__u32 key = 0;
__u32 *mtu;
mtu = bpf_map_lookup_elem(&map_mtu_rw, &key);
if (mtu)
bpf_check_mtu(ctx, 0, mtu, 0, 0);
return TCX_PASS;
}
SEC("xdp")
__failure __msg("read from map forbidden")
int xdp_writeonly_mtu(struct xdp_md *ctx)
{
__u32 key = 0;
__u32 *mtu;
mtu = bpf_map_lookup_elem(&map_mtu_wo, &key);
if (mtu)
bpf_check_mtu(ctx, 0, mtu, 0, 0);
return XDP_PASS;
}
SEC("tc/ingress")
__failure __msg("write into map forbidden")
int tc_readonly_mtu(struct __sk_buff *ctx)
{
__u32 key = 0;
__u32 *mtu;
mtu = bpf_map_lookup_elem(&map_mtu_ro, &key);
if (mtu)
bpf_check_mtu(ctx, 0, mtu, 0, 0);
return TCX_PASS;
}
SEC("tc/ingress")
__success
int tc_writeonly_output(struct __sk_buff *ctx)
{
__u32 key = 0;
__u32 *out;
out = bpf_map_lookup_elem(&map_mtu_wo, &key);
if (out)
bpf_get_current_comm(out, sizeof(*out));
return TCX_PASS;
}
char LICENSE[] SEC("license") = "GPL";

4
tools/testing/selftests/bpf/progs/verifier_raw_stack.c
View File

@ -84,6 +84,8 @@ __naked void skb_load_bytes_zero_len(void)
SEC("tc")
__description("raw_stack: skb_load_bytes, no init")
__success __retval(0)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__failure_unpriv __msg_unpriv("invalid read from stack off -8+0 size 8")
__naked void skb_load_bytes_no_init(void)
{
asm volatile (" \
@ -103,6 +105,8 @@ __naked void skb_load_bytes_no_init(void)
SEC("tc")
__description("raw_stack: skb_load_bytes, init")
__success __retval(0)
__caps_unpriv(CAP_BPF | CAP_NET_ADMIN)
__success_unpriv
__naked void stack_skb_load_bytes_init(void)
{
asm volatile (" \

52
tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
View File

@ -17,6 +17,7 @@
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/un.h>
#include <linux/unaligned.h>
#include <linux/filter.h>
#include <linux/rcupdate_trace.h>
#include <net/sock.h>
@ -1316,6 +1317,50 @@ __bpf_kfunc void bpf_kfunc_call_test_pass2(struct prog_test_pass2 *p)
{
}
__bpf_kfunc void bpf_kfunc_test_uninit_struct(struct prog_test_pass1 *out__uninit)
{
out__uninit->x0 = 1;
out__uninit->x1 = 2;
out__uninit->x2 = 3;
out__uninit->x3 = 4;
}
__bpf_kfunc void bpf_kfunc_test_uninit_mem(void *out__uninit, u32 out__sz)
{
memset(out__uninit, 0x2a, out__sz);
}
__bpf_kfunc void bpf_kfunc_test_uninit_partial(void *out__uninit, u32 out__sz)
{
if (out__sz)
*(u8 *)out__uninit = 42;
}
__bpf_kfunc int bpf_kfunc_test_uninit_alias(int *out__uninit, const int *in)
{
int value = get_unaligned(in);
put_unaligned(42, out__uninit);
return value;
}
__bpf_kfunc void bpf_kfunc_test_uninit_multi(int *a__uninit, void *b__uninit, u32 b__sz)
{
put_unaligned(42, a__uninit);
memset(b__uninit, 0x2a, b__sz);
}
__bpf_kfunc void bpf_kfunc_test_uninit_pair(struct prog_test_pair_arg p, int *out__uninit)
{
put_unaligned((int)(p.lo + p.hi), out__uninit);
}
__bpf_kfunc void bpf_kfunc_test_uninit_stack(u64 a, u64 b, u64 c, u64 d, u64 e,
int *out__uninit)
{
put_unaligned((int)(a + b + c + d + e), out__uninit);
}
__bpf_kfunc void bpf_kfunc_call_test_fail1(struct prog_test_fail1 *p)
{
}
@ -1747,6 +1792,13 @@ BTF_ID_FLAGS(func, bpf_kfunc_call_int_mem_release, KF_RELEASE)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_pass_ctx)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_pass1)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_pass2)
BTF_ID_FLAGS(func, bpf_kfunc_test_uninit_struct)
BTF_ID_FLAGS(func, bpf_kfunc_test_uninit_mem)
BTF_ID_FLAGS(func, bpf_kfunc_test_uninit_partial)
BTF_ID_FLAGS(func, bpf_kfunc_test_uninit_alias)
BTF_ID_FLAGS(func, bpf_kfunc_test_uninit_multi)
BTF_ID_FLAGS(func, bpf_kfunc_test_uninit_pair)
BTF_ID_FLAGS(func, bpf_kfunc_test_uninit_stack)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_fail1)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_fail2)
BTF_ID_FLAGS(func, bpf_kfunc_call_test_fail3)

8
tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
View File

@ -289,6 +289,14 @@ __u64 bpf_kfunc_call_stack_arg_big(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
void bpf_kfunc_call_test_pass_ctx(struct __sk_buff *skb) __ksym;
void bpf_kfunc_call_test_pass1(struct prog_test_pass1 *p) __ksym;
void bpf_kfunc_call_test_pass2(struct prog_test_pass2 *p) __ksym;
void bpf_kfunc_test_uninit_struct(struct prog_test_pass1 *out__uninit) __ksym;
void bpf_kfunc_test_uninit_mem(void *out__uninit, __u32 out__sz) __ksym;
void bpf_kfunc_test_uninit_partial(void *out__uninit, __u32 out__sz) __ksym;
int bpf_kfunc_test_uninit_alias(int *out__uninit, const int *in) __ksym;
void bpf_kfunc_test_uninit_multi(int *a__uninit, void *b__uninit, __u32 b__sz) __ksym;
void bpf_kfunc_test_uninit_pair(struct prog_test_pair_arg p, int *out__uninit) __ksym;
void bpf_kfunc_test_uninit_stack(__u64 a, __u64 b, __u64 c, __u64 d, __u64 e,
int *out__uninit) __ksym;
void bpf_kfunc_call_test_mem_len_fail2(__u64 *mem, int len) __ksym;
void bpf_kfunc_call_test_destructive(void) __ksym;

48
tools/testing/selftests/bpf/test_loader.c
View File

@ -33,6 +33,7 @@ static inline const char *str_has_pfx(const char *str, const char *pfx)
#endif
static int sysctl_unpriv_disabled = -1;
static int unpriv_mitigations_disabled = -1;
enum mode {
PRIV = 1,
@ -71,6 +72,7 @@ struct test_spec {
int load_mask;
int linear_sz;
const char *skip_reason;
bool prepare_priv;
bool auxiliary;
bool valid;
};
@ -606,6 +608,8 @@ static int parse_test_spec(struct test_loader *tester,
if (err)
goto cleanup;
spec->mode_mask |= UNPRIV;
} else if (strcmp(s, "test_prepare_priv") == 0) {
spec->prepare_priv = true;
} else if ((val = str_has_pfx(s, "load_mode="))) {
if (strcmp(val, "jited") == 0) {
load_mask = JITED;
@ -1015,10 +1019,10 @@ struct cap_state {
bool initialized;
};
static int drop_capabilities(struct cap_state *caps)
static int drop_capabilities(struct cap_state *caps, __u64 keep_caps)
{
const __u64 caps_to_drop = (1ULL << CAP_SYS_ADMIN | 1ULL << CAP_NET_ADMIN |
1ULL << CAP_PERFMON | 1ULL << CAP_BPF);
1ULL << CAP_PERFMON | 1ULL << CAP_BPF) & ~keep_caps;
int err;
err = cap_disable_effective(caps_to_drop, &caps->old_caps);
@ -1028,6 +1032,13 @@ static int drop_capabilities(struct cap_state *caps)
}
caps->initialized = true;
if (keep_caps) {
err = cap_enable_effective(keep_caps, NULL);
if (err) {
PRINT_FAIL("failed to set capabilities: %i, %s\n", err, strerror(-err));
return err;
}
}
return 0;
}
@ -1048,8 +1059,12 @@ static int restore_capabilities(struct cap_state *caps)
static bool can_execute_unpriv(struct test_loader *tester, struct test_spec *spec)
{
if (sysctl_unpriv_disabled < 0)
sysctl_unpriv_disabled = get_unpriv_disabled() ? 1 : 0;
if (sysctl_unpriv_disabled)
sysctl_unpriv_disabled = get_unpriv_sysctl_disabled();
if (sysctl_unpriv_disabled && !(spec->unpriv.caps & (1ULL << CAP_BPF)))
return false;
if (unpriv_mitigations_disabled < 0)
unpriv_mitigations_disabled = get_unpriv_mitigations_disabled();
if (unpriv_mitigations_disabled)
return false;
if ((spec->prog_flags & BPF_F_ANY_ALIGNMENT) && !EFFICIENT_UNALIGNED_ACCESS)
return false;
@ -1351,17 +1366,8 @@ void run_subtest(struct test_loader *tester,
test__end_subtest();
return;
}
if (drop_capabilities(&caps)) {
test__end_subtest();
return;
}
if (subspec->caps) {
err = cap_enable_effective(subspec->caps, NULL);
if (err) {
PRINT_FAIL("failed to set capabilities: %i, %s\n", err, strerror(-err));
goto subtest_cleanup;
}
}
if (!spec->prepare_priv && drop_capabilities(&caps, subspec->caps))
goto subtest_cleanup;
}
/* Implicitly reset to NULL if next test case doesn't specify.
@ -1414,6 +1420,18 @@ void run_subtest(struct test_loader *tester,
bpf_object__for_each_map(map, tobj)
bpf_map__set_autocreate(map, !unpriv || is_unpriv_capable_map(map));
if (unpriv && spec->prepare_priv) {
/*
* Module BTF lookup needs CAP_SYS_ADMIN. Allow tests to prepare
* their objects first, then verify programs with the requested caps.
*/
err = bpf_object__prepare(tobj);
if (!ASSERT_OK(err, "obj_prepare"))
goto tobj_cleanup;
if (drop_capabilities(&caps, subspec->caps))
goto tobj_cleanup;
}
err = bpf_object__load(tobj);
if (subspec->expect_failure) {
if (!ASSERT_ERR(err, "unexpected_load_success")) {

16
tools/testing/selftests/bpf/unpriv_helpers.c
View File

@ -111,9 +111,8 @@ static int get_mitigations_off(void)
return !enabled_in_config;
}
bool get_unpriv_disabled(void)
bool get_unpriv_sysctl_disabled(void)
{
int mitigations_off;
bool disabled;
char buf[2];
FILE *fd;
@ -127,8 +126,12 @@ bool get_unpriv_disabled(void)
disabled = true;
}
if (disabled)
return true;
return disabled;
}
bool get_unpriv_mitigations_disabled(void)
{
int mitigations_off;
/*
* Some unpriv tests rely on spectre mitigations being on.
@ -144,6 +147,11 @@ bool get_unpriv_disabled(void)
return mitigations_off;
}
bool get_unpriv_disabled(void)
{
return get_unpriv_sysctl_disabled() || get_unpriv_mitigations_disabled();
}
bool get_kasan_jit_enabled(void)
{
return config_contains("CONFIG_BPF_JIT_KASAN=y") == 1;

2
tools/testing/selftests/bpf/unpriv_helpers.h
View File

@ -5,5 +5,7 @@
#define UNPRIV_SYSCTL "kernel/unprivileged_bpf_disabled"
bool get_unpriv_disabled(void);
bool get_unpriv_sysctl_disabled(void);
bool get_unpriv_mitigations_disabled(void);
bool get_kasan_jit_enabled(void);
bool get_kasan_multi_shot_enabled(void);