github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-11 20:13:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

bpf: Clear NON_OWN_REF after RCU protection ends

Browse Source 
A local kptr load of an object containing a graph node is marked MEM_RCU
and NON_OWN_REF while protected by RCU. When the last RCU read-side critical
section ends, invalidate_rcu_protected_refs() removes MEM_RCU and marks the
pointer PTR_UNTRUSTED, but leaves NON_OWN_REF set.

The stale flag lets graph kfunc argument checks continue treating the
pointer as a live borrowed reference. In particular, bpf_rbtree_remove()
can accept a pointer after its protection ended and return it as a new
owning reference, even though the object may already have been freed.

Clear NON_OWN_REF when an RCU-protected pointer is demoted. A spin lock also
provides implicit RCU protection, so invalidate non-owning references before
demoting RCU-protected pointers when releasing the lock. Otherwise the
demotion would clear the flag before invalidate_non_owning_refs() can find
and invalidate those aliases.

The demoted pointer remains available for fault-protected reads. Exempt such
reads from the allocated-object reference-state assertion; writes through a
fault-prone pointer are already rejected, and bpf_may_fault_on_deref() makes
the surviving loads use BPF_PROBE_MEM.

Fixes: 1b12171533 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260904084325.52250-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This commit is contained in:
Kumar Kartikeya Dwivedi 2026-09-04 10:43:18 +02:00 committed by Alexei Starovoitov
parent 2edd833946
commit cd6f72d7f3
1 changed files with 14 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
kernel/bpf/verifier.c
View File

@ -6045,7 +6045,13 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,
return -EACCES;
}
if (type_is_alloc(reg->type) && !type_is_non_owning_ref(reg->type) &&
/*
* A fault-prone allocated object may still be read through a
* BPF_PROBE_MEM load after its lifetime protection ends. Writes
* through such pointers were rejected above.
*/
if (type_is_alloc(reg->type) && !bpf_may_fault_on_deref(reg->type) &&
!type_is_non_owning_ref(reg->type) &&
!(reg->type & MEM_RCU) && !reg_is_referenced(env, reg)) {
verifier_bug(env, "allocated object must have a referenced id");
return -EFAULT;
@ -7423,10 +7429,14 @@ static int process_spin_lock(struct bpf_verifier_env *env, struct bpf_reg_state
lock);
return -EINVAL;
}
/*
* Invalidate non-owning refs before RCU demotion clears their
* NON_OWN_REF flag.
*/
invalidate_non_owning_refs(env);
if (!in_rcu_cs(env))
invalidate_rcu_protected_refs(env);
invalidate_non_owning_refs(env);
}
return 0;
}
@ -9526,7 +9536,7 @@ static void invalidate_rcu_protected_refs(struct bpf_verifier_env *env)
bpf_for_each_reg_in_vstate_mask(env->cur_state, state, reg, stack, clear_mask, ({
if (reg->type & MEM_RCU) {
bpf_diag_mod_begin(env, reg, NULL, BPF_DIAG_MOD_WRITE);
reg->type &= ~(MEM_RCU | PTR_MAYBE_NULL);
reg->type &= ~(MEM_RCU | PTR_MAYBE_NULL | NON_OWN_REF);
reg->type |= PTR_UNTRUSTED;
bpf_diag_mod_end(env);
}