mirror of
https://github.com/torvalds/linux.git
synced 2026-10-23 23:51:02 +02:00
aeb709c767
53539 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Alexei Starovoitov
|
fbd97dd1d3
|
bpf: Fix objects stuck in free_by_rcu_ttrace
do_call_rcu_ttrace() returns early when call_rcu_ttrace_in_progress is set
and leaves the objects in free_by_rcu_ttrace. __free_rcu() frees
waiting_for_gp_ttrace only and clears the flag. Hence the objects that
free_bulk() or __free_by_rcu() added while RCU tasks trace GP was in flight
stay in free_by_rcu_ttrace until free_bulk() or alloc_bulk() is called for
the same bpf_mem_cache again, which may never happen. The number of such
objects is not bounded.
Turn call_rcu_ttrace_in_progress into three states:
0 - idle
1 - __free_rcu() is queued
2 - __free_rcu() is queued and free_by_rcu_ttrace got more objects since
do_call_rcu_ttrace() sets 2. __free_rcu() does cmpxchg(1 -> 0) and starts
the next GP when it fails. It cannot clear the flag first and check
free_by_rcu_ttrace later, since bpf_mem_alloc_destroy() frees bpf_mem_cache
without waiting for RCU callbacks when the flag is zero.
Now __free_rcu() queues itself, so the one that didn't see 'draining' may
do call_rcu_tasks_trace() after rcu_barrier_tasks_trace() in
free_mem_alloc(). Queue it under rcu_read_lock() and do synchronize_rcu()
before the barriers. Calling rcu_barrier_tasks_trace() twice works too, but
creating and destroying hash maps in a loop on many cpus slows down to one
free_mem_alloc() per GP and kworkers pile up.
Fixes:
|
||
|
Alexei Starovoitov
|
47f4f695ce
|
bpf: Factor out __do_call_rcu_ttrace()
Move the part of do_call_rcu_ttrace() that runs after call_rcu_ttrace_in_progress is set into __do_call_rcu_ttrace(). The next patch will call it from __free_rcu(). No functional change. Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20260930095920.601738-2-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> |
||
|
Alexei Starovoitov
|
b88822d258
|
bpf: Fix packet range of pointers sharing an id
Since commit |
||
|
Florent Revest (Anthropic)
|
17637e1a58
|
bpf: Skip detached progs in trampoline images that are still in use
bpf_tramp_image_put() makes sure a trampoline image is not freed while
a task may still be running in it, but nothing similar is done for the
progs called by that image. Detach drops the last prog reference right
away and the prog is freed after grace periods, on the basis that a
task still in the traced function skips the fexit progs once the nop at
ip_after_call is patched to a jump.
That leaves out a task sleeping in a sleepable prog that runs before
the detached one, which no grace period waits for:
CPU 0 CPU 1
in image I, sleeping in prog S
detach P from I's trampoline
-> new image, bpf_tramp_image_put(I)
bpf_prog_put(P), last ref
grace periods, P freed
back from S
__bpf_prog_enter(P)
call P->bpf_func
If S and P are fexit progs the task is already past the patched jump,
and fentry only images don't have one. On x86 this is an int3 in
poisoned bpf_prog_pack memory:
Oops: int3: 0000 [#1] SMP NOPTI
CPU: 18 UID: 0 PID: 94573 Comm: x169 Not tainted 6.18.44 #1 PREEMPT(lazy)
RIP: 0010:0xffffffffc0601d8d
Call Trace:
<TASK>
? bpf_trampoline_6442515411+0x1a4/0x21b
bpf_lsm_bprm_committed_creds+0x5/0x10
security_bprm_committed_creds+0x5f/0x70
begin_new_exec+0x2d6/0x410
...
We hit this in production when progs attached through trampolines got
detached while their hooks were busy, and it was independently found
with a fuzzer and KASAN.
Have the JITs emit a patchable nop in front of each prog call sequence
and record it in the image. When a prog is detached, patch its nop to a
jump over the call sequence. Progs that stay attached keep running for
the tasks that are in the image, and ip_after_call isn't needed anymore.
The task can be in any image that isn't freed yet, not only in the
current one. It sleeps in image I1 that calls S, P and Q, then P is
detached and the trampoline moves to image I2, then Q is detached and
its call is still in I1. So the trampoline keeps a list of its images
until they are freed, and detaching a prog patches its nop in all of
them. Images hold a reference on the trampoline for that long.
On riscv and loongarch a jump of any range takes several instructions,
and a task preempted in the middle of them could resume into half of the
new sequence. The nop is a single instruction there, patched to a near
branch through arch_bpf_trampoline_skip().
With the extra nops, BPF_MAX_TRAMP_LINKS progs no longer fit in a page
on x86 and arm64 (and already didn't on powerpc), so lower the limit
there like s390 does.
Fixes:
|
||
|
Florent Revest (Anthropic)
|
0fb5751a6e
|
bpf: Wait for an RCU tasks grace period before freeing trampoline progs
When a prog is detached from a trampoline, it is freed after an RCU
grace period, or an RCU tasks trace one if it is sleepable. This covers
the tasks that are running the prog, since the prog's enter helper takes
the matching read lock before the prog is called. On a preemptible
kernel, it doesn't cover a task that was preempted in the trampoline
just before the enter helper. That task holds no lock yet, and it calls
the prog after it was freed:
BUG: KASAN: vmalloc-out-of-bounds in __bpf_prog_enter_recur+0x3a5/0x3f0
Read of size 8 at addr ffffc90000055040 by task candidate/110
CPU: 1 UID: 0 PID: 110 Comm: candidate Not tainted 7.3.0-rc2-00014-g15071f2a1263-dirty #2 PREEMPT(full)
Call Trace:
<TASK>
__bpf_prog_enter_recur+0x3a5/0x3f0
bpf_trampoline_6442509193+0x37/0xf1
__x64_sys_futex+0x9/0x410
do_syscall_64+0xb0/0x530
...
Wait for an RCU tasks grace period before the existing one when freeing
a prog that was linked to a trampoline. An RCU tasks grace period only
ends once the tasks that were preempted have run again, and
bpf_tramp_image_put() already relies on it to free the image. It doesn't
wait for tasks that sleep in the trampoline, the next commit takes care
of those.
Only progs that were linked to a trampoline can be called this way, so
bpf_trampoline_add_prog() marks them and other progs are still freed as
before.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
ab39974240
|
bpf: Hold map BTF for the memory allocator destructor record
bpf_ma_set_dtor() duplicates map->record so that the bpf_mem_alloc
destructor can release the special fields of hash and rhash map elements
once the allocator frees them for good. btf_record_dup() only acquires
references on kernel and module BTF. Fields whose types live in the map
BTF keep pointing into it: a kptr to a local type refers to map->btf,
and a list_head or rb_root field carries a value_rec owned by the struct
meta table of map->btf.
That borrowed state can outlive the map. When RCU callbacks are still in
flight, bpf_mem_alloc_destroy() copies the allocator and defers the final
drain, together with the destructor context, to a workqueue.
bpf_map_free() then frees map->record and drops the map's BTF reference
as soon as map_free() returns. Once the program that loaded the BTF is
gone as well, the BTF is freed while the worker still uses the
duplicate. The worker dereferences it through btf_is_kernel() and
btf_find_struct_meta() in bpf_obj_free_fields() when destroying elements
that were freed under RCU, and through btf_is_kernel() in
btf_record_free() when releasing the context itself:
BUG: KASAN: slab-use-after-free in btf_is_kernel+0x19/0x30
Read of size 1 at addr ffff88811505dce8 by task kworker/u16:2/50
Workqueue: events_unbound free_mem_alloc_deferred
Call Trace:
btf_is_kernel+0x19/0x30
btf_record_free+0xc7/0xf0
htab_dtor_ctx_free+0x1a/0x30
free_mem_alloc_no_barrier+0x39/0x230
free_mem_alloc_deferred+0x2a/0x40
process_scheduled_works+0x5ce/0x9d0
worker_thread+0x42a/0x5e0
kthread+0x1f5/0x230
ret_from_fork+0x1dd/0x390
ret_from_fork_asm+0x1a/0x30
Allocated by task 358:
__kmalloc_cache_noprof+0x287/0x510
btf_new_fd+0xf7/0x3d0
__sys_bpf+0x487/0x840
Freed by task 0:
kfree+0x186/0x560
rcu_core+0x6f6/0xd40
Freeing a map that still holds more elements than the allocator's high
watermark is enough to get there, because the bulk free during teardown
queues the RCU callback that makes bpf_mem_alloc_destroy() defer.
Non-preallocated hash maps and resizable hash maps both register the
destructor and are affected alike.
Keep one reference on the map BTF in the destructor context. Every
non-kernel pointer in the duplicated record is either map->btf or memory
owned by it, so a single reference covers kptrs, list heads, and rb
roots, and mirrors what bpf_map_meta_alloc() does for the duplicated
record of an inner map. Release it only after the duplicated record has
been freed: the worker is preemptible and not in an RCU read-side
critical section, so dropping the last reference first would let the
BTF be freed while btf_record_free() still reads its fields.
Fixes:
|
||
|
Alexei Starovoitov
|
ad139384ec
|
bpf: Fix overflow of jump offset in constant blinding
bpf_jit_blind_insn() rewrites 'if rX op imm goto off' into:
AX = imm ^ rnd
AX ^= rnd
if rX op AX goto off
and subtracts 2 from off when the jump is backward. off is s16, so
-32767 and -32768 wrap and the jump goes 32767 insns forward, past the
end of the prog. The jump is inside the patch, hence bpf_adj_branches()
doesn't check it. The kernel crashes when such prog runs with
bpf_jit_harden=2.
Return -ERANGE in such case. bpf_jit_blind_constants() fails the same
way when blinding of other insns moves a jump out of range.
Fixes:
|
||
|
Linus Torvalds
|
5fc5768c7c |
bpf-fixes
-----BEGIN PGP SIGNATURE-----
iQJRBAABCgA7FiEE+soXsSLHKoYyzcli6rmadz2vbToFAmq1NvAdHGFsZXhlaS5z
dGFyb3ZvaXRvdkBnbWFpbC5jb20ACgkQ6rmadz2vbTpmpA//fqEoC6Sq1zxo3ADH
hV0Z9ewkNTjjH85QnispjcRkAhSHG3JscNXKRXm1NmNkwJsHJ4TDIKcjABYDjquD
wqfvL9hLXPsvud0M/PR6/CZeBAWXpukkaxeYedY+83ttTHjzR0tDq0Ne9yvIV+Nc
hS0qFIIHs8C6l2nyuNSxvgrv216orG8qd0Bi3tpDfsLqCsLLEmDyQ1H+ZzpJF2xW
RV3oMcggCeo305m8+uiofQGf8hmmRrmA7SEfF+Qe08ab2GOn9glINfVTbTE3AdQW
fkvAk8Zuio3hwwMBHDWWYXrKO0N3ykzcDk4V6JPUWiH1dOANf1tS3G1uJyxb4tsv
dHVdA0xL3sg7YnuSywfb82vTXvQz5QEFeDYxxLB2fMJe8LcCfgF1lkIr1DbdrMjI
hlozGCs3p/GVIVNhjGVPezWsUvzK4PIuKVM6U1qWojtAhXU/bJCvP0iBU83RlBL7
S0GGSC/qvkuPed9hAp2pnrrRo/9GEp1PZq8AXsp3nti0OaRxgxpjjQQFzZEWlOrR
bErtbKziHzl2xARvCRycNZ+QWT4ZdjmK++8pba7hOuFkRclOV4KRWk4OthBvcADu
NfNsVXdXpOnesg7TNIUJ8heVAYPjKaBHJBuG2Prghrnc95uqEQwOfbYT15zAbe6I
l3HqGerSa7WtbrnCwsf1DvyPPII=
=ZxKg
-----END PGP SIGNATURE-----
Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf
Pull bpf fixes from Alexei Starovoitov:
- Fix bpf_skb_change_tail() to drop the checksum offload instead of
rejecting the trim of CHECKSUM_PARTIAL skbs (Daniel Borkmann)
- Add KF_PERFMON kfunc flag and require CAP_PERFMON for kfuncs that
read arbitrary memory and for untrusted read-only memory reads
(Daniel Borkmann)
- Clear scalar delta on narrowing stack spill (Daniel Borkmann)
- Set up the frame pointer for the exception callback in arm64 JIT, and
zero-fill other CPUs when BPF_F_CPU update creates a per-cpu hash
element (Donggeun Yoo)
- Various fixes (Emil Tsalapatis):
- Fix bounds check underflow for skb-backed dynptrs
- Fix rx_queue_mapping context access code generation in bpf_sock
- Reject packet pointer arguments to subprogs that may mutate the
packet
- Reject ALU instructions that see arena and non-arena operands on
different code paths
- Fix copied_seq double-counting on sockmap self-redirect
(Geliang Tang)
- Fix divide-by-zero in btf_struct_walk() on a flexible array of
zero-sized elements, fix out-of-bounds read of rtt_min in sock_ops
(Jiayuan Chen)
- Fix bpf_sock_destroy() out-of-bounds read of sk_protocol on TIME_WAIT
and request socks, and sleeping under RCU when destroying a listener
with pending children (Jiayuan Chen)
- Fix JEQ/JNE with immediate operand in MIPS32 JIT and missing zero
extension of BSWAP 16/32 in MIPS64 JIT (Johan Almbladh)
- Avoid soft lockup in htab lookup[_and_delete] batch operations on
large maps (Jose Fernandez)
- Various fixes (Kumar Kartikeya Dwivedi):
- Verify global subprogs in each sleepability context they are
called from
- Make post-verification instruction rewrites killable
- Preserve packet pointer displacement in regsafe()
- Apply CO-RE relocations before subprogram validation, restrict
CO-RE poisoning to relocatable instructions, and reject truncated
ldimm64 CO-RE relocations in libbpf
- Assign lock identity to callback map values
- Compare stack frames in regs_exact()
- Bound ownership depth through local kptrs and graph roots
- Fix u32 overflow in map batch operations when the map size exceeds
4GB (Masoud Aghasi)
- Fix UAF in bpf memalloc due to concurrent consumption of ttrace lists
in alloc_bulk() (Pu Lehui)
- Allow gotox as the terminal instruction of a program or a subprogram
(Siddharth Chintamaneni)
- Disallow bpf_skb_pull_data() for LWT_SEG6LOCAL, skip unsettled links
in link iterator, and reject dev-bound-only programs on other devices
(Weiming Shi)
- Reject non-negative stack offsets in stack_slot_obj_get_spi()
(Xu Yunxiang)
- Check params size before reading reserved fields in
bpf_crypto_ctx_create() (Yuqi Xu)
- Reject max_entries > INT_MAX in sock_map_alloc() (Zhao Gongyi)
- Use a 32-bit compare in xsk_map_gen_lookup() (Zhiling Zou)
- Use kvfree() in xdp_test_run_teardown() (Zhixing Chen)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (58 commits)
selftests/bpf: Test per-cpu initialization of a BPF_F_CPU created element
bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element
bpf: Fix BSWAP 32 and 16 on MIPS64
bpf: Fix immediate JMP JEQ/JNE on MIPS32
bpf: Reject dev-bound-only programs on other devices
bpf, sockmap: Reject max_entries > INT_MAX in sock_map_alloc
selftests/bpf: Test for mixed arena/nonarena code paths
bpf: Prevent variable arena/non-arena register contents
selftests/bpf: Test rejection of pkt args to mutating subprogs
bpf: Reject pkt arguments in mutating subprogs
selftests/bpf: Add selftests for rx_queue_mapping context access
bpf: Fix bpf_sock context code generation
selftests/bpf: Test dynptr slices past end of skb
bpf: Fix bounds check for skb-backed dynptrs
selftests/bpf: Reject iterator destruction through fp+0
bpf: Reject non-negative offsets in stack_slot_obj_get_spi()
bpf: Check params size before reading reserved fields
selftests/bpf: Check local object ownership depth
bpf: Bound ownership depth through local kptrs and graph roots
selftests/bpf: Cover frame changes in bounded loops
...
|
||
|
Donggeun Yoo
|
c3a66e5f5b
|
bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash element
pcpu_init_value() initializes the per-cpu area of a newly created
[lru_]percpu_hash element. The area is recycled, so when the value
comes from a BPF program (onallcpus == false) it writes the running
CPU's slot and zeroes the rest.
bpf_percpu_hash_update() passes onallcpus == true, which delegates to
pcpu_copy_value(). pcpu_copy_value() writes only the CPU named in
map_flags when BPF_F_CPU is set, so on the create path the other slots
keep the recycled element's values:
update(k1, 0xdeadc0de, BPF_F_ALL_CPUS) every CPU holds 0xdeadc0de
delete(k1) element back on the freelist
update(k2, 0xc0ffee, BPF_F_CPU | 0) creates, writes CPU 0 only
lookup(k2) CPU 0 0xc0ffee, rest 0xdeadc0de
Zero-fill the other CPUs on that arm too.
Fixes:
|
||
|
Weiming Shi
|
6db1ce73e9
|
bpf: Reject dev-bound-only programs on other devices
__bpf_offload_dev_match() falls back to comparing offdev pointers after an
exact netdev mismatch. Bound-only programs normally have NULL offdevs, so
unrelated netdevs compare equal. A bound-only program on an
offload-registered netdev can instead inherit a real offdev and match a
sibling port. With CAP_BPF and CAP_NET_ADMIN, a caller can use
bpf(BPF_LINK_CREATE) with a different target ifindex to run metadata kfuncs
specialized for the bound driver on the target driver's xdp_buff. Running a
veth-bound program on tun reads beyond tun's bare stack xdp_buff as a
veth_xdp_buff.
Oops: general protection fault, probably for non-canonical address
KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017]
RIP: 0010:veth_xdp_rx_timestamp (drivers/net/veth.c:1673)
Call Trace:
...
tun_build_skb (drivers/net/tun.c:1739)
tun_get_user (drivers/net/tun.c:1856)
tun_chr_write_iter (drivers/net/tun.c:2091)
vfs_write (fs/read_write.c:595 fs/read_write.c:687)
ksys_write (fs/read_write.c:739)
do_syscall_64 (arch/x86/entry/syscall_64.c:84)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
Kernel panic - not syncing: Fatal exception in interrupt
Restrict non-offloaded programs to exact netdev matches and retain the
shared-offdev fallback only for genuinely offloaded multi-port programs.
Fixes:
|
||
|
Emil Tsalapatis
|
f85f5917aa
|
bpf: Prevent variable arena/non-arena register contents
The verifier marks ALU instructions that include at least
one arena operand with needs_zext: These instructions are
fixed up after verification to be ALU32 instructions to
ensure that the result is a valid offset into an arena.
However, different code paths may provide two non-arena
64-bit arguments to the same instruction. The result of
the operation in that code path is wrong, since it is
now unexpectedly truncated to 32 bits and zero-extended.
Add logic to the verifier to ensure every instruction either
always has at least one PTR_TO_ARENA argument, or never does.
Since needs_zext already tracks the first scenario, add a
prevent_zext field in bpf_insn_aux to track the latter.
Reject instructions that use arena arguments and have prevent_zext
set, or do not have arena arguments and have needs_zext set.
Fixes:
|
||
|
Emil Tsalapatis
|
a6c1edfbe2
|
bpf: Reject pkt arguments in mutating subprogs
The verifier tracks changes in how PTR_TO_PACKET registers'
bounds are modified across subprog boundaries. PTR_TO_PACKET
registers are actually passed as PTR_TO_MEM, which is assumed
valid for the entire call. This is not the case with packet memory,
where a pskb_* call may invalidate its memory region.
Reject BPF code that passes PTR_TO_PACKET pointers to subprogs that
may mutate a packet. We cannot pass the pointer as a true PTR_TO_PACKET
because we would also need to somehow pass the PTR_TO_PACKET_META
or PTR_TO_PACKET_END to the subprog. Since we cannot avoid representing
the pointer in the subprog as PTR_TO_MEM, only permit it if the
subprog is guaranteed not to mutate the packet.
Fixes: 80f281664f5a ("bpf: Support pointers in global func args")
Reported-by: Nicholas Carlini <nicholas@carlini.com>
Suggested-by: Nicholas Carlini <nicholas@carlini.com>
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260922172028.6269-6-emil@etsalapatis.com
|
||
|
Xu Yunxiang
|
79a9172f3a |
bpf: Reject non-negative offsets in stack_slot_obj_get_spi()
bpf_get_spi() computes (-off - 1) / BPF_REG_SIZE using C division,
which truncates toward zero. For off == 0, this produces spi 0, the
same index used by the valid stack slot at fp-8.
stack_slot_obj_get_spi() currently checks alignment and the resulting
spi bounds, but does not reject the non-negative offset itself. It can
therefore validate a PTR_TO_STACK register holding fp+0 against an
iterator stored at fp-8 even though the runtime receives the actual fp+0
pointer. An effectful iterator kfunc can then interpret memory outside
the BPF stack as iterator state.
Reject non-negative offsets before converting the offset to an spi. All
valid stack objects begin at a negative offset from the frame pointer.
Fixes:
|
||
|
Linus Torvalds
|
0a15ba6b0c |
Timer race fixes:
- Fix timer signal <-> exec() race, to prevent UAF (Thomas Gleixner)
- Clean up POSIX CPU timers right after de_thread(), to prevent UAF
(Hyunwoo Kim)
- Fix POSIX CPU timers race between expiry and timer_settime(),
to prevent UAF (Thomas Gleixner)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqvrMYRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gqUQ/+NkruN984bFynF/eZ0/2DFv91AAUP8zgH
/S3PBlwuSbYFN9JVhngDMwxQkamE56weJbFc+0QuvVT5UVw/vX9BS4QOvvzN+f8D
FEN3UqD0d1B8OwlPNTw0sFPwJDdPctTinfKOhNjNQe6RLFsNARvGyaKDIDroWTfV
dxuJ/7Ecs+5m1bmGJnPEC+IH/OnV9BEEl1NdZb+INKpBlui9LCsw4rRIj/8dPK/H
UNhvXpykKrJCDftbCzAFSNryuzcJgq4kHtMbsqiUL6y50AB69eHGi/Y0xYBAEr1h
NiDPq2PAMmH1NCCMsTtqbJZMqgCr+7DSZiCFn7bZPwg0V5tV4PFZD484q0sCbiej
Fwg+arHd0icnceIcWMsBWPUVOSLxZaWdp9a2Tj3Ill06//b5bEDBJBbpecS+so3t
8W6IvdoCYm7sz50mohnjOdx7biHPu0yhwgj+EoAV3nZKoALQAAcI7+HJzSWpGnJi
HIO0zylRAZCjk9H3QNWO+LdWgifc8DysAZOWpmbuwGgp8q483IDRDtme/kMt3+D1
1qTHa1TD/tPo8UmmgyVJQ7e1hCxBkGuuBBu5Y3/qkUEOQM6B/H2Ji7stxsLpW3JL
HLzC3kL2SBBVBO2ljqiH5IhVAL10Qm5vPxaCOjExdBt1vjMxN1DowZqD4rT/tw58
ArpD4zmr8VQ=
=KhWJ
-----END PGP SIGNATURE-----
Merge tag 'timers-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer race fixes from Ingo Molnar:
- Fix timer signal <-> exec() race, to prevent UAF (Thomas Gleixner)
- Clean up POSIX CPU timers right after de_thread(), to prevent UAF
(Hyunwoo Kim)
- Fix POSIX CPU timers race between expiry and timer_settime(),
to prevent UAF (Thomas Gleixner)
* tag 'timers-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
posix-cpu-timers: Prevent freeing a timer which is queued on the expiry list
exec: Cleanup POSIX timers right after de_thread()
signal: Prevent exec() race
|
||
|
Linus Torvalds
|
fecbe78ac0 |
Scheduler fix:
- Avoid false positive migration warning for proxy donors
(Andrea Righi)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqvqssRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gPzBAAhE9hcpkBV6vNW9xzBDKdGPxRjch2vcfu
lQbx7da1yC2rHU7RDlcdfEgkhAqTRwEAXKz26zth3sDHLk43tusuNtqG3swELhNW
YAGbHjdn87snQlmP8AOK4EaT3uE5NjWqRSIJWMK+AhWSwqO/zzK91T6yZyQY0fM4
3Edp8CFo3IeyOzCR96vsob2x1fFQhuR//5fWul3uuB0EZ8VA5FhH3ene6VCm7P/1
dauxX0rMTb2730qNXA8cHROZq+bwhqTZOUaoOZ33WxnRPkvY9mV/hZsN8JnJuzBE
ogzyyorcl8dFH8qOapos9Cp3tQj9GkTX7mXWDbuUflt/8uOXtQMf75kFGBVI5NUw
2xNfgubTYGo6Qc1C+wyOhGYJ6T5Al+083pV/vPc4y7Z1i7RgZ94QypMuZuaR/RqS
z+RQcdNQlXiRIAIILGJqq1xdbaCJvbVx3tiFZkhPse6ioOF6UNGbQiNExAq3v5BU
ocvhBuf9p/uvRmfs+ZtQNqAAjZUL7tQPvdFAsjxKjuI2Z5YGPROy1L9NdYKfzryM
yWOEkV2mdn97CwzDS+auC0HmPkGqf8we2VI5Ub4R35UPqDcV6vc5BAnwzAbuxAmc
K7mQOMDEaRkbVQ0MoSMdidIXLL0AcN+BROBUtHv8kqJur6nADE3K3HZNdhr2ViLN
eisNI6m8pLk=
=BEte
-----END PGP SIGNATURE-----
Merge tag 'sched-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fix from Ingo Molnar:
- Avoid false positive migration warning for proxy donors
(Andrea Righi)
* tag 'sched-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/core: Avoid false migration warning for proxy donors
|
||
|
Linus Torvalds
|
abb91eed94 |
Perf events fixes:
- Fix crash when probing CS CALL instructions (Jinke Han) - Fix NULL pointer crash during module unload (Vinay Belgaumkar) Signed-off-by: Ingo Molnar <mingo@kernel.org> -----BEGIN PGP SIGNATURE----- iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqvqiURHG1pbmdvQGtl cm5lbC5vcmcACgkQEnMQ0APhK1hiohAAjcvOY7M998pX1tmo1Egw9kAuI0odtNOX weQ4Wq7K3X+tg+q1wVUmE/N/y/WLYWNatjwvjc8TClYdQEiaqBMH4TSLAuY3TOxa TxwdTC20uSN4EPZ0iRhKzm3biPzzzRq4M9hhV+WfGcK7ieXRn4Q7d9S3DDe5oEfG lI4l/RefIBiINVPC7dNM7xpS/7XBEPnzNeshOMwp6ZsPziZJizgC8C7RhQFAPszo Ho36KKFQqMNouCSybQl1GxLyPw+oGtneWESHXrF6Mhp+bcx40fMtJxKNyVLsVWuM wo0Ry843pCbDofOIqg7m0AufWUhz7B4MttTXXrU2/BYMHEbxlgms1AO7lnSGzPmn vP0JHZnT3y34P5uvGaVho7t9QKKbuY47cKNmsiiLuXiBQQuKnvDmDln1Mu1KS3fg a1LI8kv043iLqAjsIWMVtKlRGUX36f4NXUWrxvO/tmup3ocJhG1oYmEe/RaFddVX 5qRbHn7Z1w8jAWldODYSrXkpMRgMtClQuqHdjxZQt5DPZSORzoFxTvSfJLdUUtVx CUiT34zcLPHfvGD+ctHe5kVesgDHCxp/z1tKrJBunB+XZVl6rKHycfiGjaUXQRcR NUp6iFlfay8b79EkMsLC8p6uAmzX2q+gEwNVy8eevBSXdsYqcY5islj3ob7sBHsH vk4gDJikpE0= =onkS -----END PGP SIGNATURE----- Merge tag 'perf-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull perf events fixes from Ingo Molnar: - Fix crash when probing CS CALL instructions (Jinke Han) - Fix NULL pointer crash during module unload (Vinay Belgaumkar) * tag 'perf-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: perf: Fix null pointer access in is_include_guest_event() x86/kprobes: Fix crash when probing CS CALL instructions |
||
|
Linus Torvalds
|
bdab18633a |
- Also allocate a default private futex hash on vfork()
as well, to avoid races with (private) futex waiters
(Peter Zijlstra)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqvp20RHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1giDg//VpFKPXgma7fW5E+T0kjWHWD7FS1vYyJ4
nMunKME8PAV0A4S7946iUAWHjViVfksTmfvcXepSFCKZKLf55g8s2aZ1VoigMtYX
bQxNrnOidebSg9npLs9NV4sjvvNy+k03gcNN+OK+ZRfYCVRqbLgfCm5RJbaId5y4
+EuizVmtJuNav6HwAEIbU4LXGIdwSL9Pn8Zitkz/H7g0ZEkv+VA4h6NttvKDNfV7
OltcUFejtU7Z1lItfH+PP9pMcjPA6OPI2h7LLmUcZGUhhQQtW2Gb1fffz45PiH8T
FB7PdFt22TLG5c6hLB7zbrFFillWKn3l3Ihi/IxqVmMulhk6RVTPdCVJcj1cGZxF
9NXZ+L81poKwEETaIk52v5jm9qNF+kHbXJuCjPbmEdPxtxfv+Ma4zXom/xKkuXpx
qf01GXxelUPdCAl0cT0pzRvbEfHNIOsE2Id7f+59jB+L8ZbYEch04cIVRqCQcOgi
9B+lXj36fkFBV6wmuP5SWShWsgMsMpgSzOz5mUdWjY3Ocn9QuzDxPkk50Cm7uL1i
q/HpGv3T849HFnCH7+mi8wSiX33La+N297+AkGO7U7h5QFldPokvbAijwfmF+1Nw
CebsAfsgJsN1GZacDTH0jmRxh4I0Wa32yiCAldeGS2w9EMEAFbA87iwsOgjwHiQl
JOv/s06Pg7o=
=qQHe
-----END PGP SIGNATURE-----
Merge tag 'locking-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull futex fix from Ingo Molnar:
- Also allocate a default private futex hash on vfork() as well, to
avoid races with (private) futex waiters (Peter Zijlstra)
* tag 'locking-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
futex: Also allocate private hash on vfork()
|
||
|
Yuqi Xu
|
a11212910c
|
bpf: Check params size before reading reserved fields
bpf_crypto_ctx_create() is a kfunc whose second argument is declared
with the __sz annotation, so the verifier only guarantees that
params__sz bytes of params are valid. The function nevertheless reads
params->reserved[0] and params->reserved[1] (offsets 14 and 15) before
comparing params__sz against the size of struct bpf_crypto_params, so a
BPF program can pass a shorter buffer and have the kernel read past the
region that was validated for it.
Move the size check in front of the reserved field reads.
Fixes:
|
||
|
Thomas Gleixner
|
c21eaa72f0 |
posix-cpu-timers: Prevent freeing a timer which is queued on the expiry list
Kijo analyzed another race in the POSIX CPU timer code: Commit |
||
|
Kumar Kartikeya Dwivedi
|
bfc888f045
|
bpf: Bound ownership depth through local kptrs and graph roots
Program-allocated objects can own other local objects through referenced
kptrs. bpf_obj_free_fields() follows those pointers through
__bpf_obj_drop_impl() synchronously, before the object storage is freed
through RCU. A self-referential local kptr type therefore permits arbitrarily
deep object chains, and dropping the head can exhaust the kernel stack.
Long acyclic type chains have the same problem.
btf_check_and_fixup_fields() still assumes referenced kptrs only point to
kernel types and checks ownership through list and rbtree roots only. Its
existing rule is sufficient for graph-only cycles: the target of each graph
edge must contain a node, so every type in a cycle has both a root and a
node. The rule rejects such a type owning another root, breaking every
cycle. It also limits graph-only chains to three types, or two if the first
type contains a node, and conservatively rejects longer acyclic chains.
The missing local-kptr edges, rather than a missed graph-only cycle, are the
bug introduced by support for bpf_kptr_xchg() into local kptrs.
Replace that restriction with one bounded ownership walk covering graph
roots and local referenced kptrs. Run it after all BTF records have been
fixed up, reject cycles and paths deeper than eight record-bearing types,
and cache each type's suffix depth while checking it against the remaining
budget. This also permits the longer acyclic graph-only layouts rejected
by the old rule; update their existing BTF tests accordingly.
Keep the bound independent of MAX_CALL_FRAMES because recursive destruction
can run below a BPF call chain. A plain local pointee without special-field
metadata adds only a final non-recursing drop. Non-owning kptrs and
kernel-BTF kptrs do not recurse through local records and remain outside the
walk. Include local percpu-kptr edges too, although allocation of percpu
objects with special fields is currently forbidden, so that relaxing that
restriction cannot bypass the ownership bound.
btf_check_and_fixup_fields() continues to initialize graph_root.value_rec,
including for separately allocated map records. The ownership relationships
belong to immutable program BTF and only need validation at BTF load time.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
8901cee931
|
bpf: Compare stack frames in regs_exact()
regs_exact() compares the register state up to id, followed by the ID
mappings, but does not compare frameno. The PTR_TO_STACK case in regsafe()
checks frameno separately, which is bypassed when exact comparison is
requested. Consequently, infinite-loop detection can treat pointers to
different stack frames as the same pointer and reject a finite loop.
For example, initialize fp-8 to zero in the caller and to one in the
callee, then pass the caller's fp-8 to the callee as r1:
loop:
r0 = *(u64 *)(r1 + 0);
if r0 != 0 goto done;
r1 = r10;
r1 += -8;
goto loop;
done:
exit;
The loop terminates after reading the callee's slot on its second
iteration. At the loop header, however, the only relevant difference is
r1's frameno, so exact comparison incorrectly reports an infinite loop.
The same problem occurs when the pointer is spilled to the stack.
Move frameno into the type-specific metadata union, ahead of id, so the
existing prefix comparison in regs_exact() covers it. Ordinary stack
pointers do not use another union member. Iterator and IRQ stack-slot
states use their dedicated union views and do not need a frame lookup.
This also keeps bpf_reg_state at 80 bytes.
Since frameno now shares storage with other pointer metadata, it is only
meaningful for PTR_TO_STACK registers. Return NULL from bpf_func() for
other register types. process_iter_arg(), get_constant_map_key() and
is_dynptr_reg_valid_init() look up the frame before checking the register
type and would otherwise index frame[] with a byte of the register's map
or BTF pointer. They dereference the frame only after their type check.
Move the states_maybe_looping() boundary from frameno to precise after the
field relocation. Its prefix comparison continues to cover the complete
value state and now includes frameno.
Continue to ignore precise. Precision marks control whether pruning may
ignore scalar ranges; they do not change the represented values, and exact
comparison already compares those ranges unconditionally. Marks can also
change through backtracking while an ancestor state is still being
explored.
Fixes:
|
||
|
Andrea Righi
|
fe3c73d7bc |
sched/core: Avoid false migration warning for proxy donors
Proxy execution can move a blocked donor's scheduling context to the
lock owner's CPU even when the donor is migration-disabled. The donor
does not execute there, and its original execution CPU remains recorded
in wake_cpu.
set_task_cpu() warns unconditionally for migration-disabled tasks, so a
subsequent proxy migration or the wakeup path returning the donor home
triggers a false positive: moving a blocked scheduling context does not
violate the migration-disabled execution context.
For example, creating a mutex owner on CPU1 and a migration-disabled
waiter on CPU0 can trigger the following warning:
proxy_migrate_repro: donor blocking on CPU0 with migration disabled
proxy_migrate_repro: donor moved from CPU0 to CPU1
WARNING: kernel/sched/core.c:3389 at set_task_cpu+0x1d3/0x280
...
Call Trace:
try_to_wake_up+0x43f/0x780
__mutex_unlock_slowpath+0x330/0x540
owner_fn+0x9f/0xc0 [proxy_migrate_repro]
...
proxy_migrate_repro: donor woke on CPU0, task_cpu=0
proxy_migrate_repro: completed
Exclude blocked proxy donors from the warning. The proxy wakeup path
restores an executable placement before clearing the blocked state.
Fixes:
|
||
|
Vinay Belgaumkar
|
88aed0422f |
perf: Fix null pointer access in is_include_guest_event()
A typical module unload occurring event when there is an active perf
connection leads to freeing of the pmu pointer. The call log is something
like:
..
__pmu_detach_event
pmu_detach_event
pmu_detach_events
perf_pmu_unregister
..
__pmu_detach_event() sets event->pmu to null. When the perf connection
finally is closed, the following stack trace is observed:
Oops: general protection fault, kernel NULL pointer dereference
...
RIP: 0010:_free_event+0x3e/0x370
...
Call Trace:
...
perf_event_release_kernel+0x260/0x2d0
perf_release+0x12/0x20
A call to mediated_pmu_unaccount_event() inside _free_event() is the root
cause of this crash. Adding a check inside is_include_guest_event() ensures
we don't accidentally access a null pmu ptr. In addition to this, we will
now call mediated_pmu_unaccount_event() before clearing the pmu ptr so that
nr_include_guest_events counts are maintained correctly.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
71919742c8 |
bpf: Assign lock identity to callback map values
A nested bpf_for_each_map_elem() callback can unlock a different element
of the same map:
static long inner(void *map, int *key, struct value *v,
struct value **outer_value)
{
bpf_spin_lock(&v->lock);
bpf_spin_unlock(&(*outer_value)->lock);
return 0;
}
static long outer(void *map, int *key, struct value *v, void *ctx)
{
bpf_for_each_map_elem(map, inner, &v, 0);
return 0;
}
Both callback values currently have ID zero and the same map_ptr.
process_spin_lock() compares those two fields, so it accepts the unlock
even though the two callbacks can receive different map elements.
Assign a fresh ID to every callback map value in the for-each,
timer/workqueue, and task-work constructors. Copies of one callback
argument retain its ID, so locking and unlocking through that argument
continues to work. Distinct callbacks also get distinct IDs for
single-element arrays, including inner arrays sharing inner_map_meta.
Preserve map_uid for every inner-map lookup and compare it through
check_ids() during state pruning. This preserves relationships between
maps, keys, and values while allowing equivalent states with different
lookup IDs to match. It avoids field-specific rules for when an inner map
needs an identity.
Move map_uid out of the metadata union and next to the other IDs, so
register comparisons can use the existing memcmp() ranges and remap the
IDs separately. Clear it when resetting a register or converting a map
lookup result to a socket pointer. Shrink frameno to u8, which is enough
for MAX_CALL_FRAMES, to make room without growing bpf_reg_state.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
c26e97721b |
bpf: Apply CO-RE relocations before subprogram validation
check_subprogs() verifies that each subprogram ends in an exit or an
unconditional jump before in-kernel CO-RE relocations are applied. An
unresolved relocation can then replace that terminal instruction with an
invalid helper call. The resulting fall-through into another subprogram
breaks the CFG invariant used by postorder and stack liveness analysis,
which can write past their per-subprogram arrays.
Apply CO-RE relocations immediately after preparing the program BTF, before
subprogram discovery and validation. Keep func_info and line_info validation
after subprogram discovery because those records depend on the complete
subprogram layout.
Reject an ldimm64 first slot at the end of the instruction stream before
CO-RE can inspect its missing second slot. check_subprogs() previously
rejected this form before relocation processing because it is not a valid
subprogram terminator. Moving CO-RE ahead of check_subprogs() removes that
implicit protection, so perform an explicit check before applying
relocations.
Include core_relo_cnt when deciding whether to prepare program BTF. A load
that supplied only CO-RE relocation metadata previously skipped both BTF
setup and relocation processing.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
fd16449a9b |
bpf: Preserve packet pointer class displacement in regsafe()
regsafe() maps packet pointer IDs between states and checks that each current register range is a subset of the corresponding explored register range. It does not, however, preserve the displacement between registers that share a packet pointer ID. This is unsound because packet range is shared by ID. A bounds check on one class member updates every member, and a later access can consume the range through another member. Commit |
||
|
Kumar Kartikeya Dwivedi
|
261b61d373 |
bpf: Make post-verification instruction rewrites killable
After do_check() returns, the verifier runs several instruction rewrite
passes. Some of them patch or remove one instruction at a time. Each
operation moves the remaining instruction and auxiliary-data arrays and
adjusts all branch offsets, making the overall work quadratic in the
program length.
A privileged loader can submit 131072 unconditional jumps by zero followed
by a valid return. Verification finishes quickly, but bpf_opt_remove_nops()
then spends a long time removing each jump separately. Since this
post-verification work neither checks for signals nor reschedules, a pending
SIGKILL cannot terminate the task until the rewrite finishes.
Make bpf_patch_insn_data() and verifier_remove_insns() common cancellation
and rescheduling points. These helpers run from BPF_PROG_LOAD process
context, and bpf_patch_insn_data() can already sleep while reallocating
auxiliary data.
Report interrupted constant blinding as -EINTR and propagate it through
both JIT paths, including kernels that permit interpreter fallback.
Other blinding failures retain the existing fallback behavior.
This does not reduce the quadratic cost of the rewrite passes, but it makes
the work preemptible and allows a killed loader to be torn down promptly.
Fixes:
|
||
|
Weiming Shi
|
50e80e2bb5 |
bpf: Skip unsettled links in link iterator
bpf_link_prime() inserts a link into link_idr before anon_inode_getfile()
succeeds and before bpf_link_settle() publishes the ID in link->id.
bpf_link_by_id() treats such an ID-zero link as unsettled, but the link
iterator takes a reference without this check.
If anon_inode_getfile() then fails, the creator removes the ID and frees
its still-private link directly. The iterator is left with a dangling
reference and its next bpf_link_put() accesses freed memory.
Treat ID-zero entries as transient in bpf_link_get_curr_or_next(), just as
bpf_link_by_id() does.
BUG: KASAN: slab-use-after-free in bpf_link_put
Write of size 8 by task exp/384
Call Trace:
bpf_link_put kernel/bpf/syscall.c:3372
bpf_link_seq_next kernel/bpf/link_iter.c:33
bpf_seq_read kernel/bpf/bpf_iter.c:158
vfs_read fs/read_write.c:572
ksys_read fs/read_write.c:716
do_syscall_64 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe arch/x86/entry/entry_64.S:121
Kernel panic - not syncing: KASAN: panic_on_warn set ...
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
40c2096961 |
bpf: Verify global subprogs in each sleepability context
Global subprograms are verified independently with a fresh verifier root. do_check_common() currently seeds that root's in_sleepable state from the program, even though a global subprogram can also run from callbacks whose execution context differs from the program's main entry point. In particular, workqueue and task-work callbacks are sleepable even when the containing program is not. A global subprogram of that program is therefore verified as non-sleepable, making in_rcu_cs() true and allowing loads of RCU-protected kptrs to produce trusted MEM_RCU pointers. The same subprogram can then be called from a sleepable callback without a classic RCU reader. It can retain such a pointer while the object is freed and use it after free. The verifier's execution-context predicates are complementary. A state is sleepable only when in_sleepable is set and no RCU, preemption, IRQ, or lock region is active. Each condition which prevents sleeping also provides RCU protection, while in_rcu_cs() treats a non-sleepable state as implicitly protected. Use this relationship to represent a global subprogram caller with only the result of in_sleepable_context(). A protected sleepable caller is normalized to in_sleepable=false at the independent verification root. This both prevents sleepable operations and makes in_rcu_cs() true without copying caller-owned lock state. Track only the contexts in which each global subprogram is actually reached. Verify it once if all reachable calls use the same context, and twice only if both sleepable and non-sleepable calls reach it. Calls found while verifying globals or asynchronous callbacks mark further contexts for checking. Repeat the existing subprogram walk until all called contexts have been verified; unreachable global calls remain unchecked. Accumulate instruction counts over those verification passes. Preserve the total recorded before each pass, since path accounting has already added this pass's synchronous instructions and its root total must also include asynchronous subprograms. This makes an unprotected callback verify the global subprogram as sleepable, turning its RCU-protected kptr load into an untrusted pointer. Protected callers and global subprograms which do not depend on implicit RCU protection remain valid. Fixes: |
||
|
Linus Torvalds
|
4aec9ad1c6 |
dma-mapping fixes for Linux 7.3
A few fixes for the DMA-mapping code:
- resolved regression in accessing encrypted memory by IOMMU-backed
devices (Aneesh Kumar K.V),
- improved failure handling and removed rare bug in swiotlb/highmem
(Donggeun Yoo).
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQSrngzkoBtlA8uaaJ+Jp1EFxbsSRAUCaquwMwAKCRCJp1EFxbsS
RMT6AP0elpdaZXNY0KwUBTwU95H604J+donqriepHABIBhIDEQD9GWZqNf/m1gEI
tR5lHQ3+NGs0Q7Vd2ed1vSe82HQSsgU=
=QxUC
-----END PGP SIGNATURE-----
Merge tag 'dma-mapping-7.3-2026-09-17' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping fixes from Marek Szyprowski:
"A few fixes for the DMA-mapping code:
- resolved regression in accessing encrypted memory by IOMMU-backed
devices (Aneesh Kumar K.V)
- improved failure handling and removed rare bug in swiotlb/highmem
(Donggeun Yoo)"
* tag 'dma-mapping-7.3-2026-09-17' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
x86/mm: Don't force unencrypted DMA for IOMMU-backed devices
dma-mapping: don't trace the DMA address when the allocation fails
swiotlb: use the adjusted address for the highmem page lookup
dma-coherent: report a failed reserved memory assignment
|
||
|
Thomas Gleixner
|
d2710c8d93 |
signal: Prevent exec() race
Hyunwoo debugged the following KASAN UAF splat:
BUG: KASAN: slab-use-after-free in __send_signal_locked+0xb27/0xba0
Write of size 8 at addr ffff888007ed80c8 by task poc/79
...
Call Trace:
__send_signal_locked+0xb27/0xba0
do_send_sig_info+0xa7/0x160
do_send_specific+0x76/0xa0
__x64_sys_tgkill+0x193/0x270
...
Allocated by task 80:
do_timer_create+0x1a4/0x1030
__x64_sys_timer_create+0x145/0x190
...
Freed by task 12:
kmem_cache_free_bulk+0x1f8/0x4a0
kvfree_rcu_bulk+0x14f/0x1c0
kfree_rcu_work+0x128/0x1a0
...
Last potentially related work creation:
kvfree_call_rcu+0x39/0x390
__flush_itimer_signals+0x211/0x320
flush_itimer_signals+0x47/0x90
begin_new_exec+0xa6b/0x28c0
It turned out that this happens with a non-leader exec() as Hyunwoo
explained:
de_thread() calls exchange_tids() before release_task(leader), so the
struct pid held by a SIGEV_THREAD_ID timer created against the leader's tid
now points to the thread which called execve(). pid_task() returns that
thread and lock_task_sighand() on it succeeds.
If the timer signal is blocked, its sigqueue stays queued on the leader's
task::pending. The next expiry of that timer can then run while
release_task() flushes the queue.
posixtimer_send_sigqueue() checks whether the sigqueue is already queued
with a plain list_empty(), which only reads list_head::next.
list_del_init() is not atomic and INIT_LIST_HEAD() stores list_head::next
before list_head::prev, so the check can pass in between. list_add_tail()
queues the entry on the task::pending of the live thread, and the
list_head::prev store from the flush then overwrites the list_head::prev
link that list_add_tail() has just set.
__flush_itimer_signals() does not undo that either. With list_head::prev
pointing at the entry itself, its list_del_init() only stores the same
values again, so the entry is not removed from the list. It is still there
after the last reference is dropped and the timer is freed by RCU, and the
list_add_tail() of a later tgkill() follows that list_head::prev into the
freed timer.
This problem surfaced with the recent commit which moved the sigqueue flush
out of the sighand lock held region.
Hyonwoo proposed to fix this by using list_del_init_careful(), but that
just papers over the problem. After some disucssions and various attempts
to solve it, Eric pointed out that there is no reason to flush
task::pending late in release_task() and it should be done in
exit_signals() already.
As nothing can collect and deliver signals which are queued in a dying
task's pending queue, there is no reason to delay it further.
But it has to be ensured that no signals can be queued into it after that
point. exit_signals() sets PF_EXITING in task::flags, which can be used as
an indicator for this.
Cure it by:
- Preventing signal queueing for task private signals (PIDTYPE_PID) when
the task has PF_EXITING set in __send_signal_locked() and in
posixtimer_send_sigqueue().
- Protecting the unlocked setting of PF_EXITING in exit_signals() for the
task group empty and the group exit case with sighand lock
- Flushing task::pending signals right there.
Optimize that by moving the whole pending list to an on-stack list head
under sighand lock and free the signals without the lock held.
There has been quite some discussion about the lockless flush and the
non-leader exec case on weakly ordered systems. The problem is that a third
party which tries to send a posix timer signal relies on the PID lookup to
find the target task and that lookup might result in the new leader when
the signal was originaly directed to the old leader. In case that the
signal was queued on the old leader then the lockless flush raised a
concern over the following situation:
old_leader new_leader third party
A: flush_list() // list_del_init() stores to sigqueue
LOCK (tasklist)
old_leader->exit_state = EXIT_ZOMBIE;
B: UNLOCK (tasklist)
C: LOCK (tasklist)
if (old_leader->exit_state)
transfer_tids()
D: store PID
posix_timer_send_sigqueue()
// Observes #D so t = new_leader
E: t = get_target()
F: LOCK (sighand)
G: if (list_empty(sigqueue))
list_add(sigqueue)
The concern was that the third party might observe #D but not observe #A
and therefore would proceed to #G while the list_del() stores (#A) in
flush_list() are not visible yet, which could result in list corruption.
That would be possible if looking at it solely from a RELEASE+ACQUIRE
ordering point of view, but B-C is a UNLOCK+LOCK hand-over, which is not
the same as RELEASE+ACQUIRE:
RELEASE+ACQUIRE: RCpc, only the CPUs involved agree on the ordering
UNLOCK+LOCK: RCtso, the hand-over is store-ordering
As B-C is UNLOCK+LOCK, which is RCtso and that does impose store order,
A stores must happen before the D store.
Combine with E-F, which has a data dependency from the LOAD to the LOCK and
thereby constraints later LOADs, those sigqueue loads in G that come after
F must in fact observe the A stores.
Fixes:
|
||
|
Peter Zijlstra
|
b61b6f95d6 |
futex: Also allocate private hash on vfork()
As Jann demonstrated, it is entirely feasible to access the mm through vfork().
Therefore we need to allocate a private hash on vfork() as well as any other
CLONE_VM user.
Specifically, it must be avoided to have (private) futex waiters before
allocating the private hash.
Fixes:
|
||
|
Linus Torvalds
|
9b87fdc9af |
sched_ext: Fixes for v7.3-rc3
- An error raised by a BPF program before the scheduler finished enabling was consumed by the disable path's pre-enable shortcut, leaving a running scheduler that couldn't be disabled and was later freed while in use. - Two compat kfuncs dereferenced a NULL scheduler when handed an exited or idle task, oopsing the kernel. - Keep-running decisions in the dispatch path used the root scheduler's flags for tasks belonging to a sub-scheduler, causing warnings and stalls. - Schedulers with their own CPU ID mapping had no way to learn which IDs are online. Add a kernel-maintained online mask to plug the hole. - Cgroup idle state: the initial cpu.idle state wasn't passed on cgroup init and same-value rewrites delivered spurious callbacks. - Example scheduler fixes for a reenqueue loop on attach, placements on CPUs without effective grants, stalled partition work and stale idle tracking. -----BEGIN PGP SIGNATURE----- iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCaql+vQ4cdGpAa2VybmVs Lm9yZwAKCRCxYfJx3gVYGUTxAP9mhhePmghF/aq6SzeaVGeUXtclXRAGE1EAqea6 kzh50AEAiWXtFQu9dAF3EaXTWJkEJZkJUs0AO0wEXgghC0py9gY= =gcDW -----END PGP SIGNATURE----- Merge tag 'sched_ext-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext Pull sched_ext fixes from Tejun Heo: - An error raised by a BPF program before the scheduler finished enabling was consumed by the disable path's pre-enable shortcut, leaving a running scheduler that couldn't be disabled and was later freed while in use. - Two compat kfuncs dereferenced a NULL scheduler when handed an exited or idle task, oopsing the kernel. - Keep-running decisions in the dispatch path used the root scheduler's flags for tasks belonging to a sub-scheduler, causing warnings and stalls. - Schedulers with their own CPU ID mapping had no way to learn which IDs are online. Add a kernel-maintained online mask to plug the hole. - Cgroup idle state: the initial cpu.idle state wasn't passed on cgroup init and same-value rewrites delivered spurious callbacks. - Example scheduler fixes for a reenqueue loop on attach, placements on CPUs without effective grants, stalled partition work and stale idle tracking. * tag 'sched_ext-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: sched_ext: Maintain an online cid mask in the scheduler arena sched_ext: scx_qmap: Restore unused idle claims from ops.dispatch() sched_ext: Close the pre-enable ops error claim window sched_ext: scx_qmap: Fix pending partition work handoff sched_ext: scx_qmap: Place only on cids whose caps are in effect sched_ext: scx_qmap: Do not add IMMED to rescue inserts sched_ext: Use @prev's scheduler for the keep decisions in dispatch_one() sched_ext: Rename sch to root_sch in dispatch_one() sched_ext: Fix NULL sched deref in kfunc sub-sched error paths sched_ext: Don't deliver duplicate ops.cgroup_set_idle() for same value sched_ext: Pass the initial cpu.idle state in scx_cgroup_init_args |
||
|
Linus Torvalds
|
6fb20c0271 |
cgroup: Fixes for v7.3-rc3
- The task iterator could pick up a dying task whose refcount had already dropped to zero and resurrect it, leading to a use-after-free when reading cgroup.procs. Skip such tasks. -----BEGIN PGP SIGNATURE----- iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCaql+xg4cdGpAa2VybmVs Lm9yZwAKCRCxYfJx3gVYGVcaAP424sQDRkkInl7APgUeQfgUheNG8WgCUt+qMgT2 6VbVEwD9HodEyKW4r/JBWcqN3m4bbNSx57gpXgQwH+ESoorqLgs= =nZ8D -----END PGP SIGNATURE----- Merge tag 'cgroup-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup Pull cgroup fix from Tejun Heo: - The task iterator could pick up a dying task whose refcount had already dropped to zero and resurrect it, leading to a use-after-free when reading cgroup.procs. Skip such tasks. * tag 'cgroup-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: cgroup: Avoid iteration of dying tasks with zero refcount |
||
|
Tejun Heo
|
a9e3760b08 |
sched_ext: Maintain an online cid mask in the scheduler arena
Schedulers on the default cid mapping treat [0, nr_online_cids) as the online set and restart on hotplug. Schedulers that install their own mapping with scx_bpf_cid_override() have no way to learn which cids are online: the count no longer identifies members and the CPU-form cpumask is unusable from cid programs. This is an obvious hole in the cid API. Add scx_bpf_online_cmask(), a kernel-maintained cmask in the scheduler's arena, allocated alongside the per-CPU scratch masks and populated after the cid mapping is finalized and before ops.init(), for child schedulers too. The pointer stays valid through ops.exit() with no reference to take. It is the arena offset as a void pointer, the same form struct_ops arena arguments arrive in. The verifier types the void return as a scalar for the program's arena cast. The mask follows the SCX hotplug notifications: seeded from cpu_active_mask and updated before ops.cid_online/offline() runs, so it lags cpu_online_mask only inside a hotplug transition. Updates walk the scheduler list under the lock that also serializes unlinking. Reads are live, not atomic snapshots. Root initialization excludes hotplug. v2: Reworded the getter kerneldoc (Andrea Righi). Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> |
||
|
Tejun Heo
|
9a0b159ff1 |
sched_ext: scx_qmap: Restore unused idle claims from ops.dispatch()
scx_qmap tracks idle cids itself. pick_direct_dispatch_cid() claims a cid by clearing its bit and the task is inserted into that cid's local DSQ, which kicks the CPU. When the task does not arrive, for example because the insert fell back to the global DSQ after an affinity change, the CPU wakes, finds nothing and picks idle again. That is not an idle transition, so ops.update_idle() is not called and the cid stays marked busy until an unrelated task runs on it. Restore the claim from ops.dispatch(). The kick guarantees a dispatch on the kicked CPU, and when it finds nothing to run with a NULL @prev, the CPU is going back to idle. Document the pattern in ops.update_idle(), which reports only actual transitions. Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> Cc: Andrea Righi <arighi@nvidia.com> |
||
|
Joel Granados
|
afdf35cfae |
sysctl: Fix type truncation in sysctl_msec_to_jiffies
Return MAX_JIFFY_OFFSET for all the values truncated when val (u64) is
passed to msecs_to_jiffies (u32). This aligns with how very large
millisecond values get translated into MAX_JIFFY_OFFSET.
Fixes:
|
||
|
Kuniyuki Iwashima
|
318012c565 |
sysctl: Check range in do_proc_ulong_conv_ms_jiffies
Add the range check back to do_proc_ulong_conv_ms_jiffies that commit |
||
|
Kuniyuki Iwashima
|
82431877d8 |
sysctl: Check range in proc_dointvec_ms_jiffies_minmax
Add the range check to do_proc_int_conv_ms_jiffies_minmax that commit |
||
|
Michal Koutný
|
057dac23d3 |
cgroup: Avoid iteration of dying tasks with zero refcount
The commit |
||
|
Masoud Aghasi
|
953824e508 |
bpf: Fix u32 overflow issue in map batch operations
Several map batch operation implementations such as generic_map_lookup_batch() use calculations in the form of "values + cp * map->value_size" to compute the desired userspace memory address for reading or writing. This can overflow the u32 type (the result of "cp * map->value_size") when the map size exceeds 4GB. generic_map_lookup_batch() may corrupt values for some keys in userspace memory, and in some cases it mismatches values for some keys while still reporting success. Other batch operations may fail to delete or update some keys, or the syscall may return unexpected errors. Add size_t casts to prevent the affected offset and size calculations from overflowing. Fixes: |
||
|
Pu Lehui
|
1c21452d02 |
bpf: Fix UAF due to concurrent consumption of ttrace lists in alloc_bulk
Syzkaller repeatedly triggered UAF splats related to nodes in
waiting_for_gp_ttrace within the bpf memalloc:
BUG: KASAN: slab-use-after-free in llist_del_first+0x85/0x110 lib/llist.c:61
Read of size 8 at addr ffff8881572cd080 by task syz.4.470/5112
...
llist_del_first+0x85/0x110 lib/llist.c:61
alloc_bulk+0x193/0x460 kernel/bpf/memalloc.c:229
bpf_mem_refill+0x386/0x560 kernel/bpf/memalloc.c:436
Freed by task 14:
...
__free_rcu kernel/bpf/memalloc.c:281 [inline]
__free_rcu_tasks_trace+0x48/0xd0 kernel/bpf/memalloc.c:291
rcu_tasks_invoke_cbs+0x1ec/0x3e0 kernel/rcu/tasks.h:571
rcu_tasks_one_gp+0x13d/0x220 kernel/rcu/tasks.h:621
rcu_tasks_kthread+0xf3/0x120 kernel/rcu/tasks.h:651
The reason is that the UAF occurs after the RCU Tasks Trace GP expires:
when the __free_rcu() callback runs, there is no synchronization
protecting llist_del_all() against concurrent alloc_bulk() operating on
waiting_for_gp_ttrace, leading to the race condition below:
CPU0 CPU1
__free_rcu (RCU Tasks Trace callback)
alloc_bulk
llist_del_first(&c->waiting_for_gp_ttrace)
entry = smp_load_acquire(&head->first);
do {
if (entry == NULL)
return NULL;
free_all(llist_del_all(&c->waiting_for_gp_ttrace))
llist_for_each_safe(pos, t, llnode)
free_one(pos);
next = READ_ONCE(entry->next); <-- trigger UAF
} while (!try_cmpxchg(&head->first, &entry, next));
In addition, there is also a theoretical race condition on the
free_by_rcu_ttrace list. This race requires two preconditions: an
in-flight Tasks Trace GP keeping c->call_rcu_ttrace_in_progress == 1,
and concurrent cross-CPU frees repopulating c->free_by_rcu_ttrace with
new nodes. Under these conditions, the following scenario triggers UAF:
// CPU0
// irq work is still busy (on PREEMPT_RT)
alloc_bulk()
llist_del_first(&c->free_by_rcu_ttrace)
entry = smp_load_acquire(&head->first);
do {
if (entry == NULL)
return NULL;
// CPU1
bpf_mem_alloc_destroy()
WRITE_ONCE(c->draining, true)
// wait for CPU0
irq_work_sync()
// CPU2
do_call_rcu_ttrace(tgt(CPU0))
if (c->draining) {
llist_del_all(&c->free_by_rcu_ttrace)
free_all()
}
// CPU0 continue
next = READ_ONCE(entry->next); <-- trigger UAF
while (!try_cmpxchg(&head->first, &entry, next));
Fix this by introducing a raw spinlock to synchronize the concurrent
consumption on waiting_for_gp_ttrace and free_by_rcu_ttrace.
Fixes:
|
||
|
Linus Torvalds
|
22098763a1 |
tracing fixes for 7.3:
- Don't destroy user event fields when removal fails
User event fields are destroyed before the event is removed from
visibility. But that can fail leaving the still visible event with no
fields. Move the destroying of the fields to after the event is
successfully removed from visibility.
- Initialize function graph state is fork before calling copy_exec_state()
For non-CLONE_VM forks, copy_exec_state() allocates a new task_exec_state.
If that allocation fails, ftrace_graph_exit_task() will free the tasks
ret_stack pointer. Since that pointer is still using the parent's
ret_stack, it mistakenly frees the parent's pointer too.
Call ftrace_graph_init() on the task first which will NULL out the new
tasks's ret_stack and if the copy fails, it will not free anything.
- Remove FGRAPH_MAX_INDEX
The macro FGRAPH_MAX_INDEX was added but never used. Remove it.
- Save ent_size in function graph printing of nested functions
The function graph tracer needs to look at the next event to see if the
next event is the return of the current function entry. If it is, it
prints a single line:
ktime_get();
Otherwise it prints it like a nested function:
tick_nohz_irq_exit() {
ktime_get();
kcpustat_irq_exit();
}
In order to look at the next event, it must save the current event so that
it has the information to print from it. It saves the event in the
iterator descriptor called "ent". What it doesn't save is the ent_size of
the event which is now used to know if the function graph arguments are to
be printed. The peek doesn't save the size so the size used happens to be
that of the size of the last event that was seen.
Save the entry event size in the iterator descriptor so that the correct
size is used.
- Fix several errors with freeing data in the histogram code
The histogram code had a lot of leaked or or incorrect accounting when
failures happen. Correct them.
- Fix histogram regression of .percent and .graph modifiers
Up until 6.3 histogram values could have "percent" or "graph" modifiers
that changed how they were printed. But a change that added restricting
histograms values from being strings, stack traces and other modifiers
inadvertently prevented them from using the percent and graph modifiers,
which were legal use cases for values.
Put back the percent and graph modifiers.
- Fix various typos in the comments
- Set the trace_clock before initializing a histogram with clock argument
The histogram API allows the user to specific which trace clock to use via
a "clock=" string. The histogram is set up first before the clock is
checked. If the passed in clock is not valid, it exits without fully
fixing up the histogram leaving it on the list and a use-after-free can
trigger.
Update the clock argument first and if it fails then exit gracefully
before the histogram trigger is placed on any lists.
- Restore :mod: trailer after parsing in ftrace_set_clr_event
The function ftrace_set_clr_event() modifies the parse string and needs to
put it back to what was passed in. It searches for ":mod:" via a strsep()
but fails to put back the first ':' in the string.
Add back the ':' in the passed in string.
- Take trace_array reference when opening a tracer options file
The options files are dynamically created and some tracers add their own
options. When a tracer adds their own list of options, the trace_array
holding them has an array to hold the list of options for each tracer.
This array increases in size via a krealloc(), and the new entry gets a
newly allocated array to hold the options of the new tracer being added.
The element in each entry of the tracer's option array holds a pointer
back to the trace_array, a pointer to the tracer it is associated to, a
pointer to the flags of the option.
The issue is that these arrays are freed when the trace_array is freed
when its instance it represents is removed from the instances directory.
There's a race that an open of one of these options files can happen when
the instance is being removed.
Add a new helper function to be called by the open function of the options
file to iterate all existing trace_arrays under a lock and find the one
that has the given option element in one of it's tracer arrays. If found,
then update the associated trace_array's reference counter to keep it from
being freed. If not found, have the open call return -ENODEV.
- Disable interrupts when acquiring the lock in rb_wake_up_waiters()
The function rb_wake_up_waiters() assumes it will be called in interrupt
context and does not disable irqs when taking cpu_buffer->reader_lock,
which can be called in hard interrupt context. The issue is in PREEMPT_RT,
this function is called in thread context leaving this lock open to a
deadlock.
Take the lock with interrupts disabled.
- Use rcu_assign_pointer() for tmp_ops filter hash
The tmp_ops used in update_ftrace_direct_mod() assigns its filter_hash
field directly, but that field is annotated as __rcu and sparse complains.
Assign it with rcu_assign_pointer()
- Fix use-after-free in enable_trigger_private_data_free()
The trace_event_call is accessed through the event_trigger_data's
trace_event_file pointer to put the trace_event_call on freeing. The issue
is that the trace_event_file data may have been freed already causing a
use-after-free. Add a field to the event_trigger_data that points directly
to the trace_event_call so that it can decrement its reference directly
without needing to go through the trace_event_file.
- Fix accounting of buffer data remote headers
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount the
number of pages is needed for the asked for size as it doesn't take into
account the meta data on each page. Add a helper function to do the
calculation properly and use that in these functions.
- Catch nr_page_va overflow in ring_buffer_desc sizing
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than what
was asked for.
- Do not resize the subbuf order if any per_cpu buffer is disabled
The mmapping of ring buffers disables resizing the subbuffers, but it is
done per-cpu whereas the subbuf size change is done for all the per_cpu
buffers under the buffer->mutex. It could change the size of some while
the mapping is happening on others. Have the resize of the subbuf order
check all the per_cpu buffers under the lock to see if any of them is
disabled before starting and causing an inconsistency between buffers that
are being mapped.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCaqbdrBQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qro9AQDF/j3VW3Uu98lVFI9AB10XYhLDd5nt
Zpf+3RviNgFpxgEAiE2+4K+4sM2SfaDDh9JMww9MKg1exL+cemE3a+JbBgY=
=jgYE
-----END PGP SIGNATURE-----
Merge tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing fixes from Steven Rostedt:
- Don't destroy user event fields when removal fails
User event fields are destroyed before the event is removed from
visibility. But that can fail leaving the still visible event with no
fields. Move the destroying of the fields to after the event is
successfully removed from visibility.
- Initialize function graph state is fork before calling
copy_exec_state()
For non-CLONE_VM forks, copy_exec_state() allocates a new
task_exec_state. If that allocation fails, ftrace_graph_exit_task()
will free the tasks ret_stack pointer. Since that pointer is still
using the parent's ret_stack, it mistakenly frees the parent's
pointer too.
Call ftrace_graph_init() on the task first which will NULL out the
new tasks's ret_stack and if the copy fails, it will not free
anything.
- Remove FGRAPH_MAX_INDEX
The macro FGRAPH_MAX_INDEX was added but never used. Remove it.
- Save ent_size in function graph printing of nested functions
The function graph tracer needs to look at the next event to see if
the next event is the return of the current function entry. If it is,
it prints a single line:
ktime_get();
Otherwise it prints it like a nested function:
tick_nohz_irq_exit() {
ktime_get();
kcpustat_irq_exit();
}
In order to look at the next event, it must save the current event so
that it has the information to print from it. It saves the event in
the iterator descriptor called "ent". What it doesn't save is the
ent_size of the event which is now used to know if the function graph
arguments are to be printed. The peek doesn't save the size so the
size used happens to be that of the size of the last event that was
seen.
Save the entry event size in the iterator descriptor so that the
correct size is used.
- Fix several errors with freeing data in the histogram code
The histogram code had a lot of leaked or or incorrect accounting
when failures happen. Correct them.
- Fix histogram regression of .percent and .graph modifiers
Up until 6.3 histogram values could have "percent" or "graph"
modifiers that changed how they were printed. But a change that added
restricting histograms values from being strings, stack traces and
other modifiers inadvertently prevented them from using the percent
and graph modifiers, which were legal use cases for values.
Put back the percent and graph modifiers.
- Fix various typos in the comments
- Set the trace_clock before initializing a histogram with clock
argument
The histogram API allows the user to specific which trace clock to
use via a "clock=" string. The histogram is set up first before the
clock is checked. If the passed in clock is not valid, it exits
without fully fixing up the histogram leaving it on the list and a
use-after-free can trigger.
Update the clock argument first and if it fails then exit gracefully
before the histogram trigger is placed on any lists.
- Restore :mod: trailer after parsing in ftrace_set_clr_event
The function ftrace_set_clr_event() modifies the parse string and
needs to put it back to what was passed in. It searches for ":mod:"
via a strsep() but fails to put back the first ':' in the string.
Add back the ':' in the passed in string.
- Take trace_array reference when opening a tracer options file
The options files are dynamically created and some tracers add their
own options. When a tracer adds their own list of options, the
trace_array holding them has an array to hold the list of options for
each tracer. This array increases in size via a krealloc(), and the
new entry gets a newly allocated array to hold the options of the new
tracer being added.
The element in each entry of the tracer's option array holds a
pointer back to the trace_array, a pointer to the tracer it is
associated to, a pointer to the flags of the option.
The issue is that these arrays are freed when the trace_array is
freed when its instance it represents is removed from the instances
directory. There's a race that an open of one of these options files
can happen when the instance is being removed.
Add a new helper function to be called by the open function of the
options file to iterate all existing trace_arrays under a lock and
find the one that has the given option element in one of it's tracer
arrays. If found, then update the associated trace_array's reference
counter to keep it from being freed. If not found, have the open call
return -ENODEV.
- Disable interrupts when acquiring the lock in rb_wake_up_waiters()
The function rb_wake_up_waiters() assumes it will be called in
interrupt context and does not disable irqs when taking
cpu_buffer->reader_lock, which can be called in hard interrupt
context. The issue is in PREEMPT_RT, this function is called in
thread context leaving this lock open to a deadlock.
Take the lock with interrupts disabled.
- Use rcu_assign_pointer() for tmp_ops filter hash
The tmp_ops used in update_ftrace_direct_mod() assigns its
filter_hash field directly, but that field is annotated as __rcu and
sparse complains. Assign it with rcu_assign_pointer()
- Fix use-after-free in enable_trigger_private_data_free()
The trace_event_call is accessed through the event_trigger_data's
trace_event_file pointer to put the trace_event_call on freeing. The
issue is that the trace_event_file data may have been freed already
causing a use-after-free. Add a field to the event_trigger_data that
points directly to the trace_event_call so that it can decrement its
reference directly without needing to go through the
trace_event_file.
- Fix accounting of buffer data remote headers
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount
the number of pages is needed for the asked for size as it doesn't
take into account the meta data on each page. Add a helper function
to do the calculation properly and use that in these functions.
- Catch nr_page_va overflow in ring_buffer_desc sizing
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than
what was asked for.
- Do not resize the subbuf order if any per_cpu buffer is disabled
The mmapping of ring buffers disables resizing the subbuffers, but it
is done per-cpu whereas the subbuf size change is done for all the
per_cpu buffers under the buffer->mutex. It could change the size of
some while the mapping is happening on others. Have the resize of the
subbuf order check all the per_cpu buffers under the lock to see if
any of them is disabled before starting and causing an inconsistency
between buffers that are being mapped.
* tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (25 commits)
ring-buffer: Check resize_disabled before publishing the new subbuf order
tracing/remotes: Catch nr_page_va overflow in ring_buffer_desc sizing
tracing/remotes: Account for ring buffer page header in size calculation
tracing: Don't dereference trace_event_file in deferred trigger free
ftrace: Use rcu_assign_pointer() for tmp_ops filter hash
ring-buffer: Acquire the lock with irqsave in rb_wake_up_waiters()
tracing: Take trace_array reference when opening a tracer options file
tracing: Fix ring_buffer_read_page_size() kernel-doc
tracing: Restore :mod: trailer after parsing in ftrace_set_clr_event()
tracing: Fix memory corruption from a "STACKTRACE" histogram key
tracing: Fix memory corruption from the histogram stacktrace modifier
tracing: Undo the registration when enabling the histogram trigger fails
tracing: Take the reference before publishing the named histogram trigger
tracing: Set the trace clock before registering the histogram trigger
tracing: Fix typo "preceeded" in comment
tracing: Fix typo "availabe" in comment
tracing: Let histogram values keep the percent and graph modifiers
tracing: Keep the entry count when the histogram stats allocation fails
tracing: Free histogram the field rejected for a bad modifier
tracing: Free histogram the var ref when its initialization fails
...
|
||
|
David Carlier
|
d860c67c05 |
ring-buffer: Check resize_disabled before publishing the new subbuf order
ring_buffer_subbuf_order_set() stores the new order and only then walks
the CPUs, returning -EBUSY if any of them has resizing disabled. A user
mapped buffer has resizing disabled, and __rb_map_vma() reads
buffer->subbuf_order without buffer->mutex, so an mmap of an already
mapped CPU racing the failing order change sizes the mapping with the
new order and inserts pages past the sub-buffer into the VMA.
Check the CPUs before storing the new order.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Vincent Donnefort
|
d059d8bf2c |
tracing/remotes: Catch nr_page_va overflow in ring_buffer_desc sizing
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than
what was asked for.
Return SIZE_MAX from trace_buffer_desc_size() on nr_page_va overflow.
Link: https://patch.msgid.link/20260911193937.602202-3-vdonnefort@google.com
Fixes:
|
||
|
Vincent Donnefort
|
442ffa742d |
tracing/remotes: Account for ring buffer page header in size calculation
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount the
required pages because every ring buffer page contains a header
(BUF_PAGE_HDR_SIZE). Account for that header to ensure allocated remote
ring buffers aren't smaller than requested by the user.
The newly introduced helper __calc_nr_pages_ring_buffer_desc() can
return a value that overflows the descriptor nr_pages field (32 bits).
Link: https://patch.msgid.link/20260911193937.602202-2-vdonnefort@google.com
Fixes:
|
||
|
Linus Torvalds
|
c874ace034 |
Misc timer fixes:
- Fix clockevents replacement race when a broadcast
device is replaced which may trigger a BUG() crash
(朱恺乾 - Zhu Kaiqian)
- Fix potential timerqueue ordering bug when rearming
a queued timer with nonzero slack (Andrea Parri)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqmXaERHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1jM9RAAnEShNuh27uYj3oxVgaLo+Dhsk1AYzNnr
WVS/Cz8HfFXMEVnOsT3CibcB6p5wydTHms/8248GZWWUuM1HtL/7zcUHWGFgs72S
WRTcw/ouzvdAKQfxlH2j96uApMWwnWnv9XRfpFel9bgIIK1POL9g0JJmcuFK5uNf
5aYvzkdLv3SKHR0BnrIF4a6jqq1Shf2sZPDXHmJDE/k/He28zvFjiOlIEChxfCfh
qUEzY036hSU0RbAONSbn88bj7dc10/Xuck/iW3WVW8cOqtVxw79biAoumUe9jqwE
hX3B6rvDEPvaEOPDm2PgUlrapFukjfImu7K9rDljbFMX1jF6eb7ZQMk4ftJXL+rM
M0RPCdrS2ZrVOKt3VFIYRH7ZzFNwtE+RHPZSD6lpVgia6xpgi6yY++AzTeCn+VpK
3AmkxMg3xHOLkISyCRUlmtTn3Cis6O7+9+9dEad24dh5mkQM7Tr6nzprYeg3fgpR
z714UKjOUvBNBtxCjdZl5/c/i8mb0IaH4DmT+/V6mIXWoHchbqgw0Or7G7XMm5XM
M1J+4RJrGhhg1eUTb254PWi/OixuXZ8XgcB1wwAiJFMTJY9YqBExKRAdZ+Al1DJP
FgHDEPyvElpeh5XFFqf8Ft9xXOTn1CSEc6G+dCO0MswcikHnw4UnmLGn9z3js2AS
SRlK75Dm49s=
=SyXt
-----END PGP SIGNATURE-----
Merge tag 'timers-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fixes from Ingo Molnar:
- Fix clockevents replacement race when a broadcast
device is replaced which may trigger a BUG() crash
(朱恺乾 - Zhu Kaiqian)
- Fix potential timerqueue ordering bug when rearming
a queued timer with nonzero slack (Andrea Parri)
* tag 'timers-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
hrtimer: Use hard expiry when updating timers on the same base
tick/broadcast: Plug clockevents replacement race
|
||
|
Linus Torvalds
|
b2a8a7669e |
Miscellaneous scheduler fixes:
- Fix EEVDF se->max_slice value on enqueueing (Vincent Guittot)
- Fix EEVDF augmented rb-trees re-balancing with
multiple fields (Vincent Guittot)
- In proxy scheduling, account cgroup CPU time to the execution
context, not the scheduling context (Hui Su)
- Likewise, call wq_worker_tick() for the execution context,
not the scheduling context (Hui Su)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqmXEMRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hchg/6A1gRkn7T5+K957U8wpB9vjtV9cKVdWpI
XFDGm60ylFUmU388Xb8mmrbDgmej6RpX6C4ccppygM3196w513tB+Zr8w6jbSszk
ddgWwfwi58FFBJZTH7JDqeJ64wvrl8KId44yM6k2JdXATxh2DGF0w+YdsA+M5HVJ
EJbjACYhePdK27wvQDtj1poDfAyiabqEnv7w62dhEU9I+ikmcPAyrhmqU0yFDNUR
sNozsDQnEJrHtllGHpr3FVxYRqob6lOtG+86VSiZ8F6i2kA3p/451mpMyyCOMUrF
kZlBIryLG0gylXIensqLox+z2ZIE4nUL0OX3o7mC+MLNERdWvsdgHi9AcZlIoFpJ
wMPBLENnnGbilmwhXjk0pL655rlVVUGwaTV4T9Pk5D5qew6B9LGe8uqiLo8U/qih
1o5Lf3ZnUi0o8XHMfNkwQ3Y0m1S7CbgJYKItE+ec+2QifmKGD5dOKo5WWDKszywF
Zb9ScP2fMKideS/JEX1/+jvLcpDmV+HE3mC58Muek96fbJG1IQ4bL5tu+fqO85rM
68dJymtMrkoeegmq4jqERt758sZnyv2QbDmr1Kc1G9vSKNAeahYldPU2gQ6qFBnO
VCMvI2BUmfiia9A6NBKKgpjijc4zWydyuaQ2zdvEBddYnLWKZ5Ge26lLrjraNtdB
GtHalAP1gVs=
=zJHp
-----END PGP SIGNATURE-----
Merge tag 'sched-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fixes from Ingo Molnar:
- Fix EEVDF se->max_slice value on enqueueing (Vincent Guittot)
- Fix EEVDF augmented rb-trees re-balancing with multiple
fields (Vincent Guittot)
- In proxy scheduling, account cgroup CPU time to the execution
context, not the scheduling context (Hui Su)
- Likewise, call wq_worker_tick() for the execution context,
not the scheduling context (Hui Su)
* tag 'sched-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/core: Call wq_worker_tick() for the execution context
sched: Account cgroup CPU time to the execution context
sched/eevdf: Fix rb augmented with multi fields
sched/eevdf: Fix augmented max_slice
|
||
|
fangqiurong
|
c7a1c6e800 |
sched_ext: Close the pre-enable ops error claim window
scx_alloc_and_add_sched() publishes ops->priv before
scx_root_enable_workfn() switches the state to SCX_ENABLING. An error
claimed via scx_bpf_error_bstr() from an associated BPF program in that
window is consumed by scx_disable_workfn(), which takes the pre-enable
shortcut in scx_root_disable(). The shortcut returns without any teardown
and restores SCX_DISABLED with an unconditional scx_set_enable_state() xchg
racing the enable workfn's own transition. The enable then completes with
the claim consumed: the scheduler stays up but can never be disabled again,
and bpf_scx_unreg() frees it while still in use, resulting in a
use-after-free. Both WARN_ON_ONCE()s fire back to back:
WARNING: kernel/sched/ext/ext.c:7522 at
scx_root_enable_workfn+0xeec/0x1be0, CPU#3: scx_enable_help/276
WARNING: kernel/sched/ext/ext.c:6398 at scx_root_disable+0xb50/0xdb8,
CPU#0: sched_ext_helpe/664
scx_root_enable_workfn() switches to SCX_ENABLING before the scheduler
allocation, so ops->priv is never visible while SCX_DISABLED. The allocation
failure path restores SCX_DISABLED.
Fixes:
|
||
|
Linus Torvalds
|
85855f85de |
Miscellaneous perf events fixes:
- Fix sched_cb_list corruption on PMU callbacks that
invoke list_del() during perf_event_overflow()
calls (Thomas Richter)
- Fix PEBS pt_regs->flags snapshot data that was
regressed with the introduction of adaptive
PEBS v4 support (Dapeng Mi)
- Fix possible drain_pebs() re-entry bug
when intel_pmu_drain_pebs_buffer() is called from
process context (Dapeng Mi)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqmWoURHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hzhA//TBu2jx/c2r5tafuW5mpPJH+3eGcMWUh9
KfVLt/FMzgt+sOtzM/eIQ2Sk4VAachTkSLjh6MQR1m9g4jXTVr5rH8lJlFXqW/74
v5XhaEryXTSOn6zpxXpKrVGYFq6RfdTijtmrAK+7ac6HChgRrMa0Eb5yVvPauLXE
+Y0RugHjF75c4iXapb68osWF+7EoVKGqLPZjdQ12D6wga7+1DRTZWV37hOXHtu1M
GbajBMTKF5Q4QCffsyY9PDks86dKLDrv8z7XxGNYz4pdcnwD4bMdYfu8FxohowaO
EXx9b0dxK1RobcsMo+wHZojTD0i4ySSIbN7lJU6dHopMFJ3nKgCDYuPvzov9T/Dm
8eFrzlQbr0m9+NqndFomjv4so1WGF1Q2DHqEOJQgSeHNECpUAx7mzIzv/AJiJm1P
AC3S8PJT5+AFQZtySqV6nI8UyzyMgoDo3EYdp3oKHq/B6SGDlAUHr3k8e5bpI8ss
JbIvyo1RH6DrB+FstHHve7zn5ueYmjxPQRS8NdIRrBKAeMryC7DYEDU7ZUwkAJS9
jJZE7wM0zhLbF5EPVEA3+rhrh1wZ0mUFsWJwWbn7QwIpSm331J8AEKdpKFa3k5op
FH9PusGL/SKeVwo1nReExLJWghdlxhbHAC8QhLMHTKOrhoqoUzpp5hRQ9rgx5z7N
hksCg2xZC+U=
=mKBo
-----END PGP SIGNATURE-----
Merge tag 'perf-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar
- Fix sched_cb_list corruption on PMU callbacks that
invoke list_del() during perf_event_overflow()
calls (Thomas Richter)
- Fix PEBS pt_regs->flags snapshot data that
regressed with the introduction of adaptive
PEBS v4 support (Dapeng Mi)
- Fix possible drain_pebs() re-entry bug when
intel_pmu_drain_pebs_buffer() is called from
process context (Dapeng Mi)
* tag 'perf-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf/x86/intel: Prevent drain_pebs() reentry
perf/x86/intel: Correct pt_regs->flags update for PEBS path
perf/core: Allow list_del during perf_event_overflow()
|