mirror of
https://github.com/torvalds/linux.git
synced 2026-10-15 20:30:02 +02:00
master
53618 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Kumar Kartikeya Dwivedi
|
fd16449a9b |
bpf: Preserve packet pointer class displacement in regsafe()
regsafe() maps packet pointer IDs between states and checks that each current register range is a subset of the corresponding explored register range. It does not, however, preserve the displacement between registers that share a packet pointer ID. This is unsound because packet range is shared by ID. A bounds check on one class member updates every member, and a later access can consume the range through another member. Commit |
||
|
Kumar Kartikeya Dwivedi
|
261b61d373 |
bpf: Make post-verification instruction rewrites killable
After do_check() returns, the verifier runs several instruction rewrite
passes. Some of them patch or remove one instruction at a time. Each
operation moves the remaining instruction and auxiliary-data arrays and
adjusts all branch offsets, making the overall work quadratic in the
program length.
A privileged loader can submit 131072 unconditional jumps by zero followed
by a valid return. Verification finishes quickly, but bpf_opt_remove_nops()
then spends a long time removing each jump separately. Since this
post-verification work neither checks for signals nor reschedules, a pending
SIGKILL cannot terminate the task until the rewrite finishes.
Make bpf_patch_insn_data() and verifier_remove_insns() common cancellation
and rescheduling points. These helpers run from BPF_PROG_LOAD process
context, and bpf_patch_insn_data() can already sleep while reallocating
auxiliary data.
Report interrupted constant blinding as -EINTR and propagate it through
both JIT paths, including kernels that permit interpreter fallback.
Other blinding failures retain the existing fallback behavior.
This does not reduce the quadratic cost of the rewrite passes, but it makes
the work preemptible and allows a killed loader to be torn down promptly.
Fixes:
|
||
|
David Carlier
|
1d653a1839 |
fprobe: Terminate the fgraph_data list when the reservation is not filled
fprobe_fgraph_entry() reserves shadow stack space for every fprobe with
an exit handler, but only fills it for those whose entry handler returns
0. fgraph_reserve_data() does not clear the area, so fprobe_return()
parses the unused tail as headers left over from an earlier call, and an
exit handler can run twice or despite its entry handler asking to skip
it.
Write a zero word after the last entry to terminate the walk. A zeroed
slot does not decode to a NULL fprobe on the arches that encode the
header into one unsigned long, since arch_decode_fprobe_header_fp() ORs
in FPROBE_HEADER_MSB_PATTERN, so make read_fprobe_header() return NULL
for a zeroed slot.
Link: https://lore.kernel.org/all/20260917212407.384468-1-devnexen@gmail.com/
Fixes:
|
||
|
Weiming Shi
|
50e80e2bb5 |
bpf: Skip unsettled links in link iterator
bpf_link_prime() inserts a link into link_idr before anon_inode_getfile()
succeeds and before bpf_link_settle() publishes the ID in link->id.
bpf_link_by_id() treats such an ID-zero link as unsettled, but the link
iterator takes a reference without this check.
If anon_inode_getfile() then fails, the creator removes the ID and frees
its still-private link directly. The iterator is left with a dangling
reference and its next bpf_link_put() accesses freed memory.
Treat ID-zero entries as transient in bpf_link_get_curr_or_next(), just as
bpf_link_by_id() does.
BUG: KASAN: slab-use-after-free in bpf_link_put
Write of size 8 by task exp/384
Call Trace:
bpf_link_put kernel/bpf/syscall.c:3372
bpf_link_seq_next kernel/bpf/link_iter.c:33
bpf_seq_read kernel/bpf/bpf_iter.c:158
vfs_read fs/read_write.c:572
ksys_read fs/read_write.c:716
do_syscall_64 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe arch/x86/entry/entry_64.S:121
Kernel panic - not syncing: KASAN: panic_on_warn set ...
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
40c2096961 |
bpf: Verify global subprogs in each sleepability context
Global subprograms are verified independently with a fresh verifier root. do_check_common() currently seeds that root's in_sleepable state from the program, even though a global subprogram can also run from callbacks whose execution context differs from the program's main entry point. In particular, workqueue and task-work callbacks are sleepable even when the containing program is not. A global subprogram of that program is therefore verified as non-sleepable, making in_rcu_cs() true and allowing loads of RCU-protected kptrs to produce trusted MEM_RCU pointers. The same subprogram can then be called from a sleepable callback without a classic RCU reader. It can retain such a pointer while the object is freed and use it after free. The verifier's execution-context predicates are complementary. A state is sleepable only when in_sleepable is set and no RCU, preemption, IRQ, or lock region is active. Each condition which prevents sleeping also provides RCU protection, while in_rcu_cs() treats a non-sleepable state as implicitly protected. Use this relationship to represent a global subprogram caller with only the result of in_sleepable_context(). A protected sleepable caller is normalized to in_sleepable=false at the independent verification root. This both prevents sleepable operations and makes in_rcu_cs() true without copying caller-owned lock state. Track only the contexts in which each global subprogram is actually reached. Verify it once if all reachable calls use the same context, and twice only if both sleepable and non-sleepable calls reach it. Calls found while verifying globals or asynchronous callbacks mark further contexts for checking. Repeat the existing subprogram walk until all called contexts have been verified; unreachable global calls remain unchecked. Accumulate instruction counts over those verification passes. Preserve the total recorded before each pass, since path accounting has already added this pass's synchronous instructions and its root total must also include asynchronous subprograms. This makes an unprotected callback verify the global subprogram as sleepable, turning its RCU-protected kptr load into an untrusted pointer. Protected callers and global subprograms which do not depend on implicit RCU protection remain valid. Fixes: |
||
|
fangqiurong
|
cb86607ada |
sched_ext: Don't run ops.dequeue() with a DSQ lock held
ops.dequeue() is invoked with the source user DSQ's lock still held on
the consume and move paths (scx_consume_dispatch_q(),
move_task_between_dsqs()). A BPF scheduler which locks the source user
DSQ from ops.dequeue() - e.g. by iterating it with bpf_iter_scx_dsq -
self-deadlocks.
ops.dequeue() can only call the "any" kfuncs and none of them can lock a
builtin DSQ, so the global and bypass paths can't deadlock; however,
all DSQ locks share one lockdep class, so iterating any user DSQ from
ops.dequeue() on those paths trips the recursion check.
Move the invocation after the DSQ unlock on all three paths.
SCX_TASK_IN_CUSTODY is cleared under the lock serializing the transfer
so that the callback is invoked exactly once.
Fixes:
|
||
|
Tejun Heo
|
df5cdc2c83 |
sched_ext: Derive SCX_RQ_IN_WAKEUP from the core enqueue flags
schedule_deferred_locked() skips scheduling a deferred action while
SCX_RQ_IN_WAKEUP is set and relies on the task_woken_scx() call that follows
a wakeup enqueue to run it. enqueue_task_scx() sets the flag from the merged
enqueue flags, which include the flags stashed for a remote activation.
move_remote_task_to_local_dsq() thus sets SCX_RQ_IN_WAKEUP on the
destination rq when the moved task was woken up, although no
task_woken_scx() follows that activation.
An IMMED insert into a busy destination requests a local reenqueue during
that enqueue. The request gets linked but not scheduled and stays pending
until an unrelated wakeup or preemption on that CPU runs the deferred
actions. The IMMED task sits behind the running task in the meantime. If
nothing runs them before the scheduler is disabled, the request outlives the
scheduler and points into its freed per-cpu area, which the next scheduler
dereferences from run_deferred().
Test the core enqueue flags for the wakeup bit. Only the core's wakeup path
is followed by task_woken_scx().
Fixes:
|
||
|
Linus Torvalds
|
4aec9ad1c6 |
dma-mapping fixes for Linux 7.3
A few fixes for the DMA-mapping code:
- resolved regression in accessing encrypted memory by IOMMU-backed
devices (Aneesh Kumar K.V),
- improved failure handling and removed rare bug in swiotlb/highmem
(Donggeun Yoo).
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQSrngzkoBtlA8uaaJ+Jp1EFxbsSRAUCaquwMwAKCRCJp1EFxbsS
RMT6AP0elpdaZXNY0KwUBTwU95H604J+donqriepHABIBhIDEQD9GWZqNf/m1gEI
tR5lHQ3+NGs0Q7Vd2ed1vSe82HQSsgU=
=QxUC
-----END PGP SIGNATURE-----
Merge tag 'dma-mapping-7.3-2026-09-17' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping fixes from Marek Szyprowski:
"A few fixes for the DMA-mapping code:
- resolved regression in accessing encrypted memory by IOMMU-backed
devices (Aneesh Kumar K.V)
- improved failure handling and removed rare bug in swiotlb/highmem
(Donggeun Yoo)"
* tag 'dma-mapping-7.3-2026-09-17' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
x86/mm: Don't force unencrypted DMA for IOMMU-backed devices
dma-mapping: don't trace the DMA address when the allocation fails
swiotlb: use the adjusted address for the highmem page lookup
dma-coherent: report a failed reserved memory assignment
|
||
|
Thomas Gleixner
|
d2710c8d93 |
signal: Prevent exec() race
Hyunwoo debugged the following KASAN UAF splat:
BUG: KASAN: slab-use-after-free in __send_signal_locked+0xb27/0xba0
Write of size 8 at addr ffff888007ed80c8 by task poc/79
...
Call Trace:
__send_signal_locked+0xb27/0xba0
do_send_sig_info+0xa7/0x160
do_send_specific+0x76/0xa0
__x64_sys_tgkill+0x193/0x270
...
Allocated by task 80:
do_timer_create+0x1a4/0x1030
__x64_sys_timer_create+0x145/0x190
...
Freed by task 12:
kmem_cache_free_bulk+0x1f8/0x4a0
kvfree_rcu_bulk+0x14f/0x1c0
kfree_rcu_work+0x128/0x1a0
...
Last potentially related work creation:
kvfree_call_rcu+0x39/0x390
__flush_itimer_signals+0x211/0x320
flush_itimer_signals+0x47/0x90
begin_new_exec+0xa6b/0x28c0
It turned out that this happens with a non-leader exec() as Hyunwoo
explained:
de_thread() calls exchange_tids() before release_task(leader), so the
struct pid held by a SIGEV_THREAD_ID timer created against the leader's tid
now points to the thread which called execve(). pid_task() returns that
thread and lock_task_sighand() on it succeeds.
If the timer signal is blocked, its sigqueue stays queued on the leader's
task::pending. The next expiry of that timer can then run while
release_task() flushes the queue.
posixtimer_send_sigqueue() checks whether the sigqueue is already queued
with a plain list_empty(), which only reads list_head::next.
list_del_init() is not atomic and INIT_LIST_HEAD() stores list_head::next
before list_head::prev, so the check can pass in between. list_add_tail()
queues the entry on the task::pending of the live thread, and the
list_head::prev store from the flush then overwrites the list_head::prev
link that list_add_tail() has just set.
__flush_itimer_signals() does not undo that either. With list_head::prev
pointing at the entry itself, its list_del_init() only stores the same
values again, so the entry is not removed from the list. It is still there
after the last reference is dropped and the timer is freed by RCU, and the
list_add_tail() of a later tgkill() follows that list_head::prev into the
freed timer.
This problem surfaced with the recent commit which moved the sigqueue flush
out of the sighand lock held region.
Hyonwoo proposed to fix this by using list_del_init_careful(), but that
just papers over the problem. After some disucssions and various attempts
to solve it, Eric pointed out that there is no reason to flush
task::pending late in release_task() and it should be done in
exit_signals() already.
As nothing can collect and deliver signals which are queued in a dying
task's pending queue, there is no reason to delay it further.
But it has to be ensured that no signals can be queued into it after that
point. exit_signals() sets PF_EXITING in task::flags, which can be used as
an indicator for this.
Cure it by:
- Preventing signal queueing for task private signals (PIDTYPE_PID) when
the task has PF_EXITING set in __send_signal_locked() and in
posixtimer_send_sigqueue().
- Protecting the unlocked setting of PF_EXITING in exit_signals() for the
task group empty and the group exit case with sighand lock
- Flushing task::pending signals right there.
Optimize that by moving the whole pending list to an on-stack list head
under sighand lock and free the signals without the lock held.
There has been quite some discussion about the lockless flush and the
non-leader exec case on weakly ordered systems. The problem is that a third
party which tries to send a posix timer signal relies on the PID lookup to
find the target task and that lookup might result in the new leader when
the signal was originaly directed to the old leader. In case that the
signal was queued on the old leader then the lockless flush raised a
concern over the following situation:
old_leader new_leader third party
A: flush_list() // list_del_init() stores to sigqueue
LOCK (tasklist)
old_leader->exit_state = EXIT_ZOMBIE;
B: UNLOCK (tasklist)
C: LOCK (tasklist)
if (old_leader->exit_state)
transfer_tids()
D: store PID
posix_timer_send_sigqueue()
// Observes #D so t = new_leader
E: t = get_target()
F: LOCK (sighand)
G: if (list_empty(sigqueue))
list_add(sigqueue)
The concern was that the third party might observe #D but not observe #A
and therefore would proceed to #G while the list_del() stores (#A) in
flush_list() are not visible yet, which could result in list corruption.
That would be possible if looking at it solely from a RELEASE+ACQUIRE
ordering point of view, but B-C is a UNLOCK+LOCK hand-over, which is not
the same as RELEASE+ACQUIRE:
RELEASE+ACQUIRE: RCpc, only the CPUs involved agree on the ordering
UNLOCK+LOCK: RCtso, the hand-over is store-ordering
As B-C is UNLOCK+LOCK, which is RCtso and that does impose store order,
A stores must happen before the D store.
Combine with E-F, which has a data dependency from the LOAD to the LOCK and
thereby constraints later LOADs, those sigqueue loads in G that come after
F must in fact observe the A stores.
Fixes:
|
||
|
Peter Zijlstra
|
b61b6f95d6 |
futex: Also allocate private hash on vfork()
As Jann demonstrated, it is entirely feasible to access the mm through vfork().
Therefore we need to allocate a private hash on vfork() as well as any other
CLONE_VM user.
Specifically, it must be avoided to have (private) futex waiters before
allocating the private hash.
Fixes:
|
||
|
Tejun Heo
|
7de9a6fb44 |
sched_ext: Wait for SCX_OPSS_DISPATCHING before reenqueueing a task
|
||
|
Linus Torvalds
|
9b87fdc9af |
sched_ext: Fixes for v7.3-rc3
- An error raised by a BPF program before the scheduler finished enabling was consumed by the disable path's pre-enable shortcut, leaving a running scheduler that couldn't be disabled and was later freed while in use. - Two compat kfuncs dereferenced a NULL scheduler when handed an exited or idle task, oopsing the kernel. - Keep-running decisions in the dispatch path used the root scheduler's flags for tasks belonging to a sub-scheduler, causing warnings and stalls. - Schedulers with their own CPU ID mapping had no way to learn which IDs are online. Add a kernel-maintained online mask to plug the hole. - Cgroup idle state: the initial cpu.idle state wasn't passed on cgroup init and same-value rewrites delivered spurious callbacks. - Example scheduler fixes for a reenqueue loop on attach, placements on CPUs without effective grants, stalled partition work and stale idle tracking. -----BEGIN PGP SIGNATURE----- iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCaql+vQ4cdGpAa2VybmVs Lm9yZwAKCRCxYfJx3gVYGUTxAP9mhhePmghF/aq6SzeaVGeUXtclXRAGE1EAqea6 kzh50AEAiWXtFQu9dAF3EaXTWJkEJZkJUs0AO0wEXgghC0py9gY= =gcDW -----END PGP SIGNATURE----- Merge tag 'sched_ext-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext Pull sched_ext fixes from Tejun Heo: - An error raised by a BPF program before the scheduler finished enabling was consumed by the disable path's pre-enable shortcut, leaving a running scheduler that couldn't be disabled and was later freed while in use. - Two compat kfuncs dereferenced a NULL scheduler when handed an exited or idle task, oopsing the kernel. - Keep-running decisions in the dispatch path used the root scheduler's flags for tasks belonging to a sub-scheduler, causing warnings and stalls. - Schedulers with their own CPU ID mapping had no way to learn which IDs are online. Add a kernel-maintained online mask to plug the hole. - Cgroup idle state: the initial cpu.idle state wasn't passed on cgroup init and same-value rewrites delivered spurious callbacks. - Example scheduler fixes for a reenqueue loop on attach, placements on CPUs without effective grants, stalled partition work and stale idle tracking. * tag 'sched_ext-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: sched_ext: Maintain an online cid mask in the scheduler arena sched_ext: scx_qmap: Restore unused idle claims from ops.dispatch() sched_ext: Close the pre-enable ops error claim window sched_ext: scx_qmap: Fix pending partition work handoff sched_ext: scx_qmap: Place only on cids whose caps are in effect sched_ext: scx_qmap: Do not add IMMED to rescue inserts sched_ext: Use @prev's scheduler for the keep decisions in dispatch_one() sched_ext: Rename sch to root_sch in dispatch_one() sched_ext: Fix NULL sched deref in kfunc sub-sched error paths sched_ext: Don't deliver duplicate ops.cgroup_set_idle() for same value sched_ext: Pass the initial cpu.idle state in scx_cgroup_init_args |
||
|
Linus Torvalds
|
6fb20c0271 |
cgroup: Fixes for v7.3-rc3
- The task iterator could pick up a dying task whose refcount had already dropped to zero and resurrect it, leading to a use-after-free when reading cgroup.procs. Skip such tasks. -----BEGIN PGP SIGNATURE----- iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCaql+xg4cdGpAa2VybmVs Lm9yZwAKCRCxYfJx3gVYGVcaAP424sQDRkkInl7APgUeQfgUheNG8WgCUt+qMgT2 6VbVEwD9HodEyKW4r/JBWcqN3m4bbNSx57gpXgQwH+ESoorqLgs= =nZ8D -----END PGP SIGNATURE----- Merge tag 'cgroup-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup Pull cgroup fix from Tejun Heo: - The task iterator could pick up a dying task whose refcount had already dropped to zero and resurrect it, leading to a use-after-free when reading cgroup.procs. Skip such tasks. * tag 'cgroup-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: cgroup: Avoid iteration of dying tasks with zero refcount |
||
|
Tejun Heo
|
a9e3760b08 |
sched_ext: Maintain an online cid mask in the scheduler arena
Schedulers on the default cid mapping treat [0, nr_online_cids) as the online set and restart on hotplug. Schedulers that install their own mapping with scx_bpf_cid_override() have no way to learn which cids are online: the count no longer identifies members and the CPU-form cpumask is unusable from cid programs. This is an obvious hole in the cid API. Add scx_bpf_online_cmask(), a kernel-maintained cmask in the scheduler's arena, allocated alongside the per-CPU scratch masks and populated after the cid mapping is finalized and before ops.init(), for child schedulers too. The pointer stays valid through ops.exit() with no reference to take. It is the arena offset as a void pointer, the same form struct_ops arena arguments arrive in. The verifier types the void return as a scalar for the program's arena cast. The mask follows the SCX hotplug notifications: seeded from cpu_active_mask and updated before ops.cid_online/offline() runs, so it lags cpu_online_mask only inside a hotplug transition. Updates walk the scheduler list under the lock that also serializes unlinking. Reads are live, not atomic snapshots. Root initialization excludes hotplug. v2: Reworded the getter kerneldoc (Andrea Righi). Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> |
||
|
Tejun Heo
|
9a0b159ff1 |
sched_ext: scx_qmap: Restore unused idle claims from ops.dispatch()
scx_qmap tracks idle cids itself. pick_direct_dispatch_cid() claims a cid by clearing its bit and the task is inserted into that cid's local DSQ, which kicks the CPU. When the task does not arrive, for example because the insert fell back to the global DSQ after an affinity change, the CPU wakes, finds nothing and picks idle again. That is not an idle transition, so ops.update_idle() is not called and the cid stays marked busy until an unrelated task runs on it. Restore the claim from ops.dispatch(). The kick guarantees a dispatch on the kicked CPU, and when it finds nothing to run with a NULL @prev, the CPU is going back to idle. Document the pattern in ops.update_idle(), which reports only actual transitions. Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> Cc: Andrea Righi <arighi@nvidia.com> |
||
|
Joel Granados
|
afdf35cfae |
sysctl: Fix type truncation in sysctl_msec_to_jiffies
Return MAX_JIFFY_OFFSET for all the values truncated when val (u64) is
passed to msecs_to_jiffies (u32). This aligns with how very large
millisecond values get translated into MAX_JIFFY_OFFSET.
Fixes:
|
||
|
Kuniyuki Iwashima
|
318012c565 |
sysctl: Check range in do_proc_ulong_conv_ms_jiffies
Add the range check back to do_proc_ulong_conv_ms_jiffies that commit |
||
|
Kuniyuki Iwashima
|
82431877d8 |
sysctl: Check range in proc_dointvec_ms_jiffies_minmax
Add the range check to do_proc_int_conv_ms_jiffies_minmax that commit |
||
|
Michal Koutný
|
057dac23d3 |
cgroup: Avoid iteration of dying tasks with zero refcount
The commit |
||
|
Masoud Aghasi
|
953824e508 |
bpf: Fix u32 overflow issue in map batch operations
Several map batch operation implementations such as generic_map_lookup_batch() use calculations in the form of "values + cp * map->value_size" to compute the desired userspace memory address for reading or writing. This can overflow the u32 type (the result of "cp * map->value_size") when the map size exceeds 4GB. generic_map_lookup_batch() may corrupt values for some keys in userspace memory, and in some cases it mismatches values for some keys while still reporting success. Other batch operations may fail to delete or update some keys, or the syscall may return unexpected errors. Add size_t casts to prevent the affected offset and size calculations from overflowing. Fixes: |
||
|
Pu Lehui
|
1c21452d02 |
bpf: Fix UAF due to concurrent consumption of ttrace lists in alloc_bulk
Syzkaller repeatedly triggered UAF splats related to nodes in
waiting_for_gp_ttrace within the bpf memalloc:
BUG: KASAN: slab-use-after-free in llist_del_first+0x85/0x110 lib/llist.c:61
Read of size 8 at addr ffff8881572cd080 by task syz.4.470/5112
...
llist_del_first+0x85/0x110 lib/llist.c:61
alloc_bulk+0x193/0x460 kernel/bpf/memalloc.c:229
bpf_mem_refill+0x386/0x560 kernel/bpf/memalloc.c:436
Freed by task 14:
...
__free_rcu kernel/bpf/memalloc.c:281 [inline]
__free_rcu_tasks_trace+0x48/0xd0 kernel/bpf/memalloc.c:291
rcu_tasks_invoke_cbs+0x1ec/0x3e0 kernel/rcu/tasks.h:571
rcu_tasks_one_gp+0x13d/0x220 kernel/rcu/tasks.h:621
rcu_tasks_kthread+0xf3/0x120 kernel/rcu/tasks.h:651
The reason is that the UAF occurs after the RCU Tasks Trace GP expires:
when the __free_rcu() callback runs, there is no synchronization
protecting llist_del_all() against concurrent alloc_bulk() operating on
waiting_for_gp_ttrace, leading to the race condition below:
CPU0 CPU1
__free_rcu (RCU Tasks Trace callback)
alloc_bulk
llist_del_first(&c->waiting_for_gp_ttrace)
entry = smp_load_acquire(&head->first);
do {
if (entry == NULL)
return NULL;
free_all(llist_del_all(&c->waiting_for_gp_ttrace))
llist_for_each_safe(pos, t, llnode)
free_one(pos);
next = READ_ONCE(entry->next); <-- trigger UAF
} while (!try_cmpxchg(&head->first, &entry, next));
In addition, there is also a theoretical race condition on the
free_by_rcu_ttrace list. This race requires two preconditions: an
in-flight Tasks Trace GP keeping c->call_rcu_ttrace_in_progress == 1,
and concurrent cross-CPU frees repopulating c->free_by_rcu_ttrace with
new nodes. Under these conditions, the following scenario triggers UAF:
// CPU0
// irq work is still busy (on PREEMPT_RT)
alloc_bulk()
llist_del_first(&c->free_by_rcu_ttrace)
entry = smp_load_acquire(&head->first);
do {
if (entry == NULL)
return NULL;
// CPU1
bpf_mem_alloc_destroy()
WRITE_ONCE(c->draining, true)
// wait for CPU0
irq_work_sync()
// CPU2
do_call_rcu_ttrace(tgt(CPU0))
if (c->draining) {
llist_del_all(&c->free_by_rcu_ttrace)
free_all()
}
// CPU0 continue
next = READ_ONCE(entry->next); <-- trigger UAF
while (!try_cmpxchg(&head->first, &entry, next));
Fix this by introducing a raw spinlock to synchronize the concurrent
consumption on waiting_for_gp_ttrace and free_by_rcu_ttrace.
Fixes:
|
||
|
Linus Torvalds
|
22098763a1 |
tracing fixes for 7.3:
- Don't destroy user event fields when removal fails
User event fields are destroyed before the event is removed from
visibility. But that can fail leaving the still visible event with no
fields. Move the destroying of the fields to after the event is
successfully removed from visibility.
- Initialize function graph state is fork before calling copy_exec_state()
For non-CLONE_VM forks, copy_exec_state() allocates a new task_exec_state.
If that allocation fails, ftrace_graph_exit_task() will free the tasks
ret_stack pointer. Since that pointer is still using the parent's
ret_stack, it mistakenly frees the parent's pointer too.
Call ftrace_graph_init() on the task first which will NULL out the new
tasks's ret_stack and if the copy fails, it will not free anything.
- Remove FGRAPH_MAX_INDEX
The macro FGRAPH_MAX_INDEX was added but never used. Remove it.
- Save ent_size in function graph printing of nested functions
The function graph tracer needs to look at the next event to see if the
next event is the return of the current function entry. If it is, it
prints a single line:
ktime_get();
Otherwise it prints it like a nested function:
tick_nohz_irq_exit() {
ktime_get();
kcpustat_irq_exit();
}
In order to look at the next event, it must save the current event so that
it has the information to print from it. It saves the event in the
iterator descriptor called "ent". What it doesn't save is the ent_size of
the event which is now used to know if the function graph arguments are to
be printed. The peek doesn't save the size so the size used happens to be
that of the size of the last event that was seen.
Save the entry event size in the iterator descriptor so that the correct
size is used.
- Fix several errors with freeing data in the histogram code
The histogram code had a lot of leaked or or incorrect accounting when
failures happen. Correct them.
- Fix histogram regression of .percent and .graph modifiers
Up until 6.3 histogram values could have "percent" or "graph" modifiers
that changed how they were printed. But a change that added restricting
histograms values from being strings, stack traces and other modifiers
inadvertently prevented them from using the percent and graph modifiers,
which were legal use cases for values.
Put back the percent and graph modifiers.
- Fix various typos in the comments
- Set the trace_clock before initializing a histogram with clock argument
The histogram API allows the user to specific which trace clock to use via
a "clock=" string. The histogram is set up first before the clock is
checked. If the passed in clock is not valid, it exits without fully
fixing up the histogram leaving it on the list and a use-after-free can
trigger.
Update the clock argument first and if it fails then exit gracefully
before the histogram trigger is placed on any lists.
- Restore :mod: trailer after parsing in ftrace_set_clr_event
The function ftrace_set_clr_event() modifies the parse string and needs to
put it back to what was passed in. It searches for ":mod:" via a strsep()
but fails to put back the first ':' in the string.
Add back the ':' in the passed in string.
- Take trace_array reference when opening a tracer options file
The options files are dynamically created and some tracers add their own
options. When a tracer adds their own list of options, the trace_array
holding them has an array to hold the list of options for each tracer.
This array increases in size via a krealloc(), and the new entry gets a
newly allocated array to hold the options of the new tracer being added.
The element in each entry of the tracer's option array holds a pointer
back to the trace_array, a pointer to the tracer it is associated to, a
pointer to the flags of the option.
The issue is that these arrays are freed when the trace_array is freed
when its instance it represents is removed from the instances directory.
There's a race that an open of one of these options files can happen when
the instance is being removed.
Add a new helper function to be called by the open function of the options
file to iterate all existing trace_arrays under a lock and find the one
that has the given option element in one of it's tracer arrays. If found,
then update the associated trace_array's reference counter to keep it from
being freed. If not found, have the open call return -ENODEV.
- Disable interrupts when acquiring the lock in rb_wake_up_waiters()
The function rb_wake_up_waiters() assumes it will be called in interrupt
context and does not disable irqs when taking cpu_buffer->reader_lock,
which can be called in hard interrupt context. The issue is in PREEMPT_RT,
this function is called in thread context leaving this lock open to a
deadlock.
Take the lock with interrupts disabled.
- Use rcu_assign_pointer() for tmp_ops filter hash
The tmp_ops used in update_ftrace_direct_mod() assigns its filter_hash
field directly, but that field is annotated as __rcu and sparse complains.
Assign it with rcu_assign_pointer()
- Fix use-after-free in enable_trigger_private_data_free()
The trace_event_call is accessed through the event_trigger_data's
trace_event_file pointer to put the trace_event_call on freeing. The issue
is that the trace_event_file data may have been freed already causing a
use-after-free. Add a field to the event_trigger_data that points directly
to the trace_event_call so that it can decrement its reference directly
without needing to go through the trace_event_file.
- Fix accounting of buffer data remote headers
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount the
number of pages is needed for the asked for size as it doesn't take into
account the meta data on each page. Add a helper function to do the
calculation properly and use that in these functions.
- Catch nr_page_va overflow in ring_buffer_desc sizing
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than what
was asked for.
- Do not resize the subbuf order if any per_cpu buffer is disabled
The mmapping of ring buffers disables resizing the subbuffers, but it is
done per-cpu whereas the subbuf size change is done for all the per_cpu
buffers under the buffer->mutex. It could change the size of some while
the mapping is happening on others. Have the resize of the subbuf order
check all the per_cpu buffers under the lock to see if any of them is
disabled before starting and causing an inconsistency between buffers that
are being mapped.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCaqbdrBQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qro9AQDF/j3VW3Uu98lVFI9AB10XYhLDd5nt
Zpf+3RviNgFpxgEAiE2+4K+4sM2SfaDDh9JMww9MKg1exL+cemE3a+JbBgY=
=jgYE
-----END PGP SIGNATURE-----
Merge tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing fixes from Steven Rostedt:
- Don't destroy user event fields when removal fails
User event fields are destroyed before the event is removed from
visibility. But that can fail leaving the still visible event with no
fields. Move the destroying of the fields to after the event is
successfully removed from visibility.
- Initialize function graph state is fork before calling
copy_exec_state()
For non-CLONE_VM forks, copy_exec_state() allocates a new
task_exec_state. If that allocation fails, ftrace_graph_exit_task()
will free the tasks ret_stack pointer. Since that pointer is still
using the parent's ret_stack, it mistakenly frees the parent's
pointer too.
Call ftrace_graph_init() on the task first which will NULL out the
new tasks's ret_stack and if the copy fails, it will not free
anything.
- Remove FGRAPH_MAX_INDEX
The macro FGRAPH_MAX_INDEX was added but never used. Remove it.
- Save ent_size in function graph printing of nested functions
The function graph tracer needs to look at the next event to see if
the next event is the return of the current function entry. If it is,
it prints a single line:
ktime_get();
Otherwise it prints it like a nested function:
tick_nohz_irq_exit() {
ktime_get();
kcpustat_irq_exit();
}
In order to look at the next event, it must save the current event so
that it has the information to print from it. It saves the event in
the iterator descriptor called "ent". What it doesn't save is the
ent_size of the event which is now used to know if the function graph
arguments are to be printed. The peek doesn't save the size so the
size used happens to be that of the size of the last event that was
seen.
Save the entry event size in the iterator descriptor so that the
correct size is used.
- Fix several errors with freeing data in the histogram code
The histogram code had a lot of leaked or or incorrect accounting
when failures happen. Correct them.
- Fix histogram regression of .percent and .graph modifiers
Up until 6.3 histogram values could have "percent" or "graph"
modifiers that changed how they were printed. But a change that added
restricting histograms values from being strings, stack traces and
other modifiers inadvertently prevented them from using the percent
and graph modifiers, which were legal use cases for values.
Put back the percent and graph modifiers.
- Fix various typos in the comments
- Set the trace_clock before initializing a histogram with clock
argument
The histogram API allows the user to specific which trace clock to
use via a "clock=" string. The histogram is set up first before the
clock is checked. If the passed in clock is not valid, it exits
without fully fixing up the histogram leaving it on the list and a
use-after-free can trigger.
Update the clock argument first and if it fails then exit gracefully
before the histogram trigger is placed on any lists.
- Restore :mod: trailer after parsing in ftrace_set_clr_event
The function ftrace_set_clr_event() modifies the parse string and
needs to put it back to what was passed in. It searches for ":mod:"
via a strsep() but fails to put back the first ':' in the string.
Add back the ':' in the passed in string.
- Take trace_array reference when opening a tracer options file
The options files are dynamically created and some tracers add their
own options. When a tracer adds their own list of options, the
trace_array holding them has an array to hold the list of options for
each tracer. This array increases in size via a krealloc(), and the
new entry gets a newly allocated array to hold the options of the new
tracer being added.
The element in each entry of the tracer's option array holds a
pointer back to the trace_array, a pointer to the tracer it is
associated to, a pointer to the flags of the option.
The issue is that these arrays are freed when the trace_array is
freed when its instance it represents is removed from the instances
directory. There's a race that an open of one of these options files
can happen when the instance is being removed.
Add a new helper function to be called by the open function of the
options file to iterate all existing trace_arrays under a lock and
find the one that has the given option element in one of it's tracer
arrays. If found, then update the associated trace_array's reference
counter to keep it from being freed. If not found, have the open call
return -ENODEV.
- Disable interrupts when acquiring the lock in rb_wake_up_waiters()
The function rb_wake_up_waiters() assumes it will be called in
interrupt context and does not disable irqs when taking
cpu_buffer->reader_lock, which can be called in hard interrupt
context. The issue is in PREEMPT_RT, this function is called in
thread context leaving this lock open to a deadlock.
Take the lock with interrupts disabled.
- Use rcu_assign_pointer() for tmp_ops filter hash
The tmp_ops used in update_ftrace_direct_mod() assigns its
filter_hash field directly, but that field is annotated as __rcu and
sparse complains. Assign it with rcu_assign_pointer()
- Fix use-after-free in enable_trigger_private_data_free()
The trace_event_call is accessed through the event_trigger_data's
trace_event_file pointer to put the trace_event_call on freeing. The
issue is that the trace_event_file data may have been freed already
causing a use-after-free. Add a field to the event_trigger_data that
points directly to the trace_event_call so that it can decrement its
reference directly without needing to go through the
trace_event_file.
- Fix accounting of buffer data remote headers
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount
the number of pages is needed for the asked for size as it doesn't
take into account the meta data on each page. Add a helper function
to do the calculation properly and use that in these functions.
- Catch nr_page_va overflow in ring_buffer_desc sizing
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than
what was asked for.
- Do not resize the subbuf order if any per_cpu buffer is disabled
The mmapping of ring buffers disables resizing the subbuffers, but it
is done per-cpu whereas the subbuf size change is done for all the
per_cpu buffers under the buffer->mutex. It could change the size of
some while the mapping is happening on others. Have the resize of the
subbuf order check all the per_cpu buffers under the lock to see if
any of them is disabled before starting and causing an inconsistency
between buffers that are being mapped.
* tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (25 commits)
ring-buffer: Check resize_disabled before publishing the new subbuf order
tracing/remotes: Catch nr_page_va overflow in ring_buffer_desc sizing
tracing/remotes: Account for ring buffer page header in size calculation
tracing: Don't dereference trace_event_file in deferred trigger free
ftrace: Use rcu_assign_pointer() for tmp_ops filter hash
ring-buffer: Acquire the lock with irqsave in rb_wake_up_waiters()
tracing: Take trace_array reference when opening a tracer options file
tracing: Fix ring_buffer_read_page_size() kernel-doc
tracing: Restore :mod: trailer after parsing in ftrace_set_clr_event()
tracing: Fix memory corruption from a "STACKTRACE" histogram key
tracing: Fix memory corruption from the histogram stacktrace modifier
tracing: Undo the registration when enabling the histogram trigger fails
tracing: Take the reference before publishing the named histogram trigger
tracing: Set the trace clock before registering the histogram trigger
tracing: Fix typo "preceeded" in comment
tracing: Fix typo "availabe" in comment
tracing: Let histogram values keep the percent and graph modifiers
tracing: Keep the entry count when the histogram stats allocation fails
tracing: Free histogram the field rejected for a bad modifier
tracing: Free histogram the var ref when its initialization fails
...
|
||
|
David Carlier
|
d860c67c05 |
ring-buffer: Check resize_disabled before publishing the new subbuf order
ring_buffer_subbuf_order_set() stores the new order and only then walks
the CPUs, returning -EBUSY if any of them has resizing disabled. A user
mapped buffer has resizing disabled, and __rb_map_vma() reads
buffer->subbuf_order without buffer->mutex, so an mmap of an already
mapped CPU racing the failing order change sizes the mapping with the
new order and inserts pages past the sub-buffer into the VMA.
Check the CPUs before storing the new order.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Vincent Donnefort
|
d059d8bf2c |
tracing/remotes: Catch nr_page_va overflow in ring_buffer_desc sizing
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than
what was asked for.
Return SIZE_MAX from trace_buffer_desc_size() on nr_page_va overflow.
Link: https://patch.msgid.link/20260911193937.602202-3-vdonnefort@google.com
Fixes:
|
||
|
Vincent Donnefort
|
442ffa742d |
tracing/remotes: Account for ring buffer page header in size calculation
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount the
required pages because every ring buffer page contains a header
(BUF_PAGE_HDR_SIZE). Account for that header to ensure allocated remote
ring buffers aren't smaller than requested by the user.
The newly introduced helper __calc_nr_pages_ring_buffer_desc() can
return a value that overflows the descriptor nr_pages field (32 bits).
Link: https://patch.msgid.link/20260911193937.602202-2-vdonnefort@google.com
Fixes:
|
||
|
Linus Torvalds
|
c874ace034 |
Misc timer fixes:
- Fix clockevents replacement race when a broadcast
device is replaced which may trigger a BUG() crash
(朱恺乾 - Zhu Kaiqian)
- Fix potential timerqueue ordering bug when rearming
a queued timer with nonzero slack (Andrea Parri)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqmXaERHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1jM9RAAnEShNuh27uYj3oxVgaLo+Dhsk1AYzNnr
WVS/Cz8HfFXMEVnOsT3CibcB6p5wydTHms/8248GZWWUuM1HtL/7zcUHWGFgs72S
WRTcw/ouzvdAKQfxlH2j96uApMWwnWnv9XRfpFel9bgIIK1POL9g0JJmcuFK5uNf
5aYvzkdLv3SKHR0BnrIF4a6jqq1Shf2sZPDXHmJDE/k/He28zvFjiOlIEChxfCfh
qUEzY036hSU0RbAONSbn88bj7dc10/Xuck/iW3WVW8cOqtVxw79biAoumUe9jqwE
hX3B6rvDEPvaEOPDm2PgUlrapFukjfImu7K9rDljbFMX1jF6eb7ZQMk4ftJXL+rM
M0RPCdrS2ZrVOKt3VFIYRH7ZzFNwtE+RHPZSD6lpVgia6xpgi6yY++AzTeCn+VpK
3AmkxMg3xHOLkISyCRUlmtTn3Cis6O7+9+9dEad24dh5mkQM7Tr6nzprYeg3fgpR
z714UKjOUvBNBtxCjdZl5/c/i8mb0IaH4DmT+/V6mIXWoHchbqgw0Or7G7XMm5XM
M1J+4RJrGhhg1eUTb254PWi/OixuXZ8XgcB1wwAiJFMTJY9YqBExKRAdZ+Al1DJP
FgHDEPyvElpeh5XFFqf8Ft9xXOTn1CSEc6G+dCO0MswcikHnw4UnmLGn9z3js2AS
SRlK75Dm49s=
=SyXt
-----END PGP SIGNATURE-----
Merge tag 'timers-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fixes from Ingo Molnar:
- Fix clockevents replacement race when a broadcast
device is replaced which may trigger a BUG() crash
(朱恺乾 - Zhu Kaiqian)
- Fix potential timerqueue ordering bug when rearming
a queued timer with nonzero slack (Andrea Parri)
* tag 'timers-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
hrtimer: Use hard expiry when updating timers on the same base
tick/broadcast: Plug clockevents replacement race
|
||
|
Linus Torvalds
|
b2a8a7669e |
Miscellaneous scheduler fixes:
- Fix EEVDF se->max_slice value on enqueueing (Vincent Guittot)
- Fix EEVDF augmented rb-trees re-balancing with
multiple fields (Vincent Guittot)
- In proxy scheduling, account cgroup CPU time to the execution
context, not the scheduling context (Hui Su)
- Likewise, call wq_worker_tick() for the execution context,
not the scheduling context (Hui Su)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqmXEMRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hchg/6A1gRkn7T5+K957U8wpB9vjtV9cKVdWpI
XFDGm60ylFUmU388Xb8mmrbDgmej6RpX6C4ccppygM3196w513tB+Zr8w6jbSszk
ddgWwfwi58FFBJZTH7JDqeJ64wvrl8KId44yM6k2JdXATxh2DGF0w+YdsA+M5HVJ
EJbjACYhePdK27wvQDtj1poDfAyiabqEnv7w62dhEU9I+ikmcPAyrhmqU0yFDNUR
sNozsDQnEJrHtllGHpr3FVxYRqob6lOtG+86VSiZ8F6i2kA3p/451mpMyyCOMUrF
kZlBIryLG0gylXIensqLox+z2ZIE4nUL0OX3o7mC+MLNERdWvsdgHi9AcZlIoFpJ
wMPBLENnnGbilmwhXjk0pL655rlVVUGwaTV4T9Pk5D5qew6B9LGe8uqiLo8U/qih
1o5Lf3ZnUi0o8XHMfNkwQ3Y0m1S7CbgJYKItE+ec+2QifmKGD5dOKo5WWDKszywF
Zb9ScP2fMKideS/JEX1/+jvLcpDmV+HE3mC58Muek96fbJG1IQ4bL5tu+fqO85rM
68dJymtMrkoeegmq4jqERt758sZnyv2QbDmr1Kc1G9vSKNAeahYldPU2gQ6qFBnO
VCMvI2BUmfiia9A6NBKKgpjijc4zWydyuaQ2zdvEBddYnLWKZ5Ge26lLrjraNtdB
GtHalAP1gVs=
=zJHp
-----END PGP SIGNATURE-----
Merge tag 'sched-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fixes from Ingo Molnar:
- Fix EEVDF se->max_slice value on enqueueing (Vincent Guittot)
- Fix EEVDF augmented rb-trees re-balancing with multiple
fields (Vincent Guittot)
- In proxy scheduling, account cgroup CPU time to the execution
context, not the scheduling context (Hui Su)
- Likewise, call wq_worker_tick() for the execution context,
not the scheduling context (Hui Su)
* tag 'sched-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/core: Call wq_worker_tick() for the execution context
sched: Account cgroup CPU time to the execution context
sched/eevdf: Fix rb augmented with multi fields
sched/eevdf: Fix augmented max_slice
|
||
|
fangqiurong
|
c7a1c6e800 |
sched_ext: Close the pre-enable ops error claim window
scx_alloc_and_add_sched() publishes ops->priv before
scx_root_enable_workfn() switches the state to SCX_ENABLING. An error
claimed via scx_bpf_error_bstr() from an associated BPF program in that
window is consumed by scx_disable_workfn(), which takes the pre-enable
shortcut in scx_root_disable(). The shortcut returns without any teardown
and restores SCX_DISABLED with an unconditional scx_set_enable_state() xchg
racing the enable workfn's own transition. The enable then completes with
the claim consumed: the scheduler stays up but can never be disabled again,
and bpf_scx_unreg() frees it while still in use, resulting in a
use-after-free. Both WARN_ON_ONCE()s fire back to back:
WARNING: kernel/sched/ext/ext.c:7522 at
scx_root_enable_workfn+0xeec/0x1be0, CPU#3: scx_enable_help/276
WARNING: kernel/sched/ext/ext.c:6398 at scx_root_disable+0xb50/0xdb8,
CPU#0: sched_ext_helpe/664
scx_root_enable_workfn() switches to SCX_ENABLING before the scheduler
allocation, so ops->priv is never visible while SCX_DISABLED. The allocation
failure path restores SCX_DISABLED.
Fixes:
|
||
|
Linus Torvalds
|
85855f85de |
Miscellaneous perf events fixes:
- Fix sched_cb_list corruption on PMU callbacks that
invoke list_del() during perf_event_overflow()
calls (Thomas Richter)
- Fix PEBS pt_regs->flags snapshot data that was
regressed with the introduction of adaptive
PEBS v4 support (Dapeng Mi)
- Fix possible drain_pebs() re-entry bug
when intel_pmu_drain_pebs_buffer() is called from
process context (Dapeng Mi)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqmWoURHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hzhA//TBu2jx/c2r5tafuW5mpPJH+3eGcMWUh9
KfVLt/FMzgt+sOtzM/eIQ2Sk4VAachTkSLjh6MQR1m9g4jXTVr5rH8lJlFXqW/74
v5XhaEryXTSOn6zpxXpKrVGYFq6RfdTijtmrAK+7ac6HChgRrMa0Eb5yVvPauLXE
+Y0RugHjF75c4iXapb68osWF+7EoVKGqLPZjdQ12D6wga7+1DRTZWV37hOXHtu1M
GbajBMTKF5Q4QCffsyY9PDks86dKLDrv8z7XxGNYz4pdcnwD4bMdYfu8FxohowaO
EXx9b0dxK1RobcsMo+wHZojTD0i4ySSIbN7lJU6dHopMFJ3nKgCDYuPvzov9T/Dm
8eFrzlQbr0m9+NqndFomjv4so1WGF1Q2DHqEOJQgSeHNECpUAx7mzIzv/AJiJm1P
AC3S8PJT5+AFQZtySqV6nI8UyzyMgoDo3EYdp3oKHq/B6SGDlAUHr3k8e5bpI8ss
JbIvyo1RH6DrB+FstHHve7zn5ueYmjxPQRS8NdIRrBKAeMryC7DYEDU7ZUwkAJS9
jJZE7wM0zhLbF5EPVEA3+rhrh1wZ0mUFsWJwWbn7QwIpSm331J8AEKdpKFa3k5op
FH9PusGL/SKeVwo1nReExLJWghdlxhbHAC8QhLMHTKOrhoqoUzpp5hRQ9rgx5z7N
hksCg2xZC+U=
=mKBo
-----END PGP SIGNATURE-----
Merge tag 'perf-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar
- Fix sched_cb_list corruption on PMU callbacks that
invoke list_del() during perf_event_overflow()
calls (Thomas Richter)
- Fix PEBS pt_regs->flags snapshot data that
regressed with the introduction of adaptive
PEBS v4 support (Dapeng Mi)
- Fix possible drain_pebs() re-entry bug when
intel_pmu_drain_pebs_buffer() is called from
process context (Dapeng Mi)
* tag 'perf-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf/x86/intel: Prevent drain_pebs() reentry
perf/x86/intel: Correct pt_regs->flags update for PEBS path
perf/core: Allow list_del during perf_event_overflow()
|
||
|
Jose Fernandez (Anthropic)
|
85136bf224 |
bpf: Avoid soft lockup in __htab_map_lookup_and_delete_batch()
__htab_map_lookup_and_delete_batch() has no rescheduling point. The batch count bounds how many entries are copied out, not how many buckets are visited, so one BPF_MAP_LOOKUP_BATCH call can walk the map end to end. The empty-bucket fast path is worse: it stays inside a single rcu_read_lock() / bpf_disable_instrumentation() section for any run of consecutive empty buckets. That holds up on small maps, but it falls apart at scale. On a 144-CPU arm64 host running a CONFIG_PREEMPT_NONE kernel, periodic BPF_MAP_LOOKUP_BATCH calls against an LRU hash map with 16,777,216 buckets held a CPU inside the batch op for 77+ seconds and triggered the soft lockup watchdog. Commit |
||
|
Ali Ahmet Memiş
|
bcfe2816e6 |
tracing: Don't dereference trace_event_file in deferred trigger free
The enable_event trigger defers trace_event_put_ref() to the
trigger free kthread, but the trace_event_file can already be freed
when the instance is removed.
Keep the trace_event_call directly in enable_trigger_data so the
deferred free does not access the freed trace_event_file.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Leon Hwang
|
b4dcc18b97 |
ftrace: Use rcu_assign_pointer() for tmp_ops filter hash
tmp_ops.func_hash->filter_hash is annotated __rcu, but
update_ftrace_direct_mod() assigns hash to it directly. Sparse reports an
address-space mismatch.
Use rcu_assign_pointer() for the assignment.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260911142512.19344-1-leon.hwang@linux.dev
Fixes:
|
||
|
Siddharth Chintamaneni
|
0d7823cd4c |
bpf: Allow terminal gotox instructions
check_subprogs() treats gotox as a direct jump and validates its reserved
zero offset. When gotox is the final instruction, this produces a
synthetic successor one instruction past the end of the subprogram and
rejects an otherwise valid program.
Skip direct-offset validation for gotox and accept it as a
non-fallthrough terminal instruction. Its actual targets remain validated
from the instruction-array jump table during CFG construction.
Fixes:
|
||
|
Sebastian Andrzej Siewior
|
815e07c8fe |
ring-buffer: Acquire the lock with irqsave in rb_wake_up_waiters()
rb_wake_up_waiters() is a irq_work callback which is initialized with
init_irq_work(). As such it will be invoked in thread context on
PREEMPT_RT. Invoking the callback in IRQ context on PREEMPT_RT is not an
option due its usage of wake_up_all(). Since this callback may run in
thread context, it needs to acquire ring_buffer_per_cpu::reader_lock with
disabling interrupts and may not assume that they are disabled.
Use raw_spinlock_irqsave() to acquire ring_buffer_per_cpu::reader_lock.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260911102152.YEtwkBj9@linutronix.de
Fixes:
|
||
|
Steven Rostedt
|
ed0aff60f8 |
tracing: Take trace_array reference when opening a tracer options file
When a tracer option file is opened, it is passed a descriptor that points
to an element on the trace_array's topts array. This element has
information to find the trace array and other information. It uses this
element to take a reference of the trace_array so that the trace_array
does not get removed while this file is opened.
Unfortunately, there's a race condition where the element itself could be
freed by the removal of the instance the trace_array represents causing a
use-after-free as this element that is used to find the trace_array to
increment its reference counter is also freed when the instance is
removed.
To solve this, add a trace_array_tracer_options_get() helper function that
will take the address of the element that is passed to the open function
by the inode->i_private pointer and search all the trace_arrays under a
lock to find the one that the element's address is in the range of the
trace_arrays topts array elements. When a match happens, that trace_array's
reference would be increased.
Note, there's a race where if an admin was deleting and creating trace
instances at the same time and the memory of the old trace_array's array
matched the memory of the new trace_array that it could in theory open the
option from the wrong trace array. But we do not care because it would be
stupid to perform that kind of action. As long as the only thing that can
happen is that the option from the wrong trace array is used and doesn't
crash the kernel it will only make the user confused. But if they are
doing something stupid like this, they are already confused, so no harm
done.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260910221209.62dad8d3@robin
Fixes:
|
||
|
Karl Mehltretter
|
7e645147df |
tracing: Fix ring_buffer_read_page_size() kernel-doc
ring_buffer_read_page_size() takes a parameter named rpage, but its
kernel-doc describes page. As a result, kernel-doc reports rpage as
undescribed and page as an excess parameter description.
Rename the documentation entry to match the function.
Link: https://patch.msgid.link/20260909062917.89482-1-kmehltretter@gmail.com
Fixes:
|
||
|
Thomas Weißschuh
|
911002e99e |
tracing: Restore :mod: trailer after parsing in ftrace_set_clr_event()
While ftrace_set_clr_event() modifies its input buffer during parsing,
before returning to the caller the buffer is supposed to be restored
to its original state.
This works correctly for the colon between the subsystem and event
but not the colon at the beginning of :mod:.
Restore the colon, so the :mod: trailer is not stripped after
ftrace_set_clr_event().
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Donggeun Yoo
|
7f711e6235 |
tracing: Fix memory corruption from a "STACKTRACE" histogram key
"cpu", "CPU", "stacktrace" and "STACKTRACE" are generic fields, defined
with an offset and a size of zero so that the filter code can match them
by name. parse_field() maps them onto their common_* equivalents for
backward compatibility, but unlike the common_* names it hands the
placeholder back to the caller instead of NULL.
create_hist_field() takes a non-NULL field as a promise that the record
carries a stacktrace and picks HIST_FIELD_FN_STACK, so the __data_loc
word is read from offset 0, that is from common_type, and its low 16
bits are followed as an offset into the record. What is found there
becomes the length of an unbounded memcpy. Pick an event whose id is
small enough that the offset stays inside its own record and the length
is a kernel text address:
# cd /sys/kernel/tracing
# echo 'hist:keys=STACKTRACE' > events/ftrace/print/trigger
# echo hello > trace_marker
Oops: general protection fault, probably for non-canonical address
RIP: 0010:rb_next+0x23/0x60
</IRQ>
RIP: 0010:memcpy+0xc/0x30
event_hist_trigger+0x2e7/0x12c0
Kernel panic - not syncing: Fatal exception in interrupt
Leave the field NULL, which is what the comment above the branch says
the code does and what common_stacktrace already does. FILTER_CPU and
FILTER_COMM are left alone, their create_hist_field() branches never
look at the field.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Donggeun Yoo
|
a5e70ba87c |
tracing: Fix memory corruption from the histogram stacktrace modifier
parse_field() sets HIST_FIELD_FL_STACKTRACE from the ".stacktrace"
modifier before it looks the field name up, and nothing afterwards
checks that the name resolved to a field which holds a stacktrace.
create_hist_field() picks HIST_FIELD_FN_STACK on the strength of the
field pointer alone, which reads a __data_loc word from the record and
follows its low 16 bits as an offset into the same record.
event_hist_trigger() takes the first word there as an entry count and
copies that many longs into a 31 entry array:
n_entries = *stack;
memcpy(entries, ++stack, n_entries * sizeof(unsigned long));
Neither end of that copy is bounded, and the count is whatever the event
holds at the offset, so any field will do:
# cd /sys/kernel/tracing/events/sched/sched_process_fork
# echo 'hist:keys=parent_pid.stacktrace' > trigger
# (true)
BUG: kernel NULL pointer dereference, address: 0000000000000008
RIP: 0010:rb_insert_color+0x18/0x130
timerqueue_linked_add+0x7e/0xd0
enqueue_hrtimer+0x39/0xb0
__hrtimer_run_queues+0x10f/0x1f0
</IRQ>
RIP: 0010:memcpy+0xc/0x30
event_hist_trigger+0x165/0x690
The timer interrupt landed on the rbtree the copy had already run over.
No debug options are needed for this; KASAN reports the same write as an
out-of-bounds read of 13835058055416381440 bytes.
Documentation/trace/histogram.rst already states the rule, "must be a
long[] type", so enforce it once the name has been resolved. Names which
resolve to no field at all, "hitcount.stacktrace" and the common_*
pseudo-fields, are refused for the same reason: they hold no stacktrace
to read.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Donggeun Yoo
|
92383cef66 |
tracing: Undo the registration when enabling the histogram trigger fails
Commit |
||
|
Donggeun Yoo
|
0fe23b8eab |
tracing: Take the reference before publishing the named histogram trigger
event_hist_trigger_named_init() puts the trigger on the global
named_triggers list and only then takes the reference on the trigger it
shares its histogram with:
data->ref++;
save_named_trigger(data->named_data->name, data);
ret = event_hist_trigger_init(data->named_data);
if (ret < 0) {
kfree(data->cmd_ops);
data->cmd_ops = &trigger_hist_cmd;
}
return ret;
event_hist_trigger_init() fails when alloc_hist_pad() cannot allocate, and
nothing takes the trigger back off the list on the way out.
event_hist_trigger_parse() frees it, and the next lookup by name reads the
freed object:
BUG: KASAN: slab-use-after-free in find_named_trigger+0xac/0xc0
Read of size 8 at addr ffff888009346860 by task init/1
find_named_trigger+0xac/0xc0
hist_register_trigger+0xc1/0xa00
event_hist_trigger_parse+0x3146/0x6af0
event_trigger_write+0xce/0x160
Freed by task 67:
kfree+0x154/0x420
trigger_kthread_fn+0xfd/0x160
Do the reference first and publish once it has succeeded, so that nothing
which can fail runs after the trigger becomes findable.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Donggeun Yoo
|
6ede78d056 |
tracing: Set the trace clock before registering the histogram trigger
hist_register_trigger() puts the trigger on the global named_triggers
list in cmd_ops->init(), and only then sets the trace clock:
if (data->cmd_ops->init) {
ret = data->cmd_ops->init(data);
if (ret < 0)
goto out;
}
if (hist_data->enable_timestamps) {
ret = tracing_set_clock(file->tr, hist_data->attrs->clock);
if (ret) {
hist_err(tr, HIST_ERR_SET_CLOCK_FAIL, errpos(clock));
goto out;
}
The clock string is not checked anywhere before that call, so a named
trigger using common_timestamp with an unknown clock fails after it has
already become findable. event_hist_trigger_parse() then frees it
without taking it off the list, and the next lookup by name reads the
freed object:
~# cd /sys/kernel/tracing/events/sched/sched_switch
~# echo 'hist:name=foo:keys=common_pid:ts=common_timestamp:clock=bogus' > trigger
bash: echo: write error: Invalid argument
~# echo 'hist:name=foo:keys=common_pid' > trigger
BUG: KASAN: slab-use-after-free in find_named_trigger+0xac/0xc0
Read of size 8 at addr ffff88800915d760 by task init/1
find_named_trigger+0xac/0xc0
hist_register_trigger+0xc1/0x900
event_hist_trigger_parse+0x3146/0x6af0
event_trigger_write+0xce/0x160
Freed by task 63:
kfree+0x154/0x420
trigger_kthread_fn+0xfd/0x160
Set the clock before the trigger is registered, so that nothing which
can fail runs after it is published, the way commit
|
||
|
Hemanth Selam
|
89b000ba07 |
tracing: Fix typo "availabe" in comment
Correct "availabe" to "available", reported by scripts/checkpatch.pl using the misspelling list in scripts/spelling.txt. Only touches comments, no code changes. Link: https://patch.msgid.link/20260907062608.13924-1-hemanth.selam@gmail.com Assisted-by: Cursor:claude-opus-5 Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org> |
||
|
Donggeun Yoo
|
3d617bfd79 |
tracing: Let histogram values keep the percent and graph modifiers
The .percent and .graph modifiers exist only for histogram values, but a
value carrying either of them has been rejected since v6.3. The example
in Documentation/trace/histogram.rst,
# echo 'hist:keys=prev_comm:vals=hitcount.percent:nohitcount' > \
events/sched/sched_switch/trigger
returns -EINVAL.
parse_field() sets the two flags only when the field is neither a key nor
a variable, that is, only on a value:
} else if (strncmp(modifier, "percent", 7) == 0) {
if (*flags & (HIST_FIELD_FL_VAR | HIST_FIELD_FL_KEY))
goto error;
*flags |= HIST_FIELD_FL_PERCENT;
__create_val_field() then rejects a value for carrying them, so no field
can reach hist_trigger_print_val(), where both are implemented.
commit
|
||
|
Donggeun Yoo
|
06f5634ec5 |
tracing: Keep the entry count when the histogram stats allocation fails
print_entries() uses n_entries both as the number of sort entries and as
its own return value, so the -ENOMEM it stores when the stats allocation
fails overwrites the count that the cleanup still needs:
n_entries = tracing_map_sort_entries(map, ...);
if (n_entries < 0)
return n_entries;
...
if (!stats) {
n_entries = -ENOMEM;
goto out;
}
...
out:
tracing_map_destroy_sort_entries(sort_entries, n_entries);
tracing_map_destroy_sort_entries() takes an unsigned int and loops up to
it, so -ENOMEM arrives as 4294967284. It walks an array of at most
map->max_elts pointers and calls destroy_sort_entry(), which dereferences
and frees, on whatever lies past the end.
Reading the hist file of a trigger with a .percent value, with that
allocation forced to fail:
BUG: KASAN: vmalloc-out-of-bounds in tracing_map_destroy_sort_entries+0xa0/0xb0
Read of size 8 at addr ffffc90000045000 by task init/1
tracing_map_destroy_sort_entries+0xa0/0xb0
hist_show+0x6f7/0x1df0
seq_read_iter+0x2b8/0x1190
vfs_read+0x176/0xa40
The buggy address belongs to a 4-page vmalloc region starting at
ffffc90000041000 allocated at tracing_map_sort_entries+0x5c/0xd50
A few pages further the fault is fatal. The registers at the oops confirm
the bound: the loop's end pointer less the array start, over the pointer
size, is 4294967284.
Return the error in a separate variable and leave n_entries holding the
count, the way tracing_map_sort_entries() does on its own error path.
The stats block is only entered for a value carrying .percent or .graph,
which __create_val_field() has rejected since v6.3, so this cannot be
reached in mainline as it stands. It becomes reachable again with
"tracing: hist: let values keep the percent and graph modifiers", so it
should be applied first.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Donggeun Yoo
|
230234d12c |
tracing: Free histogram the field rejected for a bad modifier
Writing a hist trigger whose value or variable carries a modifier that is
not allowed there leaks the fields that were built for it.
__create_val_field() takes the field from parse_expr() and stores it in
hist_data->fields[] only after the modifier checks have run:
hist_field = parse_expr(hist_data, file, field_str, flags, var_name,
&n_subexprs);
...
if (hist_field->flags & HIST_FIELD_FL_VAR) {
if (hist_field->flags & (...))
goto err;
} else {
if (hist_field->flags & (...))
goto err;
}
hist_data->fields[val_idx] = hist_field;
Both checks jump past that store, and the err label returns without
freeing anything. The error unwinds to create_hist_data(), which calls
destroy_hist_data() -> destroy_hist_fields(), and that reaches a field
only by walking fields[]. A field that never got there is unreachable.
commit
|
||
|
Donggeun Yoo
|
516001d53e |
tracing: Free histogram the var ref when its initialization fails
create_var_ref() allocates a VAR_REF hist_field and then calls init_var_ref() to fill it in. When that fails the field is leaked. commit |
||
|
Donggeun Yoo
|
4bddcb346a |
tracing: Free histogram var refs regardless of how often they are referenced
Using the same variable three or more times in one hist trigger leaks the variable reference and its strings when the trigger is removed. commit |
||
|
Donggeun Yoo
|
0701995aaf |
function_graph: Use the saved entry's size when reprinting it
When a graph entry does not fit in the trace_seq, print_graph_entry()
saves it in the iterator's fgraph_data and reprints it on the next read.
The entry has already been consumed from the ring buffer by then, so the
copy is all that is left of it.
The copy is sized with iter->ent_size, which no longer describes the
saved entry but whatever entry the iterator has moved on to. The
argument count is derived from the same field, so a 72 byte entry saved
and then reprinted ahead of a 48 byte return entry loses its arguments.
Record the size next to the failure flag, so that the two are always set
together, and restore it before reprinting.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Donggeun Yoo
|
b228454870 |
fgraph: Remove unused FGRAPH_MAX_INDEX
FGRAPH_MAX_INDEX has no user, and it expands to FGRAPH_INDEX_SIZE and
FGRAPH_RET_INDEX, neither of which is defined anywhere in the tree. It
was added in that form by commit
|
||
|
Jérémy Jean
|
08cacffeef |
ftrace: fork: Initialize function graph state before copy_exec_state()
dup_task_struct() copies the parent's task_struct, including ret_stack.
ftrace_graph_init_task() clears the copied function graph state, but it
currently runs after copy_exec_state().
For non-CLONE_VM forks, copy_exec_state() allocates a new task_exec_state.
If that allocation fails, copy_process() reaches bad_fork_free and
free_task() calls ftrace_graph_exit_task(). Since the child still carries
the parent's ret_stack pointer, the unwind frees the parent's active
function graph return stack. The parent subsequently accesses freed memory
from function_graph_enter_regs().
KASAN reports:
[ 22.190920] ==================================================================
[ 22.195899] BUG: KASAN: slab-use-after-free in function_graph_enter_regs+0xa76/0xb90
[ 22.200747] Write of size 8 at addr ff110000054dc0a8 by task repro/1
[ 22.205134]
[ 22.210770] CPU: 0 UID: 0 PID: 1 Comm: repro Not tainted 7.2.0-07732-g9328b3b03bdc-dirty #3 PREEMPT(lazy)
[ 22.212576] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[ 22.213750] Call Trace:
[ 22.215271] <TASK>
[ 22.216242] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.217774] dump_stack_lvl+0x4e/0x70
[ 22.220531] print_report+0x157/0x4b4
[ 22.223202] ? fixup_red_left+0x9/0x30
[ 22.224407] ? complete_report_info+0x83/0x110
[ 22.226679] ? function_graph_enter_regs+0xa76/0xb90
[ 22.228084] kasan_report+0xce/0x100
[ 22.230109] ? function_graph_enter_regs+0xa76/0xb90
[ 22.232860] ? stack_trace_save+0x4/0xd0
[ 22.234156] function_graph_enter_regs+0xa76/0xb90
[ 22.236090] ? kasan_save_stack+0x30/0x50
[ 22.237752] ? __pfx_function_graph_enter_regs+0x10/0x10
[ 22.238694] ? ring_buffer_lock_reserve+0x345/0xf80
[ 22.239628] ? stack_trace_save+0x4/0xd0
[ 22.242121] ? stack_trace_save+0x4/0xd0
[ 22.243588] ftrace_graph_func+0xda/0x160
[ 22.245362] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.246520] 0xffffffffa0000095
[ 22.250528] ? stack_trace_save+0x9/0xd0
[ 22.251757] ? ring_buffer_unlock_commit+0x11d/0x5c0
[ 22.253152] stack_trace_save+0x9/0xd0
[ 22.254264] kasan_save_stack+0x30/0x50
[ 22.273631] kasan_save_track+0x14/0x30
[ 22.276763] kasan_save_free_info+0x3b/0x70
[ 22.278296] __kasan_slab_free+0x43/0x70
[ 22.280157] kmem_cache_free+0xbf/0x3b0
[ 22.282963] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.284001] free_task+0xa2/0x160
[ 22.285699] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.286752] copy_process+0x2aae/0x7bc0
Initialize the child function graph state immediately after
dup_task_struct(), before the first fallible operation.
Cc: stable@vger.kernel.org
Fixes:
|
||
|
Henry Martin
|
2deb753127 |
tracing/user_events: Don't destroy fields when event removal fails
destroy_user_event() destroys the event's fields before attempting to
remove the trace event call. If user_event_set_call_visible() fails,
e.g. because the event is still enabled and trace_remove_event_call()
returns -EBUSY, the event is left registered with an irreversibly
destroyed field list. Any subsequent interaction with the event then
operates on an empty field list while it is still fully visible in
tracefs.
Move the field destruction after the call removal, and splice the
field list back onto the event when the removal fails so the event
remains in a consistent state.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260904115223.2976446-1-bsdhenrymartin@gmail.com
Fixes:
|
||
|
Andrea Parri
|
c5dcb3aadc |
hrtimer: Use hard expiry when updating timers on the same base
Rearming a queued timer with nonzero slack can leave the timerqueue out of order. remove_and_enqueue_same_base() checks the new soft expiry against its neighbours' hard expiries, then stores the new hard expiry in the node without requeueing it. For example, with A at 10 and B at 20, rearming A at 11 with slack 30 passes the neighbour check but leaves A's hard expiry of 41 before B's 20. The same function also caches the soft expiry in base->expires_next when updating or inserting the first timer, giving next-event selection an earlier deadline than the queue head's hard expiry. Set the timer expiry before handling the queue. Use its stored hard expiry for the in-place ordering check and both updates to base->expires_next. The early update is safe because remove_and_enqueue_same_base() runs with base->cpu_base->lock held. The lock keeps the queue stable while hrtimer_can_update_in_place() checks the new expiry against both neighbours. If the check fails, timerqueue_linked_del() removes the node without comparing expiry values before it is reinserted. Fixes: |
||
|
Jiayuan Chen
|
b0b3dc6652 |
bpf: Fix divide-by-zero in btf_struct_walk()
When an access goes past the struct and the last member is a flexible
array, btf_struct_walk() folds the offset back into a single element with
(off - moff) % t->size, but never checks that the element type has a size.
BTF takes an empty struct, so this in program BTF
/* event could be empty */
struct event {
#ifdef HAVE_TIMESTAMP
__u64 ts;
#endif
};
struct batch {
int nr;
struct event events[];
};
divides by zero at prog load time. Getting there needs a PTR_TO_BTF_ID that
is not MEM_ALLOC, e.g. a plain read of a local kptr stashed in a map from a
sleepable program.
Oops: divide error: 0000 [#1] SMP KASAN PTI
RIP: 0010:btf_struct_walk+0x53f/0x1570
Call Trace:
<TASK>
btf_struct_access+0x42a/0xcd0
check_ptr_to_btf_access+0x4dc/0x1160
check_mem_access+0x3a45/0x8740
check_load_mem+0x36a/0xd10
do_check_common+0x3ef0/0xb210
bpf_check+0x6d3b/0x8580
bpf_prog_load+0xf7c/0x2720
__sys_bpf+0xa83/0x3690
__x64_sys_bpf+0xc7/0x150
x64_sys_call+0x1f3f/0x27e0
do_syscall_64+0xe5/0x610
entry_SYSCALL_64_after_hwframe+0x76/0x7e
</TASK>
Reject a zero-sized element type. The fixed array path in the same function
already bails out on the same thing:
btf_struct_walk()
...
/* skip empty array */
if (moff == mtrue_end)
continue;
msize /= total_nelems;
Fixes:
|
||
|
Daniel Borkmann
|
2936aed9b0 |
bpf: Clear scalar delta on narrowing stack spill
check_stack_write_fixed_off() breaks the scalar link on a narrowing spill
by zeroing the id of the destination stack slot, but leaves the delta in
place. save_register_state() has just copied the source register verbatim,
so the slot keeps the BPF_ADD_CONST delta of a register it is no longer
linked to. This is the same shape as the case addressed in
|
||
|
Daniel Borkmann
|
f9191460cd |
bpf: Require CAP_PERFMON for untrusted read-only memory reads
Marking bpf_rdonly_cast() KF_PERFMON CAP-limits one producer of PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED, but not the type itself. A global subprogram argument tagged __arg_untrusted results in the same register with no kfunc call. Reported-by: STAR Labs SG <info@starlabs.sg> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260910213510.49358-3-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|
Daniel Borkmann
|
81c975aae3 |
bpf: Require CAP_PERFMON for kfuncs reading memory
Mark fault-safe probe reading kfuncs as KF_PERFMON, similarly as we do for the old-style BPF helper equivalents. bpf_rdonly_cast() is included in this list as well as it returns PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED for an unchecked object and is using fault-safe BPF_PROBE_MEM. Note that only the void form of bpf_rdonly_cast() produced a register that was readable without CAP_PERFMON. For a struct type id the kfunc returns PTR_TO_BTF_ID | PTR_UNTRUSTED, whose dereference has always been gated in check_ptr_to_btf_access(). The flag is not conditional on the type id, so for the latter it only moves the rejection from the dereference to the call itself, which is the better place to report it anyway. The bpf_stream_vprintk() and bpf_stream_print_stack() kfuncs are marked as well. The former ends up in the same bpf_bprintf_prepare() as the bpf_snprintf() helper, where %pks, %pus and %pI4 read through a program- supplied address and %pB resolves one into a symbol. The latter walks the stack and prints each instruction pointer via %pS. Lastly, bpf_get_kmem_cache() takes a raw scalar address that the verifier does not constrain and dereferences the page/slab metadata for it. Field reads on the returned pointer are still blocked (PTR_TO_BTF_ID | PTR_UNTRUSTED -> check_ptr_to_btf_access() results in -EPERM without the CAP), but the NULL/non-NULL result is observable. Reported-by: STAR Labs SG <info@starlabs.sg> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260910213510.49358-2-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|
Daniel Borkmann
|
88ce88e933 |
bpf: Add KF_PERFMON kfunc flag
Tracing related BPF helpers e.g. under bpf_base_func_proto() are gated behind CAP_PERFMON. However, the same is currently not true for kfuncs and they are accessible via plain CAP_BPF. Add a new KF_PERFMON flag which can be used such that check_kfunc_call() ensures env->allow_ptr_leaks is permitted. This follows similar pattern to existing KF_DESTRUCTIVE flag. The rejection returns -EPERM to match the other CAP_PERFMON gates in the verifier, that is, check_ptr_to_btf_access() and check_ptr_to_map_access(), which report the very same policy to user space. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260910213510.49358-1-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|
Linus Torvalds
|
ad724d319c |
Summary
* Replace CONFIG_PROC_SYSCTL with CONFIG_SYSCTL
CONFIG_SYSCTL is the config string that controls sysctl subsys.
* Testing
Ran through x86_64 selftest. Skipped linux-next for this trivial fix.
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEErkcJVyXmMSXOyyeQupfNUreWQU8FAmqerc0ACgkQupfNUreW
QU/cjwv/TO4x+9L4pCbz9wrlELCjhGuUWq4QZl78n/UqcNFGZ6wXQS+9WHoAa9r3
pfYHU0e7YfaYBJ+OaKHJq6IYRQ0M8zk+//K+fjdIj47poZo+Oqsv+bM5AnLtll3c
ltHjEUZTOPwGMagokOJgZiIJuf6L1Ex2DOU/+MEqFEwSoGg4IorGeT/lKRUS2/RQ
cdh+JQppSIvYeqMG1XsM7f73TuY48wyemu795sRZxtRgypv/RkN9JfVP8Qj50jBq
G/07NHc+f2Zq+m/oq20be1pphJTpj4NOkg2oTF3LJtNuNreSKtpaQL5lWw3eL3Go
qxDnZNhxCf3UAIMPpppQmvKqe/zeMALb1c6PixeCp1rezAa6Gu3AL3OV1tf2SKvc
ArNK8yhY+8Cx6n/RTdNwM8HEOy41Hy+f16uN06/EezEXpFh7KwqT1vKlc4hwarPm
3o3kJzUiam+Oz5hrnijDyDf8nPigHfqqg/hXR2p/x/6my7tA7jtjwX4DdZ2tzLcD
9rtcNpiO
=y3Vr
-----END PGP SIGNATURE-----
Merge tag 'sysctl-7.03-fixes-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl
Pull sysctl fix from Joel Granados:
"This fell through the cracks during the latest merge window. There are
no more CONFIG_PROC_SYSCTL uses after this fix:
- Replace CONFIG_PROC_SYSCTL with CONFIG_SYSCTL
CONFIG_SYSCTL is the config string that controls sysctl subsys"
* tag 'sysctl-7.03-fixes-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl:
syscall_user_dispatch: Use CONFIG_SYSCTL for sysctl guard
|
||
|
Hui Su
|
f5741d2b34 |
sched/core: Call wq_worker_tick() for the execution context
wq_worker_tick() accounts CPU time and detects CPU-intensive work for
the kworker that is actually running. With proxy execution, rq->donor
is the scheduling context while rq->curr is the execution context.
Calling the hook with rq->donor can skip workqueue accounting when a
kworker is executing on behalf of a donor task. It can also account a
blocked kworker when the donor is a worker but rq->curr is the task
actually executing. The former can delay WORKER_CPU_INTENSIVE handling
and pool concurrency management, which can delay pending kernel work
and userspace operations depending on it.
Use rq->curr for the workqueue tick hook while retaining rq->donor for
scheduler accounting.
Fixes:
|
||
|
Hui Su
|
c23810313b |
sched: Account cgroup CPU time to the execution context
Proxy execution separates the scheduling context from the execution context. Commit |
||
|
Vincent Guittot
|
51b0e68cfa |
sched/eevdf: Fix rb augmented with multi fields
The eevdf rb tree maintains 3 augmented fields but only one is currently
copied when balancing the tree.
Add a more generic define that can be used when there are several augmented
fields. In this case, we provide a function that takes care of copying all
fields.
Fixes:
|
||
|
Vincent Guittot
|
9a8bc9bb4c |
sched/eevdf: Fix augmented max_slice
Similarly to se->min_slice, init se->max_slice with se->slice before
enqueueing the entity so the augmented callback computes it correctly
at parent level.
Fixes:
|
||
|
Thomas Richter
|
59e63416f5 |
perf/core: Allow list_del during perf_event_overflow()
A PMU might use perf_sched_cb_inc() and perf_sched_cb_dec()
interface to get the PMU call back function pmu::sched_task
invoked at schedule in and schedule out. This is achieved
by walking along the list anchored by sched_cb_list.
The following scenario might lead to a list corruption.
perf_pmu_sched_task()
for_each_list_entry(..., &sched_cb_list)
+--> __perf_pmu_sched_task()
+--> event->pmu->sched_task())
+--> PMU_push_sample()
+--> perf_event_overflow()
+--> __perf_event_overflow()
+--> pmu->stop()
+--> perf_sched_cb_dec()
remove entry from sched_cb_list
while list node in use.
This happens when ioctl(fd, PERF_EVENT_IOC_REFRESH, xxx) has been
invoked and perf_event::event_limit hits zero.
Prevent the list corruption and convert for_each_list_entry()
to for_each_list_entry_safe().
Fixes:
|
||
|
Linus Torvalds
|
5e1287972b |
vfs-7.3-rc3.fixes
Please consider pulling these changes from the signed vfs-7.3-rc3.fixes tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaqFWOQAKCRCRxhvAZXjc
otLlAP9X02ybdUt9NndBK8LjslDWwB9hOXzPgYsOKYODEqODjQD/aLpbXVEsA1yy
SLdSDtbtpf+01z4KHorvAakBzk/jrw4=
=OzBX
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc3.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs fixes from Christian Brauner:
- netfs:
- Fix an uninitialized return value in netfs_unbuffered_write()
when preparing the first subrequest fails
- For partial unbuffered/DIO writes return the amount transferred
rather than an error
- Update i_size with the amount actually written when a partial
transfer ends in an error
- Fix a subrequest reference leak when the io_iter ends up empty
- Handle netfs_alloc_subrequest() failure during unbuffered writes
- Load all readahead folios into the rolling buffer upfront and
drop the readahead references once the first subrequest is
dispatched
- Mark folios for copy-to-cache while issuing subrequests
- Fix read progress reporting
- afs:
- Add the missing kunmap in the error path of afs_dir_search_bucket()
- Fix a double kunmap in afs_edit_dir_remove()
- Don't free an existing server's endpoint state when cleaning up a
candidate server in afs_lookup_server()
- Unbind peers removed from a server's address list
- ufs:
- Load the cylinder group metadata before creating the root dentry
- Validate the cylinder group index and rotor positions before
caching them
- Treat an unreadable directory block as not empty
- exec:
- Close the close-on-exec files before taking exec_update_lock
Closing a file can block on the filesystem, so a hung filesystem
blocked everything that takes exec_update_lock and a FUSE server
inspecting the calling process could deadlock
- Drop the bprm loader before closing bprm->file in free_bprm()
- exit: Hold a reference to thread_pid across proc_flush_pid()
- reboot: Fix a use-after-free on cad_pid
- nsfs: Keep the namespace tree fields out of the rcu_head used by
kfree_rcu()
- nstree: Check listing permission before taking a namespace
reference in listns()
- super: Return 0 when a nested thaw drops its hold while other
freezers remain
- ext4: Don't set I_METADATA_WRITEBACK during fastcommit replay
- adfs: Free s_fs_info in ->kill_sb()
- autofs: Free the inode info allocated in autofs_fill_super() when
the root inode allocation fails
- ovl: Return EINVAL instead of EIO on a user namespace mismatch now
that it's a plain refusal and not an internal error
- cachefiles: Don't cast the variable-length coherency data to a
__be64 in the coherency tracepoint
* tag 'vfs-7.3-rc3.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (28 commits)
nstree: check listing permission before taking a namespace reference
exec: do_close_on_exec() before taking exec_update_lock
exit: hold a reference to thread_pid across proc_flush_pid
fs: autofs: fix memory leak in autofs_fill_super()
exec: Drop bprm loader before closing bprm->file
afs: Clear stale peer app data after address list changes
afs: Fix incorrect free in candidate cleanup in afs_lookup_server()
afs: Fix double-unmap of directory block
afs: Fix missing kunmap in afs_dir_search_bucket()
ovl: return EINVAL instead of EIO in case of mismatched user_ns
reboot: fix cad_pid use-after-free race
cachefiles: Fix potential UAF/KASAN warning
netfs: Fix read progress reporting
netfs: Mark folios with COPY_TO_CACHE whilst issuing subreqs
netfs: Fix readahead synchronisation issues by loading all folios upfront
netfs: break unbuffered write when netfs_alloc_subrequest() fails
netfs: Fix subreq ref leak
netfs: Fix i_size update for partial transfer
netfs: Fix error vs transferred passed to ->ki_complete()
netfs: Fix unbuffered/DIO write partial transfer error return
...
|
||
|
Linus Torvalds
|
3f8b8c94a7 |
printk fixes for 7.3-rc3
-----BEGIN PGP SIGNATURE----- iQJPBAABCAA5FiEESH4wyp42V4tXvYsjUqAMR0iAlPIFAmqhEMMbFIAAAAAABAAO bWFudTIsMi41KzEuMTIsMiwyAAoJEFKgDEdIgJTyjjcQAI2HtdUFDsG4GifU1B1s ha1uQM9gTH+BPO0RiQwvPD8hg/WdmQ1I60JE3jmWuC+AiT0W/fpHiKvvS5ThW9X7 gessJNbKnvPW8LYoBCT3gsHO6yOL3pvsgM5fUB+SIn8i/B1TIgf/SqmejXZA0S1b 3NdU6h/2AhEbpzgp3dPfoQe4zrAIQ2IHrjYJ1JunPTtvNsboc1jML4/ViXOTsIvO S7/wsiWD42PfVJI8eXVt5fYR33ruhBqJeMRre2sSZ11NwnValOwmJIxU/RmMlA0S Vd1pNE1RMcfodYw9JXrouHo9osx3/lAiKF5J0+xehllE7mn3M8PcCPKGpXuHuKy4 s1KNXXlP0aQatsyyqYBROZeUOv2roXazY/mHqtQ+zrCrE02Hsn7WGPcSXoIbROj6 AJIjMpz5caS+twtpa4FwhCckbthoIjPMUsv1a5pm9RU10Gr2PqFnRgDxpUDyi5ve 2M1ybFeJD89tE+CHtGWVuWUS6e17OG4ydFT0GdfDej0Zlq2PckOoEROenbGFl/wk kuN7q40Tmg5DKHcSWvy40aoLNJ2kVyvmbc4I7uOkDRAIjCRAoRyyrLAndABIhb58 oQwFNaNI09qbrk9o7ol4RSruLrgzA/yzUbflzUeXs57LAAOgGXaZmiKzSabTyuCr 0O6dIPCCjtEH2YY0Hk6HaLX0 =zZPb -----END PGP SIGNATURE----- Merge tag 'printk-for-7.3-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux Pull printk fixes from Petr Mladek: - Use lazy irq_work for waking printk kthreads - Flush pending irq_work before destroying printk kthreads - Remove redundant WARN() when a printk kthread can't be created - Typo fix * tag 'printk-for-7.3-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux: printk/nbcon: Change nbcon_irq_work to IRQ_WORK_LAZY printk/nbcon: Flush nbcon_irq_work in nbcon_free() console: fix /dev/kmsg reference in flags kernel doc printk: Don't WARN on kthread_run failure. |
||
|
Norbert Szetei
|
56ea4e8683
|
nstree: check listing permission before taking a namespace reference
legitimize_ns() takes a reference on the candidate namespace before may_list_ns() has decided whether the caller may see it. The __free(ns_put) cleanup on the denied path can drop the last reference to a mount namespace while we still hold the rcu read lock, and put_mnt_ns() may sleep there. This is the same problem commit |
||
|
Petr Mladek
|
a76f015d42 | Merge branch 'for-7.4-trivial' into for-linus | ||
|
Donggeun Yoo
|
b7d7914a9a |
swiotlb: use the adjusted address for the highmem page lookup
swiotlb_bounce() reads the page frame number from the slot's recorded
orig_addr, then advances orig_addr by tlb_offset to reach the address
the caller asked about. The highmem branch mixes the two: the offset
within the page comes from the adjusted address, the page from the value
before it.
Once the adjustment crosses a page boundary the pair no longer describes
one location, and the whole copy lands one page below the intended one
for a positive tlb_offset, one above for a negative one. DMA_FROM_DEVICE
writes the device data over the wrong page and leaves the intended one
stale, DMA_TO_DEVICE feeds the device from a page the mapping may not
cover. Partial syncs through dma_sync_single_range_for_*() are what make
tlb_offset non-zero.
The branch test is picked the same way, so a slot recorded in lowmem can
be adjusted into highmem and the lowmem path then hands a highmem
address to phys_to_virt().
Take both from orig_addr once it is final and keep pfn in the branch
that uses it. PhysHighMem() asks the question straight from the address,
as dma-debug already does.
Fixes:
|
||
|
Donggeun Yoo
|
504981db4f |
dma-coherent: report a failed reserved memory assignment
rmem_dma_device_init() drops the return value of
dma_assign_coherent_memory() and always reports success. That call fails
with -EBUSY when the device already has a coherent pool, and the file
allows only "*one* such region of memory" per device.
of_reserved_mem_device_init_by_idx() reads the zero as success. It logs
"assigned reserved memory node" for a region that was not assigned and
records the pairing, so of_reserved_mem_device_release() later runs
rmem_dma_device_release() for it. That clears dev->dma_mem without
looking at which region it was called for, dropping the pool the device
did get and leaving it on ordinary memory.
dma_declare_coherent_memory() checks the same call and releases the
memory on failure, and rmem_swiotlb_device_init() propagates its own
errors. Return the error here as well, so a device tree that assigns two
pools to one device fails the probe instead of half working.
Fixes:
|
||
|
Karl Mehltretter
|
353a95f1cd |
syscall_user_dispatch: Use CONFIG_SYSCTL for sysctl guard
Commit |
||
|
Thomas Gleixner
|
113a9796ef |
tick/broadcast: Plug clockevents replacement race
朱恺乾 reported and decoded the following race condition when a broadcast
device is replaced:
CPUA CPUB
__tick_broadcast_oneshot_control()
bc = tick_broadcast_device.evtdev;
tick_install_broadcast_device(dev)
clockevents_exchange_device(cur, dev)
shutdown(cur);
detach(cur);
cur->handler = noop;
tick_broadcast_device.evtdev = dev;
tick_broadcast_set_event(bc, next_event); <- FAIL: arms a detached device.
If the original broadcast device has a restricted interrupt affinity mask
and the last CPU in that mask goes offline then the BUG() in
tick_cleanup_dead_cpu() triggers because the clockevent device is not in
detached state.
The reason for this is that tick_install_broadcast_device() is not
serialized vs. tick broadcast operations.
The obvious cure is to serialize tick_install_broadcast_device() with
tick_broadcast_lock against a concurrent tick broadcast operation.
That requires to split clockevents_exchange_device() into two parts, one
which does the exchange, shutdown and detach operation and the other which
drops the module reference count. This is required because the module
reference cannot be dropped while holding tick_broadcast_lock.
Let clockevents_exchange_device() do both operations as before, but let the
broadcast device code take the two step approach and do the device
exchange under tick_broadcast_lock and drop the module reference count
after releasing it.
Fixes:
|
||
|
Linus Torvalds
|
b1e00ffaf9 |
tracing fixes for v7.3:
- Fix several tracefs files that did not take the trace_array reference A trace instance can be created and destroyed in the tracefs "instances" directory via mkdir and rmdir respectively. The instance is represented by a trace_array descriptor. Most tracefs files pass the trace_array as the private data of the inode to the open/read/write functions. Since there is no locking between the time a task opens a file and the deletion of the instance (and the freeing of the trace_array), each open needs to get a reference to the trace_array and each close must remove it. A instance can't be removed if there's any reference taken on its trace_array. The open function uses trace_array_get() that takes a lock (preventing removal of instances) and iterates the list of all existing trace_arrays and if it finds a match, it takes the reference and releases the lock. If it doesn't find a match, it causes the open to return -ENODEV. There were some added files that did not take the trace_array reference on open that needed to be fixed. Sashiko also correctly pointed out that there were some files that took an address of an field or element of the trace_array which had a pointer back to the trace_array to take its reference on open. But this leaves a slight race between referencing this element to get the trace_array as the element itself could be freed. To solve this, some helper functions were created to look for trace_arrays with this field or element in the search so that the element did not have to be dereferenced before the trace_array's reference was taken. - Add a lock around ftrace_ops initialization When a ftrace_ops is first used by ftrace, some internal initialization is performed on the ops. But if multiple tasks were calling functions that did this initialization, it could race and perform doing the initialization more than once, corrupting the internal data. Add a lock in the initialization code to prevent this from happening. - Fix splice reads on mmapped buffers The logic in the ring buffer splice code for mmapped buffers is supposed to do a copy of the memory as the mapped buffers can't be given to splice. But there was an if statement within the copy code that would return a -1 if a request for a full page was done and it wasn't a partial read. This is because this logic was written before mmapped buffers existed and this case didn't make sense at the time. For mmapped buffers it makes perfect sense and by returning early can drop a lot of pages unnecessarily. - Have the persistent ring buffer validation check nr_subbufs Sashiko reported that the validation code was relying on the saved nr_subbufs to match the calculated nr_pages + 1 and if they were off, that the code could cause corruption. Sashiko is correct, and the saved nr_subbufs should be validated before assuming it is correct. - Do not allow more than one instance with the same name on cmdline If an admin were to add more than one trace instances with the same name they all would be created, but only the first one would be accessible via tracefs. This used to not be allowed but some restructuring of code has since made it possible. - Fix the race between subbuf resize and trace_pipe_raw readers If a task was reading trace_pipe_raw while another task was changing the ring buffer subbuf size, it could crash the reader. The trace_pipe_raw readers do get their own copy of the page from the buffer, but the code needs some restructuring to not have the resize of the subbuffers cause issues. - Cap the size of the mapped (static) ring buffer nr_pages The meta data used for ring buffer mapped buffers is 32 bit in size. A normal ring buffer could (in theory) have more than 4 billion pages. But this is not allowed by mapped buffers, so enforce it. -----BEGIN PGP SIGNATURE----- iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCap3GJRQccm9zdGVkdEBn b29kbWlzLm9yZwAKCRAp5XQQmuv6qtAfAP9hDPtzctAoz+IHZq+GYejhKTu99ICG vAxTOZivuG6KYAD/UPA4P7z5ayxGM5UlT+2U81+mpCd55F2oFiIxRuFYcwk= =/d+T -----END PGP SIGNATURE----- Merge tag 'trace-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing fixes from Steven Rostedt: - Fix several tracefs files that did not take the trace_array reference A trace instance can be created and destroyed in the tracefs "instances" directory via mkdir and rmdir respectively. The instance is represented by a trace_array descriptor. Most tracefs files pass the trace_array as the private data of the inode to the open/read/write functions. Since there is no locking between the time a task opens a file and the deletion of the instance (and the freeing of the trace_array), each open needs to get a reference to the trace_array and each close must remove it. An instance can't be removed if there's any reference taken on its trace_array. The open function uses trace_array_get() that takes a lock (preventing removal of instances) and iterates the list of all existing trace_arrays and if it finds a match, it takes the reference and releases the lock. If it doesn't find a match, it causes the open to return -ENODEV. There were some added files that did not take the trace_array reference on open that needed to be fixed. Sashiko also correctly pointed out that there were some files that took an address of an field or element of the trace_array which had a pointer back to the trace_array to take its reference on open. But this leaves a slight race between referencing this element to get the trace_array as the element itself could be freed. To solve this, some helper functions were created to look for trace_arrays with this field or element in the search so that the element did not have to be dereferenced before the trace_array's reference was taken. - Add a lock around ftrace_ops initialization When a ftrace_ops is first used by ftrace, some internal initialization is performed on the ops. But if multiple tasks were calling functions that did this initialization, it could race and perform doing the initialization more than once, corrupting the internal data. Add a lock in the initialization code to prevent this from happening. - Fix splice reads on mmapped buffers The logic in the ring buffer splice code for mmapped buffers is supposed to do a copy of the memory as the mapped buffers can't be given to splice. But there was an if statement within the copy code that would return a -1 if a request for a full page was done and it wasn't a partial read. This is because this logic was written before mmapped buffers existed and this case didn't make sense at the time. For mmapped buffers it makes perfect sense and by returning early can drop a lot of pages unnecessarily. - Have the persistent ring buffer validation check nr_subbufs Sashiko reported that the validation code was relying on the saved nr_subbufs to match the calculated nr_pages + 1 and if they were off, that the code could cause corruption. Sashiko is correct, and the saved nr_subbufs should be validated before assuming it is correct. - Do not allow more than one instance with the same name on cmdline If an admin were to add more than one trace instances with the same name they all would be created, but only the first one would be accessible via tracefs. This used to not be allowed but some restructuring of code has since made it possible. - Fix the race between subbuf resize and trace_pipe_raw readers If a task was reading trace_pipe_raw while another task was changing the ring buffer subbuf size, it could crash the reader. The trace_pipe_raw readers do get their own copy of the page from the buffer, but the code needs some restructuring to not have the resize of the subbuffers cause issues. - Cap the size of the mapped (static) ring buffer nr_pages The meta data used for ring buffer mapped buffers is 32 bit in size. A normal ring buffer could (in theory) have more than 4 billion pages. But this is not allowed by mapped buffers, so enforce it. * tag 'trace-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: ring-buffer: Use a macro for static buffer bits tracing: Fix comment in tracing_buffers_splice_read() ring-buffer: Prevent truncation of nr_pages / nr_subbufs ring-buffer: Cap static ring buffer nr_pages tracing: Fix subbuf resize races with trace_pipe_raw readers tracing: Fix to avoid creating trace instances with duplicate names ring-buffer: Add checking nr_subbufs to persistent ring buffer validation ring-buffer: Allow splice reads on static buffers tracing: Take trace_array reference when opening options file ftrace: Synchronize the initialization of ftrace_ops ftrace: Take trace_array reference before accessing its ftrace_ops tracing: Have show_event_filters/triggers files take trace array ref |
||
|
Linus Torvalds
|
2beb1b31a1 |
bpf-fixes
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE+soXsSLHKoYyzcli6rmadz2vbToFAmqdhRMACgkQ6rmadz2v
bTo8vRAAnP+x1z6FBOgHawGmmBpXtazoipRFeok/+/YGQ4SbS+evvFNvmkeCyihZ
7EeNHGuWEbA6xQBAcMf0StMjmr3jkJvUA6lsHSxBiFLpHGqNt0Fz9IXvAL/8QWPg
F7+CBs5A+J61i66LEmINbWzW1ujDf7baU/1VWui9zfPxnmPu40V/74eXzm6DmH/V
oHTsCscCYNe+N+Wix6B6nLrObY537fcJU12N56uqpkaSc0+6H1fBQJmyFGpYroTX
+4g+DNzEHyIYmt6B3+oDpbT0Zh/pP6ROq5TPDNJSSa+5B/uPo5C5pH520HwXF9hJ
04uaD7y2fpE809Nwl3OEK4ozpJ+mEGU1NKZPvFAxckY6GCiBtfbGd1etFno7r+4F
mZhNBKbQVEQX4XJUlcoqn9n7T2OCDnjiuneZAjIi4vUdevnSnakr0rCTFriiHP54
jg4fEkXxvYjPiqs5SQbxoGxmYSfg9fYaSfK4HgFdlfbwp+3Vmv4BW/tcsIJ9dcPs
L3YWkUU3znjbu2BLy0CP9fGt/1ik8p4/tA8vvZwz0yqpDMJkLy1TziGMef51UQ7k
1Pe/Ln5J3C6xnWs+3TRF11y8PuuCF0y47DGPACfwPoQxca1ufJ5fgmLXRRQToZe4
RjDmoeQvvI9w9kxh6gVhTHoiiEeD0wcQqx2CMHKTrU/PbM7d4Co=
=oz4Z
-----END PGP SIGNATURE-----
Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf
Pull bpf fixes from Alexei Starovoitov:
"This mainly contains verifier fixes that address bugs reported by
Nicholas Carlini.
- Fix incorrect non-NULL inference in pointer comparisons: pointer
types that may be NULL at runtime, pointers with unbounded offsets,
JMP32 comparisons with zero, and imprecise zero registers (Eduard
Zingerman)
- Fix precision tracking for half-dead zero spills, ld_abs/ld_ind
implicit subprog exit, bpf_loop() callbacks, linked scalar ids and
NULL call arguments (Eduard Zingerman)
- Reject BPF_PSEUDO_FUNC reference to the main program, fix zero
extension of arena 32-bit cmpxchg, don't rewrite bpf_fastcall
patterns entered by a jump (Eduard Zingerman)
- Fix percpu map update and BPF_F_CPU validation with sparse CPU IDs
(Hui Su)
- Fix NULL-ptr-derefs in bpf_snprintf_btf() for void and VAR types,
and reject key-less BTF for hash maps (Jiayuan Chen)
- Various fixes (Kumar Kartikeya Dwivedi):
- Fix out-of-bounds access in disassembler on invalid LDSX
instruction
- mark siginfo of signal tracepoints as scalar and
sched_process_wait argument as nullable
- mark faultable stack helpers as sleepable
- reject tail calls and legacy packet loads from callbacks
- enforce rbtree callback lock restrictions for resilient locks
- require MEM_PERCPU for percpu kptr stores
- clear NON_OWN_REF after RCU protection ends
- mark NULL kptr stores precise
- preserve inner map identity in callback frames
- reject non-scalar bpf_loop() iteration counts
- Fix trampoline allocation slowdown on x86 by using
EXECMEM_MODULE_DATA (Mike Rapoport)
- Keep bpf_refcount_acquire() nullable for borrowed RCU kptrs and
reject untrusted allocated-object pointers (Ning Ding)
- Fix special fields handling in recycled rhtab elements (Nuoqi Gui,
Yuan Chen)"
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (86 commits)
bpf, riscv: Make arena support depend on ZACAS
selftests/bpf: Test pointer bpf_loop iteration count rejection
bpf: Reject non-scalar bpf_loop iteration counts
bpf: use mark_arg_precision() in check_mem_size_reg()
bpf: propagate mark_chain_precision() errors out of loop_flag_is_zero()
selftests/bpf: precision of a NULL global subprogram BTF_ID argument
bpf: mark a NULL BTF_ID argument of a global subprogram precise
selftests/bpf: precision of a NULL kfunc argument
bpf: mark a NULL kfunc argument precise
selftests/bpf: precision of a NULL global subprogram memory argument
bpf: mark a NULL memory argument of a call precise
selftests/bpf: precision of a NULL helper argument
bpf: mark a NULL call argument precise
selftests/bpf: Test inner map identities in callbacks
bpf: Preserve inner map identity in callback frames
selftests/bpf: Test imprecise scalar kptr stores
bpf: Mark NULL kptr stores precise
selftests/bpf: Test rhtab kptr cancellation semantics
bpf: Cancel special fields when recycling rhtab elements
selftests/bpf: Test timer field on recycled rhtab element
...
|
||
|
Linus Torvalds
|
88405f0ad1 |
Miscellaneous scheduler fixes:
- Fix a timestamping bug in pick_task_fair() and
yield_task_fair() (Zhan Xusheng)
- Skip migrate-disabled tasks when picking a push
candidate in the RT and DL schedulers (Seiji Nishikawa)
- Skip rq->avg_idle update without a valid idle_stamp
(Shubhang Kaushik)
- Fix throttling bug in throttle_cfs_rq(), caused
by the recent single-runqueue conversion (Wanwu Li)
- Fix bandwidth calculation bug in distribute_cfs_runtime(),
caused by the single-runqueue conversion (Wanwu Li)
- Don't make x86 ITMT enablement depend on debugfs (Mario Limonciello)
- Avoid creating misfits during cache-aware load-balancing
on hybrid systems (Tim Chen)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqdTLIRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gdURAAnjK4q2xg115VyKOvg4u4epPre1XNK4a/
RmimS/2+1fm7/5zqOXOQG3qU8z2Aj+nBIZa1d1yadV7ARHSymDINobCX+2HQ1DjU
Iysk2BqbbLnQQNKh81OBWhPMyJy4yOnig/670KR5JdFAdTKFYW/Ys+pQmO9M6rEx
hc8B4SUhOGeRnonfPMTbFW7H9eopWzst8kIpF0USl58az0mla5tkooWYJsvSuUHn
g4s9IALlRe3DxgmMT3kLKoAx2+ySnrzZNRs+M8Z/sMSDKXCMFpvX36wOqBJMHwXe
ZoAow/fJh4ysoYTR7luhDIBDQAkszf4GE9a644I/Bd0sIIYKs4+1RW1T3DAoWmKv
Sx0C29MM4Ng4R3lPx2avb+c0T0VO080/3fqRFrGrck8BIpMa9HcinOxf6etI8+vt
STnz5uQZZj3V+A5vzlKCAFGUUqK/s3EJnrT5z4VPxXmY2ZSyLLub4C5SQ8RNnqRL
u+Q9ra9DYBJh/HJTgjjcAP/HxmKK5LyQXd2DzuA1wy/wwC+s7TtPrxN5ep5EdZmv
K65zGGo28BgcCKD8jc1DfVXf8cf80fMiAr4fruGdo5TVwPyBahwZBi4sbqhahtqL
D4QwCsbOfhS+evVpx29n+V7eQzZzvSdN1tFCtZ3Tg6VvT9aJ2jLSx79aCsL/cYbr
hViB9N0yhGk=
=8l0J
-----END PGP SIGNATURE-----
Merge tag 'sched-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fixes from Ingo Molnar:
- Fix a timestamping bug in pick_task_fair() and yield_task_fair()
(Zhan Xusheng)
- Skip migrate-disabled tasks when picking a push candidate in the
RT and DL schedulers (Seiji Nishikawa)
- Skip rq->avg_idle update without a valid idle_stamp (Shubhang
Kaushik)
- Fix throttling bug in throttle_cfs_rq(), caused by the recent
single-runqueue conversion (Wanwu Li)
- Fix bandwidth calculation bug in distribute_cfs_runtime(),
caused by the single-runqueue conversion (Wanwu Li)
- Don't make x86 ITMT enablement depend on debugfs (Mario Limonciello)
- Avoid creating misfits during cache-aware load-balancing on hybrid
systems (Tim Chen)
* tag 'sched-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/fair: Avoid creating misfits during cache-aware balancing
x86/itmt: Don't make ITMT enablement depend on debugfs
sched/fair: Use cfs_rq->h_curr in distribute_cfs_runtime()
sched/fair: Use cfs_rq->h_curr in throttle_cfs_rq()
sched/core: Skip rq->avg_idle update without a valid idle_stamp
sched/rt,dl: Skip migrate-disabled tasks when picking a push candidate
sched/fair: Use update_curr_eevdf() for the remaining root cfs_rq callers
|
||
|
Linus Torvalds
|
c4a3928e7d |
Misc perf events fixes:
- Skip empty AUX records with only format flags
(Leo Yan)
- Fix use-after-free when perf mmap() revival races with the
last munmap() (Yilin Zhang, Weiming Shi)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqdSokRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gRKRAAkDxFT3BNAtAypk5CvTPeO9K530u38mqc
fIvYQYbYkqIMZaFSmIthSgsPOJD/mqkccl1sd6djlS/73mI/X8eXCwZOgjLnaCsu
jRx8atD8Q7hHeyJ78qY/hFsyHPTQel5+cniEk4BjUrYIYhHklFfpQw1AZhoJieUs
vnqkDGMY/qHQ5pGs+dnz3oI8FE6ruNU0Ps6qiBqsbsvMJI+wviCidpC6BqluyxvQ
Pp6ACwqQIiH2mPwI9WYtHKmVL5ajQJPt8fbIwIvRbX5wCY566zb79oUys2pNUh/U
zzpTNOGgqtVCTgiuMiEI2SlIIMzC36P071vBTyD8YZWUNpdYXLM20RdvAUZkqefT
Vx3cMs8rgcPnjGO4XhPJJOzfyyCi25VL/p3af9S3wSJhfQNdMF9BuqRABFMBmtDi
5adP+4p5Dw9sfc2+F2/OCT7PrXsFiNsOp+iduWJYb6i8hdSjV1mfjGLyptdBCdj2
dUYr4VrFoiYUVLPWJD5F4/dDAIljN3CEoGwHiOITNCYu0fgO7GCALf8FG5U9Ec7D
guuRGnEXDFQDCOZHm5MuVYWAR7qPhe+iVoX6Dw9ZmgecGWb4eTl4A5e8LGgbcbJ1
JMfVr3OENSyGi1dPnCX4DEsIpeXN1UTMHhrwNosWcu1OrEEUkzdQ8GQ17qU6PURG
yYSXCvhdEtY=
=Ffvz
-----END PGP SIGNATURE-----
Merge tag 'perf-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar:
- Skip empty AUX records with only format flags (Leo Yan)
- Fix use-after-free when perf mmap() revival races with the
last munmap() (Yilin Zhang, Weiming Shi)
* tag 'perf-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf: Fix use-after-free when perf mmap() revival races with the last munmap()
perf/core: Skip empty AUX records with only format flags
|
||
|
Linus Torvalds
|
c8990f3179 |
Misc locking fixes:
- Fix a softirq processing delay bug in local_interrupt_disable(),
which should mostly only affect the Rust runtime (Boqun Feng)
- Remove the hardirq_disable_count() function which caused the
previous bug and is now unused & unnecessary (Boqun Feng)
- lockdep: Invalidate stale class_cache entries for zapped classes
(Eric Dumazet)
- Fix rt_mutex specific futex scheduling helpers
(Sebastian Andrzej Siewior)
- Fix rcuwait use-after-free race during futex requeue PI (Yao Kai)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqdSKMRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gh6hAAtIhAm10pkx7JG4Kl+SQArmoAxNMfW9X1
grx50dn1/LFuIF0upooJwJmLMBpPMmxcpYPWbg5nTUsx+wDqcJTT0T7Aw2iZN2f+
OdODMOGNuQU4IUxDC+qX+fkgHvRFzng0zhSXYL2kGpQJHktrFMAHn90+5aO9SEgA
XcSjmDacTxfJADNk43snrlHy6t+yOXBEmIrhicnbh11GFNIXLbvdGW/0SObU0Q5o
WdwGHTZRFAM3mpX7xMSFOKypM+Gt4pXdQ/uC1PQg0B/AT35p4zsYIdcht/cBMHp0
93eTeJd9ede410t3hL1Hnsuo1zf5qHtCgBukS+UHJWoWpnSOMgT2iaBx2X3jIsev
ptPIqV5Alq8O+tnNOSDOwD2HH2drAdUBVH+gZ/yKMDbk9jfw9eZKqRw628U2P7rG
mD5CLzpmfj0mMAuqcOhhdqSQmPhjp2Y8Ju/8UAMSmcocvqjQBxvouNS3Tby9U5v+
R4CsuUwtnYpMyy5xyRrnwVsaE3le5fvneXCmCokk8FtQ8as/4ATd+cG+RROSa3vW
/AAjrB1HBQqyseE8lki4LFFmlr01Otc8u6pASy4/pxeuKMA9hSuLC3RuEfVetW6W
BcK4BCIJBy6chcsCBxv0F7FgI/y7/bpVyBmoTkW3Sb0I23xswXzTzbucn/dPJNdl
oqFj3KQygxw=
=vEuZ
-----END PGP SIGNATURE-----
Merge tag 'locking-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking fixes from Ingo Molnar:
- Fix a softirq processing delay bug in local_interrupt_disable(),
which should mostly only affect the Rust runtime (Boqun Feng)
- Remove the hardirq_disable_count() function which caused the
previous bug and is now unused & unnecessary (Boqun Feng)
- lockdep: Invalidate stale class_cache entries for zapped classes
(Eric Dumazet)
- Fix rt_mutex specific futex scheduling helpers
(Sebastian Andrzej Siewior)
- Fix rcuwait use-after-free race during futex requeue PI (Yao Kai)
* tag 'locking-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
futex: Prevent rcuwait use-after-free during requeue PI
futex: Provide rt_mutex_.*_schedule() equivalents for futex scheduling
locking/lockdep: Invalidate stale class_cache entries for zapped classes
preempt: Remove hardirq_disable_count()
interrupt: Disable interrupt before modifying hardirq_disable counter
|
||
|
Kumar Kartikeya Dwivedi
|
c3fd8e5fd1 |
bpf: Reject non-scalar bpf_loop iteration counts
bpf_loop() declares its nr_loops argument as ARG_ANYTHING. Privileged programs may pass pointer values to such arguments, so check_func_arg() lets a pointer-valued R1 reach the helper-specific checks. Since commit |
||
|
Tejun Heo
|
90f19b2816 |
sched_ext: Use @prev's scheduler for the keep decisions in dispatch_one()
dispatch_one() tests ops flags and bypass state against the root scheduler
in both places where it decides to keep running @prev: the early keep of a
@prev with slice left tests the root's bypass state, and the keep-last at
the end tests the root's SCX_OPS_ENQ_LAST and bypass state. Both are
properties of the scheduler @prev belongs to, and put_prev_task_scx(), which
acts on the outcome, reads them from that scheduler. When @prev belongs to a
sub-scheduler the two sides disagree.
The keep-last case is visible. The root set SCX_OPS_ENQ_LAST, so a lone
@prev of a sub-scheduler is not kept and is enqueued with SCX_ENQ_LAST to a
sub-scheduler that never opted in. This trips the WARN_ON_ONCE in
put_prev_task_scx() for the missing flag, and the sub-scheduler queues the
task like any other and triggers no follow-up scheduling event, which can
lead to stalls.
Test SCX_OPS_ENQ_LAST and bypass state on @prev's sched in both places and
charge SCX_EV_DISPATCH_KEEP_LAST to it. Read the sched at each decision, as
the dispatch in between can drop the rq lock.
Fixes:
|
||
|
Tejun Heo
|
3265ef0b67 |
sched_ext: Rename sch to root_sch in dispatch_one()
dispatch_one() uses the root scheduler for everything it does, including the two decisions to keep running @prev, which are wrong when @prev belongs to a sub-scheduler. The function has to deal with @prev's scheduler too. Rename the root's local from sch to root_sch for clarity and to make room for it. No functional change. Signed-off-by: Tejun Heo <tj@kernel.org> Reviewed-by: Andrea Righi <arighi@nvidia.com> |
||
|
Kees Cook
|
3a2c4d55e3 |
treewide: refresh kmalloc_obj() conversions
This is another run of the Coccinelle script for converting kmalloc() family of allocations to kmalloc_obj() via the existing rules in scripts/coccinelle/api/kmalloc_objs.cocci This catches both the set of kmalloc() uses added since the first kmalloc_obj() conversions in v7.0 and adds a large group missed in the first pass due to Coccinelle not interacting well with the cleanup.h scoped_...() family of macros[1]. I worked around this with spatch's "--macro-file" argument to a file with all the scoped_...() macros mapped to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control flow indicator I could find. Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc, riscv, and s390 with no new warnings. Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1] Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2] Signed-off-by: Kees Cook <kees+treewide@kernel.org> |
||
|
Eduard Zingerman
|
cf2475616b |
bpf: use mark_arg_precision() in check_mem_size_reg()
Use newly added mark_arg_precision() helper in check_mem_size_reg(). Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-10-0f5a360ff15d@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|
Eduard Zingerman
|
1d7f8f191c |
bpf: propagate mark_chain_precision() errors out of loop_flag_is_zero()
Stop verification if mark_chain_precision() fails when called from
loop_flag_is_zero(). No functional change intended for the paths where
backtracking succeeds.
Fixes:
|
||
|
Eduard Zingerman
|
e726fc6b9a |
bpf: mark a NULL BTF_ID argument of a global subprogram precise
btf_check_func_arg_match() accepts a NULL register for an
ARG_PTR_TO_BTF_ID argument tagged __arg_nullable and skips
check_reg_type() and check_func_arg_reg_off() without marking the
register precise. Hence a checkpoint created on such a path would
prune against arbitrary scalar value.
Fixes:
|
||
|
Eduard Zingerman
|
506ada8962 |
bpf: mark a NULL kfunc argument precise
check_kfunc_arg() allows bpf_register_is_null() for nullable arguments
w/o marking the underlying scalar register precise. Hence a checkpoint
created on such a path would prune against arbitrary scalar value.
Fixes:
|
||
|
Eduard Zingerman
|
f1e418129f |
bpf: mark a NULL memory argument of a call precise
check_mem_reg() allows bpf_register_is_null() for nullable arguments
w/o marking the underlying scalar register precise. Hence a checkpoint
created on such a path would prune against arbitrary scalar value.
The argument may live on the stack rather than in a register when a
call has more than MAX_BPF_FUNC_REG_ARGS arguments, hence the new
mark_arg_precision() helper.
Fixes:
|
||
|
Eduard Zingerman
|
1a3a10b030 |
bpf: mark a NULL call argument precise
check_func_arg() allows bpf_register_is_null() for nullable arguments
w/o marking the underlying scalar register precise. Hence a checkpoint
created on such a path would prune against arbitrary scalar value.
check_helper_call() enforces second parameter of the
bpf_get_local_storage() to be zero, w/o marking the underlying scalar
register precise. Hence a checkpoint created on such a path would
prune against arbitrary scalar value.
Grouping these two into one patch, as they share the same fixes tag.
Fixes:
|
||
|
Steven Rostedt
|
d80e12156f |
ring-buffer: Use a macro for static buffer bits
Instead of hard coding 30 for the number of bits used for the static buffer ids in two places, create a macro. This way if it changes in the future, it will change in all the locations that use it. Link: https://patch.msgid.link/20260904151641.17eae0aa@gandalf.local.home Signed-off-by: Steven Rostedt <rostedt@goodmis.org> |
||
|
Steven Rostedt
|
5cbea50077 |
tracing: Fix comment in tracing_buffers_splice_read()
The comment about returning an error if the read fails on the first
iteration is slightly incorrect. It makes it sound like the only reason it
could fail on a later iteration is if the subbuf order changed. That is
incorrect, it could also fail if the length passed in was not a multiple
of the subbuf size. Fix the comment.
Link: https://lore.kernel.org/all/20260904143527.40e73d36@gandalf.local.home/
Link: https://patch.msgid.link/20260904144902.506862a1@gandalf.local.home
Fixes:
|
||
|
Vincent Donnefort
|
c843fd3c73 |
ring-buffer: Prevent truncation of nr_pages / nr_subbufs
Although ring_buffer_per_cpu::nr_pages is defined as unsigned long, it is capped to 32-bits in a few places, limiting the operations possible on a very large buffer. Use `unsigned long` where appropriate and prevent truncation of values using nr_pages (or nr_subbufs). While at it, subbuf_size must be at least `unsigned int`. Note that persistent, remote and user-mapped ring buffers are capping the number of pages to 30 bits already, making "int" safe in many places. Link: https://patch.msgid.link/20260904164450.1345852-5-vdonnefort@google.com Signed-off-by: Vincent Donnefort <vdonnefort@google.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org> |
||
|
Vincent Donnefort
|
f2b2b64559 |
ring-buffer: Cap static ring buffer nr_pages
Static ring buffers (i.e. persistent, user-mapped and remote) rely on
the bpage::id field. The number of pages for those ring buffers must fit
into that variable. Enforce this limit on ring buffer creation or
user-mapping.
While at it, prevent nr_pages underflow when allocating a persistent
buffer.
Link: https://patch.msgid.link/20260904164450.1345852-4-vdonnefort@google.com
Fixes:
|
||
|
Vincent Donnefort
|
dae8dda341 |
tracing: Fix subbuf resize races with trace_pipe_raw readers
Concurrent subbuffer resizes may crash trace_pipe_raw readers or leak
uninitialized memory to userspace due to stale size values.
Modify ring_buffer_alloc_read_page() to handle the resizing of an
existing buffer_data_read_page if necessary and add a new
ring_buffer_read_page_size(). This new function enables ring-buffer
buffer_data_read_page users to not call the racy
ring_buffer_subbuf_size_get(). This makes the spare_size member of
ftrace_buffer_info redundant.
Finally, handle buffer_data_read_page/reader_page order discrepancy in
ring_buffer_read_page(). On a mismatch simply copy manually the data to
the buffer_data_read_page.
Link: https://lore.kernel.org/all/20260817140812.2C7D41F00A3A@smtp.kernel.org/
Link: https://patch.msgid.link/20260904164450.1345852-3-vdonnefort@google.com
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
b90c5d770d |
bpf: Preserve inner map identity in callback frames
Callback frame constructors initialize map-typed argument registers with __mark_reg_known_zero() and then restore map_ptr. This clears map_uid, which is the only field distinguishing inner maps that share an inner_map_meta template. When a timer callback invokes bpf_for_each_map_elem() on a second inner map, both the saved first map and the second map value can reach the nested callback as the same template with map_uid zero. bpf_timer_init() then accepts pairing the timer from the second map with the first map. The runtime records the first map in the timer without taking a reference. Freeing that map does not find the timer stored in the second map, so a later timer callback dereferences the freed map. Copy map_uid from the same caller register as map_ptr when constructing for-each, timer/workqueue, and task-work callback arguments. The existing identity check can then reject mismatched inner maps while allowing a callback value to be paired with its actual map. Fixes: |
||
|
Kumar Kartikeya Dwivedi
|
ecdc504379 |
bpf: Mark NULL kptr stores precise
check_map_kptr_access() permits a scalar store into an untrusted kptr
field only when the register is known to contain zero. Unlike other
verifier checks whose outcome depends on a scalar value, it does not mark
that register precise.
A state checkpoint reached with an imprecise zero can therefore prune a
second path that reaches the store with an arbitrary nonzero scalar. The
program can write attacker-controlled bits into the kptr field and load
them back as a PTR_TO_BTF_ID.
Call mark_chain_precision() before accepting a known-zero register. This
forces state equivalence to compare its scalar range and makes the verifier
visit and reject a path carrying a nonzero value.
Fixes:
|
||
|
Nuoqi Gui
|
65cc95eba9 |
bpf: Cancel special fields when recycling rhtab elements
rhtab_map_update_existing() and rhtab_delete_elem() call
bpf_obj_free_fields() when replacing or deleting a value. These map
operations can run from BPF programs in NMI context, where releasing a
referenced kptr or another complex field is not generally safe.
Array and hash maps avoid that problem by cancelling only the asynchronous
fields which can be stopped safely in the caller context. Other ownership
state remains attached to the allocation until its memory allocator
destructor performs the final cleanup.
Use bpf_obj_cancel_fields() for the corresponding rhtab paths as well. This
cancels timers, workqueues, and task work while allowing rhtab_mem_dtor() to
release referenced kptrs when the allocation is eventually destroyed.
Fixes:
|
||
|
Yuan Chen
|
5df46ddcb7 |
bpf: Preserve special fields in recycled rhtab elements
rhtab_map_update_elem() initializes special fields after obtaining an element from bpf_mem_cache_alloc(). The allocator can return a fresh, zeroed unit, or recycle one from its RCU-pending lists before the registered destructor has run. A BPF program can retain a map-value pointer after deleting its element and initialize and arm a timer through that pointer. If the deleted unit is recycled, check_and_init_map_value() clears the only pointer to the timer. Neither a later deletion nor rhtab_mem_dtor() can then cancel it, and the callback can run with its key and value pointing into freed memory. Do not reinitialize special fields on insertion. Fresh allocator units are already zeroed. For recycled units, the special fields are ownership state that must remain visible to the eventual destructor. copy_map_value() already skips those fields, matching the non-preallocated hash-map path and the lifecycle established by commit |
||
|
Masami Hiramatsu (Google)
|
d7dbdd2ee0 |
tracing: Fix to avoid creating trace instances with duplicate names
Since commit |
||
|
Linus Torvalds
|
421066905c |
Probes fixes for v7.3-rc1:
- kprobes: Protect kprobe_blacklist with RCU
. RCU-protect kprobe_blacklist and use kfree_rcu() to prevent UAF
races during module unloading and enable safe atomic lookups.
- tracing/probes: Fix multi-probe field use-after-free and BTF parsing
. Multi-probe UAF fix: Duplicate field and type strings on
trace_probe_event to prevent UAF when freeing primary probe.
. BTF member lookup fixes:
- Check the containing inner struct/union kflag when resolving
anonymous members to ensure correct bitfield offset calculation.
- Prevent unnamed bitfields from being pushed to anon_stack in
btf_find_struct_member(), avoiding false lookup errors.
- Fix code block indentation in get_bitoffset_of_field().
- uprobes: Error pointer safety
. Guard free_trace_uprobe() with IS_ERR_OR_NULL() to avoid crashing
during automatic cleanup when an error pointer is returned.
-----BEGIN PGP SIGNATURE-----
iQFPBAABCgA5FiEEh7BulGwFlgAOi5DV2/sHvwUrPxsFAmqahXQbHG1hc2FtaS5o
aXJhbWF0c3VAZ21haWwuY29tAAoJENv7B78FKz8bOJIH/1RuAq2y8fvfqWKwDBNG
9CrSIMmZ0915s4LVSGQrrjNYfpj2rFYkEMsJcFo2pavKwWNyaxXFjXu8Vy9JGckx
VFAHA52x2QaEYdwBeoo/Jd3+7Ks/3zH1XwfSILFa0PMn86/JCKHx/+5ah6Sk4vcu
/he61Auyp6lJtvv88n95j1evCJNouU6lJ3fnvm8mNYTeLOIvPZ3qku6SsiOqNdeQ
Ln8bcNP2Iis33PqfeydiRWv9nPog/ifH4a9WJ+fdqKA+06AHKVsfHB+fP0hxyUDp
TS4oxKrIk6HIbEQRjgo8YcOPWurHAm0GQ2ZslgLFuWDE9rDQCtPdOYv/YzJe0ByG
nHc=
=hX9F
-----END PGP SIGNATURE-----
Merge tag 'probes-fixes-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull probes fixes from Masami Hiramatsu:
- Protect kprobe_blacklist with RCU
RCU-protect kprobe_blacklist and use kfree_rcu() to prevent UAF races
during module unloading and enable safe atomic lookups.
- Fix multi-probe field use-after-free
Duplicate field and type strings on trace_probe_event to prevent UAF
when freeing primary probe
- Fix probe BTF member lookup:
Check the containing inner struct/union kflag when resolving
anonymous members to ensure correct bitfield offset calculation
Prevent unnamed bitfields from being pushed to anon_stack in
btf_find_struct_member(), avoiding false lookup errors
Fix code block indentation in get_bitoffset_of_field()
- uprobes error pointer safety
Guard free_trace_uprobe() with IS_ERR_OR_NULL() to avoid crashing
during automatic cleanup when an error pointer is returned
* tag 'probes-fixes-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
kprobes: Protect kprobe_blacklist with RCU
tracing/probes: Fix use-after-free on field name/type of events with multiple probes
tracing/probes: Fix code indent in get_bitoffset_of_field()
tracing/probes: Fix BTF kflag check for anonymous struct member access
tracing/probes: Fix anon_stack check for unnamed bitfields in btf_find_struct_member
uprobes: guard trace cleanup against error pointers
|
||
|
Kumar Kartikeya Dwivedi
|
cd6f72d7f3 |
bpf: Clear NON_OWN_REF after RCU protection ends
A local kptr load of an object containing a graph node is marked MEM_RCU
and NON_OWN_REF while protected by RCU. When the last RCU read-side critical
section ends, invalidate_rcu_protected_refs() removes MEM_RCU and marks the
pointer PTR_UNTRUSTED, but leaves NON_OWN_REF set.
The stale flag lets graph kfunc argument checks continue treating the
pointer as a live borrowed reference. In particular, bpf_rbtree_remove()
can accept a pointer after its protection ended and return it as a new
owning reference, even though the object may already have been freed.
Clear NON_OWN_REF when an RCU-protected pointer is demoted. A spin lock also
provides implicit RCU protection, so invalidate non-owning references before
demoting RCU-protected pointers when releasing the lock. Otherwise the
demotion would clear the flag before invalidate_non_owning_refs() can find
and invalidate those aliases.
The demoted pointer remains available for fault-protected reads. Exempt such
reads from the allocated-object reference-state assertion; writes through a
fault-prone pointer are already rejected, and bpf_may_fault_on_deref() makes
the surviving loads use BPF_PROBE_MEM.
Fixes:
|
||
|
Ning Ding
|
dc36739e5c |
bpf: Keep refcount_acquire nullable for borrowed RCU kptrs
bpf_refcount_acquire() is fallible for a borrowed reference because the
object may have reached a zero refcount. The verifier therefore keeps
KF_RET_NULL on the return value unless the argument is an owning reference.
An RCU-protected load of a local kptr is marked MEM_ALLOC, but it only
receives NON_OWN_REF when the pointee contains a graph node. A refcounted
object without a graph node consequently looks like an owning reference
even though the loaded register has no acquired reference state. If the
program drops the last real reference while remaining in the RCU critical
section, refcount_inc_not_zero() returns NULL while the verifier treats the
result as non-NULL.
Only classify the argument as owning when it is backed by a verifier-tracked
reference. This retains the non-NULL return for pointers from bpf_obj_new(),
bpf_kptr_xchg(), or an earlier successful acquisition, while requiring a
NULL check for borrowed RCU kptrs.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
048029ba1c |
bpf: Require MEM_PERCPU for percpu kptr stores
map_kptr_match_type() treats perm_flags as the set of register type flags
that a kptr field permits. Adding MEM_PERCPU to that set for
BPF_KPTR_PERCPU does not require the source register to carry it, however.
The subset test consequently accepts both a plain bpf_obj_new() allocation
and a referenced kernel pointer into a __percpu_kptr map field.
Loads from the field are always marked MEM_PERCPU. Consumers then treat the
stored value as the cookie returned by bpf_percpu_obj_new(): per-CPU pointer
helpers relocate it, and map teardown selects the per-CPU free path. A plain
allocation can therefore provide an arbitrary kernel read/write, while a
kernel pointer can be relocated into an invalid address or sent through a
missing destructor.
Require the source MEM_PERCPU flag to match the destination field kind.
This preserves valid bpf_percpu_obj_new() stores and rejects both the
program-BTF and kernel-BTF variants.
Fixes:
|
||
|
Eduard Zingerman
|
6aed0134d3
|
bpf: Mark the zero register precise for a register-form NULL check
check_cond_jmp_op() accepts "if rA <op> rB" as a NULL check for a
nullable pointer rA when rB is a scalar known to be zero,
lifts PTR_MAYBE_NULL from rA in the corresponding branch and does not
mark rB precise. Consider the following program:
r0 = bpf_get_prandom_u32();
r6 = 1; /* the r6 == 0 path is explored first */
if (r0 == 0) goto 1f;
r6 = 0;
1:
r0 = bpf_map_lookup_elem(map, &0); /* absent, NULL at runtime */
if (r0 == r6) goto 2f; /* taken as a NULL check for r0 */
*(u8 *)(r0 + 0); /* verifier: map value; runtime: zero */
2:
return 0;
The r6 == 0 path is explored first and the dereference is accepted.
The r6 == 1 path is pruned at the checkpoint recorded for (1),
so the comparison is never verified with a non-zero r6. At runtime a
failed lookup returns NULL, NULL != 1 takes the non-NULL edge and the
program dereferences a pointer that is zero.
Fixes:
|
||
|
Eduard Zingerman
|
e51179a4e0
|
bpf: Don't predict JMP32 pointer vs zero comparisons
Consider the following program:
r1 = map_value; /* low 32 bits are zero at runtime */
r6 = 0xdead000000000000;
if w1 != 0 goto l1;
l0: r1 += r6;
r2 = *(u64 *)(r1 + 0);
exit;
l1: r6 = 0;
goto l0;
At the moment is_branch_taken() reports the jump as always taken,
because it does not distinguish between BPF_JMP and BPF_JMP32
comparisons when processing 'if w1 != 0 ...'.
Fixes:
|
||
|
Eduard Zingerman
|
73a98f9681
|
bpf: Don't resurrect a scalar id dropped by collect_linked_regs()
check_cond_jmp_op() copies the compared registers into
env->{false,true}_reg{1,2} before collect_linked_regs() runs and copies
those snapshots back into both branch states afterwards.
collect_linked_regs() records at most LINKED_REGS_MAX members of a
linked registers group in the jump history and calls clear_scalar_id()
for every member that does not fit. The compared register is not exempt
from that.
As a consequence, sync_linked_regs() might adjust ranges for more
registers than bpf_bt_sync_linked_regs() can propagate precision to.
Collect the linked registers before the snapshots are taken instead.
This might lead to some unnecessary clear_scalar_id's, but from
previous testing situations with many linked registers are
extremely rare.
Fixes:
|
||
|
Eduard Zingerman
|
67b529f521
|
bpf: Don't infer non-NULL from a pointer with an unbounded offset
reg_not_null() decides that a register holds a non-NULL value by
looking at its type alone. For pointer types that allow arithmetic the
type only guarantees a non-NULL base, in case of an unbound offset
the runtime offset value might still add up to NULL.
Consider the followng program:
r6 = bpf_map_lookup_elem(map, &0); /* present */
if (r6 == 0) return 0;
r7 = bpf_map_lookup_elem(map, &1); /* absent, NULL at runtime */
r8 = r7;
r8 -= r6; /* pointer - pointer: unknown scalar, -r6 */
r8 <<= 1;
r8 >>= 1; /* any non-negative offset is accepted by */
/* check_reg_sane_offset_ptr() */
r6 += r8; /* verifier: map value; runtime: zero */
if (r7 != r6) return 0;
*(u8 *)(r7 + 0); /* r7 is inferred non-NULL, both are zero */
At runtime both registers are zero, the comparison is true and the
load faults with NULL pointer dereference.
Require the offset to be within +-BPF_MAX_VAR_OFF in reg_not_null().
Fixes:
|
||
|
Daehyeon Ko
|
cdd812d068
|
exit: hold a reference to thread_pid across proc_flush_pid
Commit |
||
|
Yao Kai
|
a3b8d46fe4 |
futex: Prevent rcuwait use-after-free during requeue PI
On PREEMPT_RT, FUTEX_CMP_REQUEUE_PI can trigger a KASAN report
(slab-out-of-bounds) in futex_requeue_pi_complete() invocation of
rcuwait_wake_up().
The futex_q used by futex_wait_requeue_pi() is allocated on the waiter's
stack. An early wakeup can race with a PI requeue as follows:
waiter requeue task
------ ------------
futex_wait_requeue_pi()
futex_do_wait()
schedule()
futex_requeue
futex_proxy_trylock_atomic()
futex_requeue_pi_prepare()
Q_REQUEUE_PI_NONE -> Q_REQUEUE_PI_IN_PROGRESS
* timeout/ signal wakes waiter *
futex_requeue_pi_wakeup_sync()
Q_REQUEUE_PI_IN_PROGRESS -> Q_REQUEUE_PI_WAIT
requeue_pi_wake_futex
futex_requeue_pi_complete()
cmpxchg Q_REQUEUE_PI_WAIT -> Q_REQUEUE_PI_LOCKED
rcuwait_wait_event()
if (atomic_read(&q->requeue_state) != Q_REQUEUE_PI_WAIT)
break /* no schedule() */
/* q.pi_state->owner == current */
futex_private_hash_put()
/* return from syscall */
rcuwait_wake_up(&q->requeue_wait)
/* q is gone */
futex_requeue_pi_complete() publishes Q_REQUEUE_PI_LOCKED before
calling rcuwait_wake_up(). The waiter observes this state in
rcuwait_wait_event() before invoking schedule() in rcuwait_wait_event().
Here, the waiter is free leave the syscall before requeue task can
complete the wake.
To address this race skip rcuwait_wake_up() in the Q_REQUEUE_PI_LOCKED
case.
This state is only published by requeue_pi_wake_futex(), which saves
q->task before futex_requeue_pi_complete() and wakes the waiter via
wake_up_state().
This wake is intended to wake the waiter from its futex_do_wait() sleep.
If the waiter is still sleeping there, it can not get into the
Q_REQUEUE_PI_WAIT state (and require this removed wake).
Should the waiter be woken up from futex_do_wait() by other means (as in
this example) and sleep in futex_requeue_pi_wakeup_sync() then the
wake_up_state() from requeue_pi_wake_futex() will wake it, too.
Should the waiter task terminate before wake_up_state() had a chance to
wake the task then the task pointer does not become invalid because the
futex_hash_bucket::lock is held and the task pointer is RCU protected.
[bigeasy: Updated comment and commit message]
Fixes:
|
||
|
Sebastian Andrzej Siewior
|
912edebe85 |
futex: Provide rt_mutex_.*_schedule() equivalents for futex scheduling
There is rt_mutex_{pre|post}_schedule() around
rt_mutex_wait_proxy_lock() to ensure that sched_submit_work()/
sched_update_worker() is invoked before we schedule out and block on
rt_mutex while waiting for it become available.
The reason is that blocking on rt_mutex assigns a pi_waiter for the PI
chain and sched_submit_work() will also assign a pi_waiter if it blocks
on lock but a this point we already have a waiter assigned.
We can't skip sched_submit_work() entirely because I/O relies on the
fact that I/O queue is flushed while it blocks on a sleeping lock.
Therefore sched_submit_work() is moved before we block on the lock.
Sleeping lock in this context means mutex or rw_semaphore not spinlock_t
on PREEMPT_RT. Because the mutex abstraction on PREEMPT_RT uses the same
abstraction as the futex proxy lock, the futex code ended up using
rt_mutex_{pre|post}_schedule(), too.
Using it is/ was just to keep the task_struct::sched_rt_mutex assertion
happy. Futex proxy lock is used only in the syscall context of a task.
At this point it never got any I/O that needs to be flushed and it can't
be a workqueue that needs to notify that it will be scheduled out.
Therefore sched_submit_work() does nothing here.
By mistake futex_wait_requeue_pi() -> rt_mutex_wait_proxy_lock() did not
get the rt_mutex_{pre|post}_schedule() annotation. This was not noticed
because in this callchain the lock is (usually) not contended and so
rt_mutex_slowlock_block() does not schedule, triggering the assert.
Adding rt_mutex_pre_schedule() here looks wrong (as noted by PeterZ)
because at this point there is a pi_waiter recorded and invoking
sched_submit_work() with a possible lock contention would be wrong.
Add rt_mutex_futex_{pre|post}_schedule() which toggles the
sched_rt_mutex assert and does not involve sched_submit_work(). Add
asserts here to ensure that sched_submit_work() would do nothing. Use it
only in futex proxy lock case which is rt_mutex_wait_proxy_lock().
Remove it from futex_lock_pi().
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
e7d28823c6 |
bpf: Reject legacy packet loads from callbacks
check_ld_abs() models a failed BPF_LD_ABS or BPF_LD_IND in a
subprogram as an implicit return with R0 set to zero. It calls
prepare_func_exit() to explore this synthesized path.
When the load is reached directly from a synchronous callback,
prepare_func_exit() enforces the callback return contract and marks R0
precise. R0 is not derived from a real instruction on this path, so
precision backtracking reaches the callback call with R0 still requested
and triggers the "callback unexpected regs" verifier bug. A privileged
program loader can therefore cause a verifier warning and an -EFAULT
BPF_PROG_LOAD.
These legacy packet-load instructions are deprecated. Reject them from
callbacks rather than complicating their implicit-return model. Check all
active frames before constructing the implicit return so nested static
subprograms cannot hide the callback context.
Global functions are verified independently with a fresh frame zero, so
an active-frame check cannot identify a global function called from a
callback. Also check the complete subprogram call graph during stack-depth
validation and reject a function containing a legacy load when any caller
is a callback. This covers global and static descendants without making
has_ld_abs transitive, preserving its per-function BTF return-type check.
Ordinary uses outside callbacks remain supported.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
9d02927fdf |
bpf: Mark faultable stack helpers as sleepable
The faultable variants of bpf_get_stack() and bpf_get_task_stack() pass
may_fault=true into the common stack collection code. Resolving user-space
build IDs may then call build_id_parse_file() and block on filesystem
reads.
Neither helper prototype sets might_sleep. Since prototype selection uses
the sleepability of the whole program, the verifier can still allow these
helpers from a non-sleepable region within that program, such as an
explicit RCU or preemption-disabled region. The task-stack helper can also
be called from a non-sleepable timer callback of a sleepable program.
Mark both faultable prototypes as sleepable. The existing helper context
check then rejects these calls while continuing to allow them in genuinely
sleepable contexts.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
620614bf76 |
bpf: Mark bpf_btf_find_by_name_kind() as sleepable
When bpf_btf_find_by_name_kind() finds a type in module BTF, it
returns a new BTF object fd through __btf_new_fd(). This reaches
anon_inode_getfd(), which can sleep while allocating or expanding the
current task fd table.
The helper prototype does not set might_sleep, so the verifier allows
the helper in non-sleepable contexts such as BPF timer callbacks. The
fd allocation can then sleep in softirq context and install the fd into
the interrupted task.
Mark the helper as sleepable. This preserves calls from the main body
of a sleepable syscall program while rejecting calls from its
non-sleepable regions.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
369f4ce734 |
bpf: Check ancestor frames for rbtree callbacks
bpf_rbtree_add() invokes its comparator while the caller holds the root
lock. The native insertion code retains raw parent and link pointers across
the callback, so the verifier prohibits unlocking, consuming tree nodes,
or changing RCU state from that callback.
in_rbtree_lock_required_cb() only checks the innermost verifier frame.
Static subprogram calls are permitted while holding a spin lock, and such a
call pushes a frame without in_callback_fn set. Consequently, all callback
restrictions disappear in the nested frame. The subprogram can unlock the
tree, remove and drop the node being compared, then relock. Native insertion
resumes with the stale parent pointer and links freed memory into the tree.
Walk all active frames for the rbtree callback instead. Benign static
subprograms remain permitted, while callback restrictions follow execution
into nested frames.
Fixes:
|
||
|
Eduard Zingerman
|
0b1c83dc3c |
bpf: don't rewrite bpf_fastcall patterns entered by a jump
mark_fastcall_pattern_for_call() must ensure that matched
"spill; call; fill" instruction series is not interrupted by a jump.
Otherwise the rewrite applied by bpf_remove_fastcall_spills_fills()
is not sound.
Record the instructions targeted by jumps in
insn_aux_data[*].jump_target when the CFG is built and use this flag
to stop growing a pattern at such an instruction. Jumps to the first
spill are fine.
Note that existing insn_aux_data[*].jmp_point field can't be reused,
as it marks subprogram return instructions.
Fixes:
|
||
|
Eduard Zingerman
|
1f3cd9719c |
bpf: update disasm.c to print BPF_PROBE_ATOMIC as atomics
bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, this patch adjusts print_bpf_insn() to print such instructions as regular atomics with a 'probe_' prefix (instead of printing them as BUG_XX). Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260903171542.1438050-2-eddyz87@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|
Eduard Zingerman
|
4814ed6406 |
bpf: zero extend the result of an arena 32-bit cmpxchg
bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from
BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, and it runs before
bpf_opt_subreg_zext_lo32_rnd_hi32().
That pass emits an explicit zero extension for a 32-bit cmpxchg even
when bpf_jit_needs_zext() is false. This is done because on some
architectures 32-bit cmpxchg requires explicit zero extension for the
dst register. E.g. on x86-64 'lock cmpxchg' does not change the %eax
if comparison is successful, while BPF semantics declare that each
operation on a 32-bit register zero extends it's upper half.
is_cmpxchg_insn() matches BPF_MODE == BPF_ATOMIC only, so an arena
cmpxchg misses said zero extension adjustment. This patch adjusts
is_cmpxchg_insn() to match BPF_PROBE_ATOMIC alongside BPF_ATOMIC.
Fixes:
|
||
|
Steven Rostedt
|
6c001a62c3 |
ring-buffer: Add checking nr_subbufs to persistent ring buffer validation
Sashiko reported that the code was using meta->nr_subbufs without making
sure that it matched the nr_pages + 1 on data that was assuming the two
were the same.
Add a check to the persistent ring buffer validation code to make sure
that the saved nr_subbufs matches what we expect.
Link: https://patch.msgid.link/20260903132728.7fb27d34@gandalf.local.home
Fixes:
|
||
|
Wanwu Li
|
0a85182723 |
sched_ext: Fix NULL sched deref in kfunc sub-sched error paths
When the root scheduler has sub-scheds attached, the COMPAT kfunc
wrappers scx_bpf_select_cpu_and() and scx_bpf_dsq_insert_vtime() refuse
the call and report to @p's scheduler:
scx_error(scx_task_sched(p), "... must be used");
The wrappers are reachable with tasks that have no scheduler.
scx_bpf_select_cpu_and() is in the select_cpu kfunc group, which
scx_kfunc_context_filter() opens to BPF_PROG_TYPE_SYSCALL programs;
scx_bpf_dsq_insert_vtime() is in the enqueue_dispatch group, which
ops.enqueue() and ops.dispatch() may call with any KF_RCU task -- the
group has no kf_tasks validation, and scx_dsq_insert_preamble() checks
task ownership with scx_task_on_sched() precisely because @p may be an
arbitrary task.
scx_task_sched(p) is p->scx.sched, which is NULL for tasks past
sched_ext_dead() -- which clears it via scx_disable_and_exit_task() on
exit -- and for idle tasks, which the enable paths skip as they are
never scheduled through SCX. It is also an rcu_dereference_protected()
that expects @p's pi_lock or rq lock, which neither wrapper holds.
Passing NULL to scx_error() reaches scx_vexit(), which dereferences
sch->exit_info, oopsing the kernel.
One concrete trigger exercised while developing the fix: a
BPF_PROG_TYPE_SYSCALL program calling the select_cpu_and wrapper on an
exited-but-not-reaped task while a sub-scheduler was attached (its pid
stays findable while the zombie is unreaped; faulting instruction is
the scx_vexit() prologue "mov r15,[rdi+0x398]" with RDI=NULL and 0x398
the offset of sch->exit_info):
sched_ext: BPF scheduler "kfunc_subsched_null" enabled
sched_ext: BPF sub-scheduler "kfunc_subsched_null" enabled
sched_ext: Unassociated program run_select_cpu_ (id 76)
BUG: kernel NULL pointer dereference, address: 0000000000000398
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
Oops: Oops: 0000 [#1] SMP NOPTI
CPU: 7 UID: 0 PID: 8201 Comm: kfunc_test_runn Tainted: G W
RIP: 0010:scx_vexit+0x25/0xa0
Code: ... <4c> 8b bf 98 03 00 00 ...
CR2: 0000000000000398
Call Trace:
<TASK>
__scx_exit+0x4f/0x70
scx_bpf_select_cpu_and+0xab/0xb0
bpf_prog_430ed61a7b66e03a_run_select_cpu_and+0x9c/0xe7
? __x64_sys_bpf+0x2c/0x40
bpf_prog_test_run_syscall+0x130/0x2f0
__sys_bpf+0x930/0x10d0
? __x64_sys_bpf+0x2c/0x40
__x64_sys_bpf+0x2c/0x40
do_syscall_64+0xbc/0x460
entry_SYSCALL_64_after_hwframe+0x76/0x7e
</TASK>
Read @p's scheduler under RCU instead, which the wrappers can do from
their guard(rcu)(): fault it when it can be determined, and when it
can't be determined -- @p is a task past sched_ext_dead() or an idle
task -- there is nothing obviously wrong to report, so just refuse the
call as before without faulting any scheduler.
These COMPAT wrappers are scheduled for eventual removal once the
deprecation grace period elapses, but until then -- and regardless of
their removal timeline -- they must not oops the kernel on a task they
are handed.
Cc: stable@vger.kernel.org # v7.1+
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
d055247942 |
bpf: Mark syscall helpers as sleepable
bpf_sys_bpf() executes the bpf(2) syscall body, which can take mutexes, allocate with GFP_KERNEL, and wait for an RCU grace period. bpf_sys_close() reaches close_fd() and filp_close(), which can sleep as well. Both helpers are limited to BPF_PROG_TYPE_SYSCALL, whose main program is sleepable. That does not make every callback sleepable: a syscall program can register a bpf_timer callback, and the verifier checks that callback in a non-sleepable context while retaining the syscall helper set. Without .might_sleep on the prototypes, such a callback can invoke bpf_sys_bpf() from hrtimer softirq context and trigger a scheduling-while-atomic failure. bpf_sys_close() is exposed through the same missing context check. Set .might_sleep on both prototypes so the existing helper-context check rejects them from timer callbacks and other atomic regions. Calls from the sleepable main body remain valid. Fixes: |
||
|
Kumar Kartikeya Dwivedi
|
a453d6e3b8 |
bpf: Mark sched_process_wait argument as nullable
do_wait() passes wo->wo_pid to the sched_process_wait tracepoint.
kernel_wait4() leaves wo_pid NULL for wait4(-1), and
kernel_waitid_prepare() does likewise for waitid(P_ALL).
btf_ctx_access() currently types argument 0 as PTR_TO_BTF_ID |
PTR_TRUSTED. Without PTR_MAYBE_NULL, the verifier accepts an unchecked
dereference. Trusted pointer loads have no fault protection, so a wait for
any child can then cause a NULL pointer dereference in JITed BPF code.
Add sched_process_wait to raw_tp_null_args[] with argument 0 marked
nullable. The verifier rejects an unchecked dereference while preserving
access after the program checks the pointer for NULL.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
7b7b8b5960 |
bpf: Reject resilient lock operations in rbtree callbacks
__bpf_rbtree_add() keeps parent and link pointers live across calls to the
program-supplied comparison callback. The verifier therefore requires the
root's lock to remain held throughout the callback.
The helper path enforces this rule for bpf_spin_lock() and
bpf_spin_unlock(), but the resilient lock kfunc argument path does not.
Since resilient locks may protect BPF rbtree roots, a callback can release
the root lock and let another CPU remove and free the node referenced by
the in-progress tree walk. The walk then resumes using freed pointers.
Reject resilient lock kfuncs in an rbtree comparison callback, matching
the existing policy for the spin lock helpers. Resilient-lock-protected
trees remain valid when their comparison callbacks leave lock state alone.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
266aa4ad0b |
bpf: Reject tail calls directly from callback frames
A tail call from a non-zero frame is modeled as a return from that frame.
The verifier makes R0 unknown and calls prepare_func_exit() for the taken
branch.
When the current frame is a synchronous callback, prepare_func_exit()
enforces the callback return-value contract and marks R0 precise. Since the
tail-call path synthesized R0 rather than deriving it from an instruction,
precision backtracking reaches the callback-calling instruction with R0
still requested and triggers the "callback unexpected regs" verifier bug.
A CAP_BPF task can therefore cause a WARN and an -EFAULT BPF_PROG_LOAD.
Tail calls reachable from callbacks are already rejected later by
check_max_stack_depth(). Reject a tail call made directly by a callback
before constructing the inconsistent return state, using the existing
diagnostic. Tail calls from ordinary subprograms keep their current
behavior.
Fixes:
|
||
|
Kumar Kartikeya Dwivedi
|
77515ab12e |
bpf: Mark signal tracepoint siginfo arguments as scalar
The signal_generate and signal_deliver tracepoints declare their info
argument as a struct kernel_siginfo pointer. btf_ctx_access() therefore
treats it as a trusted pointer for tp_btf programs.
Signal delivery also uses SEND_SIG_NOINFO and SEND_SIG_PRIV as special
values for this argument. Those values are zero and one respectively,
and are not pointers. A tp_btf program can currently dereference either
value and fault the kernel. In particular, signal_generate can run from
timer interrupt context, turning the fault into a kernel panic.
Record both tracepoints in raw_tp_null_args[] and mark argument one as
a non-pointer. This preserves scalar access to the cookie while rejecting
direct and helper-mediated pointer use. Merely marking it nullable would
not suffice because SEND_SIG_PRIV is nonzero.
Fixes:
|
||
|
Jiayuan Chen
|
5403a383f5 |
bpf: Fix NULL-ptr-deref in btf_var_show()
btf_var_show() calls btf_type_id_resolve() unconditionally, which
dereferences btf->resolved_ids. That is NULL for a base BTF - e.g. the
vmlinux BTF that bpf_snprintf_btf() renders against - since base BTF is
not resolved during parsing. btf_modifier_show() guards this with
'if (btf->resolved_ids)', but btf_var_show() does not.
A BPF program that passes the type_id of a BTF_KIND_VAR from the vmlinux
BTF to bpf_snprintf_btf() thus NULL-derefs:
KASAN: probably user-memory-access in range [0x46638-0x4663f]
RIP: 0010:btf_var_show (kernel/bpf/btf.c:2929)
Call Trace:
<TASK>
btf_type_show (kernel/bpf/btf.c:8259)
btf_type_snprintf_show (kernel/bpf/btf.c:8329)
bpf_snprintf_btf (kernel/trace/bpf_trace.c:1047)
bpf_prog_test_run_raw_tp (net/bpf/test_run.c:829)
__sys_bpf (kernel/bpf/syscall.c:4804)
do_syscall_64 (arch/x86/entry/syscall_64.c:84)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
Resolve the var's type directly with btf_type_skip_modifiers() when
resolved_ids is NULL, mirroring btf_modifier_show().
Fixes:
|
||
|
Jiayuan Chen
|
4ea508b9eb |
bpf: Fix NULL-ptr-deref when showing a void BTF type
btf_modifier_show() resolves the modifier and then calls
btf_type_ops(t)->show() unconditionally. For the void type (type_id 0,
BTF_KIND_UNKN) kind_ops[] has no entry, so ->show is NULL.
A "const void" (a modifier resolving to void) cannot be a map key or
value - map_check_btf() rejects it because void has no size - so the map
dump path does not reach it. But bpf_snprintf_btf() takes a type_id
straight from the BPF program, and passing such a "const void" from the
vmlinux BTF NULL-derefs:
KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
RIP: 0010:btf_modifier_show (kernel/bpf/btf.c:2914)
Call Trace:
<TASK>
btf_type_show (kernel/bpf/btf.c:8251)
btf_type_snprintf_show (kernel/bpf/btf.c:8321)
bpf_snprintf_btf (kernel/trace/bpf_trace.c:1047)
bpf_prog_test_run_raw_tp (net/bpf/test_run.c:829)
__sys_bpf (kernel/bpf/syscall.c:4804)
do_syscall_64 (arch/x86/entry/syscall_64.c:94)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
Fall back to btf_df_show() when the resolved type has no show op; it
emits the "<unsupported kind:N>" placeholder already used for kinds like
FWD and FUNC. bpf_snprintf_btf() then returns the length as usual.
Fixes:
|
||
|
Jiayuan Chen
|
0895a0c073 |
bpf: Reject key-less BTF for hash maps
map_check_btf() allows a key-less BTF (btf_key_type_id == 0) only for maps that have a ->map_check_btf callback, and leaves the actual decision to that callback. Hash maps used to have no ->map_check_btf, so a key-less BTF was rejected outright. That changed when htab and rhtab gained a ->map_check_btf to register a dtor - htab in commit |
||
|
Vincent Donnefort
|
6365c44a82 |
ring-buffer: Allow splice reads on static buffers
ring_buffer_read_page() rejects splice (full=1) reads on static buffers
(that is user-mapped, persistent or remote) because !read check assumes
unread pages must be swapped. However for those buffers we have no other
choice than memcpy the data.
For the memcpy case, only return an error when the writer is still on
the reader page for the splice interface to wait.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260901155445.1475405-2-vdonnefort@google.com
Fixes:
|
||
|
Eduard Zingerman
|
374b2c5561 |
bpf: reject BPF_PSEUDO_FUNC reference to the main program
fixups.c:jit_subprogs() rewrites BPF_PSEUDO_FUNC loads to contain real
function addresses. This function is invoked from bpf_jit_subprogs()
only when env->subprog_cnt > 1. Meaning that for any program like
below:
int main(void *ctx) {
void *ptr = main;
...
bpf_timer_set_callback(..., ptr);
...
}
The 'ptr' won't be ever converted to contain an address.
In combination with e.g. bpf_timer_set_callback() this would lead to a
function call at a bogus address.
Instead of complicating the implementation, just assume that no useful
program needs main to be a sync or async callback and reject
BPF_PSEUDO_FUNC loads for the main subprogram.
Fixes:
|
||
|
Masami Hiramatsu (Google)
|
0c4256196b |
kprobes: Protect kprobe_blacklist with RCU
__within_kprobe_blacklist() traverses kprobe_blacklist without holding
kprobe_mutex. When a module is unloaded, kprobe_remove_area_blacklist()
removes blacklist entries and immediately frees them with kfree().
A concurrent call to within_kprobe_blacklist() can therefore dereference
freed memory.
Furthermore, within_kprobe_blacklist() can be called in atomic or
non-preemptible contexts where the sleeping kprobe_mutex cannot be taken.
Protect kprobe_blacklist with RCU. Use guard(rcu)() and
list_for_each_entry_rcu() for traversal, list_add_tail_rcu() for
insertions, list_del_rcu() for deletions, and kfree_rcu() to reclaim
entries safely after a grace period.
Link: https://lore.kernel.org/all/178810004323.64882.16493230858653316962.stgit@devnote2/
Fixes:
|
||
|
Henry Martin
|
86b7a239ec |
tracing/probes: Fix use-after-free on field name/type of events with multiple probes
The fields of a probe-based dynamic event (kprobe, uprobe, eprobe and
fprobe events) are created in traceprobe_define_arg_fields() by handing
the probe_arg name/type strings to trace_define_field(), which only
stores the pointers without copying. Those strings are owned by the
trace_probe and are freed when that probe is removed.
An event can have several probes attached. The field list is defined
only once, by the first probe that registers the event, but it is kept
alive by any surviving sibling probe. Deleting just that first probe by
symbol -
# primary A: fields are defined from A's args
echo 'p:kprobes/ev vfs_read a1=$arg1' > kprobe_events
# append B: shares A's event call
echo 'p:kprobes/ev vfs_write a1=$arg1' >> kprobe_events
# delete only A (matched by symbol), B survives
echo '-:kprobes/ev vfs_read' >> kprobe_events
frees A's args (trace_probe_cleanup() -> traceprobe_free_probe_arg()),
but trace_probe_unlink() keeps the trace_probe_event because the probe
list is not empty. The event call stays registered via B while its
fields now reference freed memory. Any field lookup then reads it, e.g.
echo 'a1 == 1' > events/kprobes/ev/filter
BUG: KASAN: slab-use-after-free in strcmp+0xa7/0xb0
Call Trace:
strcmp
trace_find_event_field
parse_pred
process_preds
create_filter
apply_event_filter
event_filter_write
field->name references parg->name (kstrdup'd, freed with the probe) and,
for array arguments, field->type references parg->fmt (kmalloc'd, freed
with the probe) - the scalar type otherwise points at the static
fmttype rodata, which is safe.
Have traceprobe_define_arg_fields() duplicate the name and type strings
and anchor the copies on the trace_probe_event, which embeds the event
call and outlives every individual probe; trace_probe_event_free()
releases them.
The reproducer above triggers reliably; the field lookup and the delete
both run under event_mutex, so this is a dangling reference after
removal rather than a race.
The issue was found by the autokbug dynamic kernel fuzzer at Tencent
Yunding Lab.
Link: https://lore.kernel.org/all/20260826030009.1855331-1-bsdhenrymartin@gmail.com/
Fixes:
|
||
|
Masami Hiramatsu (Google)
|
871e07b6e3 |
tracing/probes: Fix code indent in get_bitoffset_of_field()
Fix code block indentation introduced by commit
|
||
|
Masami Hiramatsu (Google)
|
47e93045a2 |
tracing/probes: Fix BTF kflag check for anonymous struct member access
btf_find_struct_member() traverses into nested anonymous structures and
unions to find a struct member. However, get_bitoffset_of_field() in
trace_probe.c checked btf_type_kflag(type) using the outer parent type
instead of the actual anonymous structure/union that directly contains
the found member.
If the parent structure and anonymous structure have mismatched kflags
(e.g., the parent has kflag=0 while the anonymous structure has kflag=1
because it contains bitfields), the bitfield size encoded in the upper
8 bits of member->offset is erroneously treated as part of the byte/bit
offset, corrupting the resolved offset and failing to set last_bitsize.
Similarly, btf_find_struct_member() pushed anonymous member offsets
onto anon_stack without masking BTF_MEMBER_BIT_OFFSET() when kflag is set.
To fix this problem, update btf_find_struct_member() to return actual
containing structure/union type via member_type, use appropriate
__btf_member_bit_offset() to get bit offset, and use member_type for
btf_type_kflag() in get_bitoffset_of_field().
Link: https://lore.kernel.org/all/178827250904.123716.17452648791331881284.stgit@devnote2/
Fixes:
|
||
|
Masami Hiramatsu (Google)
|
f36d94a20c |
tracing/probes: Fix anon_stack check for unnamed bitfields in btf_find_struct_member
btf_find_struct_member() traverses into nested anonymous structures
and unions by pushing members with !member->name_off onto anon_stack.
However, it does not consider the unnamed bitfields (e.g. `int : 5`
or `unsigned int : 0`) which also have member->name_off == 0.
If such an unnamed bitfield is pushed to anon_stack, the
btf_find_struct_member() return an error even if there are other
valid entries in anon_stack.
To fix this, only push unnamed struct/union members to anon_stack.
Also move the btf_type_is_struct() check to the entry of this function
because now it is sure only struct/union are pushed to anon_stack.
Link: https://lore.kernel.org/all/178827249775.123716.7813217688423513612.stgit@devnote2/
Fixes:
|
||
|
Steven Rostedt
|
f2951ebd15 |
tracing: Take trace_array reference when opening options file
The options files do not take the trace_array reference for the options
they represent. This could cause a use-after-free kernel crash if one of
these files is opened by one task and another task removes the instance
that the option is for. Because it doesn't take a reference upon opening,
it will not stop the removal which will free the options descriptor that
is being used.
As the options are somewhat dynamic in their creation at boot up, each
file represents a flag in the trace_array. The trace_array has an array of
indexes to represent each of these flags that is stored in the
trace_flags_index array. The address of the index array element is used to
pass to the inode->i_private pointer. Then that element is read which
holds the index (which represents the flag) and then the index is used to
calculate the trace_array descriptor from its trace_flags_index array.
One issue is that the index element can not be referenced until the
trace_array's reference is taken. To handle this, create a new helper
function called: trace_array_options_get() that will iterate all the
existing trace_arrays in the ftrace_trace_arrays list (under the
trace_types_lock), and compare the passed in address of the index element
with the entire array of the trace_array's trace_flags_index array.
If it matches, then up the corresponding trace_array's reference and
return.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260902121918.5a9e9d1b@gandalf.local.home
Fixes:
|
||
|
Eduard Zingerman
|
e3e4f66cc4 |
bpf: backtracking shouldn't clear outer frame R1-R5 for callbacks
When processing calls to bpf_loop() verifier marks R1 (and R4) as
precise. R1 tracks loop iterations number and because of the
'callback_depth < R1' mechanics in check_helper_call() must be marked
precise. However, precision propagation for R1 was broken,
when bpf_loop() call was verified on a second iteration.
Consider the following verification trace:
- main: bpf_loop(nr_loops, callback ...)
- callback: BPF_EXIT
- main: bpf_loop(nr_loops, callback ...)
- ...
While the first visit of the call to bpf_loop() propagated R1
precision as expected, the second call to mark_chain_precision() in
the check_helper_call() set R1, but it was immediately reset when
backtrack_insn() processed preceding BPF_EXIT in the loop deleted in
this patch.
Because of that, the second visit of the call to bpf_loop() injected
checkpoint with R1 not marked as precise. Which could trick the
verifier into accepting unsafe programs. See the next patch for an
example of such program.
Commit is structured in a way to minimize conflicts when
'bpf' would be eventually merged with 'bpf-next'.
Fixes:
|
||
|
Steven Rostedt
|
4617721c50 |
ftrace: Synchronize the initialization of ftrace_ops
There's some internal state that ftrace_ops needs to have set, but since
it can be declared outside of the ftrace.c code, it calls
ftrace_ops_init() on the ops in every global function. The issue is that
if two tasks call it on the same ops at the same time it is possible to
have the initialization of one corrupt the initialization of the other
call.
Create a ops_mutex to use to synchronize every initialization of the
ftrace_ops. The mutex is taken within checking the ftrace_ops flag that
states it was initializied but the flag is checked again after the mutex
has been taken. Checking first outside the mutex allows it to shortcut
having to take the mutex. But then the check needs to be done again after
the mute is taken in case of races.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260902095501.6b59af20@gandalf.local.home
Fixes:
|
||
|
Eduard Zingerman
|
387b1baefb |
bpf: backtrack_insn(): Handle ld_{abs,ind} subprog exit edge
Nicholas Carlini reported a bug in precision backtracking mechanism
for BPF_LD | BPF_{IND,ABS} instructions. These instructions are
modelled as two branches:
- fallthrough;
- implicit exit from current subprogram.
The implicit exit case was not handled by the backtrack_insn()
function. When backtracking such a path backtrack_insn() did not
call bt_subprog_enter(), which meant that backtracking continued
manipulating precision marks in a caller frame, while looking at
instructions in a callee frame.
This lead to segmentation faults during verification (see the
selftest), or unsound state pruning.
Fixes:
|
||
|
John Ogness
|
560f4deda3 |
printk/nbcon: Change nbcon_irq_work to IRQ_WORK_LAZY
Change the nbcon_irq_work to be IRQ_WORK_LAZY, thus not raising
an IRQ upon irq_work queuing. The irq_work is then handled on the
next kernel tick. This additional delay is acceptable because
nbcon_irq_work is only responsible for non-emergency deferred
printing, which is delayed anyway. This has the benefit of not
needing to raise an IRQ for each printk() call.
On a side note, the Tegra20 and Tegra30 platforms can hang if an
irq_work IRQ is raised while entering cpuidle states. This problem
was reproducible by calling printk() while entering cpuidle. So
this change also provides a workaround for these platforms (as long
as they are not running tickless).
Link: https://lore.kernel.org/lkml/f3757a75-0ba1-4558-bf57-f19ab7e59a4c@nvidia.com
Fixes:
|
||
|
John Ogness
|
0433632bbe |
printk/nbcon: Flush nbcon_irq_work in nbcon_free()
Ensure any pending nbcon_irq_work is flushed before allowing the console to be recycled. Signed-off-by: John Ogness <john.ogness@linutronix.de> Reviewed-by: Petr Mladek <pmladek@suse.com> Link: https://patch.msgid.link/20260901093245.344455-2-john.ogness@linutronix.de Signed-off-by: Petr Mladek <pmladek@suse.com> |
||
|
Eric Dumazet
|
02c6be7d67 |
locking/lockdep: Invalidate stale class_cache entries for zapped classes
syzbot reported a lockdep splat hitting DEBUG_LOCKS_WARN_ON(1) in
hlock_class() due to an invalid class_idx:
WARNING: kernel/locking/lockdep.c:238 at __lock_acquire+0x382/0x2cf0 kernel/locking/lockdep.c:5203
Workqueue: wg-crypt-wg0 wg_packet_tx_worker
RIP: 0010:hlock_class kernel/locking/lockdep.c:238 [inline]
RIP: 0010:check_wait_context kernel/locking/lockdep.c:4870 [inline]
RIP: 0010:__lock_acquire+0x389/0x2cf0 kernel/locking/lockdep.c:5203
Call Trace:
<IRQ>
lock_acquire+0x106/0x350 kernel/locking/lockdep.c:5886
_raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:173
tcp_tsq_handler+0x29/0x200 net/ipv4/tcp_output.c:1291
tcp_tsq_workfn+0x384/0x410 net/ipv4/tcp_output.c:1325
...
When a lock class is zapped (e.g. during module unload or key
unregistration), zap_class() clears the class's bit in
lock_classes_in_use and removes it from the class hash table.
However, existing lockdep_map instances embedded in data structures
may still retain a pointer to the zapped class in their class_cache[]
array.
When __lock_acquire() subsequently runs on such a lock, it finds
lock->class_cache[subclass] != NULL, skipping register_lock_class()
and assigning hlock->class_idx to the index of the zapped class. When
check_wait_context() or hlock_class() inspects the held_lock, it finds
!test_bit(class_idx, lock_classes_in_use) and warns. Furthermore, if
the zapped slot is subsequently re-allocated to an unrelated lock key,
the stale class_cache entry would erroneously match the unrelated
class (ABA issue).
Add lock_class_cache_is_valid() to validate that the cached class is
within lock_classes bounds, still allocated in lock_classes_in_use
(using uninstrumented arch_test_bit() in __always_inline context so it
is safe in noinstr contexts like match_held_lock()), and that
class->key matches the expected subkey (taking lockdep_set_subclass()
overrides into account). Also use READ_ONCE()/WRITE_ONCE() when
accessing class_cache[]. If the entry is invalid or stale, fall back
to register_lock_class() / look_up_lock_class().
Fixes:
|
||
|
Yilin Zhang
|
58a8108bc7 |
perf: Fix use-after-free when perf mmap() revival races with the last munmap()
perf_mmap_close() drops rb->mmap_count *without* holding
event->mmap_mutex (the refcount_dec_and_test() right before the
refcount_dec_and_mutex_lock() of event->mmap_count). A concurrent
perf_mmap_rb() can slot its entire "revival" path into that window
(perf_mmap holds event->mmap_mutex for its whole duration, including
rb_alloc):
munmap side (perf_mmap_close) mmap side (perf_mmap_rb)
----------------------------------- --------------------------------
rb->mmap_count 1 -> 0 (no lock) (holds event->mmap_mutex)
inc_not_zero(rb->mmap_count) fails
ring_buffer_attach(event, NULL)
rb_alloc() + attach new rb
refcount_set(&event->mmap_count, 1)
lock; event->mmap_count 1 -> 0
ring_buffer_attach(event, NULL)
ring_buffer_put() -> frees the *new* rb
The revival's refcount_set(&event->mmap_count, 1) is an invisible
1 -> 1 write: the close frees the just-revived buffer although the
other process still has it mapped -- a page-level use-after-free
allowing local privilege escalation to root by any unprivileged user
(default kernel.perf_event_paranoid=2).
Swap the order of the two counter updates: event->mmap_count is
dropped first via refcount_dec_and_mutex_lock(), so its 1 -> 0
transition and the ring_buffer_attach() stay serialized with
perf_mmap(). rb->mmap_count == 0 then implies every event using the
buffer is detached already, so the result of the rb->mmap_count drop
can gate the remaining teardown directly and detach_rest is no longer
needed.
An earlier fix for this race from Kyle Zeng and David Lee takes
event->mmap_mutex around both counter updates [0]; here the not-last
close stays lockless.
Fixes:
|
||
|
Leo Yan
|
8a7f5b5e86 |
perf/core: Skip empty AUX records with only format flags
perf_aux_output_end() emits a PERF_RECORD_AUX when the recorded size is
nonzero or when any flag other than PERF_AUX_FLAG_OVERWRITE is set.
PMU format flags describe how an AUX payload is encoded. TRBE driver
sets PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW for raw trace buffers, causing
an AUX record to be emitted even when no trace data.
This is noticeable when tracing a task with strace. Ptrace stops
repeatedly end empty AUX transactions, producing many zero-sized
PERF_RECORD_AUX records. For example:
perf record -e cs_etm//u -m,128M -- strace ls
perf script -D 2>&1 |
awk '/PERF_RECORD_AUX offset/ {
for (i = 1; i <= NF; i++)
if ($i == "size:" && $(i + 1) == "0")
count++
}
END { print count }'
165
This recording contains 165 zero-sized AUX records which provide no
useful information to userspace.
Ignore PERF_AUX_FLAG_PMU_FORMAT_TYPE_MASK, together with
PERF_AUX_FLAG_OVERWRITE, when deciding whether an empty AUX record is
useful. Zero-sized records carrying TRUNCATED, PARTIAL or COLLISION
are still emitted.
Fixes:
|
||
|
Tim Chen
|
f0d243a96f |
sched/fair: Avoid creating misfits during cache-aware balancing
Cache-aware load balancing biases tasks toward their preferred LLC. On asymmetric CPU capacity systems (e.g. big.LITTLE) the destination LLC may contain CPUs that are too small to run the task. Pulling the task there turns it into a misfit, trading a cache-locality gain for a capacity loss that's more detrimental to performance. Guard both cache-aware migration entry points against this: - can_migrate_llc_task(): forbid the LLC migration when the task fits its source CPU but would not fit the destination CPU. - alb_break_llc(): veto the active balance under the same condition so the runnable task is not pushed onto a CPU that cannot accommodate it. Both checks are gated with checks for hybrid processors, so symmetric systems are unaffected. Tasks that already do not fit their source CPU are left to the existing LLC policy, since the move cannot make their fitness worse (this also preserves misfit up-migration to bigger CPUs). Additionally, if there are misfit tasks found in the load balancing classification phase, prioritize misfit task migrations over LLC load aggregation on asymmetric systems. A better fitting CPU will boost performance more than better cache locality. Reviewed-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Tested-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Reviewed-by: Chen Yu <yu.c.chen@intel.com> Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/edbb2503d554c63dc9b72e201fb4a17e1cb119e7.camel@linux.intel.com |
||
|
Wanwu Li
|
b038383526 |
sched/fair: Use cfs_rq->h_curr in distribute_cfs_runtime()
distribute_cfs_runtime() refreshes the rq clock and accounts elapsed runtime with update_curr() before redistributing bandwidth, but gates this on cfs_rq->curr. Since commit |
||
|
Wanwu Li
|
f8610c57f4 |
sched/fair: Use cfs_rq->h_curr in throttle_cfs_rq()
After commit |
||
|
Shubhang Kaushik (Ampere)
|
c6dcd97c8b |
sched/core: Skip rq->avg_idle update without a valid idle_stamp
Commit |
||
|
Seiji Nishikawa
|
dae5c02920 |
sched/rt,dl: Skip migrate-disabled tasks when picking a push candidate
A migrate_disable()'d RT task cannot be moved to another CPU, but the scheduler still keeps such a task on that CPU's pushable list (rq->rt.pushable_tasks) and still marks the runqueue RT-overloaded (rq->rt.overloaded = 1). So the RT balancer keeps treating this CPU as having a task to move away, and keeps trying to move the task, but the push can never succeed. When the head is pinned, push_rt_task() does not give up either. It falls back to pushing rq->curr instead, using the per-CPU stopper, as added by commit |
||
|
Zhan Xusheng
|
1719d035a6 |
sched/fair: Use update_curr_eevdf() for the remaining root cfs_rq callers
pick_task_fair() and yield_task_fair() call update_curr(&rq->cfs) to bring curr up to date before they look at the eevdf state. With cgroups that does not happen: update_curr() reads ->h_curr, which on the root cfs_rq is the top level group entity, and returns at the !entity_is_task() check before touching vruntime. Both then read ->curr, so the guard and the update disagree about which entity they mean. Counting how often ->h_curr and ->curr differ at pick_task_fair(), on one CPU for 10s with three busy tasks and one 200us-periodic task: all tasks in the root cgroup 43321 calls, 0 no-ops busy tasks in G0, periodic in G1 45211 calls, 45193 no-ops Whether that matters depends on what precedes the pick. Since commit |
||
|
Tao Cui
|
29871903f3 |
sched_ext: Don't deliver duplicate ops.cgroup_set_idle() for same value
ops.cgroup_set_idle() is documented to be invoked when a cgroup
transitions between idle and non-idle states, and scx_group_set_weight()
already skips value-preserving writes. scx_group_set_idle() delivers
every write unconditionally, so rewriting an already-correct cpu.idle
value feeds the BPF scheduler a transition callback each time, which
toggle- or accounting-based schedulers miscount. Mirror the weight
guard and only deliver on an actual change.
Verified with a probe scheduler printing each callback: rewriting
cpu.idle=1 twice on an already-idle cgroup delivered two callbacks
before and none after.
Fixes:
|
||
|
Steven Rostedt
|
9100191e5a |
ftrace: Take trace_array reference before accessing its ftrace_ops
The trace instance files set_ftrace_filter and set_ftrace_notrace was
updated to work with specific trace instances (trace_arrays). The issue is
that when these files are opened, there is a small race window where it
will use the ftrace_ops from the inode->private pointer to get a reference
to the trace_array and then take its reference. The problem is that the
ftrace_ops itself could be freed. If the rmdir on the instance happens at
the same time the set_ftrace_filter file is opened, the rmdir could have
also freed the ftrace_ops and referencing it will cause a use-after-free
bug and crash the kernel.
Instead, pass in the trace_array as the file private data (NULL for the
top level instance), and then pass both the trace_array and the ftrace_ops
to the ftrace_regex_open() function. If the trace_array is NULL, then it
just uses the ftrace_ops without the need to take its reference (like
normal). If the ftrace_ops is NULL, that is only the case for the top
level instance and the global_ops can be used.
This allows the trace_array to have its reference incremented before
touching the ftrace_ops that could also be freed when the instance is.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260828223901.29e26edb@robin
Fixes:
|
||
|
Steven Rostedt
|
f4a771cc68 |
tracing: Have show_event_filters/triggers files take trace array ref
The newly added files show_event_filters and show_event_triggers that show all filters or triggers that are set within the trace array do not take a reference for the trace array it is showing. Without taking a reference, the trace_array may be freed via "rmdir" while a task is reading one of theses files. Those files iterate all the events within an instance (trace_array) and nothing prevents that instance from being freed while its data is being read. This causes a use-after-free crash. Have the open of both those files take the trace_array reference via the trace_array_get() that prevents the trace_array from being freed while the files are opened. Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260828094153.17b95037@gandalf.local.home Fixes: |