Proxy execution separates the scheduling context from the execution
context. Commit aa4f74dfd4 ("sched: Fix runtime accounting w/ split
exec & sched contexts") made per-task and thread-group runtime
accounting follow the task that actually executes, while cgroup CPU
usage is charged to the donor.
When the donor and execution task belong to different cgroups, this
makes a task's execution time count against a different cgroup from the
one the task belongs to.
Cgroup CPU usage should follow the execution context, matching the
per-task, thread-group, and cgroup user/system accounting. Keep
scheduling state associated with the donor, but charge cgroup CPU
usage to rq->curr.
A reproducer with the donor and execution task in separate cgroups
showed the execution task accumulating runtime while cgroup CPU usage
was charged to the donor's cgroup. With this change, the execution
task's cgroup accumulates the CPU usage instead. The same behavior was
verified with an RT donor and with legacy cpuacct accounting.
Fixes: aa4f74dfd4 ("sched: Fix runtime accounting w/ split exec & sched contexts")
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Hui Su <sh_def@163.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260904034707.268416-1-sh_def@163.com
The eevdf rb tree maintains 3 augmented fields but only one is currently
copied when balancing the tree.
Add a more generic define that can be used when there are several augmented
fields. In this case, we provide a function that takes care of copying all
fields.
Fixes: aef6987d89 ("sched/eevdf: Propagate min_slice up the cgroup hierarchy")
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Link: https://patch.msgid.link/20260909150522.858312-1-vincent.guittot@linaro.org
Similarly to se->min_slice, init se->max_slice with se->slice before
enqueueing the entity so the augmented callback computes it correctly
at parent level.
Fixes: 6e3c0a4e1a ("sched/fair: Fix lag clamp")
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Link: https://patch.msgid.link/20260907123855.1297976-1-vincent.guittot@linaro.org
A PMU might use perf_sched_cb_inc() and perf_sched_cb_dec()
interface to get the PMU call back function pmu::sched_task
invoked at schedule in and schedule out. This is achieved
by walking along the list anchored by sched_cb_list.
The following scenario might lead to a list corruption.
perf_pmu_sched_task()
for_each_list_entry(..., &sched_cb_list)
+--> __perf_pmu_sched_task()
+--> event->pmu->sched_task())
+--> PMU_push_sample()
+--> perf_event_overflow()
+--> __perf_event_overflow()
+--> pmu->stop()
+--> perf_sched_cb_dec()
remove entry from sched_cb_list
while list node in use.
This happens when ioctl(fd, PERF_EVENT_IOC_REFRESH, xxx) has been
invoked and perf_event::event_limit hits zero.
Prevent the list corruption and convert for_each_list_entry()
to for_each_list_entry_safe().
Fixes: bd27568117 ("perf: Rewrite core context handling")
Signed-off-by: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260908105637.627004-1-tmricht@linux.ibm.com
Please consider pulling these changes from the signed vfs-7.3-rc3.fixes tag.
Thanks!
Christian
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCaqFWOQAKCRCRxhvAZXjc
otLlAP9X02ybdUt9NndBK8LjslDWwB9hOXzPgYsOKYODEqODjQD/aLpbXVEsA1yy
SLdSDtbtpf+01z4KHorvAakBzk/jrw4=
=OzBX
-----END PGP SIGNATURE-----
Merge tag 'vfs-7.3-rc3.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs fixes from Christian Brauner:
- netfs:
- Fix an uninitialized return value in netfs_unbuffered_write()
when preparing the first subrequest fails
- For partial unbuffered/DIO writes return the amount transferred
rather than an error
- Update i_size with the amount actually written when a partial
transfer ends in an error
- Fix a subrequest reference leak when the io_iter ends up empty
- Handle netfs_alloc_subrequest() failure during unbuffered writes
- Load all readahead folios into the rolling buffer upfront and
drop the readahead references once the first subrequest is
dispatched
- Mark folios for copy-to-cache while issuing subrequests
- Fix read progress reporting
- afs:
- Add the missing kunmap in the error path of afs_dir_search_bucket()
- Fix a double kunmap in afs_edit_dir_remove()
- Don't free an existing server's endpoint state when cleaning up a
candidate server in afs_lookup_server()
- Unbind peers removed from a server's address list
- ufs:
- Load the cylinder group metadata before creating the root dentry
- Validate the cylinder group index and rotor positions before
caching them
- Treat an unreadable directory block as not empty
- exec:
- Close the close-on-exec files before taking exec_update_lock
Closing a file can block on the filesystem, so a hung filesystem
blocked everything that takes exec_update_lock and a FUSE server
inspecting the calling process could deadlock
- Drop the bprm loader before closing bprm->file in free_bprm()
- exit: Hold a reference to thread_pid across proc_flush_pid()
- reboot: Fix a use-after-free on cad_pid
- nsfs: Keep the namespace tree fields out of the rcu_head used by
kfree_rcu()
- nstree: Check listing permission before taking a namespace
reference in listns()
- super: Return 0 when a nested thaw drops its hold while other
freezers remain
- ext4: Don't set I_METADATA_WRITEBACK during fastcommit replay
- adfs: Free s_fs_info in ->kill_sb()
- autofs: Free the inode info allocated in autofs_fill_super() when
the root inode allocation fails
- ovl: Return EINVAL instead of EIO on a user namespace mismatch now
that it's a plain refusal and not an internal error
- cachefiles: Don't cast the variable-length coherency data to a
__be64 in the coherency tracepoint
* tag 'vfs-7.3-rc3.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (28 commits)
nstree: check listing permission before taking a namespace reference
exec: do_close_on_exec() before taking exec_update_lock
exit: hold a reference to thread_pid across proc_flush_pid
fs: autofs: fix memory leak in autofs_fill_super()
exec: Drop bprm loader before closing bprm->file
afs: Clear stale peer app data after address list changes
afs: Fix incorrect free in candidate cleanup in afs_lookup_server()
afs: Fix double-unmap of directory block
afs: Fix missing kunmap in afs_dir_search_bucket()
ovl: return EINVAL instead of EIO in case of mismatched user_ns
reboot: fix cad_pid use-after-free race
cachefiles: Fix potential UAF/KASAN warning
netfs: Fix read progress reporting
netfs: Mark folios with COPY_TO_CACHE whilst issuing subreqs
netfs: Fix readahead synchronisation issues by loading all folios upfront
netfs: break unbuffered write when netfs_alloc_subrequest() fails
netfs: Fix subreq ref leak
netfs: Fix i_size update for partial transfer
netfs: Fix error vs transferred passed to ->ki_complete()
netfs: Fix unbuffered/DIO write partial transfer error return
...
legitimize_ns() takes a reference on the candidate namespace before
may_list_ns() has decided whether the caller may see it. The
__free(ns_put) cleanup on the denied path can drop the last reference to a
mount namespace while we still hold the rcu read lock, and put_mnt_ns()
may sleep there. This is the same problem commit 2ec2aff3c8 ("ns: make
sure reference are dropped outside of rcu lock") fixed for the put_user()
path. Neither ns_requested() nor may_list_ns() needs a reference, both
only look at the namespace type and at the caller's own namespaces, so do
the checks first and take the reference last.
Splat:
Voluntary context switch within RCU read-side critical section!
WARNING: kernel/rcu/tree_plugin.h:332 at rcu_note_context_switch+0x238/0x2a0, CPU#5: a/3442
CPU: 5 UID: 1000 PID: 3442 Comm: a Not tainted 7.0.0-30-generic #30-Ubuntu PREEMPT(lazy)
RIP: 0010:rcu_note_context_switch+0x238/0x2a0
Call Trace:
<TASK>
__schedule+0xcf/0x650
schedule+0x27/0x90
schedule_preempt_disabled+0x15/0x30
__mutex_lock.constprop.0+0x550/0xaf0
__mutex_lock_slowpath+0x13/0x20
mutex_lock+0x3b/0x50
exp_funnel_lock+0xb2/0x260
synchronize_rcu_expedited+0xe7/0x220
namespace_unlock+0x26a/0x320
put_mnt_ns+0xd3/0x120
mntns_put+0xe/0x20
do_listns+0x13e/0x560
__do_sys_listns+0x126/0x2d0
__x64_sys_listns+0x20/0x30
x64_sys_call+0x2366/0x2390
do_syscall_64+0x105/0x5a0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
</TASK>
Fixes: 76b6f5dfb3 ("nstree: add listns()")
Signed-off-by: Norbert Szetei <norbert@doyensec.com>
Link: https://patch.msgid.link/ABA32239-733B-438C-B95A-B13ED69FF0F3@doyensec.com
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
swiotlb_bounce() reads the page frame number from the slot's recorded
orig_addr, then advances orig_addr by tlb_offset to reach the address
the caller asked about. The highmem branch mixes the two: the offset
within the page comes from the adjusted address, the page from the value
before it.
Once the adjustment crosses a page boundary the pair no longer describes
one location, and the whole copy lands one page below the intended one
for a positive tlb_offset, one above for a negative one. DMA_FROM_DEVICE
writes the device data over the wrong page and leaves the intended one
stale, DMA_TO_DEVICE feeds the device from a page the mapping may not
cover. Partial syncs through dma_sync_single_range_for_*() are what make
tlb_offset non-zero.
The branch test is picked the same way, so a slot recorded in lowmem can
be adjusted into highmem and the lowmem path then hands a highmem
address to phys_to_virt().
Take both from orig_addr once it is final and keep pfn in the branch
that uses it. PhysHighMem() asks the question straight from the address,
as dma-debug already does.
Fixes: 5f89468e2f ("swiotlb: manipulate orig_addr when tlb_addr has offset")
Cc: stable@vger.kernel.org
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Link: https://lore.kernel.org/r/20260905084210.148255-1-donggeunyoo.kernel@gmail.com
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
rmem_dma_device_init() drops the return value of
dma_assign_coherent_memory() and always reports success. That call fails
with -EBUSY when the device already has a coherent pool, and the file
allows only "*one* such region of memory" per device.
of_reserved_mem_device_init_by_idx() reads the zero as success. It logs
"assigned reserved memory node" for a region that was not assigned and
records the pairing, so of_reserved_mem_device_release() later runs
rmem_dma_device_release() for it. That clears dev->dma_mem without
looking at which region it was called for, dropping the pool the device
did get and leaving it on ordinary memory.
dma_declare_coherent_memory() checks the same call and releases the
memory on failure, and rmem_swiotlb_device_init() propagates its own
errors. Return the error here as well, so a device tree that assigns two
pools to one device fails the probe instead of half working.
Fixes: 7bfa5ab6fa ("drivers: dma-coherent: add initialization from device tree")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Link: https://lore.kernel.org/r/20260905074727.108029-1-donggeunyoo.kernel@gmail.com
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
Commit 8d75c338f0 ("sysctl: remove CONFIG_PROC_SYSCTL, it just mirrors
CONFIG_SYSCTL") removed CONFIG_PROC_SYSCTL, but the sysctl added by
commit 5b6e32ba7b ("syscall_user_dispatch: Add
kernel.syscall_user_dispatch sysctl") is still guarded by it. Now that
both commits are merged, kernel.syscall_user_dispatch is no longer
registered.
syscall_user_dispatch_allowed defaults to true. SUD therefore remains
available, but administrators cannot disable new activations.
Use CONFIG_SYSCTL for the guard and documentation.
Fixes: 5b6e32ba7b ("syscall_user_dispatch: Add kernel.syscall_user_dispatch sysctl")
Assisted-by: Codex:gpt-5.6-sol
Acked-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Joel Granados <joel.granados@kernel.org>
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
朱恺乾 reported and decoded the following race condition when a broadcast
device is replaced:
CPUA CPUB
__tick_broadcast_oneshot_control()
bc = tick_broadcast_device.evtdev;
tick_install_broadcast_device(dev)
clockevents_exchange_device(cur, dev)
shutdown(cur);
detach(cur);
cur->handler = noop;
tick_broadcast_device.evtdev = dev;
tick_broadcast_set_event(bc, next_event); <- FAIL: arms a detached device.
If the original broadcast device has a restricted interrupt affinity mask
and the last CPU in that mask goes offline then the BUG() in
tick_cleanup_dead_cpu() triggers because the clockevent device is not in
detached state.
The reason for this is that tick_install_broadcast_device() is not
serialized vs. tick broadcast operations.
The obvious cure is to serialize tick_install_broadcast_device() with
tick_broadcast_lock against a concurrent tick broadcast operation.
That requires to split clockevents_exchange_device() into two parts, one
which does the exchange, shutdown and detach operation and the other which
drops the module reference count. This is required because the module
reference cannot be dropped while holding tick_broadcast_lock.
Let clockevents_exchange_device() do both operations as before, but let the
broadcast device code take the two step approach and do the device
exchange under tick_broadcast_lock and drop the module reference count
after releasing it.
Fixes: f8381cba04 ("[PATCH] tick-management: broadcast functionality")
Reported-by: 朱恺乾 <zhukaiqian@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Tested-by: 刘术高 <liushugao@xiaomi.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/87cymdsu0r.ffs@tglx
- Fix several tracefs files that did not take the trace_array reference
A trace instance can be created and destroyed in the tracefs "instances"
directory via mkdir and rmdir respectively. The instance is represented by
a trace_array descriptor. Most tracefs files pass the trace_array as the
private data of the inode to the open/read/write functions. Since there is
no locking between the time a task opens a file and the deletion of the
instance (and the freeing of the trace_array), each open needs to get a
reference to the trace_array and each close must remove it. A instance
can't be removed if there's any reference taken on its trace_array. The
open function uses trace_array_get() that takes a lock (preventing removal
of instances) and iterates the list of all existing trace_arrays and if it
finds a match, it takes the reference and releases the lock. If it doesn't
find a match, it causes the open to return -ENODEV.
There were some added files that did not take the trace_array reference
on open that needed to be fixed. Sashiko also correctly pointed out that
there were some files that took an address of an field or element of the
trace_array which had a pointer back to the trace_array to take its
reference on open. But this leaves a slight race between referencing this
element to get the trace_array as the element itself could be freed. To
solve this, some helper functions were created to look for trace_arrays
with this field or element in the search so that the element did not have
to be dereferenced before the trace_array's reference was taken.
- Add a lock around ftrace_ops initialization
When a ftrace_ops is first used by ftrace, some internal initialization is
performed on the ops. But if multiple tasks were calling functions that
did this initialization, it could race and perform doing the
initialization more than once, corrupting the internal data. Add a lock in
the initialization code to prevent this from happening.
- Fix splice reads on mmapped buffers
The logic in the ring buffer splice code for mmapped buffers is supposed
to do a copy of the memory as the mapped buffers can't be given to splice.
But there was an if statement within the copy code that would return a -1
if a request for a full page was done and it wasn't a partial read. This
is because this logic was written before mmapped buffers existed and this
case didn't make sense at the time. For mmapped buffers it makes perfect
sense and by returning early can drop a lot of pages unnecessarily.
- Have the persistent ring buffer validation check nr_subbufs
Sashiko reported that the validation code was relying on the saved
nr_subbufs to match the calculated nr_pages + 1 and if they were off, that
the code could cause corruption. Sashiko is correct, and the saved
nr_subbufs should be validated before assuming it is correct.
- Do not allow more than one instance with the same name on cmdline
If an admin were to add more than one trace instances with the same name
they all would be created, but only the first one would be accessible via
tracefs. This used to not be allowed but some restructuring of code has
since made it possible.
- Fix the race between subbuf resize and trace_pipe_raw readers
If a task was reading trace_pipe_raw while another task was changing the
ring buffer subbuf size, it could crash the reader. The trace_pipe_raw
readers do get their own copy of the page from the buffer, but the code
needs some restructuring to not have the resize of the subbuffers cause
issues.
- Cap the size of the mapped (static) ring buffer nr_pages
The meta data used for ring buffer mapped buffers is 32 bit in size. A
normal ring buffer could (in theory) have more than 4 billion pages.
But this is not allowed by mapped buffers, so enforce it.
-----BEGIN PGP SIGNATURE-----
iIoEABYKADIWIQRRSw7ePDh/lE+zeZMp5XQQmuv6qgUCap3GJRQccm9zdGVkdEBn
b29kbWlzLm9yZwAKCRAp5XQQmuv6qtAfAP9hDPtzctAoz+IHZq+GYejhKTu99ICG
vAxTOZivuG6KYAD/UPA4P7z5ayxGM5UlT+2U81+mpCd55F2oFiIxRuFYcwk=
=/d+T
-----END PGP SIGNATURE-----
Merge tag 'trace-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing fixes from Steven Rostedt:
- Fix several tracefs files that did not take the trace_array reference
A trace instance can be created and destroyed in the tracefs
"instances" directory via mkdir and rmdir respectively. The instance
is represented by a trace_array descriptor.
Most tracefs files pass the trace_array as the private data of the
inode to the open/read/write functions. Since there is no locking
between the time a task opens a file and the deletion of the instance
(and the freeing of the trace_array), each open needs to get a
reference to the trace_array and each close must remove it.
An instance can't be removed if there's any reference taken on its
trace_array. The open function uses trace_array_get() that takes a
lock (preventing removal of instances) and iterates the list of all
existing trace_arrays and if it finds a match, it takes the reference
and releases the lock. If it doesn't find a match, it causes the open
to return -ENODEV.
There were some added files that did not take the trace_array
reference on open that needed to be fixed. Sashiko also correctly
pointed out that there were some files that took an address of an
field or element of the trace_array which had a pointer back to the
trace_array to take its reference on open. But this leaves a slight
race between referencing this element to get the trace_array as the
element itself could be freed. To solve this, some helper functions
were created to look for trace_arrays with this field or element in
the search so that the element did not have to be dereferenced before
the trace_array's reference was taken.
- Add a lock around ftrace_ops initialization
When a ftrace_ops is first used by ftrace, some internal
initialization is performed on the ops. But if multiple tasks were
calling functions that did this initialization, it could race and
perform doing the initialization more than once, corrupting the
internal data. Add a lock in the initialization code to prevent this
from happening.
- Fix splice reads on mmapped buffers
The logic in the ring buffer splice code for mmapped buffers is
supposed to do a copy of the memory as the mapped buffers can't be
given to splice. But there was an if statement within the copy code
that would return a -1 if a request for a full page was done and it
wasn't a partial read. This is because this logic was written before
mmapped buffers existed and this case didn't make sense at the time.
For mmapped buffers it makes perfect sense and by returning early can
drop a lot of pages unnecessarily.
- Have the persistent ring buffer validation check nr_subbufs
Sashiko reported that the validation code was relying on the saved
nr_subbufs to match the calculated nr_pages + 1 and if they were off,
that the code could cause corruption. Sashiko is correct, and the
saved nr_subbufs should be validated before assuming it is correct.
- Do not allow more than one instance with the same name on cmdline
If an admin were to add more than one trace instances with the same
name they all would be created, but only the first one would be
accessible via tracefs. This used to not be allowed but some
restructuring of code has since made it possible.
- Fix the race between subbuf resize and trace_pipe_raw readers
If a task was reading trace_pipe_raw while another task was changing
the ring buffer subbuf size, it could crash the reader. The
trace_pipe_raw readers do get their own copy of the page from the
buffer, but the code needs some restructuring to not have the resize
of the subbuffers cause issues.
- Cap the size of the mapped (static) ring buffer nr_pages
The meta data used for ring buffer mapped buffers is 32 bit in size.
A normal ring buffer could (in theory) have more than 4 billion
pages. But this is not allowed by mapped buffers, so enforce it.
* tag 'trace-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
ring-buffer: Use a macro for static buffer bits
tracing: Fix comment in tracing_buffers_splice_read()
ring-buffer: Prevent truncation of nr_pages / nr_subbufs
ring-buffer: Cap static ring buffer nr_pages
tracing: Fix subbuf resize races with trace_pipe_raw readers
tracing: Fix to avoid creating trace instances with duplicate names
ring-buffer: Add checking nr_subbufs to persistent ring buffer validation
ring-buffer: Allow splice reads on static buffers
tracing: Take trace_array reference when opening options file
ftrace: Synchronize the initialization of ftrace_ops
ftrace: Take trace_array reference before accessing its ftrace_ops
tracing: Have show_event_filters/triggers files take trace array ref
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE+soXsSLHKoYyzcli6rmadz2vbToFAmqdhRMACgkQ6rmadz2v
bTo8vRAAnP+x1z6FBOgHawGmmBpXtazoipRFeok/+/YGQ4SbS+evvFNvmkeCyihZ
7EeNHGuWEbA6xQBAcMf0StMjmr3jkJvUA6lsHSxBiFLpHGqNt0Fz9IXvAL/8QWPg
F7+CBs5A+J61i66LEmINbWzW1ujDf7baU/1VWui9zfPxnmPu40V/74eXzm6DmH/V
oHTsCscCYNe+N+Wix6B6nLrObY537fcJU12N56uqpkaSc0+6H1fBQJmyFGpYroTX
+4g+DNzEHyIYmt6B3+oDpbT0Zh/pP6ROq5TPDNJSSa+5B/uPo5C5pH520HwXF9hJ
04uaD7y2fpE809Nwl3OEK4ozpJ+mEGU1NKZPvFAxckY6GCiBtfbGd1etFno7r+4F
mZhNBKbQVEQX4XJUlcoqn9n7T2OCDnjiuneZAjIi4vUdevnSnakr0rCTFriiHP54
jg4fEkXxvYjPiqs5SQbxoGxmYSfg9fYaSfK4HgFdlfbwp+3Vmv4BW/tcsIJ9dcPs
L3YWkUU3znjbu2BLy0CP9fGt/1ik8p4/tA8vvZwz0yqpDMJkLy1TziGMef51UQ7k
1Pe/Ln5J3C6xnWs+3TRF11y8PuuCF0y47DGPACfwPoQxca1ufJ5fgmLXRRQToZe4
RjDmoeQvvI9w9kxh6gVhTHoiiEeD0wcQqx2CMHKTrU/PbM7d4Co=
=oz4Z
-----END PGP SIGNATURE-----
Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf
Pull bpf fixes from Alexei Starovoitov:
"This mainly contains verifier fixes that address bugs reported by
Nicholas Carlini.
- Fix incorrect non-NULL inference in pointer comparisons: pointer
types that may be NULL at runtime, pointers with unbounded offsets,
JMP32 comparisons with zero, and imprecise zero registers (Eduard
Zingerman)
- Fix precision tracking for half-dead zero spills, ld_abs/ld_ind
implicit subprog exit, bpf_loop() callbacks, linked scalar ids and
NULL call arguments (Eduard Zingerman)
- Reject BPF_PSEUDO_FUNC reference to the main program, fix zero
extension of arena 32-bit cmpxchg, don't rewrite bpf_fastcall
patterns entered by a jump (Eduard Zingerman)
- Fix percpu map update and BPF_F_CPU validation with sparse CPU IDs
(Hui Su)
- Fix NULL-ptr-derefs in bpf_snprintf_btf() for void and VAR types,
and reject key-less BTF for hash maps (Jiayuan Chen)
- Various fixes (Kumar Kartikeya Dwivedi):
- Fix out-of-bounds access in disassembler on invalid LDSX
instruction
- mark siginfo of signal tracepoints as scalar and
sched_process_wait argument as nullable
- mark faultable stack helpers as sleepable
- reject tail calls and legacy packet loads from callbacks
- enforce rbtree callback lock restrictions for resilient locks
- require MEM_PERCPU for percpu kptr stores
- clear NON_OWN_REF after RCU protection ends
- mark NULL kptr stores precise
- preserve inner map identity in callback frames
- reject non-scalar bpf_loop() iteration counts
- Fix trampoline allocation slowdown on x86 by using
EXECMEM_MODULE_DATA (Mike Rapoport)
- Keep bpf_refcount_acquire() nullable for borrowed RCU kptrs and
reject untrusted allocated-object pointers (Ning Ding)
- Fix special fields handling in recycled rhtab elements (Nuoqi Gui,
Yuan Chen)"
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (86 commits)
bpf, riscv: Make arena support depend on ZACAS
selftests/bpf: Test pointer bpf_loop iteration count rejection
bpf: Reject non-scalar bpf_loop iteration counts
bpf: use mark_arg_precision() in check_mem_size_reg()
bpf: propagate mark_chain_precision() errors out of loop_flag_is_zero()
selftests/bpf: precision of a NULL global subprogram BTF_ID argument
bpf: mark a NULL BTF_ID argument of a global subprogram precise
selftests/bpf: precision of a NULL kfunc argument
bpf: mark a NULL kfunc argument precise
selftests/bpf: precision of a NULL global subprogram memory argument
bpf: mark a NULL memory argument of a call precise
selftests/bpf: precision of a NULL helper argument
bpf: mark a NULL call argument precise
selftests/bpf: Test inner map identities in callbacks
bpf: Preserve inner map identity in callback frames
selftests/bpf: Test imprecise scalar kptr stores
bpf: Mark NULL kptr stores precise
selftests/bpf: Test rhtab kptr cancellation semantics
bpf: Cancel special fields when recycling rhtab elements
selftests/bpf: Test timer field on recycled rhtab element
...
- Fix a timestamping bug in pick_task_fair() and
yield_task_fair() (Zhan Xusheng)
- Skip migrate-disabled tasks when picking a push
candidate in the RT and DL schedulers (Seiji Nishikawa)
- Skip rq->avg_idle update without a valid idle_stamp
(Shubhang Kaushik)
- Fix throttling bug in throttle_cfs_rq(), caused
by the recent single-runqueue conversion (Wanwu Li)
- Fix bandwidth calculation bug in distribute_cfs_runtime(),
caused by the single-runqueue conversion (Wanwu Li)
- Don't make x86 ITMT enablement depend on debugfs (Mario Limonciello)
- Avoid creating misfits during cache-aware load-balancing
on hybrid systems (Tim Chen)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqdTLIRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gdURAAnjK4q2xg115VyKOvg4u4epPre1XNK4a/
RmimS/2+1fm7/5zqOXOQG3qU8z2Aj+nBIZa1d1yadV7ARHSymDINobCX+2HQ1DjU
Iysk2BqbbLnQQNKh81OBWhPMyJy4yOnig/670KR5JdFAdTKFYW/Ys+pQmO9M6rEx
hc8B4SUhOGeRnonfPMTbFW7H9eopWzst8kIpF0USl58az0mla5tkooWYJsvSuUHn
g4s9IALlRe3DxgmMT3kLKoAx2+ySnrzZNRs+M8Z/sMSDKXCMFpvX36wOqBJMHwXe
ZoAow/fJh4ysoYTR7luhDIBDQAkszf4GE9a644I/Bd0sIIYKs4+1RW1T3DAoWmKv
Sx0C29MM4Ng4R3lPx2avb+c0T0VO080/3fqRFrGrck8BIpMa9HcinOxf6etI8+vt
STnz5uQZZj3V+A5vzlKCAFGUUqK/s3EJnrT5z4VPxXmY2ZSyLLub4C5SQ8RNnqRL
u+Q9ra9DYBJh/HJTgjjcAP/HxmKK5LyQXd2DzuA1wy/wwC+s7TtPrxN5ep5EdZmv
K65zGGo28BgcCKD8jc1DfVXf8cf80fMiAr4fruGdo5TVwPyBahwZBi4sbqhahtqL
D4QwCsbOfhS+evVpx29n+V7eQzZzvSdN1tFCtZ3Tg6VvT9aJ2jLSx79aCsL/cYbr
hViB9N0yhGk=
=8l0J
-----END PGP SIGNATURE-----
Merge tag 'sched-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fixes from Ingo Molnar:
- Fix a timestamping bug in pick_task_fair() and yield_task_fair()
(Zhan Xusheng)
- Skip migrate-disabled tasks when picking a push candidate in the
RT and DL schedulers (Seiji Nishikawa)
- Skip rq->avg_idle update without a valid idle_stamp (Shubhang
Kaushik)
- Fix throttling bug in throttle_cfs_rq(), caused by the recent
single-runqueue conversion (Wanwu Li)
- Fix bandwidth calculation bug in distribute_cfs_runtime(),
caused by the single-runqueue conversion (Wanwu Li)
- Don't make x86 ITMT enablement depend on debugfs (Mario Limonciello)
- Avoid creating misfits during cache-aware load-balancing on hybrid
systems (Tim Chen)
* tag 'sched-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/fair: Avoid creating misfits during cache-aware balancing
x86/itmt: Don't make ITMT enablement depend on debugfs
sched/fair: Use cfs_rq->h_curr in distribute_cfs_runtime()
sched/fair: Use cfs_rq->h_curr in throttle_cfs_rq()
sched/core: Skip rq->avg_idle update without a valid idle_stamp
sched/rt,dl: Skip migrate-disabled tasks when picking a push candidate
sched/fair: Use update_curr_eevdf() for the remaining root cfs_rq callers
- Skip empty AUX records with only format flags
(Leo Yan)
- Fix use-after-free when perf mmap() revival races with the
last munmap() (Yilin Zhang, Weiming Shi)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqdSokRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gRKRAAkDxFT3BNAtAypk5CvTPeO9K530u38mqc
fIvYQYbYkqIMZaFSmIthSgsPOJD/mqkccl1sd6djlS/73mI/X8eXCwZOgjLnaCsu
jRx8atD8Q7hHeyJ78qY/hFsyHPTQel5+cniEk4BjUrYIYhHklFfpQw1AZhoJieUs
vnqkDGMY/qHQ5pGs+dnz3oI8FE6ruNU0Ps6qiBqsbsvMJI+wviCidpC6BqluyxvQ
Pp6ACwqQIiH2mPwI9WYtHKmVL5ajQJPt8fbIwIvRbX5wCY566zb79oUys2pNUh/U
zzpTNOGgqtVCTgiuMiEI2SlIIMzC36P071vBTyD8YZWUNpdYXLM20RdvAUZkqefT
Vx3cMs8rgcPnjGO4XhPJJOzfyyCi25VL/p3af9S3wSJhfQNdMF9BuqRABFMBmtDi
5adP+4p5Dw9sfc2+F2/OCT7PrXsFiNsOp+iduWJYb6i8hdSjV1mfjGLyptdBCdj2
dUYr4VrFoiYUVLPWJD5F4/dDAIljN3CEoGwHiOITNCYu0fgO7GCALf8FG5U9Ec7D
guuRGnEXDFQDCOZHm5MuVYWAR7qPhe+iVoX6Dw9ZmgecGWb4eTl4A5e8LGgbcbJ1
JMfVr3OENSyGi1dPnCX4DEsIpeXN1UTMHhrwNosWcu1OrEEUkzdQ8GQ17qU6PURG
yYSXCvhdEtY=
=Ffvz
-----END PGP SIGNATURE-----
Merge tag 'perf-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar:
- Skip empty AUX records with only format flags (Leo Yan)
- Fix use-after-free when perf mmap() revival races with the
last munmap() (Yilin Zhang, Weiming Shi)
* tag 'perf-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf: Fix use-after-free when perf mmap() revival races with the last munmap()
perf/core: Skip empty AUX records with only format flags
- Fix a softirq processing delay bug in local_interrupt_disable(),
which should mostly only affect the Rust runtime (Boqun Feng)
- Remove the hardirq_disable_count() function which caused the
previous bug and is now unused & unnecessary (Boqun Feng)
- lockdep: Invalidate stale class_cache entries for zapped classes
(Eric Dumazet)
- Fix rt_mutex specific futex scheduling helpers
(Sebastian Andrzej Siewior)
- Fix rcuwait use-after-free race during futex requeue PI (Yao Kai)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqdSKMRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gh6hAAtIhAm10pkx7JG4Kl+SQArmoAxNMfW9X1
grx50dn1/LFuIF0upooJwJmLMBpPMmxcpYPWbg5nTUsx+wDqcJTT0T7Aw2iZN2f+
OdODMOGNuQU4IUxDC+qX+fkgHvRFzng0zhSXYL2kGpQJHktrFMAHn90+5aO9SEgA
XcSjmDacTxfJADNk43snrlHy6t+yOXBEmIrhicnbh11GFNIXLbvdGW/0SObU0Q5o
WdwGHTZRFAM3mpX7xMSFOKypM+Gt4pXdQ/uC1PQg0B/AT35p4zsYIdcht/cBMHp0
93eTeJd9ede410t3hL1Hnsuo1zf5qHtCgBukS+UHJWoWpnSOMgT2iaBx2X3jIsev
ptPIqV5Alq8O+tnNOSDOwD2HH2drAdUBVH+gZ/yKMDbk9jfw9eZKqRw628U2P7rG
mD5CLzpmfj0mMAuqcOhhdqSQmPhjp2Y8Ju/8UAMSmcocvqjQBxvouNS3Tby9U5v+
R4CsuUwtnYpMyy5xyRrnwVsaE3le5fvneXCmCokk8FtQ8as/4ATd+cG+RROSa3vW
/AAjrB1HBQqyseE8lki4LFFmlr01Otc8u6pASy4/pxeuKMA9hSuLC3RuEfVetW6W
BcK4BCIJBy6chcsCBxv0F7FgI/y7/bpVyBmoTkW3Sb0I23xswXzTzbucn/dPJNdl
oqFj3KQygxw=
=vEuZ
-----END PGP SIGNATURE-----
Merge tag 'locking-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking fixes from Ingo Molnar:
- Fix a softirq processing delay bug in local_interrupt_disable(),
which should mostly only affect the Rust runtime (Boqun Feng)
- Remove the hardirq_disable_count() function which caused the
previous bug and is now unused & unnecessary (Boqun Feng)
- lockdep: Invalidate stale class_cache entries for zapped classes
(Eric Dumazet)
- Fix rt_mutex specific futex scheduling helpers
(Sebastian Andrzej Siewior)
- Fix rcuwait use-after-free race during futex requeue PI (Yao Kai)
* tag 'locking-urgent-2026-09-06' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
futex: Prevent rcuwait use-after-free during requeue PI
futex: Provide rt_mutex_.*_schedule() equivalents for futex scheduling
locking/lockdep: Invalidate stale class_cache entries for zapped classes
preempt: Remove hardirq_disable_count()
interrupt: Disable interrupt before modifying hardirq_disable counter
bpf_loop() declares its nr_loops argument as ARG_ANYTHING. Privileged
programs may pass pointer values to such arguments, so check_func_arg()
lets a pointer-valued R1 reach the helper-specific checks.
Since commit bb124da69c ("bpf: keep track of max number of bpf_loop
callback iterations"), the verifier marks R1 precise and reads its upper
bound to limit callback simulation. Precision backtracking only accepts
scalar registers, so passing a pointer instead triggers the "backtracking
misuse" verifier warning. Kernels with panic_on_warn enabled subsequently
panic.
Introduce ARG_SCALAR for helper arguments that only accept scalar values
and use it for bpf_loop() nr_loops. Generic helper argument validation then
rejects pointers before loop inlining and precision processing.
Fixes: bb124da69c ("bpf: keep track of max number of bpf_loop callback iterations")
Reported-by: syzbot+7b47f87674e9a1569110@syzkaller.appspotmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260905014735.1452988-2-memxor@gmail.com
Closes: https://lore.kernel.org/bpf/6a9ad24c.b5d4176b.238c3e.0001.GAE@google.com/
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
dispatch_one() tests ops flags and bypass state against the root scheduler
in both places where it decides to keep running @prev: the early keep of a
@prev with slice left tests the root's bypass state, and the keep-last at
the end tests the root's SCX_OPS_ENQ_LAST and bypass state. Both are
properties of the scheduler @prev belongs to, and put_prev_task_scx(), which
acts on the outcome, reads them from that scheduler. When @prev belongs to a
sub-scheduler the two sides disagree.
The keep-last case is visible. The root set SCX_OPS_ENQ_LAST, so a lone
@prev of a sub-scheduler is not kept and is enqueued with SCX_ENQ_LAST to a
sub-scheduler that never opted in. This trips the WARN_ON_ONCE in
put_prev_task_scx() for the missing flag, and the sub-scheduler queues the
task like any other and triggers no follow-up scheduling event, which can
lead to stalls.
Test SCX_OPS_ENQ_LAST and bypass state on @prev's sched in both places and
charge SCX_EV_DISPATCH_KEEP_LAST to it. Read the sched at each decision, as
the dispatch in between can drop the rq lock.
Fixes: 88234b075c ("sched_ext: Introduce scx_task_sched[_rcu]()")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
dispatch_one() uses the root scheduler for everything it does, including the
two decisions to keep running @prev, which are wrong when @prev belongs to a
sub-scheduler. The function has to deal with @prev's scheduler too. Rename
the root's local from sch to root_sch for clarity and to make room for it.
No functional change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci
This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.
Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.
Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
Stop verification if mark_chain_precision() fails when called from
loop_flag_is_zero(). No functional change intended for the paths where
backtracking succeeds.
Fixes: 1ade237119 ("bpf: Inline calls to bpf_loop when callback is known")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-9-0f5a360ff15d@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
btf_check_func_arg_match() accepts a NULL register for an
ARG_PTR_TO_BTF_ID argument tagged __arg_nullable and skips
check_reg_type() and check_func_arg_reg_off() without marking the
register precise. Hence a checkpoint created on such a path would
prune against arbitrary scalar value.
Fixes: e2b3c4ff5d ("bpf: add __arg_trusted global func arg tag")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-7-0f5a360ff15d@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
check_kfunc_arg() allows bpf_register_is_null() for nullable arguments
w/o marking the underlying scalar register precise. Hence a checkpoint
created on such a path would prune against arbitrary scalar value.
Fixes: 3bda08b636 ("bpf: Allow NULL buffers in bpf_dynptr_slice(_rw)")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-5-0f5a360ff15d@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
check_mem_reg() allows bpf_register_is_null() for nullable arguments
w/o marking the underlying scalar register precise. Hence a checkpoint
created on such a path would prune against arbitrary scalar value.
The argument may live on the stack rather than in a register when a
call has more than MAX_BPF_FUNC_REG_ARGS arguments, hence the new
mark_arg_precision() helper.
Fixes: e5069b9c23 ("bpf: Support pointers in global func args")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-3-0f5a360ff15d@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
check_func_arg() allows bpf_register_is_null() for nullable arguments
w/o marking the underlying scalar register precise. Hence a checkpoint
created on such a path would prune against arbitrary scalar value.
check_helper_call() enforces second parameter of the
bpf_get_local_storage() to be zero, w/o marking the underlying scalar
register precise. Hence a checkpoint created on such a path would
prune against arbitrary scalar value.
Grouping these two into one patch, as they share the same fixes tag.
Fixes: b5dc0163d8 ("bpf: precise scalar_value tracking")
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-1-0f5a360ff15d@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Instead of hard coding 30 for the number of bits used for the static
buffer ids in two places, create a macro. This way if it changes in the
future, it will change in all the locations that use it.
Link: https://patch.msgid.link/20260904151641.17eae0aa@gandalf.local.home
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The comment about returning an error if the read fails on the first
iteration is slightly incorrect. It makes it sound like the only reason it
could fail on a later iteration is if the subbuf order changed. That is
incorrect, it could also fail if the length passed in was not a multiple
of the subbuf size. Fix the comment.
Link: https://lore.kernel.org/all/20260904143527.40e73d36@gandalf.local.home/
Link: https://patch.msgid.link/20260904144902.506862a1@gandalf.local.home
Fixes: dae8dda341 ("tracing: Fix subbuf resize races with trace_pipe_raw readers")
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Although ring_buffer_per_cpu::nr_pages is defined as unsigned long, it
is capped to 32-bits in a few places, limiting the operations possible
on a very large buffer. Use `unsigned long` where appropriate and
prevent truncation of values using nr_pages (or nr_subbufs).
While at it, subbuf_size must be at least `unsigned int`.
Note that persistent, remote and user-mapped ring buffers are capping
the number of pages to 30 bits already, making "int" safe in many
places.
Link: https://patch.msgid.link/20260904164450.1345852-5-vdonnefort@google.com
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Static ring buffers (i.e. persistent, user-mapped and remote) rely on
the bpage::id field. The number of pages for those ring buffers must fit
into that variable. Enforce this limit on ring buffer creation or
user-mapping.
While at it, prevent nr_pages underflow when allocating a persistent
buffer.
Link: https://patch.msgid.link/20260904164450.1345852-4-vdonnefort@google.com
Fixes: be68d63a13 ("ring-buffer: Add ring_buffer_alloc_range()")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Concurrent subbuffer resizes may crash trace_pipe_raw readers or leak
uninitialized memory to userspace due to stale size values.
Modify ring_buffer_alloc_read_page() to handle the resizing of an
existing buffer_data_read_page if necessary and add a new
ring_buffer_read_page_size(). This new function enables ring-buffer
buffer_data_read_page users to not call the racy
ring_buffer_subbuf_size_get(). This makes the spare_size member of
ftrace_buffer_info redundant.
Finally, handle buffer_data_read_page/reader_page order discrepancy in
ring_buffer_read_page(). On a mismatch simply copy manually the data to
the buffer_data_read_page.
Link: https://lore.kernel.org/all/20260817140812.2C7D41F00A3A@smtp.kernel.org/
Link: https://patch.msgid.link/20260904164450.1345852-3-vdonnefort@google.com
Fixes: bce761d757 ("ring-buffer: Read and write to ring buffers with custom sub buffer size")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Callback frame constructors initialize map-typed argument registers with
__mark_reg_known_zero() and then restore map_ptr. This clears map_uid,
which is the only field distinguishing inner maps that share an
inner_map_meta template.
When a timer callback invokes bpf_for_each_map_elem() on a second inner
map, both the saved first map and the second map value can reach the nested
callback as the same template with map_uid zero. bpf_timer_init() then
accepts pairing the timer from the second map with the first map.
The runtime records the first map in the timer without taking a reference.
Freeing that map does not find the timer stored in the second map, so a
later timer callback dereferences the freed map.
Copy map_uid from the same caller register as map_ptr when constructing
for-each, timer/workqueue, and task-work callback arguments. The existing
identity check can then reject mismatched inner maps while allowing a
callback value to be paired with its actual map.
Fixes: 3e8ce29850 ("bpf: Prevent pointer mismatch in bpf_timer_init.")
Fixes: 69c087ba62 ("bpf: Add bpf_for_each_map_elem() helper")
Fixes: 5c8fd7e2b5 ("bpf: bpf task work plumbing")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260904104203.345917-8-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
check_map_kptr_access() permits a scalar store into an untrusted kptr
field only when the register is known to contain zero. Unlike other
verifier checks whose outcome depends on a scalar value, it does not mark
that register precise.
A state checkpoint reached with an imprecise zero can therefore prune a
second path that reaches the store with an arbitrary nonzero scalar. The
program can write attacker-controlled bits into the kptr field and load
them back as a PTR_TO_BTF_ID.
Call mark_chain_precision() before accepting a known-zero register. This
forces state equivalence to compare its scalar range and makes the verifier
visit and reject a path carrying a nonzero value.
Fixes: 61df10c779 ("bpf: Allow storing unreferenced kptr in map")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260904104203.345917-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
rhtab_map_update_existing() and rhtab_delete_elem() call
bpf_obj_free_fields() when replacing or deleting a value. These map
operations can run from BPF programs in NMI context, where releasing a
referenced kptr or another complex field is not generally safe.
Array and hash maps avoid that problem by cancelling only the asynchronous
fields which can be stopped safely in the caller context. Other ownership
state remains attached to the allocation until its memory allocator
destructor performs the final cleanup.
Use bpf_obj_cancel_fields() for the corresponding rhtab paths as well. This
cancels timers, workqueues, and task work while allowing rhtab_mem_dtor() to
release referenced kptrs when the allocation is eventually destroyed.
Fixes: 6905f86012 ("bpf: Allow special fields in resizable hashtab")
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
[ kkd: Rebased, used direct helper calls, and rewrote the commit log ]
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260904104203.345917-4-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
rhtab_map_update_elem() initializes special fields after obtaining an
element from bpf_mem_cache_alloc(). The allocator can return a fresh,
zeroed unit, or recycle one from its RCU-pending lists before the
registered destructor has run.
A BPF program can retain a map-value pointer after deleting its element
and initialize and arm a timer through that pointer. If the deleted unit
is recycled, check_and_init_map_value() clears the only pointer to the
timer. Neither a later deletion nor rhtab_mem_dtor() can then cancel it,
and the callback can run with its key and value pointing into freed memory.
Do not reinitialize special fields on insertion. Fresh allocator units are
already zeroed. For recycled units, the special fields are ownership state
that must remain visible to the eventual destructor. copy_map_value()
already skips those fields, matching the non-preallocated hash-map path and
the lifecycle established by commit 275c30bcee ("bpf: Don't reinit map
value in prealloc_lru_pop").
Fixes: 6905f86012 ("bpf: Allow special fields in resizable hashtab")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
[ kkd: Split out the fix and rewrote the commit log ]
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260904104203.345917-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Since commit e645535a95 ("tracing: Add option to use memmapped
memory for trace boot instance") changed trace_array_get_by_name() to
trace_array_create_systems(), enable_instances() does not reuse the
same name instance. Therefore, if an administrator mistakenly specifies
multiple `trace_instance=` options with duplicate names, all are
created but only the first is accessible via tracefs.
Check whether an instance with the same name already exists before
creating a new one, and reject duplicates with a warning.
Link: https://patch.msgid.link/178847790399.283263.5313150997200138426.stgit@devnote2
Fixes: e645535a95 ("tracing: Add option to use memmapped memory for trace boot instance")
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
- kprobes: Protect kprobe_blacklist with RCU
. RCU-protect kprobe_blacklist and use kfree_rcu() to prevent UAF
races during module unloading and enable safe atomic lookups.
- tracing/probes: Fix multi-probe field use-after-free and BTF parsing
. Multi-probe UAF fix: Duplicate field and type strings on
trace_probe_event to prevent UAF when freeing primary probe.
. BTF member lookup fixes:
- Check the containing inner struct/union kflag when resolving
anonymous members to ensure correct bitfield offset calculation.
- Prevent unnamed bitfields from being pushed to anon_stack in
btf_find_struct_member(), avoiding false lookup errors.
- Fix code block indentation in get_bitoffset_of_field().
- uprobes: Error pointer safety
. Guard free_trace_uprobe() with IS_ERR_OR_NULL() to avoid crashing
during automatic cleanup when an error pointer is returned.
-----BEGIN PGP SIGNATURE-----
iQFPBAABCgA5FiEEh7BulGwFlgAOi5DV2/sHvwUrPxsFAmqahXQbHG1hc2FtaS5o
aXJhbWF0c3VAZ21haWwuY29tAAoJENv7B78FKz8bOJIH/1RuAq2y8fvfqWKwDBNG
9CrSIMmZ0915s4LVSGQrrjNYfpj2rFYkEMsJcFo2pavKwWNyaxXFjXu8Vy9JGckx
VFAHA52x2QaEYdwBeoo/Jd3+7Ks/3zH1XwfSILFa0PMn86/JCKHx/+5ah6Sk4vcu
/he61Auyp6lJtvv88n95j1evCJNouU6lJ3fnvm8mNYTeLOIvPZ3qku6SsiOqNdeQ
Ln8bcNP2Iis33PqfeydiRWv9nPog/ifH4a9WJ+fdqKA+06AHKVsfHB+fP0hxyUDp
TS4oxKrIk6HIbEQRjgo8YcOPWurHAm0GQ2ZslgLFuWDE9rDQCtPdOYv/YzJe0ByG
nHc=
=hX9F
-----END PGP SIGNATURE-----
Merge tag 'probes-fixes-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull probes fixes from Masami Hiramatsu:
- Protect kprobe_blacklist with RCU
RCU-protect kprobe_blacklist and use kfree_rcu() to prevent UAF races
during module unloading and enable safe atomic lookups.
- Fix multi-probe field use-after-free
Duplicate field and type strings on trace_probe_event to prevent UAF
when freeing primary probe
- Fix probe BTF member lookup:
Check the containing inner struct/union kflag when resolving
anonymous members to ensure correct bitfield offset calculation
Prevent unnamed bitfields from being pushed to anon_stack in
btf_find_struct_member(), avoiding false lookup errors
Fix code block indentation in get_bitoffset_of_field()
- uprobes error pointer safety
Guard free_trace_uprobe() with IS_ERR_OR_NULL() to avoid crashing
during automatic cleanup when an error pointer is returned
* tag 'probes-fixes-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
kprobes: Protect kprobe_blacklist with RCU
tracing/probes: Fix use-after-free on field name/type of events with multiple probes
tracing/probes: Fix code indent in get_bitoffset_of_field()
tracing/probes: Fix BTF kflag check for anonymous struct member access
tracing/probes: Fix anon_stack check for unnamed bitfields in btf_find_struct_member
uprobes: guard trace cleanup against error pointers
A local kptr load of an object containing a graph node is marked MEM_RCU
and NON_OWN_REF while protected by RCU. When the last RCU read-side critical
section ends, invalidate_rcu_protected_refs() removes MEM_RCU and marks the
pointer PTR_UNTRUSTED, but leaves NON_OWN_REF set.
The stale flag lets graph kfunc argument checks continue treating the
pointer as a live borrowed reference. In particular, bpf_rbtree_remove()
can accept a pointer after its protection ended and return it as a new
owning reference, even though the object may already have been freed.
Clear NON_OWN_REF when an RCU-protected pointer is demoted. A spin lock also
provides implicit RCU protection, so invalidate non-owning references before
demoting RCU-protected pointers when releasing the lock. Otherwise the
demotion would clear the flag before invalidate_non_owning_refs() can find
and invalidate those aliases.
The demoted pointer remains available for fault-protected reads. Exempt such
reads from the allocated-object reference-state assertion; writes through a
fault-prone pointer are already rejected, and bpf_may_fault_on_deref() makes
the surviving loads use BPF_PROBE_MEM.
Fixes: 1b12171533 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260904084325.52250-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_refcount_acquire() is fallible for a borrowed reference because the
object may have reached a zero refcount. The verifier therefore keeps
KF_RET_NULL on the return value unless the argument is an owning reference.
An RCU-protected load of a local kptr is marked MEM_ALLOC, but it only
receives NON_OWN_REF when the pointee contains a graph node. A refcounted
object without a graph node consequently looks like an owning reference
even though the loaded register has no acquired reference state. If the
program drops the last real reference while remaining in the RCU critical
section, refcount_inc_not_zero() returns NULL while the verifier treats the
result as non-NULL.
Only classify the argument as owning when it is backed by a verifier-tracked
reference. This retains the non-NULL return for pointers from bpf_obj_new(),
bpf_kptr_xchg(), or an earlier successful acquisition, while requiring a
NULL check for borrowed RCU kptrs.
Fixes: 1b12171533 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Ning Ding <dingning04@gmail.com>
[ kkd: Rewrote commit log ]
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260904084325.52250-4-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
map_kptr_match_type() treats perm_flags as the set of register type flags
that a kptr field permits. Adding MEM_PERCPU to that set for
BPF_KPTR_PERCPU does not require the source register to carry it, however.
The subset test consequently accepts both a plain bpf_obj_new() allocation
and a referenced kernel pointer into a __percpu_kptr map field.
Loads from the field are always marked MEM_PERCPU. Consumers then treat the
stored value as the cookie returned by bpf_percpu_obj_new(): per-CPU pointer
helpers relocate it, and map teardown selects the per-CPU free path. A plain
allocation can therefore provide an arbitrary kernel read/write, while a
kernel pointer can be relocated into an invalid address or sent through a
missing destructor.
Require the source MEM_PERCPU flag to match the destination field kind.
This preserves valid bpf_percpu_obj_new() stores and rejects both the
program-BTF and kernel-BTF variants.
Fixes: 36d8bdf75a ("bpf: Add alloc/xchg/direct_access support for local percpu kptr")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260904084325.52250-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
check_cond_jmp_op() accepts "if rA <op> rB" as a NULL check for a
nullable pointer rA when rB is a scalar known to be zero,
lifts PTR_MAYBE_NULL from rA in the corresponding branch and does not
mark rB precise. Consider the following program:
r0 = bpf_get_prandom_u32();
r6 = 1; /* the r6 == 0 path is explored first */
if (r0 == 0) goto 1f;
r6 = 0;
1:
r0 = bpf_map_lookup_elem(map, &0); /* absent, NULL at runtime */
if (r0 == r6) goto 2f; /* taken as a NULL check for r0 */
*(u8 *)(r0 + 0); /* verifier: map value; runtime: zero */
2:
return 0;
The r6 == 0 path is explored first and the dereference is accepted.
The r6 == 1 path is pruned at the checkpoint recorded for (1),
so the comparison is never verified with a non-zero r6. At runtime a
failed lookup returns NULL, NULL != 1 takes the non-NULL edge and the
program dereferences a pointer that is zero.
Fixes: 2f4cb53eed ("bpf: detect non null pointer with register operand in JEQ/JNE.")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-7-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Consider the following program:
r1 = map_value; /* low 32 bits are zero at runtime */
r6 = 0xdead000000000000;
if w1 != 0 goto l1;
l0: r1 += r6;
r2 = *(u64 *)(r1 + 0);
exit;
l1: r6 = 0;
goto l0;
At the moment is_branch_taken() reports the jump as always taken,
because it does not distinguish between BPF_JMP and BPF_JMP32
comparisons when processing 'if w1 != 0 ...'.
Fixes: cac616db39 ("bpf: Verifier track null pointer branch_taken with JNE and JEQ")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-5-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
check_cond_jmp_op() copies the compared registers into
env->{false,true}_reg{1,2} before collect_linked_regs() runs and copies
those snapshots back into both branch states afterwards.
collect_linked_regs() records at most LINKED_REGS_MAX members of a
linked registers group in the jump history and calls clear_scalar_id()
for every member that does not fit. The compared register is not exempt
from that.
As a consequence, sync_linked_regs() might adjust ranges for more
registers than bpf_bt_sync_linked_regs() can propagate precision to.
Collect the linked registers before the snapshots are taken instead.
This might lead to some unnecessary clear_scalar_id's, but from
previous testing situations with many linked registers are
extremely rare.
Fixes: ec1d77cb0e ("bpf: Use bpf_verifier_env buffers for reg_set_min_max")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-3-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
reg_not_null() decides that a register holds a non-NULL value by
looking at its type alone. For pointer types that allow arithmetic the
type only guarantees a non-NULL base, in case of an unbound offset
the runtime offset value might still add up to NULL.
Consider the followng program:
r6 = bpf_map_lookup_elem(map, &0); /* present */
if (r6 == 0) return 0;
r7 = bpf_map_lookup_elem(map, &1); /* absent, NULL at runtime */
r8 = r7;
r8 -= r6; /* pointer - pointer: unknown scalar, -r6 */
r8 <<= 1;
r8 >>= 1; /* any non-negative offset is accepted by */
/* check_reg_sane_offset_ptr() */
r6 += r8; /* verifier: map value; runtime: zero */
if (r7 != r6) return 0;
*(u8 *)(r7 + 0); /* r7 is inferred non-NULL, both are zero */
At runtime both registers are zero, the comparison is true and the
load faults with NULL pointer dereference.
Require the offset to be within +-BPF_MAX_VAR_OFF in reg_not_null().
Fixes: cac616db39 ("bpf: Verifier track null pointer branch_taken with JNE and JEQ")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260904083325.2083493-1-eddyz87@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Commit 0a36bad017 ("release_task: kill the no longer needed
get/put_pid(thread_pid)") removed the reference around proc_flush_pid().
It assumed that free_pids(post.pids) at the end of release_task() would
keep thread_pid alive until then.
That assumption is wrong. __change_pid() only records a detached PID in
post.pids when pid_has_task() is false for every PIDTYPE. If another task
still uses the exiting task's PID as its process group or session ID,
__unhash_process() removes the exiting task's PIDTYPE_PID link but leaves
the PID out of post.pids. release_task() therefore holds no reference to
it after dropping tasklist_lock.
The other task can then remove the remaining PIDTYPE links. Its
free_pids() call schedules delayed_put_pid(), and the RCU callback can free
the PID before the first release_task() reaches proc_flush_pid().
An unprivileged reproducer races wait4(-1) against setsid() to trigger this
ordering. Three of three fresh v7.2 KASAN boots reported:
BUG: KASAN: slab-use-after-free in
proc_invalidate_siblings_dcache+0x3e2/0x3f0
Read of size 8 by task h7_pid_reaper/1921
Call Trace:
proc_invalidate_siblings_dcache
release_task
wait_consider_task
__do_wait
do_wait
kernel_wait4
Freed by task 0:
kmem_cache_free
put_pid
delayed_put_pid
rcu_core
Last potentially related work creation:
__call_rcu_common
free_pids
ksys_setsid
KASAN identified a 144-byte object from the pid cache and located the bad
read 80 bytes into the freed object, matching pid->inodes. With an
explicit reference, three of three fresh boots completed without a KASAN
report. The concurrent RCU callback dropped its reference while
proc_flush_pid() was protected, and the balancing put_pid() performed the
final free afterward.
Take a reference before __unhash_process() clears p->thread_pid and release
it after proc_flush_pid() completes.
A tested source reproducer is available privately on request. No
controlled read or write, information leak, or privilege escalation is
claimed. The mainline patch applies directly to v6.19.y and newer;
v6.16.y through v6.18.y need a context-adjusted backport.
Fixes: 0a36bad017 ("release_task: kill the no longer needed get/put_pid(thread_pid)")
Reported-by: syzbot+0aee5e8066eddbbe7397@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=0aee5e8066eddbbe7397
Reported-by: syzbot+e8b3520b53e78e90034e@syzkaller.appspotmail.com
Link: https://syzkaller.appspot.com/bug?extid=e8b3520b53e78e90034e
Cc: stable@vger.kernel.org # see patch description, needs adjustments for 6.16.y-6.18.y
Signed-off-by: Daehyeon Ko <4ncienth@gmail.com>
Link: https://patch.msgid.link/20260831001221.3755948-1-4ncienth@gmail.com
Acked-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
On PREEMPT_RT, FUTEX_CMP_REQUEUE_PI can trigger a KASAN report
(slab-out-of-bounds) in futex_requeue_pi_complete() invocation of
rcuwait_wake_up().
The futex_q used by futex_wait_requeue_pi() is allocated on the waiter's
stack. An early wakeup can race with a PI requeue as follows:
waiter requeue task
------ ------------
futex_wait_requeue_pi()
futex_do_wait()
schedule()
futex_requeue
futex_proxy_trylock_atomic()
futex_requeue_pi_prepare()
Q_REQUEUE_PI_NONE -> Q_REQUEUE_PI_IN_PROGRESS
* timeout/ signal wakes waiter *
futex_requeue_pi_wakeup_sync()
Q_REQUEUE_PI_IN_PROGRESS -> Q_REQUEUE_PI_WAIT
requeue_pi_wake_futex
futex_requeue_pi_complete()
cmpxchg Q_REQUEUE_PI_WAIT -> Q_REQUEUE_PI_LOCKED
rcuwait_wait_event()
if (atomic_read(&q->requeue_state) != Q_REQUEUE_PI_WAIT)
break /* no schedule() */
/* q.pi_state->owner == current */
futex_private_hash_put()
/* return from syscall */
rcuwait_wake_up(&q->requeue_wait)
/* q is gone */
futex_requeue_pi_complete() publishes Q_REQUEUE_PI_LOCKED before
calling rcuwait_wake_up(). The waiter observes this state in
rcuwait_wait_event() before invoking schedule() in rcuwait_wait_event().
Here, the waiter is free leave the syscall before requeue task can
complete the wake.
To address this race skip rcuwait_wake_up() in the Q_REQUEUE_PI_LOCKED
case.
This state is only published by requeue_pi_wake_futex(), which saves
q->task before futex_requeue_pi_complete() and wakes the waiter via
wake_up_state().
This wake is intended to wake the waiter from its futex_do_wait() sleep.
If the waiter is still sleeping there, it can not get into the
Q_REQUEUE_PI_WAIT state (and require this removed wake).
Should the waiter be woken up from futex_do_wait() by other means (as in
this example) and sleep in futex_requeue_pi_wakeup_sync() then the
wake_up_state() from requeue_pi_wake_futex() will wake it, too.
Should the waiter task terminate before wake_up_state() had a chance to
wake the task then the task pointer does not become invalid because the
futex_hash_bucket::lock is held and the task pointer is RCU protected.
[bigeasy: Updated comment and commit message]
Fixes: 07d91ef510 ("futex: Prevent requeue_pi() lock nesting issue on RT")
Signed-off-by: Yao Kai <yaokai34@huawei.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260901135453.3121948-3-bigeasy@linutronix.de
There is rt_mutex_{pre|post}_schedule() around
rt_mutex_wait_proxy_lock() to ensure that sched_submit_work()/
sched_update_worker() is invoked before we schedule out and block on
rt_mutex while waiting for it become available.
The reason is that blocking on rt_mutex assigns a pi_waiter for the PI
chain and sched_submit_work() will also assign a pi_waiter if it blocks
on lock but a this point we already have a waiter assigned.
We can't skip sched_submit_work() entirely because I/O relies on the
fact that I/O queue is flushed while it blocks on a sleeping lock.
Therefore sched_submit_work() is moved before we block on the lock.
Sleeping lock in this context means mutex or rw_semaphore not spinlock_t
on PREEMPT_RT. Because the mutex abstraction on PREEMPT_RT uses the same
abstraction as the futex proxy lock, the futex code ended up using
rt_mutex_{pre|post}_schedule(), too.
Using it is/ was just to keep the task_struct::sched_rt_mutex assertion
happy. Futex proxy lock is used only in the syscall context of a task.
At this point it never got any I/O that needs to be flushed and it can't
be a workqueue that needs to notify that it will be scheduled out.
Therefore sched_submit_work() does nothing here.
By mistake futex_wait_requeue_pi() -> rt_mutex_wait_proxy_lock() did not
get the rt_mutex_{pre|post}_schedule() annotation. This was not noticed
because in this callchain the lock is (usually) not contended and so
rt_mutex_slowlock_block() does not schedule, triggering the assert.
Adding rt_mutex_pre_schedule() here looks wrong (as noted by PeterZ)
because at this point there is a pi_waiter recorded and invoking
sched_submit_work() with a possible lock contention would be wrong.
Add rt_mutex_futex_{pre|post}_schedule() which toggles the
sched_rt_mutex assert and does not involve sched_submit_work(). Add
asserts here to ensure that sched_submit_work() would do nothing. Use it
only in futex proxy lock case which is rt_mutex_wait_proxy_lock().
Remove it from futex_lock_pi().
Fixes: d14f9e930b ("locking/rtmutex: Use rt_mutex specific scheduler helpers")
Reported-by: Yao Kai <yaokai34@huawei.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260901135453.3121948-2-bigeasy@linutronix.de
Closes: https://lore.kernel.org/all/20260717084922.4153317-2-yaokai34@huawei.com
check_ld_abs() models a failed BPF_LD_ABS or BPF_LD_IND in a
subprogram as an implicit return with R0 set to zero. It calls
prepare_func_exit() to explore this synthesized path.
When the load is reached directly from a synchronous callback,
prepare_func_exit() enforces the callback return contract and marks R0
precise. R0 is not derived from a real instruction on this path, so
precision backtracking reaches the callback call with R0 still requested
and triggers the "callback unexpected regs" verifier bug. A privileged
program loader can therefore cause a verifier warning and an -EFAULT
BPF_PROG_LOAD.
These legacy packet-load instructions are deprecated. Reject them from
callbacks rather than complicating their implicit-return model. Check all
active frames before constructing the implicit return so nested static
subprograms cannot hide the callback context.
Global functions are verified independently with a fresh frame zero, so
an active-frame check cannot identify a global function called from a
callback. Also check the complete subprogram call graph during stack-depth
validation and reject a function containing a legacy load when any caller
is a callback. This covers global and static descendants without making
has_ld_abs transitive, preserving its per-function BTF return-type check.
Ordinary uses outside callbacks remain supported.
Fixes: ee861486e3 ("bpf: Fix ld_{abs,ind} failure path analysis in subprogs")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://lore.kernel.org/bpf/20260903152147.C0E241F00A3A@smtp.kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-8-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The faultable variants of bpf_get_stack() and bpf_get_task_stack() pass
may_fault=true into the common stack collection code. Resolving user-space
build IDs may then call build_id_parse_file() and block on filesystem
reads.
Neither helper prototype sets might_sleep. Since prototype selection uses
the sleepability of the whole program, the verifier can still allow these
helpers from a non-sleepable region within that program, such as an
explicit RCU or preemption-disabled region. The task-stack helper can also
be called from a non-sleepable timer callback of a sleepable program.
Mark both faultable prototypes as sleepable. The existing helper context
check then rejects these calls while continuing to allow them in genuinely
sleepable contexts.
Fixes: d4dd9775ec ("bpf: wire up sleepable bpf_get_stack() and bpf_get_task_stack() helpers")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When bpf_btf_find_by_name_kind() finds a type in module BTF, it
returns a new BTF object fd through __btf_new_fd(). This reaches
anon_inode_getfd(), which can sleep while allocating or expanding the
current task fd table.
The helper prototype does not set might_sleep, so the verifier allows
the helper in non-sleepable contexts such as BPF timer callbacks. The
fd allocation can then sleep in softirq context and install the fd into
the interrupted task.
Mark the helper as sleepable. This preserves calls from the main body
of a sleepable syscall program while rejecting calls from its
non-sleepable regions.
Fixes: 3d78417b60 ("bpf: Add bpf_btf_find_by_name_kind() helper.")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://lore.kernel.org/bpf/20260903155150.D57251F000E9@smtp.kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-4-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_rbtree_add() invokes its comparator while the caller holds the root
lock. The native insertion code retains raw parent and link pointers across
the callback, so the verifier prohibits unlocking, consuming tree nodes,
or changing RCU state from that callback.
in_rbtree_lock_required_cb() only checks the innermost verifier frame.
Static subprogram calls are permitted while holding a spin lock, and such a
call pushes a frame without in_callback_fn set. Consequently, all callback
restrictions disappear in the nested frame. The subprogram can unlock the
tree, remove and drop the node being compared, then relock. Native insertion
resumes with the stale parent pointer and links freed memory into the tree.
Walk all active frames for the rbtree callback instead. Benign static
subprograms remain permitted, while callback restrictions follow execution
into nested frames.
Fixes: a44b1334aa ("bpf: Allow calling static subprogs while holding a bpf_spin_lock")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903214758.2727663-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
mark_fastcall_pattern_for_call() must ensure that matched
"spill; call; fill" instruction series is not interrupted by a jump.
Otherwise the rewrite applied by bpf_remove_fastcall_spills_fills()
is not sound.
Record the instructions targeted by jumps in
insn_aux_data[*].jump_target when the CFG is built and use this flag
to stop growing a pattern at such an instruction. Jumps to the first
spill are fine.
Note that existing insn_aux_data[*].jmp_point field can't be reused,
as it marks subprogram return instructions.
Fixes: 5b5f51bff1 ("bpf: no_caller_saved_registers attribute for helper calls")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903205820.1743087-1-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from
BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, this patch adjusts
print_bpf_insn() to print such instructions as regular atomics with a
'probe_' prefix (instead of printing them as BUG_XX).
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903171542.1438050-2-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpf_convert_ctx_accesses() rewrites an atomic on an arena pointer from
BPF_STX | BPF_ATOMIC to BPF_STX | BPF_PROBE_ATOMIC, and it runs before
bpf_opt_subreg_zext_lo32_rnd_hi32().
That pass emits an explicit zero extension for a 32-bit cmpxchg even
when bpf_jit_needs_zext() is false. This is done because on some
architectures 32-bit cmpxchg requires explicit zero extension for the
dst register. E.g. on x86-64 'lock cmpxchg' does not change the %eax
if comparison is successful, while BPF semantics declare that each
operation on a 32-bit register zero extends it's upper half.
is_cmpxchg_insn() matches BPF_MODE == BPF_ATOMIC only, so an arena
cmpxchg misses said zero extension adjustment. This patch adjusts
is_cmpxchg_insn() to match BPF_PROBE_ATOMIC alongside BPF_ATOMIC.
Fixes: d503a04f8b ("bpf: Add support for certain atomics in bpf_arena to x86 JIT")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260903171542.1438050-1-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When the root scheduler has sub-scheds attached, the COMPAT kfunc
wrappers scx_bpf_select_cpu_and() and scx_bpf_dsq_insert_vtime() refuse
the call and report to @p's scheduler:
scx_error(scx_task_sched(p), "... must be used");
The wrappers are reachable with tasks that have no scheduler.
scx_bpf_select_cpu_and() is in the select_cpu kfunc group, which
scx_kfunc_context_filter() opens to BPF_PROG_TYPE_SYSCALL programs;
scx_bpf_dsq_insert_vtime() is in the enqueue_dispatch group, which
ops.enqueue() and ops.dispatch() may call with any KF_RCU task -- the
group has no kf_tasks validation, and scx_dsq_insert_preamble() checks
task ownership with scx_task_on_sched() precisely because @p may be an
arbitrary task.
scx_task_sched(p) is p->scx.sched, which is NULL for tasks past
sched_ext_dead() -- which clears it via scx_disable_and_exit_task() on
exit -- and for idle tasks, which the enable paths skip as they are
never scheduled through SCX. It is also an rcu_dereference_protected()
that expects @p's pi_lock or rq lock, which neither wrapper holds.
Passing NULL to scx_error() reaches scx_vexit(), which dereferences
sch->exit_info, oopsing the kernel.
One concrete trigger exercised while developing the fix: a
BPF_PROG_TYPE_SYSCALL program calling the select_cpu_and wrapper on an
exited-but-not-reaped task while a sub-scheduler was attached (its pid
stays findable while the zombie is unreaped; faulting instruction is
the scx_vexit() prologue "mov r15,[rdi+0x398]" with RDI=NULL and 0x398
the offset of sch->exit_info):
sched_ext: BPF scheduler "kfunc_subsched_null" enabled
sched_ext: BPF sub-scheduler "kfunc_subsched_null" enabled
sched_ext: Unassociated program run_select_cpu_ (id 76)
BUG: kernel NULL pointer dereference, address: 0000000000000398
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
Oops: Oops: 0000 [#1] SMP NOPTI
CPU: 7 UID: 0 PID: 8201 Comm: kfunc_test_runn Tainted: G W
RIP: 0010:scx_vexit+0x25/0xa0
Code: ... <4c> 8b bf 98 03 00 00 ...
CR2: 0000000000000398
Call Trace:
<TASK>
__scx_exit+0x4f/0x70
scx_bpf_select_cpu_and+0xab/0xb0
bpf_prog_430ed61a7b66e03a_run_select_cpu_and+0x9c/0xe7
? __x64_sys_bpf+0x2c/0x40
bpf_prog_test_run_syscall+0x130/0x2f0
__sys_bpf+0x930/0x10d0
? __x64_sys_bpf+0x2c/0x40
__x64_sys_bpf+0x2c/0x40
do_syscall_64+0xbc/0x460
entry_SYSCALL_64_after_hwframe+0x76/0x7e
</TASK>
Read @p's scheduler under RCU instead, which the wrappers can do from
their guard(rcu)(): fault it when it can be determined, and when it
can't be determined -- @p is a task past sched_ext_dead() or an idle
task -- there is nothing obviously wrong to report, so just refuse the
call as before without faulting any scheduler.
These COMPAT wrappers are scheduled for eventual removal once the
deprecation grace period elapses, but until then -- and regardless of
their removal timeline -- they must not oops the kernel on a task they
are handed.
Cc: stable@vger.kernel.org # v7.1+
Fixes: a5fa0708cb ("sched_ext: Enforce scheduling authority in dispatch and select_cpu operations")
Suggested-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
bpf_sys_bpf() executes the bpf(2) syscall body, which can take mutexes,
allocate with GFP_KERNEL, and wait for an RCU grace period.
bpf_sys_close() reaches close_fd() and filp_close(), which can sleep as
well.
Both helpers are limited to BPF_PROG_TYPE_SYSCALL, whose main program is
sleepable. That does not make every callback sleepable: a syscall program
can register a bpf_timer callback, and the verifier checks that callback
in a non-sleepable context while retaining the syscall helper set.
Without .might_sleep on the prototypes, such a callback can invoke
bpf_sys_bpf() from hrtimer softirq context and trigger a
scheduling-while-atomic failure. bpf_sys_close() is exposed through the
same missing context check.
Set .might_sleep on both prototypes so the existing helper-context check
rejects them from timer callbacks and other atomic regions. Calls from the
sleepable main body remain valid.
Fixes: 79a7f8bdb1 ("bpf: Introduce bpf_sys_bpf() helper and program type.")
Fixes: 3abea08924 ("bpf: Add bpf_sys_close() helper.")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-10-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
do_wait() passes wo->wo_pid to the sched_process_wait tracepoint.
kernel_wait4() leaves wo_pid NULL for wait4(-1), and
kernel_waitid_prepare() does likewise for waitid(P_ALL).
btf_ctx_access() currently types argument 0 as PTR_TO_BTF_ID |
PTR_TRUSTED. Without PTR_MAYBE_NULL, the verifier accepts an unchecked
dereference. Trusted pointer loads have no fault protection, so a wait for
any child can then cause a NULL pointer dereference in JITed BPF code.
Add sched_process_wait to raw_tp_null_args[] with argument 0 marked
nullable. The verifier rejects an unchecked dereference while preserving
access after the program checks the pointer for NULL.
Fixes: 838a10bd2e ("bpf: Augment raw_tp arguments with PTR_MAYBE_NULL")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-8-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
__bpf_rbtree_add() keeps parent and link pointers live across calls to the
program-supplied comparison callback. The verifier therefore requires the
root's lock to remain held throughout the callback.
The helper path enforces this rule for bpf_spin_lock() and
bpf_spin_unlock(), but the resilient lock kfunc argument path does not.
Since resilient locks may protect BPF rbtree roots, a callback can release
the root lock and let another CPU remove and free the node referenced by
the in-progress tree walk. The walk then resumes using freed pointers.
Reject resilient lock kfuncs in an rbtree comparison callback, matching
the existing policy for the spin lock helpers. Resilient-lock-protected
trees remain valid when their comparison callbacks leave lock state alone.
Fixes: 0de2046137 ("bpf: Implement verifier support for rqspinlock")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-6-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
A tail call from a non-zero frame is modeled as a return from that frame.
The verifier makes R0 unknown and calls prepare_func_exit() for the taken
branch.
When the current frame is a synchronous callback, prepare_func_exit()
enforces the callback return-value contract and marks R0 precise. Since the
tail-call path synthesized R0 rather than deriving it from an instruction,
precision backtracking reaches the callback-calling instruction with R0
still requested and triggers the "callback unexpected regs" verifier bug.
A CAP_BPF task can therefore cause a WARN and an -EFAULT BPF_PROG_LOAD.
Tail calls reachable from callbacks are already rejected later by
check_max_stack_depth(). Reject a tail call made directly by a callback
before constructing the inconsistent return state, using the existing
diagnostic. Tail calls from ordinary subprograms keep their current
behavior.
Fixes: e3245f8990 ("bpf: properly verify tail call behavior")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-4-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The signal_generate and signal_deliver tracepoints declare their info
argument as a struct kernel_siginfo pointer. btf_ctx_access() therefore
treats it as a trusted pointer for tp_btf programs.
Signal delivery also uses SEND_SIG_NOINFO and SEND_SIG_PRIV as special
values for this argument. Those values are zero and one respectively,
and are not pointers. A tp_btf program can currently dereference either
value and fault the kernel. In particular, signal_generate can run from
timer interrupt context, turning the fault into a kernel panic.
Record both tracepoints in raw_tp_null_args[] and mark argument one as
a non-pointer. This preserves scalar access to the cookie while rejecting
direct and helper-mediated pointer use. Merely marking it nullable would
not suffice because SEND_SIG_PRIV is nonzero.
Fixes: 838a10bd2e ("bpf: Augment raw_tp arguments with PTR_MAYBE_NULL")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20260903144433.1716731-2-memxor@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
btf_var_show() calls btf_type_id_resolve() unconditionally, which
dereferences btf->resolved_ids. That is NULL for a base BTF - e.g. the
vmlinux BTF that bpf_snprintf_btf() renders against - since base BTF is
not resolved during parsing. btf_modifier_show() guards this with
'if (btf->resolved_ids)', but btf_var_show() does not.
A BPF program that passes the type_id of a BTF_KIND_VAR from the vmlinux
BTF to bpf_snprintf_btf() thus NULL-derefs:
KASAN: probably user-memory-access in range [0x46638-0x4663f]
RIP: 0010:btf_var_show (kernel/bpf/btf.c:2929)
Call Trace:
<TASK>
btf_type_show (kernel/bpf/btf.c:8259)
btf_type_snprintf_show (kernel/bpf/btf.c:8329)
bpf_snprintf_btf (kernel/trace/bpf_trace.c:1047)
bpf_prog_test_run_raw_tp (net/bpf/test_run.c:829)
__sys_bpf (kernel/bpf/syscall.c:4804)
do_syscall_64 (arch/x86/entry/syscall_64.c:84)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
Resolve the var's type directly with btf_type_skip_modifiers() when
resolved_ids is NULL, mirroring btf_modifier_show().
Fixes: c4d0bfb450 ("bpf: Add bpf_snprintf_btf helper")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-4-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
btf_modifier_show() resolves the modifier and then calls
btf_type_ops(t)->show() unconditionally. For the void type (type_id 0,
BTF_KIND_UNKN) kind_ops[] has no entry, so ->show is NULL.
A "const void" (a modifier resolving to void) cannot be a map key or
value - map_check_btf() rejects it because void has no size - so the map
dump path does not reach it. But bpf_snprintf_btf() takes a type_id
straight from the BPF program, and passing such a "const void" from the
vmlinux BTF NULL-derefs:
KASAN: null-ptr-deref in range [0x0000000000000028-0x000000000000002f]
RIP: 0010:btf_modifier_show (kernel/bpf/btf.c:2914)
Call Trace:
<TASK>
btf_type_show (kernel/bpf/btf.c:8251)
btf_type_snprintf_show (kernel/bpf/btf.c:8321)
bpf_snprintf_btf (kernel/trace/bpf_trace.c:1047)
bpf_prog_test_run_raw_tp (net/bpf/test_run.c:829)
__sys_bpf (kernel/bpf/syscall.c:4804)
do_syscall_64 (arch/x86/entry/syscall_64.c:94)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
</TASK>
Fall back to btf_df_show() when the resolved type has no show op; it
emits the "<unsupported kind:N>" placeholder already used for kinds like
FWD and FUNC. bpf_snprintf_btf() then returns the length as usual.
Fixes: c4d0bfb450 ("bpf: Add bpf_snprintf_btf helper")
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-3-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
map_check_btf() allows a key-less BTF (btf_key_type_id == 0) only for
maps that have a ->map_check_btf callback, and leaves the actual
decision to that callback. Hash maps used to have no ->map_check_btf,
so a key-less BTF was rejected outright.
That changed when htab and rhtab gained a ->map_check_btf to register a
dtor - htab in commit 1df97a7453 ("bpf: Register dtor for freeing
special fields") and rhtab in commit 6905f86012 ("bpf: Allow special
fields in resizable hashtab"). Neither looks at the key, so a key-less
hash map now passes map_check_btf() and gets created. Reading it back
through bpffs feeds the key type_id 0 into btf_type_seq_show();
btf_type_by_id() returns the void type, kind_ops[BTF_KIND_UNKN] is NULL,
and btf_type_show() dereferences it:
RIP: 0010:btf_type_show+0x223/0x2e0 kernel/bpf/btf.c:8232
RSP: 0018:ffffc9000399f868 EFLAGS: 00010206
RAX: dffffc0000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000005 RSI: 0000000000000000 RDI: 0000000000000028
RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000
R10: ffffc9000399f970 R11: 0000000000000001 R12: ffffffff9b96b140
R13: ffffc9000399f8e0 R14: ffff88803d393c00 R15: 0000000000000003
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000200000000000 CR3: 000000003d213000 CR4: 0000000000352ef0
DR0: 0000000039ae8f55 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
btf_type_seq_show_flags+0xca/0x120 kernel/bpf/btf.c:8250
htab_map_seq_show_elem+0x12e/0x350 kernel/bpf/hashtab.c:1669
map_seq_show+0x13d/0x1e0 kernel/bpf/inode.c:293
traverse.part.0.constprop.0+0x107/0x650 fs/seq_file.c:112
traverse fs/seq_file.c:99 [inline]
seq_read_iter+0x93f/0x1270 fs/seq_file.c:196
seq_read+0x344/0x4d0 fs/seq_file.c:163
vfs_read+0x1e4/0xb40 fs/read_write.c:572
ksys_pread64 fs/read_write.c:764 [inline]
__do_sys_pread64 fs/read_write.c:772 [inline]
__se_sys_pread64 fs/read_write.c:769 [inline]
__x64_sys_pread64+0x1eb/0x250 fs/read_write.c:769
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x123/0x790 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Reject a key-less BTF in htab_map_check_btf() and rhtab_map_check_btf(),
restoring the previous behavior.
Fixes: 1df97a7453 ("bpf: Register dtor for freeing special fields")
Fixes: 6905f86012 ("bpf: Allow special fields in resizable hashtab")
Reported-by: syzbot+37b56485bbbf90ad8489@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a8f4e88.27659fcc.2ceef7.0008.GAE@google.com/T/
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260901104924.346187-2-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
ring_buffer_read_page() rejects splice (full=1) reads on static buffers
(that is user-mapped, persistent or remote) because !read check assumes
unread pages must be swapped. However for those buffers we have no other
choice than memcpy the data.
For the memcpy case, only return an error when the writer is still on
the reader page for the splice interface to wait.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260901155445.1475405-2-vdonnefort@google.com
Fixes: 117c39200d ("ring-buffer: Introducing ring-buffer mapping functions")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
fixups.c:jit_subprogs() rewrites BPF_PSEUDO_FUNC loads to contain real
function addresses. This function is invoked from bpf_jit_subprogs()
only when env->subprog_cnt > 1. Meaning that for any program like
below:
int main(void *ctx) {
void *ptr = main;
...
bpf_timer_set_callback(..., ptr);
...
}
The 'ptr' won't be ever converted to contain an address.
In combination with e.g. bpf_timer_set_callback() this would lead to a
function call at a bogus address.
Instead of complicating the implementation, just assume that no useful
program needs main to be a sync or async callback and reject
BPF_PSEUDO_FUNC loads for the main subprogram.
Fixes: 69c087ba62 ("bpf: Add bpf_for_each_map_elem() helper")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260902233658.1186477-1-eddyz87@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
__within_kprobe_blacklist() traverses kprobe_blacklist without holding
kprobe_mutex. When a module is unloaded, kprobe_remove_area_blacklist()
removes blacklist entries and immediately frees them with kfree().
A concurrent call to within_kprobe_blacklist() can therefore dereference
freed memory.
Furthermore, within_kprobe_blacklist() can be called in atomic or
non-preemptible contexts where the sleeping kprobe_mutex cannot be taken.
Protect kprobe_blacklist with RCU. Use guard(rcu)() and
list_for_each_entry_rcu() for traversal, list_add_tail_rcu() for
insertions, list_del_rcu() for deletions, and kfree_rcu() to reclaim
entries safely after a grace period.
Link: https://lore.kernel.org/all/178810004323.64882.16493230858653316962.stgit@devnote2/
Fixes: 376e242429 ("kprobes: Introduce NOKPROBE_SYMBOL() macro to maintain kprobes blacklist")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260807155802.F06041F000E9@smtp.kernel.org/
Assisted-by: Antigravity:gemini-3.7-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
The fields of a probe-based dynamic event (kprobe, uprobe, eprobe and
fprobe events) are created in traceprobe_define_arg_fields() by handing
the probe_arg name/type strings to trace_define_field(), which only
stores the pointers without copying. Those strings are owned by the
trace_probe and are freed when that probe is removed.
An event can have several probes attached. The field list is defined
only once, by the first probe that registers the event, but it is kept
alive by any surviving sibling probe. Deleting just that first probe by
symbol -
# primary A: fields are defined from A's args
echo 'p:kprobes/ev vfs_read a1=$arg1' > kprobe_events
# append B: shares A's event call
echo 'p:kprobes/ev vfs_write a1=$arg1' >> kprobe_events
# delete only A (matched by symbol), B survives
echo '-:kprobes/ev vfs_read' >> kprobe_events
frees A's args (trace_probe_cleanup() -> traceprobe_free_probe_arg()),
but trace_probe_unlink() keeps the trace_probe_event because the probe
list is not empty. The event call stays registered via B while its
fields now reference freed memory. Any field lookup then reads it, e.g.
echo 'a1 == 1' > events/kprobes/ev/filter
BUG: KASAN: slab-use-after-free in strcmp+0xa7/0xb0
Call Trace:
strcmp
trace_find_event_field
parse_pred
process_preds
create_filter
apply_event_filter
event_filter_write
field->name references parg->name (kstrdup'd, freed with the probe) and,
for array arguments, field->type references parg->fmt (kmalloc'd, freed
with the probe) - the scalar type otherwise points at the static
fmttype rodata, which is safe.
Have traceprobe_define_arg_fields() duplicate the name and type strings
and anchor the copies on the trace_probe_event, which embeds the event
call and outlives every individual probe; trace_probe_event_free()
releases them.
The reproducer above triggers reliably; the field lookup and the delete
both run under event_mutex, so this is a dangling reference after
removal rather than a race.
The issue was found by the autokbug dynamic kernel fuzzer at Tencent
Yunding Lab.
Link: https://lore.kernel.org/all/20260826030009.1855331-1-bsdhenrymartin@gmail.com/
Fixes: ca89bc071d ("tracing/kprobe: Add multi-probe per event support")
Signed-off-by: Henry Martin <bsdhenrymartin@gmail.com>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
btf_find_struct_member() traverses into nested anonymous structures and
unions to find a struct member. However, get_bitoffset_of_field() in
trace_probe.c checked btf_type_kflag(type) using the outer parent type
instead of the actual anonymous structure/union that directly contains
the found member.
If the parent structure and anonymous structure have mismatched kflags
(e.g., the parent has kflag=0 while the anonymous structure has kflag=1
because it contains bitfields), the bitfield size encoded in the upper
8 bits of member->offset is erroneously treated as part of the byte/bit
offset, corrupting the resolved offset and failing to set last_bitsize.
Similarly, btf_find_struct_member() pushed anonymous member offsets
onto anon_stack without masking BTF_MEMBER_BIT_OFFSET() when kflag is set.
To fix this problem, update btf_find_struct_member() to return actual
containing structure/union type via member_type, use appropriate
__btf_member_bit_offset() to get bit offset, and use member_type for
btf_type_kflag() in get_bitoffset_of_field().
Link: https://lore.kernel.org/all/178827250904.123716.17452648791331881284.stgit@devnote2/
Fixes: c440adfbe3 ("tracing/probes: Support BTF based data structure field access")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260822095110.0772E1F000E9@smtp.kernel.org/
Assisted-by: Antigravity:gemini-3.7-flash
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
btf_find_struct_member() traverses into nested anonymous structures
and unions by pushing members with !member->name_off onto anon_stack.
However, it does not consider the unnamed bitfields (e.g. `int : 5`
or `unsigned int : 0`) which also have member->name_off == 0.
If such an unnamed bitfield is pushed to anon_stack, the
btf_find_struct_member() return an error even if there are other
valid entries in anon_stack.
To fix this, only push unnamed struct/union members to anon_stack.
Also move the btf_type_is_struct() check to the entry of this function
because now it is sure only struct/union are pushed to anon_stack.
Link: https://lore.kernel.org/all/178827249775.123716.7813217688423513612.stgit@devnote2/
Fixes: 302db0f5b3 ("tracing/probes: Add a function to search a member of a struct/union")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260830143859.D56991F00A3D@smtp.kernel.org/
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
The options files do not take the trace_array reference for the options
they represent. This could cause a use-after-free kernel crash if one of
these files is opened by one task and another task removes the instance
that the option is for. Because it doesn't take a reference upon opening,
it will not stop the removal which will free the options descriptor that
is being used.
As the options are somewhat dynamic in their creation at boot up, each
file represents a flag in the trace_array. The trace_array has an array of
indexes to represent each of these flags that is stored in the
trace_flags_index array. The address of the index array element is used to
pass to the inode->i_private pointer. Then that element is read which
holds the index (which represents the flag) and then the index is used to
calculate the trace_array descriptor from its trace_flags_index array.
One issue is that the index element can not be referenced until the
trace_array's reference is taken. To handle this, create a new helper
function called: trace_array_options_get() that will iterate all the
existing trace_arrays in the ftrace_trace_arrays list (under the
trace_types_lock), and compare the passed in address of the index element
with the entire array of the trace_array's trace_flags_index array.
If it matches, then up the corresponding trace_array's reference and
return.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260902121918.5a9e9d1b@gandalf.local.home
Fixes: 577b785f55 ("tracing: add tracer dependent options to options directory")
Reported-by: sashiko-bot@kernel.org
Closes: https://lore.kernel.org/linux-trace-kernel/20260828135858.2AC501F000E9@smtp.kernel.org/
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
When processing calls to bpf_loop() verifier marks R1 (and R4) as
precise. R1 tracks loop iterations number and because of the
'callback_depth < R1' mechanics in check_helper_call() must be marked
precise. However, precision propagation for R1 was broken,
when bpf_loop() call was verified on a second iteration.
Consider the following verification trace:
- main: bpf_loop(nr_loops, callback ...)
- callback: BPF_EXIT
- main: bpf_loop(nr_loops, callback ...)
- ...
While the first visit of the call to bpf_loop() propagated R1
precision as expected, the second call to mark_chain_precision() in
the check_helper_call() set R1, but it was immediately reset when
backtrack_insn() processed preceding BPF_EXIT in the loop deleted in
this patch.
Because of that, the second visit of the call to bpf_loop() injected
checkpoint with R1 not marked as precise. Which could trick the
verifier into accepting unsafe programs. See the next patch for an
example of such program.
Commit is structured in a way to minimize conflicts when
'bpf' would be eventually merged with 'bpf-next'.
Fixes: ab5cfac139 ("bpf: verify callbacks as if they are called unknown number of times")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20260831-bug-015-backtrack-cb-args-precise-v1-1-68a8e2a821e0@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
There's some internal state that ftrace_ops needs to have set, but since
it can be declared outside of the ftrace.c code, it calls
ftrace_ops_init() on the ops in every global function. The issue is that
if two tasks call it on the same ops at the same time it is possible to
have the initialization of one corrupt the initialization of the other
call.
Create a ops_mutex to use to synchronize every initialization of the
ftrace_ops. The mutex is taken within checking the ftrace_ops flag that
states it was initializied but the flag is checked again after the mutex
has been taken. Checking first outside the mutex allows it to shortcut
having to take the mutex. But then the check needs to be done again after
the mute is taken in case of races.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260902095501.6b59af20@gandalf.local.home
Fixes: f04f24fb7e ("ftrace, kprobes: Fix a deadlock on ftrace_regex_lock")
Reported-by: sashiko-bot@kernel.org
Close: https://lore.kernel.org/all/20260829025528.49A831F000E9@smtp.kernel.org/
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Nicholas Carlini reported a bug in precision backtracking mechanism
for BPF_LD | BPF_{IND,ABS} instructions. These instructions are
modelled as two branches:
- fallthrough;
- implicit exit from current subprogram.
The implicit exit case was not handled by the backtrack_insn()
function. When backtracking such a path backtrack_insn() did not
call bt_subprog_enter(), which meant that backtracking continued
manipulating precision marks in a caller frame, while looking at
instructions in a callee frame.
This lead to segmentation faults during verification (see the
selftest), or unsound state pruning.
Fixes: ee861486e3 ("bpf: Fix ld_{abs,ind} failure path analysis in subprogs")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260901-bug-016-backtrack-ld-abs-v1-1-59368f1be435@gmail.com
Change the nbcon_irq_work to be IRQ_WORK_LAZY, thus not raising
an IRQ upon irq_work queuing. The irq_work is then handled on the
next kernel tick. This additional delay is acceptable because
nbcon_irq_work is only responsible for non-emergency deferred
printing, which is delayed anyway. This has the benefit of not
needing to raise an IRQ for each printk() call.
On a side note, the Tegra20 and Tegra30 platforms can hang if an
irq_work IRQ is raised while entering cpuidle states. This problem
was reproducible by calling printk() while entering cpuidle. So
this change also provides a workaround for these platforms (as long
as they are not running tickless).
Link: https://lore.kernel.org/lkml/f3757a75-0ba1-4558-bf57-f19ab7e59a4c@nvidia.com
Fixes: 76f258bf3f ("printk: nbcon: Introduce printer kthreads")
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260901093245.344455-3-john.ogness@linutronix.de
Signed-off-by: Petr Mladek <pmladek@suse.com>
Ensure any pending nbcon_irq_work is flushed before allowing the
console to be recycled.
Signed-off-by: John Ogness <john.ogness@linutronix.de>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Link: https://patch.msgid.link/20260901093245.344455-2-john.ogness@linutronix.de
Signed-off-by: Petr Mladek <pmladek@suse.com>
syzbot reported a lockdep splat hitting DEBUG_LOCKS_WARN_ON(1) in
hlock_class() due to an invalid class_idx:
WARNING: kernel/locking/lockdep.c:238 at __lock_acquire+0x382/0x2cf0 kernel/locking/lockdep.c:5203
Workqueue: wg-crypt-wg0 wg_packet_tx_worker
RIP: 0010:hlock_class kernel/locking/lockdep.c:238 [inline]
RIP: 0010:check_wait_context kernel/locking/lockdep.c:4870 [inline]
RIP: 0010:__lock_acquire+0x389/0x2cf0 kernel/locking/lockdep.c:5203
Call Trace:
<IRQ>
lock_acquire+0x106/0x350 kernel/locking/lockdep.c:5886
_raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:173
tcp_tsq_handler+0x29/0x200 net/ipv4/tcp_output.c:1291
tcp_tsq_workfn+0x384/0x410 net/ipv4/tcp_output.c:1325
...
When a lock class is zapped (e.g. during module unload or key
unregistration), zap_class() clears the class's bit in
lock_classes_in_use and removes it from the class hash table.
However, existing lockdep_map instances embedded in data structures
may still retain a pointer to the zapped class in their class_cache[]
array.
When __lock_acquire() subsequently runs on such a lock, it finds
lock->class_cache[subclass] != NULL, skipping register_lock_class()
and assigning hlock->class_idx to the index of the zapped class. When
check_wait_context() or hlock_class() inspects the held_lock, it finds
!test_bit(class_idx, lock_classes_in_use) and warns. Furthermore, if
the zapped slot is subsequently re-allocated to an unrelated lock key,
the stale class_cache entry would erroneously match the unrelated
class (ABA issue).
Add lock_class_cache_is_valid() to validate that the cached class is
within lock_classes bounds, still allocated in lock_classes_in_use
(using uninstrumented arch_test_bit() in __always_inline context so it
is safe in noinstr contexts like match_held_lock()), and that
class->key matches the expected subkey (taking lockdep_set_subclass()
overrides into account). Also use READ_ONCE()/WRITE_ONCE() when
accessing class_cache[]. If the entry is invalid or stale, fall back
to register_lock_class() / look_up_lock_class().
Fixes: a0b0fd53e1 ("locking/lockdep: Free lock classes that are no longer in use")
Closes: https://lore.kernel.org/netdev/6a8c66dc.4d75e56a.c9a88.0050.GAE@google.com/T/#u
Reported-by: syzbot+2d770620059281e225a4@syzkaller.appspotmail.com
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260824155129.676096-1-edumazet@google.com
perf_mmap_close() drops rb->mmap_count *without* holding
event->mmap_mutex (the refcount_dec_and_test() right before the
refcount_dec_and_mutex_lock() of event->mmap_count). A concurrent
perf_mmap_rb() can slot its entire "revival" path into that window
(perf_mmap holds event->mmap_mutex for its whole duration, including
rb_alloc):
munmap side (perf_mmap_close) mmap side (perf_mmap_rb)
----------------------------------- --------------------------------
rb->mmap_count 1 -> 0 (no lock) (holds event->mmap_mutex)
inc_not_zero(rb->mmap_count) fails
ring_buffer_attach(event, NULL)
rb_alloc() + attach new rb
refcount_set(&event->mmap_count, 1)
lock; event->mmap_count 1 -> 0
ring_buffer_attach(event, NULL)
ring_buffer_put() -> frees the *new* rb
The revival's refcount_set(&event->mmap_count, 1) is an invisible
1 -> 1 write: the close frees the just-revived buffer although the
other process still has it mapped -- a page-level use-after-free
allowing local privilege escalation to root by any unprivileged user
(default kernel.perf_event_paranoid=2).
Swap the order of the two counter updates: event->mmap_count is
dropped first via refcount_dec_and_mutex_lock(), so its 1 -> 0
transition and the ring_buffer_attach() stay serialized with
perf_mmap(). rb->mmap_count == 0 then implies every event using the
buffer is detached already, so the result of the rb->mmap_count drop
can gate the remaining teardown directly and detach_rest is no longer
needed.
An earlier fix for this race from Kyle Zeng and David Lee takes
event->mmap_mutex around both counter updates [0]; here the not-last
close stays lockless.
Fixes: 59741451b4 ("perf: Identify the 0->1 transition for event::mmap_count")
Reported-by: Kimi Security Team <bug-report@moonshot.ai>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Co-developed-by: Weiming Shi <shiweiming@moonshot.ai>
Signed-off-by: Weiming Shi <shiweiming@moonshot.ai>
Signed-off-by: Yilin Zhang <yilinzhang@moonshot.ai>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/linux-perf-users/20260804060931.711308-1-david.lee@trailofbits.com/ [0]
Cc: <stable@vger.kernel.org>
Cc: stable@vger.kernel.org # 6.18+
Link: https://patch.msgid.link/20260831162155.1437652-1-yilinzhang@moonshot.ai
perf_aux_output_end() emits a PERF_RECORD_AUX when the recorded size is
nonzero or when any flag other than PERF_AUX_FLAG_OVERWRITE is set.
PMU format flags describe how an AUX payload is encoded. TRBE driver
sets PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW for raw trace buffers, causing
an AUX record to be emitted even when no trace data.
This is noticeable when tracing a task with strace. Ptrace stops
repeatedly end empty AUX transactions, producing many zero-sized
PERF_RECORD_AUX records. For example:
perf record -e cs_etm//u -m,128M -- strace ls
perf script -D 2>&1 |
awk '/PERF_RECORD_AUX offset/ {
for (i = 1; i <= NF; i++)
if ($i == "size:" && $(i + 1) == "0")
count++
}
END { print count }'
165
This recording contains 165 zero-sized AUX records which provide no
useful information to userspace.
Ignore PERF_AUX_FLAG_PMU_FORMAT_TYPE_MASK, together with
PERF_AUX_FLAG_OVERWRITE, when deciding whether an empty AUX record is
useful. Zero-sized records carrying TRUNCATED, PARTIAL or COLLISION
are still emitted.
Fixes: 547b60988e ("perf: aux: Add flags for the buffer format")
Reported-by: Tamas Petz <tamas.petz@arm.com>
Signed-off-by: Leo Yan <leo.yan@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260825-perf_core_fix_zero_aux_records-v1-1-23b95e8d5df3@arm.com
Cache-aware load balancing biases tasks toward their preferred LLC. On
asymmetric CPU capacity systems (e.g. big.LITTLE) the destination LLC may
contain CPUs that are too small to run the task. Pulling the task there
turns it into a misfit, trading a cache-locality gain for a capacity loss
that's more detrimental to performance.
Guard both cache-aware migration entry points against this:
- can_migrate_llc_task(): forbid the LLC migration when the task fits its
source CPU but would not fit the destination CPU.
- alb_break_llc(): veto the active balance under the same condition so the
runnable task is not pushed onto a CPU that cannot accommodate it.
Both checks are gated with checks for hybrid processors, so symmetric
systems are unaffected. Tasks that already do not fit their source CPU
are left to the existing LLC policy, since the move cannot make their
fitness worse (this also preserves misfit up-migration to bigger CPUs).
Additionally, if there are misfit tasks found in the load balancing
classification phase, prioritize misfit task migrations
over LLC load aggregation on asymmetric systems. A better fitting
CPU will boost performance more than better cache locality.
Reviewed-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Tested-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Reviewed-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/edbb2503d554c63dc9b72e201fb4a17e1cb119e7.camel@linux.intel.com
distribute_cfs_runtime() refreshes the rq clock and accounts elapsed
runtime with update_curr() before redistributing bandwidth, but gates
this on cfs_rq->curr. Since commit 85570f10a4 ("sched/eevdf: Move to
a single runqueue") cfs_rq->curr is only maintained on the root
cfs_rq, so for the cgroup cfs_rqs it walks, the check never fires and
the refresh is dead code.
Use cfs_rq->h_curr, the per-level current entity, restoring the
intended behaviour: only refresh when something is actually running at
the throttled level, i.e. within the deferred throttle window.
Without this, runtime consumed by a still-running task of the
throttled hierarchy is not docked before redistribution;
unthrottle_cfs_rq() catches up unconditionally since
commit 28ad542768 ("sched/fair: Call update_curr() before
unthrottling the hierarchy"), so this is not a correctness hole today,
but the refresh the check was written for is gone.
Fixes: 85570f10a4 ("sched/eevdf: Move to a single runqueue")
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Aaron Lu <ziqianlu@bytedance.com>
Tested-by: Aaron Lu <ziqianlu@bytedance.com>
Link: https://patch.msgid.link/20260831101141.391382-3-liwanwu@kylinos.cn
After commit 85570f10a4 ("sched/eevdf: Move to a single runqueue"),
cfs_rq->curr is only maintained on the root cfs_rq (set/cleared from
set_next_task_fair()/put_prev_task_fair()), while cfs_rq->h_curr is
the per-level current entity, set by set_next_entity() at every level
of the hierarchy. For an intermediate cfs_rq (a cgroup), cfs_rq->curr
is always NULL, but cfs_rq->h_curr is the group entity at that level.
throttle_cfs_rq() reads cfs_rq->curr to decide whether there is a
running entity at the throttled level, in which case it should request
a full sched_cfs_bandwidth_slice() of runtime and arm the deferred
throttle task_work via task_throttle_setup_work(). For intermediate
cfs_rqs the check is always false, so bandwidth-controlled cgroups
always get just 1ns of runtime and never arm the deferred throttle
work; the running task then escapes throttling until the next pick
arms the work instead, even though there is an on-rq entity at this
level.
Switch the read to cfs_rq->h_curr so intermediate bandwidth-controlled
cgroups behave consistently with the root cfs_rq, matching the
existing usage of cfs_rq->h_curr in update_curr() and
check_enqueue_throttle().
Fixes: 85570f10a4 ("sched/eevdf: Move to a single runqueue")
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Aaron Lu <ziqianlu@bytedance.com>
Tested-by: Aaron Lu <ziqianlu@bytedance.com>
Link: https://patch.msgid.link/20260831101141.391382-2-liwanwu@kylinos.cn
Commit 4b603f1551 ("sched: Update rq->avg_idle when a task is moved
to an idle CPU") moved rq->avg_idle accounting out of the wakeup path and
into put_prev_task_idle(), so that the idle interval is consumed whenever
the idle task is switched out.
The wakeup-side accounting that it replaced only updated rq->avg_idle
when rq->idle_stamp was non-zero. The new helper lost that validity
check and unconditionally computes:
rq_clock(rq) - rq->idle_stamp
If rq->idle_stamp is zero, this uses rq_clock(rq) as the sample. That is
not a valid idle duration and can immediately drive rq->avg_idle to its
clamp.
This can happen when sched_balance_newidle() returns before setting
rq->idle_stamp, for example when this_rq->ttwu_pending is set. In that
case the rq can switch to the idle task with idle_stamp still zero and
leave idle again when the pending wakeup is processed.
Other paths can also switch to the idle task without setting
rq->idle_stamp via newidle_balance(), for example find_proxy_task() or
force-idling.
Restore the idle_stamp validity check in update_rq_avg_idle() and skip
the rq->avg_idle update when there is no measured idle interval.
Fixes: 4b603f1551 ("sched: Update rq->avg_idle when a task is moved to an idle CPU")
Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260807-master-v3-1-c328354efed3@gentwo.org
A migrate_disable()'d RT task cannot be moved to another CPU, but the
scheduler still keeps such a task on that CPU's pushable list
(rq->rt.pushable_tasks) and still marks the runqueue RT-overloaded
(rq->rt.overloaded = 1). So the RT balancer keeps treating this CPU as
having a task to move away, and keeps trying to move the task, but the
push can never succeed. When the head is pinned, push_rt_task() does not
give up either. It falls back to pushing rq->curr instead, using the
per-CPU stopper, as added by commit a7c81556ec ("sched: Fix
migrate_disable() vs rt/dl balancing").
The CPU spends tens of milliseconds in this retry loop. The core is
isolated for real-time work, but during the loop nearly half of its time
is consumed by pushes that cannot succeed.
An ftrace capture of the affected CPU, with sched_switch enabled and
commit 94894c9c47 ("sched/rt: Skip currently executing CPU in
rto_next_cpu()") applied, shows where the CPU time went. Two SCHED_FIFO
tasks at equal priority shared the CPU, taskA migrate_disable()'d and
queued, taskB as rq->curr. In one 89 ms window, taskB got only 52 ms of
CPU. The other 37 ms went to the stopper thread.
The scheduler kept trying to push taskA, the pinned head of the pushable
list, fell back to pushing taskB instead, and woke the stopper 5204
times. Every one of those pushes failed and no task was moved. taskA
stayed runnable and queued the whole time, and never ran.
Pushing taskB fails on a re-check. find_lock_lowest_rq() drops the rq
lock to take the target rq lock, then checks again with
"task != pick_next_pushable_task(rq)".
The task being pushed is taskB, but the pick returns taskA, the head of
the pushable list. taskB is rq->curr, and set_next_task_rt() removes the
running task from that list, so taskB can never be the head. The check
expects a candidate taken from the pushable list, but the fallback
pushes rq->curr, which is never on that list. So the check fails every
time.
.--> push-IPI arrives
| |
| v
| pushable head = taskA -> pinned, cannot be pushed
| |
| v
| so push taskB instead -> wake migration/N, a stop-class
| | thread, so it preempts taskB
| v
| re-check compares taskB against the pushable head,
| which is still taskA -> give up
| |
| v
| nothing moved, taskA still queued, rq still overloaded
| |
'----------'
repeats every ~17 us, 5204 times, for 89 ms
The loop cannot stop itself. Every round leaves the runqueue
exactly as it was, so the next push-IPI does the same thing. In
the capture it ended only when taskB went to sleep on its own.
taskA was then picked locally and left the pushable list.
CPU time per task in the window, from sched_switch:
taskB 51.95 ms real work
migration/N 37.18 ms nothing moved
taskA 0.00 ms queued the whole time, never picked
idle 0.01 ms
Counts over the same window:
7667 push-IPIs handled on this CPU
17481 pick_next_pushable_task() returned taskA, still pinned
5204 find_lock_lowest_rq() gave up on the re-check
1 push that actually completed
0 migrations of taskA
The CPU times and the window length come from the standard
sched_switch tracepoint. The counts needed tracepoints added inside
the RT balancer for this investigation.
The self-IPI path is closed by the rto_next_cpu() fix above, and that
part works. But the runqueue is still marked overloaded, because the
pinned task is still advertised as pushable. Other CPUs now send the
push-IPIs during their own RT balancing, and the same loop runs again.
Closing the self-IPI path did not stop a pinned task from triggering
push balancing.
A pinned task should never have been returned as a push candidate in the
first place. A migrate_disable()'d task cannot be migrated, so it
belongs in the same skip that was added for on_cpu tasks by
commit e0ca8991b2 ("sched: Make class_schedulers avoid pushing
current, and get rid of proxy_tag_curr()"). Add is_migration_disabled()
to the skip condition in pick_next_pushable_task() and
pick_next_pushable_dl_task().
With the skip in place, if the pinned task is the only extra runnable
task the helpers return NULL, push_rt_task() and push_dl_task() give up
early, and no stopper is woken. The pinned task then runs locally once
curr yields. If a task that really can be migrated is queued behind the
pinned head, it is now picked and pushed for real.
This makes the fallback that pushes rq->curr unreachable when the
pushable head is migrate-disabled. Nothing is lost, because that path
was always stopped by the re-check described above. In the capture it
ran 5204 times and moved nothing.
Fixes: a7c81556ec ("sched: Fix migrate_disable() vs rt/dl balancing")
Signed-off-by: Seiji Nishikawa <snishika@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260830073746.2189355-1-snishika@redhat.com
pick_task_fair() and yield_task_fair() call update_curr(&rq->cfs) to bring
curr up to date before they look at the eevdf state. With cgroups that
does not happen: update_curr() reads ->h_curr, which on the root cfs_rq is
the top level group entity, and returns at the !entity_is_task() check
before touching vruntime. Both then read ->curr, so the guard and the
update disagree about which entity they mean.
Counting how often ->h_curr and ->curr differ at pick_task_fair(), on one
CPU for 10s with three busy tasks and one 200us-periodic task:
all tasks in the root cgroup 43321 calls, 0 no-ops
busy tasks in G0, periodic in G1 45211 calls, 45193 no-ops
Whether that matters depends on what precedes the pick. Since
commit 68e3748781 ("sched/fair: Fix flat hierarchy") the tick and
enqueue/dequeue all update curr correctly, so on the normal reschedule
path only the microseconds between those and the pick are missing, and I
could not measure a latency difference there. Three paths have nothing
before them on that rq though:
- pick_task() on the sibling rqs of a core under core scheduling
(kernel/sched/core.c), which updates that rq's clock first for
exactly this reason
- fair_server_pick_task()
- yield_task_fair(), where the stale value feeds the entity_eligible()
test that guards forfeiting the remaining vruntime
There curr can be a full tick behind, as it was before that commit.
No new behaviour for the entity being updated: without cgroups ->h_curr
is already the task, so these two call sites already run the full
update_curr() including update_deadline(), dl_server_update() and the
resched_curr_lazy() at the end. This makes the cgroup case do the same.
Fixes: 85570f10a4 ("sched/eevdf: Move to a single runqueue")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://patch.msgid.link/20260822105930.2352761-1-zhanxusheng1024@gmail.com
ops.cgroup_set_idle() is documented to be invoked when a cgroup
transitions between idle and non-idle states, and scx_group_set_weight()
already skips value-preserving writes. scx_group_set_idle() delivers
every write unconditionally, so rewriting an already-correct cpu.idle
value feeds the BPF scheduler a transition callback each time, which
toggle- or accounting-based schedulers miscount. Mirror the weight
guard and only deliver on an actual change.
Verified with a probe scheduler printing each callback: rewriting
cpu.idle=1 twice on an already-idle cgroup delivered two callbacks
before and none after.
Fixes: 347ed2d566 ("sched/ext: Implement cgroup_set_idle() callback")
Link: https://lore.kernel.org/r/b53c61a1-4d7d-4232-941f-d48b0563d4ed@linux.dev
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
The trace instance files set_ftrace_filter and set_ftrace_notrace was
updated to work with specific trace instances (trace_arrays). The issue is
that when these files are opened, there is a small race window where it
will use the ftrace_ops from the inode->private pointer to get a reference
to the trace_array and then take its reference. The problem is that the
ftrace_ops itself could be freed. If the rmdir on the instance happens at
the same time the set_ftrace_filter file is opened, the rmdir could have
also freed the ftrace_ops and referencing it will cause a use-after-free
bug and crash the kernel.
Instead, pass in the trace_array as the file private data (NULL for the
top level instance), and then pass both the trace_array and the ftrace_ops
to the ftrace_regex_open() function. If the trace_array is NULL, then it
just uses the ftrace_ops without the need to take its reference (like
normal). If the ftrace_ops is NULL, that is only the case for the top
level instance and the global_ops can be used.
This allows the trace_array to have its reference incremented before
touching the ftrace_ops that could also be freed when the instance is.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260828223901.29e26edb@robin
Fixes: 591dffdade ("ftrace: Allow for function tracing instance to filter functions")
Reported-by: Breno Leitao <leitao@debian.org>
Tested-by: Breno Leitao <leitao@debian.org>
Closes: https://lore.kernel.org/all/apGORjltZgAiAYHT@gmail.com/
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
The newly added files show_event_filters and show_event_triggers that show
all filters or triggers that are set within the trace array do not take a
reference for the trace array it is showing. Without taking a reference,
the trace_array may be freed via "rmdir" while a task is reading one of
theses files. Those files iterate all the events within an instance
(trace_array) and nothing prevents that instance from being freed while
its data is being read. This causes a use-after-free crash.
Have the open of both those files take the trace_array reference via the
trace_array_get() that prevents the trace_array from being freed while the
files are opened.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260828094153.17b95037@gandalf.local.home
Fixes: 729757b96a ("tracing: Add show_event_filters to expose active event filters")
Fixes: 6a80838814 ("tracing: Add show_event_triggers to expose active event triggers")
Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Closes: https://lore.kernel.org/all/CA+0ovCjerKZJLwXScM9bF2ga2rLi4_XOpUfK41NDbENpeu98jA@mail.gmail.com/
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Sashiko pointed out the some of the scope cleanups for free_uprobe
could get an error pointer. Handle this case in free_uprobe
to prevent a crash.
On the other hand the macro doesn't need the guard because
free_uprobe itself already does the check.
Link: https://lore.kernel.org/all/20260831150651.1134594-2-ak@kernel.org/
Assisted-by: omp:gpt-5.6-luna sashiko
Signed-off-by: Andi Kleen <ak@kernel.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
scx_cgroup_init_args carries the initial weight and bandwidth control
parameters of a cgroup to ops.cgroup_init(), but not its cpu.idle
state. A cgroup that was already configured idle before the scheduler
was loaded (or before it was onlined under it) is presented as
non-idle, and the BPF scheduler only learns about it if cpu.idle is
written again later.
Add the sched_idle state to scx_cgroup_init_args and fill it in all
four places that build the args: scx_tg_online() for cgroups onlined
under the scheduler, scx_cgroup_init() for cgroups that already exist
when the scheduler is loaded, and the sub-scheduler handover paths
scx_cgroup_claim_subtree() and scx_cgroup_return_subtree().
Verified in a VM with a probe scheduler printing the init args: a
cgroup configured cpu.idle=1 before loading shows sched_idle=1 in
ops.cgroup_init(), the default shows 0, and later cpu.idle writes
still come through ops.cgroup_set_idle(). The sub-scheduler paths
are compile-tested only.
Fixes: 347ed2d566 ("sched/ext: Implement cgroup_set_idle() callback")
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
- An unbound worker pool could be freed while still reachable through the
pending-activation list, leading to a use-after-free. Unlink before
dropping the reference.
- On PREEMPT_RT, the BH workqueue kick raised softirqs from preemptible
context, tripping a lockdep assertion and possibly losing concurrently
raised softirq bits.
- Draining BH work off a dead CPU nests two pools' callback locks, which
lockdep misreported as recursive locking. The nesting cannot deadlock.
Annotate it.
- Reject watchdog thresholds that overflow the conversion to jiffies.
- Make the drgn workqueue dump script work again on kernels and vmcores
from before the workqueue attrs field rename.
-----BEGIN PGP SIGNATURE-----
iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCapXs/Q4cdGpAa2VybmVs
Lm9yZwAKCRCxYfJx3gVYGd75AP9VTI8d7dor4mn82j7J6l8Xcy8U1ePM/K5v1PqU
n9lfUwD/aopS+dp/uCuqR6pLBxineFAPxNoEgxxO2bDv9OqhoAY=
=fj3U
-----END PGP SIGNATURE-----
Merge tag 'wq-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
Pull workqueue fixes from Tejun Heo:
- An unbound worker pool could be freed while still reachable through
the pending-activation list, leading to a use-after-free. Unlink
before dropping the reference
- On PREEMPT_RT, the BH workqueue kick raised softirqs from preemptible
context, tripping a lockdep assertion and possibly losing
concurrently raised softirq bits
- Draining BH work off a dead CPU nests two pools' callback locks,
which lockdep misreported as recursive locking. The nesting cannot
deadlock. Annotate it
- Reject watchdog thresholds that overflow the conversion to jiffies
- Make the drgn workqueue dump script work again on kernels and vmcores
from before the workqueue attrs field rename
* tag 'wq-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq:
tools/workqueue/wq_dump.py: Support backward compatibility for wq->attrs rename
workqueue: reject watchdog thresholds that overflow jiffies
workqueue: Fix unbound pool lifetime for pending pwqs
workqueue: Use raise_softirq() to trigger softirq in irq_work handler
workqueue: Annotate cb_lock nesting when draining a dead BH pool
- After cgroup.kill was written to a cgroup, every child cloned into it
with CLONE_INTO_CGROUP was spuriously killed because the fork path
snapshotted the kill counter before resolving the target cgroup.
- Releasing an isolated cpuset partition dropped the isolation of CPUs
isolated on the kernel command line.
- Selftest and documentation fixes.
-----BEGIN PGP SIGNATURE-----
iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCapXm8w4cdGpAa2VybmVs
Lm9yZwAKCRCxYfJx3gVYGbhtAQCfUc4oanF94uiAGzH2UAA2HIorLT4TDjuDj/oM
qrqeLQEA1p2XQz56SYXhK8fG+fy+Ep3xxqS2gStmtYL20ie4lQc=
=i4QU
-----END PGP SIGNATURE-----
Merge tag 'cgroup-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
- After cgroup.kill was written to a cgroup, every child cloned into it
with CLONE_INTO_CGROUP was spuriously killed because the fork path
snapshotted the kill counter before resolving the target cgroup
- Releasing an isolated cpuset partition dropped the isolation of CPUs
isolated on the kernel command line
- Selftest and documentation fixes
* tag 'cgroup-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
selftests/cgroup: test clone3() into a previously killed cgroup
cgroup: fix spurious SIGKILL of CLONE_INTO_CGROUP children
selftests/cgroup: Add test for preserving boot-isolated CPUs
cgroup/cpuset: Preserve boot-isolated CPUs on partition release
selftests/cgroup: Drop invalid boot isolation comparison
docs: cgroup-v2: fix misc.events key format description
selftests/cgroup: Fix cg_run_in_subcgroups ignoring arg parameter
selftests/cgroup: set the test plan after the setup checks
- The task ownership check in the dispatch queue move operation raced
against the task exiting or moving to a different sub-scheduler,
spuriously triggering scheduler aborts. Fix by moving the check under
the queue lock.
- The cgroup bandwidth change callback runs in a sleepable context but
sleepable implementations were rejected at load time. Allow them and add
a marker so userspace can detect the capability.
- Sync tooling headers with the scx repo for accumulated compatibility
improvements.
- Example scheduler fixes: ignored timer re-arm failures and vtime credit
loss on cgroup migration.
- Documentation and comment fixes.
-----BEGIN PGP SIGNATURE-----
iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCapXcug4cdGpAa2VybmVs
Lm9yZwAKCRCxYfJx3gVYGZFfAQCMdpcFMqLdaHkRMiWK+SopQ703AnnpoX9xC81t
vy81mQD/QlIEZimidIS2xju3/HkHfVXX6hHxAbMy8hdclhYHlgM=
=256l
-----END PGP SIGNATURE-----
Merge tag 'sched_ext-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
- The task ownership check in the dispatch queue move operation raced
against the task exiting or moving to a different sub-scheduler,
spuriously triggering scheduler aborts. Fix by moving the check under
the queue lock
- The cgroup bandwidth change callback runs in a sleepable context but
sleepable implementations were rejected at load time. Allow them and
add a marker so userspace can detect the capability
- Sync tooling headers with the scx repo for accumulated compatibility
improvements
- Example scheduler fixes: ignored timer re-arm failures and vtime
credit loss on cgroup migration
- Documentation and comment fixes
* tag 'sched_ext-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Fix missing @slice and @vtime descriptions in finish_dispatch() kernel-doc
sched_ext: Fix several comment issues
sched_ext: Check bpf_timer_start return values in scx_qmap
sched_ext: Fix vtime delta loss in scx_flatcg cgroup migration
sched_ext: Fix timer pinning and return value in scx_central
docs/sched_ext: document that cgroup CPU knobs are scheduler-dependent
sched_ext: Fix spurious aborts in scx_bpf_dsq_move() on ownership change races
sched_ext: Sync common and compat headers from the scx repo
sched_ext: Sync tools autogen enum headers from the scx repo
Docs/admin-guide/cgroup-v2: document BPF scheduler callbacks for cpu.max and cpu.idle
sched_ext: Fix nonexistent field in sched-ext.rst example
sched_ext: Allow ops.cgroup_set_bandwidth() to be sleepable
Signed-off-by: Carlos Maiolino <cem@kernel.org>
-----BEGIN PGP SIGNATURE-----
iJUEABMJAB0WIQSmtYVZ/MfVMGUq1GNcsMJ8RxYuYwUCapUNuQAKCRBcsMJ8RxYu
Y/QVAX9SDXNSP3dw04wAuYgwSH5Ftm+WAnwusAsSvJkQdTvU0nEpAHyjb6WokS5a
EbOGy5UBfRyqJFOmOw6wF5Ax0Aoxrt+lN8CuoDoh6aEhtYlh0jvd50ustYX8QSas
W2R9B6IFIw==
=JWP4
-----END PGP SIGNATURE-----
Merge tag 'xfs-fixes-7.3-rc2' of gitolite.kernel.org:/pub/scm/fs/xfs/xfs-linux
Pull xfs fixes from Carlos Maiolino:
"This contains a few fixes for the zoned storage support, a possible
deadlock vector fix, some code refactoring patches and a quota evasion
fix on XFS while exporting it via NFS.
Please note that for the quota evasion fix, a couple patches for the
capability subsystem are included in the pull request. Those have been
ack'ed by the respective maintainer which also agreed to have them
going through the xfs tree.
This also includes a patch for the quota subsystem to stop issuing
audit messages during quota enforcing. Quota maintainer also ack'ed
and agreed with this going through xfs tree"
* tag 'xfs-fixes-7.3-rc2' of gitolite.kernel.org:/pub/scm/fs/xfs/xfs-linux:
capability: unexport has_capability_noaudit
xfs: replace ns_capable_noaudit
quota: Don't issue audit messages on quota enforcing
capability: Add new capable_noaudit
xfs: fix capability check in xfs
xfs: restore bi_bdev in xfs_zone_gc_write_chunk
xfs: split ioend handling into a separate source file
xfs: factor out a xfs_iomap_set_anon_write helper
xfs: fix zoned write iomap flags assignments
xfs: fix racy open zone caching
xfs: handle NULL open_zone for merged ioends in xfs_ioend_put_open_zones
xfs: use inode_init_always_gfp with __GFP_NOFAIL in xfs_inode_alloc
xfs: remove kmem_to_page()
xfs: don't flush and invalidate internal RT device twice in xfs_shutdown_devices
xfs: split an assert in xfs_trans_log_buf
xfs: don't hold buffer locks across sync transaction commit in xfs_sync_sb_buf
The watchdog threshold is supplied in seconds but is multiplied by HZ
before being used as a jiffies interval. Reject values that exceed
MAX_JIFFY_OFFSET / HZ so the multiplication cannot wrap and the
time_after() comparisons remain within their supported range.
The check is performed before changing the threshold or watchdog timer.
Zero remains the value used to disable the watchdog.
Fixes: 82607adcf9 ("workqueue: implement lockup detector")
Signed-off-by: Jiacheng Xu <stitch@zju.edu.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
Commit 13f1eae3b6 ("sched_ext: Synchronize slice and dsq_vtime
writes") added the slice and vtime parameters to finish_dispatch() but
did not update its kernel-doc, which produces warnings:
Warning: function parameter 'slice' not described in 'finish_dispatch'
Warning: function parameter 'vtime' not described in 'finish_dispatch'
Describe both parameters using the same wording as
dispatch_to_local_dsq(), which receives the same values.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
Since commit b69bb476de ("cgroup: fix race between fork and
cgroup.kill"), the fork path snapshots the kill_seq of the child's
future cgroup into kargs->kill_seq, and cgroup_post_fork() SIGKILLs
the child if that cgroup's kill_seq has changed in the meantime, to
catch forks racing with a cgroup.kill sweep.
For CLONE_INTO_CGROUP, however, the snapshot in cgroup_css_set_fork()
is taken before the target cgroup has been resolved: kargs->cgrp is
always NULL at this point (it is only set at the end of the function).
So the "if (kargs->cgrp)" branch is dead code and the snapshot always
records the kill_seq of the parent's cgroup. cgroup_post_fork() then
compares it with the kill_seq of the target cgroup, so the child gets
SIGKILLed whenever the two cgroups have been killed a different number
of times.
As a result, once cgroup.kill has been written to a cgroup, every
child subsequently cloned into it with clone3(CLONE_INTO_CGROUP) is
killed on the spot, for as long as the cgroup exists: kill_seq is not
exposed to userspace and never resets.
Re-snapshot kill_seq from the target cgroup once it has been resolved,
and drop the dead branch at the early snapshot site.
This does not reopen the race fixed by b69bb476de. For
CLONE_INTO_CGROUP, everything from the snapshot to the check in
cgroup_post_fork() runs with cgroup_mutex held, and kill_seq is
only ever incremented under cgroup_mutex.
tj: Updated the comment above kill_seq to reflect the new serialization
rules as suggested by Shakeel Butt.
Fixes: b69bb476de ("cgroup: fix race between fork and cgroup.kill")
Cc: stable@vger.kernel.org
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Assisted-by: LLM
Signed-off-by: Etienne Perot <eperot@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
