mirror of
https://github.com/torvalds/linux.git
synced 2026-09-22 04:34:03 +02:00
v7.3-rc1
5717 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Linus Torvalds
|
637836563d |
- Revert a commit to spinlock cleanup guards that got caught up
in the subtle limitations & fragility of guards (again...) and
caused a regression (Peter Zijlstra)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqTmG0RHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1iV+A/+J2IN9xHNPv0O9rLKaJvLsNnlMPQ4QJJB
kTbEOLDrAE7ozTmP2sLfUu75w1FQz0Hp00/tf1V7VFYaSeMBdMoqJs3itCtuqBrU
qkVFrx7splaNauvFgxdaHd5dYGszwFOjixMwhezQC1wn46ckW29bEHbIrmY5j1xC
zF7iSTaVp2zHBbkh0n6ozN28zSbHZcLEAD8mvGoscv3Bnb+9HRMAr4weTHC94kq5
4hDaRS48CgofxEfXT0erp6Rc7lS1YTUltSvkICORTv0tYtDqDf1GCiZKIjaW8FqW
N7Nre77fSlH6HfzzujNgNOhGAGoVO+Ki9vfqgnYhnFRc864g9OTREmdNLlJ2V5XB
yC0SCbAeAjOvbtTLQlRmDlKWJPlwKgXkatGupczkCna3EeL9eXx33fOO6kOGxNNs
RpQXv+wPv0S9EDdIkkuMwhx0dq6yUCfrA22+wlXUzFhXvHTbhqZlDneI9OtzubZm
kU+Vf8dgVU0z59C+2ZOwlp5YHVXquRfNXOz8lrRIeb0y0Iyvlo7pDuQ/t8w3Bh3c
KVlAEEQsK8maBHgHHKMv8ml4W6XJHC9KG6x0TtKDv+ntmrNBDhvq2sxWrhBA7JT4
kXdab1QG/hP2FEY3M6bChEadT64U1BeGlcI9xMkhX886PwrnOMzTd9xiAN7UtdCR
dITQG+bHeQo=
=EtPf
-----END PGP SIGNATURE-----
Merge tag 'locking-urgent-2026-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking fix from Ingo Molnar:
- Revert a commit to spinlock cleanup guards that got caught up
in the subtle limitations & fragility of guards (again...) and
caused a regression (Peter Zijlstra)
* tag 'locking-urgent-2026-08-30' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
locking: Revert switching guards to _irq_{disable,enable}()
|
||
|
Linus Torvalds
|
85671b807f |
More power management updates for 7.3-rc1
- Fix a kernel panic during PMU unbind in the intel_rapl power capping
driver and sign-extend the PMU delta on counter wraparound in it to
avoid misreporting energy (Sumeet Pawnikar and Yifan Li)
- Unblock runtime PM when device prepare fails that was not done by
mistake (Shibo Zhu)
- Fix possible rate limit overflow on 32-bit systems in the schedutil
cpufreq governor (Hui Su)
- Consolidate HWP P-states initialization in the intel_pstate cpufreq
driver and make that driver avoid using the DESIRED_PERF HWP hint
when the Dynamic Efficiency Control (DEC) is enabled in the processor
to avoid inconsistent behavior (Rafael Wysocki)
-----BEGIN PGP SIGNATURE-----
iQFGBAABCAAwFiEEcM8Aw/RY0dgsiRUR7l+9nS/U47UFAmqPLkgSHHJqd0Byand5
c29ja2kubmV0AAoJEO5fvZ0v1OO1rjkIAJHjnU5/ak8dVrKfDNdK7vcP656sNJkx
gsVjdrU0ki4JSE9n/PE2Z1SjqSz0DvWnk1RYxXBKYbwTswuBE7xqcT5M2g1RycbA
LyEQRRUbFUHJANpNko1y431BPyiSzX/YjdFadC9vRi/IhTVxJ4SpEp0aXnqy7ANV
JppXhyRDpgEcH2OjXhjKmKFnYD2VBw0zcIidok5uTZg6rftlxLRpzabMiGJ+T4x4
h5l5ZDejoMnL/A6MUyrJO5cXi5E+moYXFZj7ofTgX5aNzxxu0rdQ130VMlChQI19
nc1GdStcYO74xTguexlxU6nKWL3eiLaLqulEqghInAvVq+HEQyghsLc=
=kZl3
-----END PGP SIGNATURE-----
Merge tag 'pm-7.3-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
Pull more power management updates from Rafael Wysocki:
"These fix two issues in the intel_rapl power capping driver, fix a
potential issue in the schedutil cpufreq governor on 32-bit systems,
fix a runtime PM issue related to failing system suspend, and update
the intel_pstate cpufreq driver:
- Fix a kernel panic during PMU unbind in the intel_rapl power
capping driver and sign-extend the PMU delta on counter wraparound
in it to avoid misreporting energy (Sumeet Pawnikar and Yifan Li)
- Unblock runtime PM when device prepare fails that was not done by
mistake (Shibo Zhu)
- Fix possible rate limit overflow on 32-bit systems in the schedutil
cpufreq governor (Hui Su)
- Consolidate HWP P-states initialization in the intel_pstate cpufreq
driver and make that driver avoid using the DESIRED_PERF HWP hint
when the Dynamic Efficiency Control (DEC) is enabled in the
processor to avoid inconsistent behavior (Rafael Wysocki)"
* tag 'pm-7.3-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
powercap: intel_rapl: Fix kernel panic during PMU unbind
PM: sleep: Unblock runtime PM when device prepare fails
powercap: intel_rapl: Sign-extend the PMU delta on counter wraparound
cpufreq: intel_pstate: Avoid using DESIRED_PERF when DEC is enabled
cpufreq: intel_pstate: Consolidate HWP P-states initialization
cpufreq: schedutil: Fix rate limit overflow
|
||
|
Rafael J. Wysocki
|
76854b339b |
Merge branches 'pm-cpufreq' and 'pm-sleep'
Merge additional cpufreq updates and one update related to system sleep for 7.3-rc1: - Unblock runtime PM when device prepare fails that was not done by mistake (Shibo Zhu) - Fix possible rate limit overflow on 32-bit systems in the schedutil cpufreq governor (Hui Su) - Consolidate HWP P-states initialization in the intel_pstate cpufreq driver and make that driver avoid using the DESIRED_PERF HWP hint when the Dynamic Efficiency Control (DEC) is enabled in the processor to avoid inconsistent behavior (Rafael Wysocki) * pm-cpufreq: cpufreq: intel_pstate: Avoid using DESIRED_PERF when DEC is enabled cpufreq: intel_pstate: Consolidate HWP P-states initialization cpufreq: schedutil: Fix rate limit overflow * pm-sleep: PM: sleep: Unblock runtime PM when device prepare fails |
||
|
Peter Zijlstra
|
46094a7708 |
locking: Revert switching guards to _irq_{disable,enable}()
Revert commit |
||
|
Linus Torvalds
|
2709dd5ae3 |
Misc scheduler fixes:
- Add missing cpus_read_lock locking to rebuild_sched_domains()
(Sebastian Andrzej Siewior)
- Fix division by zero bug in tg_cpus() that can be triggered
with empty cpusets (Jake Steinman)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqKGdMRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1jQ3w/9Hpnbu78VJjicFBH/De/xXJ10FBJrZMI2
WzS7XXs2AeSPM9aUrhwkVgus/6x0rkAShht0tb9+ax6/DcDDxcSW2tBzCUDZ++Wu
g1oUDlbEP8OAtotJQLx3ss+/vgEjTEv7gidMNEIdqGRVlMyfUmAvBZ0C9REdFZT1
Lq8I0KtEVNKCLMi5LfcafQTHld4o67Wb2d6DqMDbSCkT45sjGvlVOmYnJPhVtUqs
JaOdvit7uheOgc/mtlWsdJXd4yYtKlMpqq5tzQSCGEfReoMdsFa8FXpaZiT/kn6N
XuCaqgtUG5J94bREzNxpsfqKclK4QWqUj3MmD2i5vzoGJkEiXDyY6B6WNqGex2KG
gJvglaW+HfyDjh8RB5n3KOGX6ikYtDBYc8HtmJewOfEGQnX/c+E7Xhl4/9u1CUnN
6z47IP80Ch8tWQNz5XZUplzbm8DL+gYMFdh8L8oRXab3rFby72h+Ftxz/epNP14W
1D1mKZE+TkeyGnKk1cidf/cz60qPBPrLp3gYzbOeaJPk5y388t/gmO+V07Mwzl2Q
O17hkIk0+4ySiRvnd8IxDowTWz53FE9qPoky5zPRGf+SvZXu9GAbY1Hwo+jl6Qle
DTg1jptYkTe5Mnttmxgi2gSTPofrGloUqXAMP/T+U+Qos+dF/ujm4pgVdWsLLD/F
rhBjdJRybrE=
=RQRK
-----END PGP SIGNATURE-----
Merge tag 'sched-urgent-2026-08-22' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fixes from Ingo Molnar:
- Add missing cpus_read_lock locking to rebuild_sched_domains()
(Sebastian Andrzej Siewior)
- Fix division by zero bug in tg_cpus() that can be triggered with
empty cpusets (Jake Steinman)
* tag 'sched-urgent-2026-08-22' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/fair: Floor tg_cpus() at 1
sched/topology: Add a cpus_read_lock to rebuild_sched_domains()
|
||
|
Linus Torvalds
|
0d78592583 |
Miscellaneous futex fixes:
- Series to enforce that private futex owner shares the mm when
attaching. (Kyle Zeng, Thomas Gleixner)
- Fix race on the initial mm->futex.phash.ref allocation
(Hyunwoo Kim)
- Fix might_sleep() warning in futex_pivot_pending()
(Peter Zijlstra)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqKGJwRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1iKxhAAj+A3L8a1OHRCKCMIE1gbk+bL5CYYrDqf
TO642/rGmvvrMdNtocK3IbzIady0umhZFrPqIRJcN5SAnoZf6mL5uD4XaR4xgaTp
prbNTXJrxzPMN5u3DcqnlWfGwFYXZxJIN6WBZCT7N6D/czr0Mqh7+Isdxey6nDWp
zWZfJ4DeVbSbf+2zft6pjwzXQAljyvEYEilV8+xgzwNfJjmG1QRS+MisPjjuGMZh
4C5bTTo966WgyKCu0MdtuP9a/WFT4ZUMWXXCB89Dz3mXImntltu+0rYEsg00I/NA
lSjEOnTpozqQwLPAUGBwGQRCP8tbPxRW9k+dmWUKY7/Ox5QtoNRzMt1n2hcG2uXk
VkefzABd0HWvUOYy8LL1o32rxdv78iqmfVxKyORHZKBS59OM+CAahCvgV9bVgE+g
Rbp1RzsPZJOEBkDY6bncyUN0Qm7bQqoCXt4ZEaX/Cj7lPRrCjzj+NXzuYKUReib1
z79Xnrps4r5QlY1PVICF4lKAWnUsLRSr/A+vPYgTJRQKTo0DwDaPTcEs4DrcAZ1j
0uTlk2B/mcThEDEd+kB7vBZcc7QnNsh8Qp5objXNPsq/7+8cJdHwXY1/qxsXgLFf
aQMhKJY/kfoE24yyb18SosXqMoxc8rDZkMKv91z597PlqIWbPMhNeg/dGlx8ptcl
S4DIaIZgJ5E=
=9ORp
-----END PGP SIGNATURE-----
Merge tag 'locking-urgent-2026-08-22' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull futex fixes from Ingo Molnar:
- Enforce that the private futex owner shares the mm when attaching
(Kyle Zeng, Thomas Gleixner)
- Fix race on the initial mm->futex.phash.ref allocation (Hyunwoo Kim)
- Fix might_sleep() warning in futex_pivot_pending() (Peter Zijlstra)
* tag 'locking-urgent-2026-08-22' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
futex: Fix might_sleep() warning in futex_pivot_pending()
futex: Fix race on the initial mm->futex.phash.ref allocation
futex: Clean up the redundant exit/exec functions
futex/pi: Plug private futex exec() race
futex: Sanitize and document task_struct::futex::state transitions
futex/pi: Reject cross-mm private futex owners
|
||
|
Linus Torvalds
|
55ab7e1422 |
for-7.3/block-20260819
-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAmqGD2AQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpomLD/9rr3SIQvdQRTnSCJlw1cFypSrnRhGAx1g8
v1F7UXuf4Abt4GIerbdXmBTx9zZRctpcOC41ReQk3YKkYuukuvna+Z6tBGf5I5+e
3SR3Zkrj2k4zyNC49g0/QkUJkTTS4tlsBFhNJtY3Q5BBIRMRd/sPvZ+cLnD7g/wH
3zM9IaEMk9xu9mF1nGI7LRl1qY0IOr4ynQpqohl5sEL3TQWgIa0r2B0jhGacBAvC
GGDwIwGDrM6oGwCn6mLN78O5odhLyTufIMNy3EnVw7BcDpLApcnJzYYjGzv+ElCM
WLBJwWYj+PYeoenuJuFhbRihbOPg5gZT3Mn8CUhLxUYzwZvIdFGWOjcGffD9sVu/
JO4nB6A+2AG/wgsb2YVKefhN86FoD0F/2hhdEdgZPJE6a+7+xeyyp9IzyQEWd0cg
Seo06fxE4pwF+dczXT3jzU3LXSZb/Jbh9hzu163VMSWbGYDCpDzfipqZGeN6DWE7
nG2NssAQtjtZrgdggYbZXtwecLZhu3cMj3jbhS4wSUzhQFW42yceJstCx4wpRhW9
m9EXjHQnRFMAb2TjUC3FzEncjbxL+uACjcKhTetd1hLbTI5I8KArZXjam1OdJOHV
urdtsru79LanQLAgRwWYPeKCZGNhX7I2o3OsmVDLQgHIDaZguTcPUVQs3wVui7Dk
IDmmrtwzpg==
=G1l4
-----END PGP SIGNATURE-----
Merge tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Enable Clang context analysis for the nvme host driver, adding
context annotations across core, fabrics, rdma, tcp and pci
- nvmet reservation state exposed through a new namespace-level
debugfs directory, plus ABI documentation for the host sysfs and
target configfs interfaces
- nvme-tcp host memory disclosure fixes on the read path: reject a
read that transferred too few bytes, don't accept C2HData based
on blk_rq_payload_bytes() alone, and fix the R2T case for a read
command
- Parallelize nvme-rdma I/O queue allocation and startup (Surabhi)
- Apple nvme fixes and quirks: page aligned admin queue buffers,
destroy the admin queue on removal, and various DMA/NVMMU
correctness fixes
- A large pile of nvmet and host fixes for out-of-bounds reads,
refcount/resource leaks, and NULL derefs across auth, zns,
passthru, pci-epf, rdma and configfs
- Various other fixes and cleanups
- MD updates via Yu Kuai:
- llbitmap reshape support, the large series wiring exact bitmap
mapping and reshape lifecycle through raid5 and raid10, growing
the page cache in place, and remapping checkpointed bits as
reshape progresses
- raid5 fixes for lockless max_nr_stripes and recovery_offset
accesses, a reshape deadlock with more failed devices than max
degraded, and bitmap batch counter consistency
- Atomic write handling for raid1/raid10, and removal of the
REQ_NOWAIT support from raid1/10/456
- raid5-ppl use-after-free fix in ppl_do_flush()
- A batch of smaller fixes across md core and the bitmap code
- s390/dasd ESE full-track write support and the surrounding
infrastructure, plus enabling CONTEXT_ANALYSIS for s390/block
- RWF_DONTCACHE support for block devices, built on new task-context
bio completion infrastructure, and wiring it up for the iomap and
buffer dropbehind writeback paths
- Async io_uring zone reset all, plus zone management command cleanups
allowing REQ_NOWAIT and tightening conventional zone rejection
- Block integrity refactoring: lift BIP_CHECK_FLAGS to the shared
header, handle nogenerate/noverify properly in fs-integrity, and drop
the blk-integrity.h include from bdev.c
- Split out a new blk_plug.h header
- ublk improvements: add UBLK_F_IO_DESC_SIZE, split request validation
from io_desc init, reject non-power-of-2 zone sizes in SET_PARAMS,
and a series of hardening fixes around map/unmap and auto buf reg
- null_blk cleanups and configfs serialization fixes
- nbd queue freeze removal on the setup paths, and a new
pre_defined_connections module parameter for pre-created devices
- blk-cgroup fixes for the race between policy activation and blkg
destruction, and accounting per-cpu stats over possible CPUs across
blk-stat, iolatency, iocost and kyber
- Various dio fixes: leak on metadata mapping error, validate user
space vectors during extraction, and set dma_alignment from the
backing file for loop and zloop direct I/O
- bio cleanups
- Various other fixes and cleanups all over
* tag 'for-7.3/block-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (241 commits)
nbd: add pre_defined_connections module parameter for pre-created devices
nbd: remove queue freeze for newly created nbd from netlink path
nbd: factor out a nbd_genl_foreach_sock
nbd: skip queue freeze when setting size at device startup
nbd: remove queue freeze in nbd_add_socket
nbd: clear queue limits on disconnect
nbd: disallow NBD_SET_SOCK on an active device
nbd: simplify find_fallback() by removing redundant logic
blk-mq: add missing call to srcu_barrier() in blk_mq_free_tag_set()
block: mtip32xx: synchronize ioctls with device removal
ublk: avoid teardown retry loop on xarray allocation failure
null_blk: fix UBSAN shift-out-of-bounds when zone_size is 0 or overflows
block: don't include blk-integrity.h in bdev.c
xfs: avoid double deferrals for RWF_DONTCACHE writes
loop: Fix recently introduced lock inversion
block: set QUEUE_FLAG_DYING unconditionally in blk_mark_disk_dead()
swim3: Add missing MODULE_DESCRIPTION
selftests: ublk: add SET_PARAMS validation test
selftests: ublk: add helper for SET_PARAMS
ublk: reject non-power-of-2 zone sizes in SET_PARAMS
...
|
||
|
Linus Torvalds
|
11260c335e |
sched_ext: Changes for v7.3
This depends on the arena argument support in the BPF tree and should be
pulled after the scheduler core and BPF pulls. The patches based on bpf-next
were kept on a separate branch which was merged into for-7.3 just now. The
same merged result was in linux-next for several days.
Most of this cycle completes the enqueue-path support for hierarchical
sub-scheduling, which makes sub-scheduler support feature complete: a root
BPF scheduler can now hand a cgroup subtree over to a nested sub-scheduler
together with revocable CPU grants, and the sub-scheduler owns all
scheduling decisions for its tasks on those CPUs.
Development volume was high and a number of changes plugging holes in the
new support landed late in the cycle. Also included are core scheduling
fixes that were completed too late for the v7.2 release and are routed
through this pull request.
- Sub-scheduler CPU delegation:
- Parent schedulers now grant and revoke per-CPU capabilities (enqueueing,
preemption, CPU frequency control) on their children, enforced on every
path a scheduler can reach a CPU through. Previously only dispatching
could be delegated; this lets sub-schedulers fully schedule their CPUs.
- Rescue execution: a task whose scheduler doesn't have access to the CPUs
the task needs to run on starved until the watchdog ejected the whole
scheduler. The kernel now runs such tasks directly on a small bandwidth
budget, turning a scheduler-killing failure into bounded degradation.
- Cgroup integration: tasks migrating across a sub-scheduler boundary
weren't re-homed to the new owner, causing wrong-scheduler scheduling
and a use-after-free. Sub-schedulers now take over their cgroup subtree
and receive its cgroup callbacks.
- Arena objects now cross the kernel/BPF boundary as typed pointer
arguments, translated transparently by the BPF tree's new arena argument
support, replacing untyped arguments with manual translation.
- scx_qmap now demonstrates full hierarchical sub-scheduling.
- Robustness improvements: the abort path is now NMI-safe, fixing deadlocks
when errors are raised from NMI context and making hardlockup recovery
direct. Reenqueue loops that could monopolize a CPU ahead of the watchdog
now eject the offending scheduler, and stalls are blamed on the scheduler
actually responsible.
- Hardening: BPF-writable arena memory is validated before kernel use, and
task slice and vtime writes got explicit synchronization rules, closing
corruption vectors open to buggy or malicious schedulers.
- Core scheduling: sched_ext dispatching can drop the rq lock inside the
core-wide pick, which let interleaving selections corrupt each other's
state and hard-hang the machine. The selection now restarts when the lock
was released. The task ordering callback was also invoked with its
arguments swapped, and the default ordering is updated to work across
sub-scheduler boundaries. The fixes are marked for stable.
- Other fixes headed for stable: a task init leak on fork failure during
enable, tooling compat macros that silently failed to detect newer
kernels, and a crash on reenqueueing against a destroyed dispatch queue.
- Tooling: scx_pair moves off deprecated callbacks, and the deprecated
scx_bpf_cpu_rq() kfunc is removed.
-----BEGIN PGP SIGNATURE-----
iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCaoOA7w4cdGpAa2VybmVs
Lm9yZwAKCRCxYfJx3gVYGQWqAP9Sy8GwS7dRdGze/eHwYlDBt5U9ayd2ntR0Z+H1
1Hd23AEA5kYPaEN68OgCXh/XqmFljkvEgEisXtMtw8XsZA+pHQA=
=OwIm
-----END PGP SIGNATURE-----
Merge tag 'sched_ext-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext updates from Tejun Heo:
"Most of this cycle completes the enqueue-path support for hierarchical
sub-scheduling, which makes sub-scheduler support feature complete: a
root BPF scheduler can now hand a cgroup subtree over to a nested
sub-scheduler together with revocable CPU grants, and the
sub-scheduler owns all scheduling decisions for its tasks on those
CPUs.
Development volume was high and a number of changes plugging holes in
the new support landed late in the cycle. Also included are core
scheduling fixes that were completed too late for the v7.2 release and
are routed through this pull request.
Sub-scheduler CPU delegation:
- Parent schedulers now grant and revoke per-CPU capabilities
(enqueueing, preemption, CPU frequency control) on their children,
enforced on every path a scheduler can reach a CPU through.
Previously only dispatching could be delegated; this lets
sub-schedulers fully schedule their CPUs.
- Rescue execution: a task whose scheduler doesn't have access to the
CPUs the task needs to run on starved until the watchdog ejected
the whole scheduler. The kernel now runs such tasks directly on a
small bandwidth budget, turning a scheduler-killing failure into
bounded degradation.
- Cgroup integration: tasks migrating across a sub-scheduler boundary
weren't re-homed to the new owner, causing wrong-scheduler
scheduling and a use-after-free. Sub-schedulers now take over their
cgroup subtree and receive its cgroup callbacks.
- Arena objects now cross the kernel/BPF boundary as typed pointer
arguments, translated transparently by the BPF tree's new arena
argument support, replacing untyped arguments with manual
translation.
- scx_qmap now demonstrates full hierarchical sub-scheduling.
Other fixes and updates:
- Robustness improvements: the abort path is now NMI-safe, fixing
deadlocks when errors are raised from NMI context and making
hardlockup recovery direct. Reenqueue loops that could monopolize a
CPU ahead of the watchdog now eject the offending scheduler, and
stalls are blamed on the scheduler actually responsible.
- Hardening: BPF-writable arena memory is validated before kernel
use, and task slice and vtime writes got explicit synchronization
rules, closing corruption vectors open to buggy or malicious
schedulers.
- Core scheduling: sched_ext dispatching can drop the rq lock inside
the core-wide pick, which let interleaving selections corrupt each
other's state and hard-hang the machine. The selection now restarts
when the lock was released. The task ordering callback was also
invoked with its arguments swapped, and the default ordering is
updated to work across sub-scheduler boundaries. The fixes are
marked for stable.
- Other fixes headed for stable: a task init leak on fork failure
during enable, tooling compat macros that silently failed to detect
newer kernels, and a crash on reenqueueing against a destroyed
dispatch queue.
- Tooling: scx_pair moves off deprecated callbacks, and the
deprecated scx_bpf_cpu_rq() kfunc is removed"
* tag 'sched_ext-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: (144 commits)
sched_ext: Drop the dead SCX_DEQ_CORE_SCHED_EXEC test in dequeue_task_scx()
sched_ext: Make core-sched task ordering hierarchy-aware
sched_ext: Use runnable_at for the default core-sched task ordering
sched_ext: Fix inverted ops.core_sched_before() invocation
sched_ext: Move the config-off sub-cap kfunc stubs into sub.c
sched_ext: Rename balance-era identifiers to dispatch terms
sched_ext: Drop the stale keep_prev fixup in dispatch_pick()
sched_ext: Keep kick_sync waiting on the rq's own CPU
sched_ext: Make SCHED_CLASS_EXT select GENERIC_ALLOCATOR
sched_ext/scx_flatcg: Fix cvtime true-up on slice expiry
sched_ext: Don't BUG_ON a destroyed DSQ in process_deferred_reenq_users
sched_ext: Fix scx_bpf_dsq_move_to_local___v2 compat detection
sched_ext: Make scx_bpf_events() read the calling scheduler's counters
sched_ext: Drop unlocked scx_rq_clock_invalidate() from scx_root_disable()
selftests/sched_ext: Fix flaky ddsp failure tests on busy systems
selftests/sched_ext: Make numa idle validation race-free
sched_ext: Fix scx_bpf_dsq_reenq___compat kfunc extern prototype
sched_ext/scx_flatcg: expire cached hweights on weight changes
sched_ext: Fix exit_task leak on fork failure during enable
sched_ext: fix stale references in doc comments
...
|
||
|
Peter Zijlstra
|
d8aa5dd979 |
futex: Fix might_sleep() warning in futex_pivot_pending()
A younger me put a WARN in might_sleep() to warn about nested sleep loops. This
younger me also build a wait-loop variant that can deal with it. This wait-loop
variant doesn't have all the fancy wrappers, since it isn't used much. It also
lacks wait-bit support.
Add the wait-bit support and use it to fix the nested wait issue.
Fixes:
|
||
|
Linus Torvalds
|
9d2ed026f0 |
Summary
* Documentation
Fix kernel-doc warnings by adjusting in file documentation.
* Consolidate do_proc_* function into do_proc_vec.
Consolidate three slightly different implementations of applying a converter
on all elements of a vector. Fixes to this function now propagate to the three
types.
* Miscellaneous
Replaced CONFIG_PROC_SYSCTL with CONFIG_SYSCTL (they were the same) and
restrict cad_pid modifications to global root (GLOBAL_ROOT_UID)
* Testing
This series was run through sysctl selftests/kunit test suite in x86_64. And
went into linux-next around rc2, giving it a bit more than a month of testing.
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEErkcJVyXmMSXOyyeQupfNUreWQU8FAmp0ga8ACgkQupfNUreW
QU9RZQv/UFBopK15/HcUPWAp8h0Yq68Reij0ryPhuOuoqRaWjLrz5iT9SvNW4IZH
WfAHal9+gVyAov2TvZUDHE0iojYa1S7l1MSBSvzlEijZnbglnt+63Ijfria3FTr2
FIqk7gOxCpDg7kRh1GkIm8yc6Ke7T2IcbMT8XW5PAs43QqMPgGfeUry14sr6H89u
pc1vXS/PcRDdsKkYKADCuljGG3ywL/7frDhSRcc+5WkCACOujzosmXTL10TVVs7H
6Lz3vujtQyCnv3UxlHicsJQXD60m2gqbCLcqa5gndfE5jO2nS3ZTqRBALT9Cimuk
43k+cKgJwOa5ry9OFXmp+in57hgOg8AZqWWENY2W0RaU6Tw7Ui5x6z36ZtE5PR9u
NlNAOprv6UnKOQ6Bc4GGtfSMh+EviqQBSNQroEz4UxLdrFx2X7SxkSgz5I6gLbzf
w+KZpQ3Y3Kjvr9ywkQUftekWqaeapv/OfrjGXRwiIBYhlurmKqvFpjjSQXmjow9u
RD1bfR9X
=aYrQ
-----END PGP SIGNATURE-----
Merge tag 'sysctl-7.03-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl
Pull sysctl updates from Joel Granados:
- Fix kernel-doc warnings by adjusting in file documentation
- Consolidate do_proc_* function into do_proc_vec
Consolidate three slightly different implementations of applying a
converter on all elements of a vector. Fixes to this function now
propagate to the three types.
- Replace CONFIG_PROC_SYSCTL with CONFIG_SYSCTL (they were the same)
and restrict cad_pid modifications to global root (GLOBAL_ROOT_UID)
* tag 'sysctl-7.03-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl:
sysctl: remove CONFIG_PROC_SYSCTL, it just mirrors CONFIG_SYSCTL
sysctl: move the "cad_pid" entry from pid_table[] to kern_reboot_table[]
sysctl: repair some kernel-doc comments
sysctl: add Returns: kernel-doc for all functions
sysctl: Update API function documentation
sysctl: Rename proc_doulongvec_minmax_conv to proc_doulongvec_conv
sysctl: Group proc_handler declarations and document
sysctl: Replace do_proc_do{int,ulong,uint}vec with do_proc_vec
sysctl: Add negp parameter to douintvec converter functions
sysctl: Move default converter assignment out of do_proc_dointvec
|
||
|
Jake Steinman
|
23906f3a16 |
sched/fair: Floor tg_cpus() at 1
tg_cpus() returns cpuset_num_cpus() unfloored, while its sibling
tg_tasks() already floors its result at 1. calc_concur_shares() feeds
nr = min(tg_tasks(tg), tg_cpus(tg))
into __calc_smp_shares() as shares_max, so an nr of 0 makes shares_max 0.
__calc_smp_shares() ends with
return clamp_t(long, shares, MIN_SHARES, shares_max);
and clamp() yields hi when hi < lo, so a zero shares_max silently defeats
the MIN_SHARES floor and returns 0 -- the exact case the comment above
that line says must return MIN_SHARES instead of 0.
That leaves a group sched_entity with load.weight == 0, and
__calc_prop_weight() then divides by cfs_rq->load.weight:
weight *= se->load.weight;
if (parent_entity(se))
weight /= cfs_rq->load.weight;
which takes a #DE inside enqueue_task_fair():
Oops: divide error: 0000 [#1] SMP NOPTI
RIP: 0010:enqueue_task_fair+0x422/0x950
Call Trace:
<TASK>
enqueue_task+0x8e/0x250
wake_up_new_task+0x148/0x2e0
kernel_clone+0x1c6/0x390
__x64_sys_clone+0xcc/0x100
do_syscall_64+0x147/0x3c0
</TASK>
This is not survivable in practice: with panic_on_oops=0 the kernel took
the first #DE and continued for 476 ms, then faulted at the same RIP with
identical register state and an identical RSP, because the oops recovery
path (kill task -> schedule()) re-enters the same enqueue while the rq
lock is held mid-enqueue. The second fault escalates to a panic.
Flooring tg_cpus() at 1 makes it symmetric with tg_tasks() and keeps
shares_max >= tg_shares, so the MIN_SHARES floor in __calc_smp_shares()
can no longer be bypassed.
Note this only removes the division hazard. Whether cpuset_num_cpus() can
legitimately return 0 -- via the cpu hotplug/suspend path where a v2
cpuset may transiently become empty, or via an RCU race -- is a separate
question still open on the report thread.
Fixes:
|
||
|
Linus Torvalds
|
3b4128b9f3 |
Timers and timekeeping core updates:
- Fix a subtly inconsistency in the timekeeping code, which fails to
account for the monotonicity adjustment in ntp_error. For small changes
of the clocksource multiplicator (+/-1) which are typically used by the
NTP PLL this is hardly to observe. But for larger adjustments,
e.g. caused by a direct frequency setting through adjtimex() the
one-time uncompensated offset is significant.
Cure this by adjusting ntp_error with the resulting offset so that the
discrepancy is smoothed away over time
- Make tick length calculations correct in NTP. The timekeeping core
takes the quantisation of the clocksource into account when calculating
the tick length to compensate for the deviation of the nominal
NTP_INTERVAL_LENGTH. While timekeeping gets this right, NTP is not
aware of that, which means it operates on the nominal value and not on
the actual value which is determined by the clock source frequency. The
rounding of a coarse clocksource like the ACPI PM timer results in a
+127 PPM deviation.
Cure this by exposing the deviation to the NTP code so that it can
operate on the same data as the timekeeping core. This is purely kernel
internal. User space still sees the nominal tick lenght via adjtimex().
- The accuracy of the NTP adjustments is fairly approximate as the code
assumes that the invocations are precisely in NTP interval frequency
ticks and the final adjustment can over and under-run.
Cure this by adjusting ntp_error by the intended skew on each tick to
achieve the desired rate.
- Handle the two competing skews of time offset and time adjustment
correctly by calculating the conflict portion between the skews and
adjusting both accordingly.
- A set of updates and improvements for the selftests
- The usual small fixes and improvements all over the place
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmqCwxwQHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoaOwD/45XBW2s5/L+GSy52r/APrzYOqgjYQdn2UQ
XI85iTjDqg9sfZQNFbr7toJ84gY/wQ0ZxykLMi+LlOSZVe5azSepoUHMg9XGk8S0
4kI+GLznmnDLFypTezpzk3DUgNgu8rhOYvPsgwQEgx0DuzWHRZgNoNDk4AfJ8HP6
8cRgPIJ7CpIbP7/p0Kc5IjW3ybo849wBirl7ZDxNRRao+Dkc56ArZvSfQgnQu5Kw
D3Ei+AFcxpBQ2GEJT4Z/92yNf1x2AXjkhfijgO6hIFzPAR+c049fG5sgneo3GoLQ
riduEPQHvoiJO3gVGzoICF2DAuVOwMHwIBlGtNEDd+FcMAvMU4V3lQi8lJV+9CpH
SJwa5R6KJHg2aTuoPup8lmfJN7BqmthrkZwcYVrv2Ikt5yWGYuoW0/e00So9IfUO
R7UG/VlruckK+T0GCuKBAAfm/+zm6Sbir+4liPDZwDz4dW2lkvkZnAb6xHPCev8S
ioYL/Jqr3GmIMgK7UelUkQ/gy5YCek89JAszdlD+blLAjUMcDeYsZaAN9rSdjzSu
dbEA4Hze6Q8HpIW35bHkytyyjNNhvL6xobjKBNRuOPUHTFMe+s5itMApRlhsu32J
6EL/D1TuwkiHlqaSek7ZqUAiitZoNWxa1NyreM9EuZ7qWPjUWOzlsDXkP2v298+t
p27xt+/8sA==
=prJt
-----END PGP SIGNATURE-----
Merge tag 'timers-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer and timekeeping core updates from Thomas Gleixner:
- Fix a subtly inconsistency in the timekeeping code, which fails to
account for the monotonicity adjustment in ntp_error.
For small changes of the clocksource multiplicator (+/-1) which are
typically used by the NTP PLL this is hard to observe. But for larger
adjustments, e.g. caused by a direct frequency setting through
adjtimex() the one-time uncompensated offset is significant.
Cure this by adjusting ntp_error with the resulting offset so that
the discrepancy is smoothed away over time
- Make tick length calculations correct in NTP.
The timekeeping core takes the quantisation of the clocksource into
account when calculating the tick length to compensate for the
deviation of the nominal NTP_INTERVAL_LENGTH.
While timekeeping gets this right, NTP is not aware of that, which
means it operates on the nominal value and not on the actual value
which is determined by the clock source frequency. The rounding of a
coarse clocksource like the ACPI PM timer results in a +127 PPM
deviation.
Cure this by exposing the deviation to the NTP code so that it can
operate on the same data as the timekeeping core. This is purely
kernel internal. User space still sees the nominal tick lenght via
adjtimex().
- The accuracy of the NTP adjustments is fairly approximate as the code
assumes that the invocations are precisely in NTP interval frequency
ticks and the final adjustment can over and under-run.
Cure this by adjusting ntp_error by the intended skew on each tick to
achieve the desired rate.
- Handle the two competing skews of time offset and time adjustment
correctly by calculating the conflict portion between the skews and
adjusting both accordingly.
- A set of updates and improvements for the selftests
- The usual small fixes and improvements all over the place
* tag 'timers-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (58 commits)
selftests: timers: nsleep-lat: Check all calls to clock_nanosleep() and clock_gettime()
selftests: timers: nsleep-lat: Reuse kselftest error numbers
selftests: timers: nsleep-lat: Explicitly list the tested clocks
selftests: timers: nsleep-lat: Use NSEC_PER_MSEC define for unreasonable latency
selftests: timers: nanosleep: Report each test separately
selftests: timers: nanosleep: Explicitly handle timer_delete() failure
selftests: timers: nanosleep: Move all single clock tests out of the loop in main()
selftests: timers: nanosleep: Reuse kselftest error numbers
selftests: timers: nanosleep: Explicitly list the tested clocks
selftests: timers: nanosleep: Drop output alignment
selftests: timers: Use clock_name() and constants from clock-helpers.h
selftests: Add clock-helpers.h
timer_list: Use ktime_t over nanoseconds
timer_list: Use standard 'long long' format placeholders
hrtimer: Add a lockdep assertion to hrtimer_update_base()
timekeeping: Use u32 for clock_was_set_seq
timekeeping: Rename clockid_aux_valid() to clockid_is_aux_clock()
hrtimer: Account nr_retries on recovered interrupt retries
timers/itimer: Zero-init old itimerval before copy to userspace
nohz: Replace dead select with choice default
...
|
||
|
Linus Torvalds
|
e2457a664e |
Scheduler updates for v7.3:
Load-balancing updates:
- Series to 'flatten the pick': improve cgroup scheduling, which has
always been problematic and painful, which has caused various
scheduling misbehavior such as the mishandling of reniced tasks et al.
Add various cgroup weight distribution methods via cgroup_mode:
'up', 'max', 'concur' and 'tasks' - with the default being 'concur'
which is the most precise yet also most expensive version.
Finally, change cgroup scheduling to a single runqueue.
(Peter Zijlstra)
- Series to improve the scheduling latency of short slice tasks
(Vincent Guittot)
- Series to fix cluster scheduling in the presence of asymmetric capacity
(Ricardo Neri)
- Prefer fully idle cores for NOHZ balancing (Andrea Righi)
- Don't trigger active load-balancing if src_rq->curr is not on_rq
(Xin Zhao)
PSI updates:
- Skip irqtime accounting when no new irq time has elapsed
(Usama Arif)
Scheduler debugging updates:
- Remove unused schedstats (Shrikanth Hegde)
- Defer freeing of cpumask memblock memory to initcall
(Waiman Long)
Misc fixes and updates by Yu C Chen, K Prateek Nayak, Peter Zijlstra,
Vincent Guittot, Xin Zhao, Yury Norov, Zhan Xusheng.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqC3WARHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hawRAAuVAreXtsp7Ok6aRHmiOMR+Va2EsHIL4B
3MY5do2oUfiJSCDtIn3BXqjM3djOXdbQLQwm6r9/NkyJ6hsOsRgZaP1L/w07FiY0
33ilIhZIH9QZ2RzvEjZcFXVwnWH6zP8Ag/gTsch3g8TIX2JJcZBzcJjQM8ZIdcBL
oxwWNe9z3Mh0/x2N+N/s1xstaZf95bWQKeBVBGjiOAakGRWMEaQmMtOUujJNmx/p
YFtNcJL6pLGfD+ASOtLxYWts4z1JyDmwvje8SZmRsgjZdhoxRlnEIgU6jrNjmPqu
NfR5w6R193Pa+FapawYi0ttuawapHvvSDDUxjl+eIccPMw0mBdjzn1XuNIt6pclC
Q4VDUQEDBYvbeznNfLuvvsLVBq/QIjim391lg9DuAGreuRx00Ij0hXOCPk6TM4H8
tfUZ8N25vIAbYrFajSDyOYdgAa2Z2VnM1Q1ko29LPb3WFfZsEqgIg7+ukoXCZ5n8
67pUrOYLuSlLXJGlJY9WDz1nbeUh3e0f2390fif3VBcQ7roPEmoL8CYZhh8Vd2rq
ug8kOpL/91/PDgSPbxRRIEYhe9X1kzWZxqpjvO93tZ+GhXUxtLFG4PjnbQhINy1A
BMNlxcq4ktVHk9gwLaG9wcySjINnN/wK91o3dQ9YjwTuCAxFi/AqUVjY+OOmguAh
TO0mkEq99M0=
=HTPk
-----END PGP SIGNATURE-----
Merge tag 'sched-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler updates from Ingo Molnar:
"Load-balancing updates:
- 'flatten the pick': improve cgroup scheduling, which has always
been problematic and painful, which has caused various scheduling
misbehavior such as the mishandling of reniced tasks et al.
Add various cgroup weight distribution methods via cgroup_mode:
'up', 'max', 'concur' and 'tasks' - with the default being 'concur'
which is the most precise yet also most expensive version.
Finally, change cgroup scheduling to a single runqueue (Peter
Zijlstra)
- Series to improve the scheduling latency of short slice tasks
(Vincent Guittot)
- Series to fix cluster scheduling in the presence of asymmetric
capacity (Ricardo Neri)
- Prefer fully idle cores for NOHZ balancing (Andrea Righi)
- Don't trigger active load-balancing if src_rq->curr is not on_rq
(Xin Zhao)
PSI updates:
- Skip irqtime accounting when no new irq time has elapsed (Usama
Arif)
Scheduler debugging updates:
- Remove unused schedstats (Shrikanth Hegde)
- Defer freeing of cpumask memblock memory to initcall (Waiman Long)
Misc fixes and updates by Yu C Chen, K Prateek Nayak, Peter Zijlstra,
Vincent Guittot, Xin Zhao, Yury Norov, Zhan Xusheng"
* tag 'sched-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (33 commits)
sched/fair: Fix flat hierarchy
sched/isolation: Defer freeing of cpumask memblock memory to initcall
sched/topology: Restore SD_PREFER_SIBLING in domains with asymmetric capacity
sched/fair: Allow load balancing between CPUs of identical capacity
sched/fair: Skip misfit load accounting when the destination CPU cannot help
sched/fair: Check CPU capacity before comparing group types during load balance
sched/fair: Also gate overloaded status update for SD_ASYM_CPUCAPACITY
sched/fair: Do not skip CPUs of similar capacity with busy SMT siblings
sched/fair: Prefer fully idle cores for NOHZ balancing
stop_machine: Make stop_one_cpu_nowait() return void
sched/eevdf: Delayed dequeue task can't preempt
sched/fair: Fix stale comments referring to removed CFS concepts
sched/debug: Remove unused schedstats
sched/psi: skip irqtime accounting when no new irq time has elapsed
sched/fair: Reflow sched_balance_rq()
sched/fair: Simplify balance_interval reset logic in sched_balance_rq()
sched/fair: Don't trigger active lb if src_rq->curr is not on_rq
sched/eevdf: Speedup short slice task scheduling
sched/eevdf: Always update slice protection
sched/eevdf: Cancel slice protection if short slice task is eligible
...
|
||
|
Linus Torvalds
|
dfa35434d7 |
Locking updates for v7.3:
Futexes:
- Use runtime constants for futex_hash computation
(K Prateek Nayak, Peter Zijlstra)
- Optimise the size check get_futex_key() (Sebastian Andrzej Siewior)
- Avoid private hash use-after-free on final put (Felix Hoffmann)
- Tell kmemleak we're not leaking __futex_queues (Peter Zijlstra)
Rust integration updates:
- Implement refcounted interrupt disable and SpinLockIrq for Rust
(Boqun Feng, Heiko Carstens, Joel Fernandes, Lyude Paul)
- Rust sync: add helpers for mb, dma_mb and friends;
add generic memory barriers and use LKMM atomics
instead of Rust atomics in the revocable code (Gary Guo)
- Add abstraction and integrate synchronize_rcu() (Philipp Stanner)
Lock debugging:
- Add qspinlock contended_release tracepoint
(Dmitry Ilvokhin, Peter Zijlstra)
- Enable the printing of held locks of remote running tasks and print
task CPU (Ingo Molnar)
- percpu-rwsem: Annotate intentional data race in readers_active_check()
(Sun Shaojie)
Misc fixes and updates by Boqun Feng, Peter Zijlstra, Fangrui Song,
Naveen Kumar Chaudhary and Thomas Huth.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmqC2KMRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1gNwg//awvTQONfhPanAyTgl7CLDSlMSHdqmlyh
Ue0/Q8Ef1Cy4jwXY2FE2A0b1VcM6cGpDPoryVdg/wMdUXRNwinzAEXmxIkRy9kve
4LybrZwDShgLxJ7pJ6KKhgjgDiat8EdYmOwCBEE3LnP7AYhkAb8BFetA3YZJvzPa
KfA2BRYCgvBTid6yOAuXWm55Ev92AczOBamBzTxCadcaDGtNGXtQO6LfnqiQDOav
X5tVoANBeaQtSs1+LxE41WdNOiRoBuy0IFFvXtZRal6PZYuGGmZ5tbQvscD099em
haVwQyzDHQrqzglv71M0KRTXvYzdGveMRg/Au1SQnuLO3V6Vd5rMQ1g7I2M9Ln0f
Pg+tlRvQ77mLoqcgrtl0W/u0fRR4eDkiJ1pmG+98oniPwau23RdbFhC0vKFz3ikF
WHMgk3/9TcULylgF1Tj6QLmNrBY3Vx8LBdsFjhflEw7bG4cW42D91npmXIiEDE6K
tJc9CcaVdyE75o59z2Dtjj+qQVBlNPlfKQFXFL7p3jU/gFw2SzYuqon66X3kGmr0
mKJ9UNJdkLdiCjxS/QiMcDeYhwJksJqxFBkH50z3Kzmo84JsSpUFkoa6GM4aSiGn
HEwgC0Q7oOXVNIKUBYk5QaRW0HSk55hbsX2TWkvpeBYkE1zXshVZCCmgpaTSJgb5
oFmiwrGfUjo=
=slqC
-----END PGP SIGNATURE-----
Merge tag 'locking-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking updates from Ingo Molnar:
"Futexes:
- Use runtime constants for futex_hash computation (K Prateek Nayak,
Peter Zijlstra)
- Optimise the size check get_futex_key() (Sebastian Andrzej Siewior)
- Avoid private hash use-after-free on final put (Felix Hoffmann)
- Tell kmemleak we're not leaking __futex_queues (Peter Zijlstra)
Rust integration updates:
- Implement refcounted interrupt disable and SpinLockIrq for Rust
(Boqun Feng, Heiko Carstens, Joel Fernandes, Lyude Paul)
- Rust sync: add helpers for mb, dma_mb and friends; add generic
memory barriers and use LKMM atomics instead of Rust atomics in the
revocable code (Gary Guo)
- Add abstraction and integrate synchronize_rcu() (Philipp Stanner)
Lock debugging:
- Add qspinlock contended_release tracepoint (Dmitry Ilvokhin, Peter
Zijlstra)
- Enable the printing of held locks of remote running tasks and print
task CPU (Ingo Molnar)
- percpu-rwsem: Annotate intentional data race in readers_active_check()
(Sun Shaojie)
Misc fixes and updates by Boqun Feng, Peter Zijlstra, Fangrui Song,
Naveen Kumar Chaudhary and Thomas Huth"
* tag 'locking-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (44 commits)
rust: sync: Introduce SpinLockIrq::lock_with() and friends
rust: sync: Add SpinLockIrq
rust: sync: Use super::* in spinlock.rs
rust: helper: Add spin_{un,}lock_irq_{enable,disable}() helpers
rust: Introduce interrupt module
s390/preempt: Enable HAS_SEPARATE_PREEMPT_RESCHED_BITS
arm64: sched/preempt: Enable HAS_SEPARATE_PREEMPT_RESCHED_BITS
preempt: Introduce HAS_SEPARATE_PREEMPT_RESCHED_BITS
sched: Avoid signed comparison of preempt_count() in __cant_migrate()
sched: Remove the unused preempt_offset parameter of __cant_sleep()
locking: Switch to _irq_{disable,enable}() variants in cleanup guards
irq: Add KUnit test for refcounted interrupt enable/disable
irq,spin_lock: Add counted interrupt disabling/enabling
openrisc: Include <linux/cpumask.h> in smp.h
preempt: Introduce __preempt_count_{sub,add}_return()
preempt: Introduce HARDIRQ_DISABLE_BITS
preempt: Track NMI nesting to separate per-CPU counter
futex: Tell kmemleak we're not leaking __futex_queues
x86/paravirt: Trace contended_release on unlock
tracing/lock: Use TRACE_EVENT_FN() for contended_release
...
|
||
|
Linus Torvalds
|
a5778046a0 |
Power management updates for 7.3-rc1
- Minor fixes and cleanups in assorted cpufreq drivers (Dan Carpenter,
Guru Das Srinagesh, Haoxiang Li, Karl Mehltretter, Sasha Finkelstein,
and Pan Chuang)
- Fix cpufreq table creation and bios_limits() callback in the Rust
bindings (Priya Bala Govindasamy)
- Add IPQ5210 support to qcom-nvmem driver (Varadarajan Narayanan)
- Adjust the .adjust_perf() cpufreq driver callback to allow the
maximum performance value to be passed to drivers and update the
intel_pstate driver to use it (Rafael Wysocki)
- Set policy->cur to the actual requested frequency in the intel_pstate
driver when the performance policy is used (Rafael Wysocki)
- Simplify HWP handling on Broadwell processors in intel_pstate (Rafael
Wysocki)
- Fix setting minimum P-state at init time in intel_pstate (Rafael
Wysocki)
- Consolidate frequency values computation in intel_pstate and clean up
code in that driver (Rafael Wysocki)
- Add missing kernel-doc descriptions for structure and union members
in the amd-pstate driver (David Vernet)
- Handle missing policy in dynamic EPP callbacks in the amd-pstate
driver (EDAMAMEX)
- Introduce EXPORT_SYMBOL_FOR_PSTATE_UT() to export amd-pstate driver
symbols to the amd-pstate-ut subdriver (K Prateek Nayak)
- Add dynamic EPP as an "energy_performance_preference" mode in
amd-pstate, remove the "amd_dynamic_epp" kernel command line option
and the "dynamic_epp" sysfs attribute, and update the dynamic_epp
documentation accordingly (K Prateek Nayak)
- Add unit tests for CPPC Performance Priority and the "dynamic" EPP
mode in the amd-pstate driver (K Prateek Nayak)
- Set min_limit_freq based on bios_min_perf in amd-pstate and remove
the defensive check for bios_min_perf from it (K Prateek Nayak)
- Fix EPP return type and handle errors in amd-pstate during
initialization, toggle auto_sel in active mode on shared memory
systems, and cache the firmware programmed EPP value (Marco Scardovi)
- Skip tests in amd-pstate-ut if the amd-pstate driver is not in active
use (Qianheng Peng)
- Replace sprintf() with sysfs_emit() in sysfs show in the cpufreq
schedutil governor and fix a self-contradictory comment in
sugov_iowait_apply() (Zhongqiu Han)
- Fix the usage example for the sampling_rate tunable of the ondemand
cpufreq governor in admin-guide (wangxiaodong)
- Avoid using deep idle states during initialization in the intel_idle
driver to work around device handling issues (Rafael Wysocki)
- Fix and refactor the ACPI processor driver code related to ACPI _LPI
support and add ACPI _LPI support to intel_idle based on that ACPI
processor driver update (Rafael Wysocki)
- Backup and restore governor for cpufreq sptests (Yiwei Lin)
- Remove unnecessary sudo from quick_shuffle() and remove unused local
variables from switch_show_governor() in cpufreq selftests (Jinseok
Kim)
- Rename the PM core module parameter prefix to "pm" and allow the PM
transition (DPM) watchdog to be disabled by default (Tzung-Bi Shih)
- Fix off-by-one in wakelocks number limit check in the system sleep
sysfs interface (Haowen Tu)
- Remove kernel-doc markings from helper descriptions in the core
hibernation code (Adi Nata)
- Use %pe to print error pointer values in the hibernation core (Ronan
Marchal)
- Fix memory leak in snapshot_write_next() error path (Malaya Kumar
Rout)
- Delay allocating and linking the next swap_map_page in the
hibernation image saving code until another image page actually
needs to be recorded (Haesung Kim)
- Fix cleanup ordering around scope-based pointers in OPP (Gregor
Herburger).
- Use clk_get_optional() for optional clocks in OPP (Praveen Talari).
- Stop setting runtime_error on runtime resume callback failures to
allow drivers to recover from resume issues (Praveen Talari)
- Handle PMU registration failure during probe in the intel_rapl_tpmi
driver (Sumeet Pawnikar)
- Avoid optional imports in intel_pstate_tracer unless they are really
needed (Yousef Alhouseen)
- Add generic CPPC performance display to the cpupower utility, build
and call CPPC information on non-AMD processors, make cpupower print
kernel and hardware frequency information, and add libm to cpupower
for generic CPPC view (Jeremy Linton)
- Remove conditional return with no effect from cpupower (Sang-Heon
Jeon)
-----BEGIN PGP SIGNATURE-----
iQFGBAABCAAwFiEEcM8Aw/RY0dgsiRUR7l+9nS/U47UFAmp+DXsSHHJqd0Byand5
c29ja2kubmV0AAoJEO5fvZ0v1OO10Z4IAIYwM9RKwbwCqPMKGvWExIb+1kpEy1aw
wyFnYSiQd09XCt8/YjERFR9gB7CfGduisTxEOqOdCIpGX2rT9Me5xO5aR5Gz4FE3
bML7KyulS9tVyaEVBjFx1yDZxGTNooTQVTLIxBYbCVB1RrgiCGdLyC2okn79XHlk
olTZemq59Kn0B6t7BP7UQ0WCCCAs3RlbrQIZSrVh0HydoO0DHlJvttN2fc7bjeiM
Je/dSm14o+vjglkVpY1kbxzBHJOH7/XH5nqSQRoHoAZWdllWK4+iQtll63Un4A72
2KG0O5+nFrS3d166TEjDEInnEwR012Sod/8Gk72M0IZZhG/naqs5ekE=
=mn7E
-----END PGP SIGNATURE-----
Merge tag 'pm-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
Pull power management updates from Rafael Wysocki:
"As has been the case for quite some time, this set of changes is
dominated by cpufreq updates including intel-pstate and amd-pstate
driver updates, minor fixes and cleanups of other assorted cpufreq
drivers, schedutil governor updates, fixes of the Rust bindings, new
hardware support (IPQ5210 in qcom-nvmem), and some updates of self
tests related to cpufreq.
The second largest group of changes are cpuidle updates consisting of
intel_idle driver updates and ACPI processor idle driver updates, both
mostly related to ACPI _LPI support.
There are also updates related to system sleep, mostly in the
hibernation core code, two operating performance points (OPP) updates,
one runtime PM framework update, one power capping update, and some
tools updates including the addition of ACPI CPPC support to cpupower.
Specifics:
- Minor fixes and cleanups in assorted cpufreq drivers (Dan
Carpenter, Guru Das Srinagesh, Haoxiang Li, Karl Mehltretter, Sasha
Finkelstein, and Pan Chuang)
- Fix cpufreq table creation and bios_limits() callback in the Rust
bindings (Priya Bala Govindasamy)
- Add IPQ5210 support to qcom-nvmem driver (Varadarajan Narayanan)
- Adjust the .adjust_perf() cpufreq driver callback to allow the
maximum performance value to be passed to drivers and update the
intel_pstate driver to use it (Rafael Wysocki)
- Set policy->cur to the actual requested frequency in the
intel_pstate driver when the performance policy is used (Rafael
Wysocki)
- Simplify HWP handling on Broadwell processors in intel_pstate
(Rafael Wysocki)
- Fix setting minimum P-state at init time in intel_pstate (Rafael
Wysocki)
- Consolidate frequency values computation in intel_pstate and clean
up code in that driver (Rafael Wysocki)
- Add missing kernel-doc descriptions for structure and union members
in the amd-pstate driver (David Vernet)
- Handle missing policy in dynamic EPP callbacks in the amd-pstate
driver (EDAMAMEX)
- Introduce EXPORT_SYMBOL_FOR_PSTATE_UT() to export amd-pstate driver
symbols to the amd-pstate-ut subdriver (K Prateek Nayak)
- Add dynamic EPP as an "energy_performance_preference" mode in
amd-pstate, remove the "amd_dynamic_epp" kernel command line option
and the "dynamic_epp" sysfs attribute, and update the dynamic_epp
documentation accordingly (K Prateek Nayak)
- Add unit tests for CPPC Performance Priority and the "dynamic" EPP
mode in the amd-pstate driver (K Prateek Nayak)
- Set min_limit_freq based on bios_min_perf in amd-pstate and remove
the defensive check for bios_min_perf from it (K Prateek Nayak)
- Fix EPP return type and handle errors in amd-pstate during
initialization, toggle auto_sel in active mode on shared memory
systems, and cache the firmware programmed EPP value (Marco
Scardovi)
- Skip tests in amd-pstate-ut if the amd-pstate driver is not in
active use (Qianheng Peng)
- Replace sprintf() with sysfs_emit() in sysfs show in the cpufreq
schedutil governor and fix a self-contradictory comment in
sugov_iowait_apply() (Zhongqiu Han)
- Fix the usage example for the sampling_rate tunable of the ondemand
cpufreq governor in admin-guide (wangxiaodong)
- Avoid using deep idle states during initialization in the
intel_idle driver to work around device handling issues (Rafael
Wysocki)
- Fix and refactor the ACPI processor driver code related to ACPI
_LPI support and add ACPI _LPI support to intel_idle based on that
ACPI processor driver update (Rafael Wysocki)
- Backup and restore governor for cpufreq sptests (Yiwei Lin)
- Remove unnecessary sudo from quick_shuffle() and remove unused
local variables from switch_show_governor() in cpufreq selftests
(Jinseok Kim)
- Rename the PM core module parameter prefix to "pm" and allow the PM
transition (DPM) watchdog to be disabled by default (Tzung-Bi Shih)
- Fix off-by-one in wakelocks number limit check in the system sleep
sysfs interface (Haowen Tu)
- Remove kernel-doc markings from helper descriptions in the core
hibernation code (Adi Nata)
- Use %pe to print error pointer values in the hibernation core
(Ronan Marchal)
- Fix memory leak in snapshot_write_next() error path (Malaya Kumar
Rout)
- Delay allocating and linking the next swap_map_page in the
hibernation image saving code until another image page actually
needs to be recorded (Haesung Kim)
- Fix cleanup ordering around scope-based pointers in OPP (Gregor
Herburger).
- Use clk_get_optional() for optional clocks in OPP (Praveen Talari).
- Stop setting runtime_error on runtime resume callback failures to
allow drivers to recover from resume issues (Praveen Talari)
- Handle PMU registration failure during probe in the intel_rapl_tpmi
driver (Sumeet Pawnikar)
- Avoid optional imports in intel_pstate_tracer unless they are
really needed (Yousef Alhouseen)
- Add generic CPPC performance display to the cpupower utility, build
and call CPPC information on non-AMD processors, make cpupower
print kernel and hardware frequency information, and add libm to
cpupower for generic CPPC view (Jeremy Linton)
- Remove conditional return with no effect from cpupower (Sang-Heon
Jeon)"
* tag 'pm-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (76 commits)
cpufreq: imx6q: fix out-of-bounds write when probed more than once
cpufreq: imx6q: fix devres accumulation across driver rebind
rust: cpufreq: Fix temporary write in Registration::bios_limit_callback
rust: cpufreq: Add CPUFREQ_TABLE_END as last table entry in TableBuilder::to_table
opp: Use clk_get_optional() to avoid leaving opp_table->clk as an error pointer
intel_idle: Avoid using deep idle states during initialization
cpupower: remove conditional return with no effect
cpufreq: intel_pstate: Adjust policy->cur in active mode to policy
cpufreq/amd-pstate: Document missing kernel-doc members
cpufreq/amd-pstate-ut: Add unit test for CPPC Performance Priority
cpufreq/amd-pstate-ut: Add unit test for "dynamic" EPP mode
cpufreq/amd-pstate: Reduce the scope of exported symbols
Documentation/amd-pstate: Update dynamic_epp documentation with new behavior
cpufreq/amd-pstate: Remove "amd_dynamic_epp" cmdline and "dynamic_epp" sysfs
cpufreq/amd-pstate: Add dynamic EPP as an "energy_performance_preference" mode
cpufreq/amd-pstate: Extract platform profile to EPP conversion into a helper
cpufreq/amd-pstate: Remove the defensive check for bios_min_perf
cpufreq/amd-pstate: Set min_limit_freq based on bios_min_perf
powercap: intel_rapl_tpmi: Handle PMU registration failure during probe
PM: sleep: Allow disabling DPM watchdog by default
...
|
||
|
Tejun Heo
|
fab183d632 |
sched_ext: Merge branch 'for-7.3-arena-args' into for-7.3
Pull to receive the __arena argument conversion: |
||
|
Tejun Heo
|
d023aa69c3 |
sched_ext: Drop the dead SCX_DEQ_CORE_SCHED_EXEC test in dequeue_task_scx()
dequeue_task_scx() masks SCX_DEQ_CORE_SCHED_EXEC out of the
SCX_DEQ_SCHED_CHANGE decision, but the test can never fire: the incoming
flags are an int of generic DEQUEUE_* bits while the flag is bit 32, and the
core-sched execute path never goes through class dequeue anyway -
set_next_task_scx() calls ops_dequeue() with the flag directly. The test was
live when the SCX_DEQ_SCHED_CHANGE computation sat in ops_dequeue() and
became dead when
|
||
|
Tejun Heo
|
006dd4d04b |
sched_ext: Make core-sched task ordering hierarchy-aware
With sub-schedulers, tasks of different schedulers routinely share rqs and SMT siblings, but scx_prio_less() consults ops.core_sched_before() only when both tasks belong to the same scheduler. Every pair spanning two schedulers falls back to the default ordering, so no scheduler can express ordering across a scheduler boundary, including a root over its sub-schedulers' tasks. Order a pair spanning schedulers by the nearest common ancestor that implements ops.core_sched_before(): both tasks are in its subtree, making this the one op where a scheduler is called on tasks it delegated to its sub-schedulers and may not be scheduling anymore. Same-scheduler pairs keep using the owning scheduler's op so a parent never orders inside a subtree it delegated. The op is skipped when the deciding scheduler is bypassing on either task's CPU. Update scx_qmap to fall back to the kernel's default ordering when handed a delegated task it has no task_ctx for. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
0ec5dd0669 |
sched_ext: Use runnable_at for the default core-sched task ordering
The default core-sched ordering runs the longest waiting task first by comparing p->scx.core_sched_at stamps. The stamp is maintained under two rules. touch_core_sched() stamps when a task starts waiting for a CPU and when its slice runs out. If the scheduler implements ops.core_sched_before(), touch_core_sched_dispatch() re-stamps on every dispatch. A comparison can see one stamp taken under each rule, which isn't a meaningful ordering. The dispatch rule also buys little - it only aligns bypass-mode comparisons with the local DSQ order. Multiple schedulers make the mixed comparisons more common. Wait time is what p->scx.runnable_at already tracks for the stall watchdog. Delete core_sched_at with both touch functions and compare runnable_at in the scx_prio_less() fallback. runnable_at is refreshed only on enqueue and goes stale while a task keeps occupying its CPU. Instead of re-stamping, order a running task after every waiting task as it is the most recently serviced. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
f7b6d128dd |
sched_ext: Fix inverted ops.core_sched_before() invocation
scx_prio_less() implements prio_less() semantics - %true means that @a is
the lower priority and should run after @b. ops.core_sched_before() is
documented to return %true when @a should run before @b. scx_prio_less()
returns the op's value as-is, inverting the documented semantics at runtime.
Call the op with the arguments swapped.
scx_qmap followed the wiring instead of the documentation and returned %true
for the younger task, so the two inversions canceled out and it behaved as
intended. Flip its comparison to match. scx_qmap is likely the only current
user in or out of the kernel tree. Any scheduler written the same way needs
the same flip, while schedulers following the documentation are fixed by
this change.
Fixes:
|
||
|
Tejun Heo
|
c384ab8a0b |
sched_ext: Move the config-off sub-cap kfunc stubs into sub.c
The EOPNOTSUPP stubs for the sub-cap kfuncs live in ext.c under #ifndef CONFIG_EXT_SUB_SCHED while the real definitions live in sub.c. Move the stubs into sub.c so all sub kfunc definitions live in one file. Pure code move, no functional change. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Hui Su
|
3bff8f8e95 |
cpufreq: schedutil: Fix rate limit overflow
rate_limit_us is an unsigned int, while NSEC_PER_USEC is defined as
1000L. On 32-bit systems, the multiplication is therefore performed
using 32-bit unsigned arithmetic before the result is assigned to
freq_update_delay_ns.
For example, writing 4294968 to rate_limit_us wraps the delay from
4294968000 ns to 704 ns. This makes schedutil update far more often
than configured.
Add sugov_update_rate_limit_us() to widen rate_limit_us to s64 before
converting it to nanoseconds. Use the helper when updating the tunable
through sysfs and when starting the governor, so both paths perform the
conversion without overflow.
Fixes:
|
||
|
Tejun Heo
|
3167bd3e0c |
sched_ext: Rename balance-era identifiers to dispatch terms
sched_class->balance() is gone from sched_ext and what balance_one() does is run dispatch to produce something pickable. Update the balance-era names to dispatch terms: - balance_one() -> dispatch_one() - SCX_RQ_IN_BALANCE -> SCX_RQ_IN_DISPATCH No BPF scheduler reads the flag. The enum autogen headers gain the new name with the old entry retained like other removed enumerators, zero-filling at load time. No functional changes. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
307a7b7ae5 |
sched_ext: Drop the stale keep_prev fixup in dispatch_pick()
The fixup demoting a keep verdict when @prev is not on ext_sched_class guarded against the rq-level SCX_RQ_BAL_KEEP flag going stale back when balancing and picking were separate operations. The verdict now travels in the return value, created and consumed in one invocation against the @prev it evaluated, and every keep decision tests SCX_TASK_QUEUED under the rq lock, which implies ext_sched_class as a class switch dequeues first. Drop the fixup along with dispatch_core_pick()'s copy. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
e0253dd04b |
sched_ext: Keep kick_sync waiting on the rq's own CPU
kick_sync_wait_bal_cb() assumes it runs on the rq's CPU from the
__schedule() tail: the snapshots it compares against live in that CPU's
percpu area and the busy-wait runs with the rq lock dropped and IRQs
enabled.
However, dispatch can now drop the rq lock while the callback sits queued,
and rq lock takers in that window (the sched class change paths, the scx
task iterator) flush pending balance callbacks on release, running the
callback on a foreign CPU. Such a run compares against unrelated snapshots
and can deadlock when the executing CPU is itself a wait target.
Bail on a foreign CPU and leave the wait state alone. The wait only observes
progress that the resched kicks already guarantee and the rq's next wait
picks up the stale cpus_to_sync bits.
Fixes:
|
||
|
Tao Cui
|
8d8dd8ae89 |
sched_ext: Don't BUG_ON a destroyed DSQ in process_deferred_reenq_users
scx_bpf_dsq_reenq() queues a deferred reenq (dru) that runs from
run_deferred(), not ops.dispatch(). If the DSQ is destroyed before the dru
runs, process_deferred_reenq_users() sees dsq->id == SCX_DSQ_INVALID and
hits the BUG_ON. destroy_dsq() doesn't flush pending drus, so just skip.
tj: Read dsq->id once with READ_ONCE(). Reading it separately in the INVALID
check and the BUG_ON would leave a window where destroy_dsq() can
invalidate the id between the two reads and still trigger the BUG_ON.
Fixes:
|
||
|
Tejun Heo
|
b27dfc7d8d |
sched_ext: Make scx_bpf_events() read the calling scheduler's counters
scx_bpf_events() always reads the root scheduler's event counters, so a sub-scheduler program querying its own events silently gets the root's instead and has no BPF-visible way to read its own (the per-scheduler sysfs "events" file is the only interface). Resolve the scheduler from the calling program with scx_prog_sched(). Unassociated programs follow the usual scx_prog_sched() resolution: the root scheduler under a pre-sub-attach compat root and zeroed counters otherwise. Also fix up the malformed comment into proper kerneldoc. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
d440933231 |
sched_ext: Drop unlocked scx_rq_clock_invalidate() from scx_root_disable()
scx_root_disable() invalidates each rq's clock before taking the rq lock.
scx_rq_clock_invalidate() is a plain read-modify-write of rq->scx.flags and
every other writer of the word runs under the rq lock, so the unlocked
update can race a concurrent flags update and lose one side's bits.
The invalidation doesn't matter in the first place. The cached clock is read
only by scx_bpf_now() from a loaded scheduler's BPF programs, nothing can
re-validate the clock while sched_ext is disabled as scx_rq_clock_update()
is gated on scx_enabled() too, and the usual rq lock cycles under the next
scheduler refresh or invalidate it before it's practically observable. Drop
the invalidation instead of fixing the locking.
v2: Description and comment updated - the invalidation is unnecessary rather
than subsumed by the rq lock cycle below.
Fixes:
|
||
|
fangqiurong
|
03506edca6 |
sched_ext: Fix exit_task leak on fork failure during enable
scx_fork() initializes tasks when scx_init_task_enabled is set, but
scx_cancel_fork() only exits them when scx_enabled() is true. A fork
that fails in the enable window (between releasing scx_fork_rwsem and
setting __scx_enabled) runs ops.init_task() but never ops.exit_task().
Gate scx_cancel_fork() on scx_init_task_enabled.
Fixes:
|
||
|
Tao Cui
|
1d0a73dd0c |
sched_ext: fix stale references in doc comments
- inlines.h: scx_bpf_dispatch() doesn't exist; the comment means scx_bpf_sub_dispatch() - internal.h: name %SCX_DEQ_SCHED_CHANGE instead of the never-defined %SCX_DEQ_SAVE - internal.h: @name shows up in the ops file in the scheduler's sysfs directory, not a "kernel.sched_ext_ops" sysctl Signed-off-by: Tao Cui <cuitao@kylinos.cn> Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
d7832ba1a4 |
sched_ext: Set up ops.sub_ecaps_updated() dispatch context on the executing CPU
scx_process_sync_ecaps() sets up the dispatch context for
ops.sub_ecaps_updated() in the target cpu's pcpu context recovered from the
llist node. However, the context is per executing cpu: the dispatch kfuncs
resolve it with this_cpu_ptr() and the dispatch buffer lives in it. What the
dispatches target is determined by the rq recorded in the context, not by
which cpu's context it is. Under core scheduling the pick runs balance_one()
for sibling rqs, so a sync processed for a sibling invokes the op with the
executing cpu's context not set up and its dispatch kfuncs misoperate on a
NULL or stale rq.
Set up the executing cpu's dsp_ctx instead, matching scx_dispatch_sched().
The recorded rq keeps the dispatches targeting the synced cpu.
Fixes:
|
||
|
Vincent Guittot
|
68e3748781 |
sched/fair: Fix flat hierarchy
When a fair task is enqueued, we must update curr and more precisely
its vruntime before placing the enqueued task so avg vruntime will take
into account the last exec phase.
Example:
TA is an always running task in cgroup G0.
TB is a short running task (cyclictest) in cgroup G1.
The lag of TB always increases up the clamp limit because TB is placed
before TA(curr) is updated (since the last tick). When curr(TA) is
finally updated, its last exec phase provide positive lag to TB
Because TA and TB don't belong to the same group, enqueue_hierarchy() will not
update TA's entity when updating curr but only G0's entity at root level.
The same applies when dequeuing.
This is because update_curr() uses ->h_curr, rather than ->curr, and therefore,
while it is invoked on the root cfs_rq, which contains all the eevdf bits, it
does not do the right thing.
Fixes:
|
||
|
Peter Zijlstra
|
3f8fa8fe90 |
Merge branch 'sched/urgent'
Pull in dependents, the flat hierarchy fix depends on this. Signed-off-by: Peter Zijlstra <peterz@infradead.org> |
||
|
Vincent Guittot
|
333238da9a |
sched: Update time before requeueing delayed entities
In order to compute the right lag, it is required to update time to 'now'. Without this, the delayed entity might appear younger than it really is and receive less compensation for having waited. Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> |
||
|
Waiman Long
|
2b58c749b8 |
sched/isolation: Defer freeing of cpumask memblock memory to initcall
When testing a linux-next kernel with commit |
||
|
Tejun Heo
|
a05c5b5cb5 |
sched_ext: Convert scx_bpf_cid_override() to __arena array arguments
scx_bpf_cid_override() predates the cid-form arena transition and takes its arrays as verifier-checked mem+size buffers, forcing scx_qmap to keep the cpu_to_cid and shard_start arrays in writable bss while the rest of its state lives in the arena. Unify on arena arguments before cid-form schedulers start seeing real use. BPF now translates between BPF and kernel arena addresses for __arena arguments. Take the arrays as __arena arguments, with the counts passed in entries. The counts now size the snapshot copies and are bounds-checked before them. scx_qmap moves the arrays into struct qmap_arena. As the arena is mmapped at load, the loader populates them between load and attach instead of before load. The arena argument address translation is currently implemented only on x86-64. Schedulers calling this kfunc load only there for now. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
a8dc810968 |
sched_ext: Convert sub-cap kfuncs to __arena cmask arguments
The sub-cap kfuncs take their cmask arguments as __ign pointers. The values cross the kfunc boundary as unchecked scalars and scx_cmask_ref_init() rebases them into the arena by hand. BPF now translates between BPF and kernel arena addresses for __arena arguments. Tag the cmask arguments __arena so the kfuncs receive kernel addresses and scx_cmask_ref_init() loses the hand-rolled conversion. The optional denied_out keeps its NULL not-provided signal via __arena__nullable. The mandatory masks use plain __arena. scx_qmap's call sites drop the (void *)(long) casts since the BPF-side declarations type the cmask arguments __arena and take arena pointers directly. The arena argument address translation is currently implemented only on x86-64. Schedulers calling these kfuncs load only there for now. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
67f1f4a48c |
sched_ext: Pass kernel arena pointers to ops_cid callbacks
The cid-form set_cmask() and sub_caps_updated() callbacks receive cmasks that the kernel builds in the arena, and the kernel converts the kernel addresses to the BPF arena pointer form by hand before each call. BPF now translates between BPF and kernel arena addresses for __arena arguments. Tag the arguments __arena in the cfi stubs and the ops_cid member declarations and pass the kernel arena addresses directly, dropping the manual scx_kaddr_to_arena() conversions and the now-unused helper. The delivered value is unchanged and existing BPF-side code works as before. The arena argument address translation is currently implemented only on x86-64. cid-form schedulers implementing these callbacks load only there for now. Signed-off-by: Tejun Heo <tj@kernel.org> |
||
|
Tejun Heo
|
294d95ff25 | Merge branch 'for-7.3' into for-7.3-arena-args | ||
|
fangqiurong
|
0c09d1ad81 |
sched_ext: Gate cid kfuncs behind the SCX struct_ops check
scx_bpf_cid_to_cpu(), scx_bpf_cpu_to_cid() and scx_bpf_cid_topo() live in
the scx_kfunc_ids_cid set, but scx_kfunc_context_filter() doesn't check
that set. The filter's first test treats any kfunc outside its known sets
as non-SCX and allows it, so these three kfuncs can be called from any
struct_ops program - e.g. a TCP congestion control program.
Add scx_kfunc_ids_cid to the filter's known sets, matching how in_any and
in_idle are handled.
Fixes:
|
||
|
Tejun Heo
|
1be10bb070 |
sched_ext: Merge branch 'for-7.2-fixes' into for-7.3
Pull to receive: |
||
|
Tejun Heo
|
d954004205 |
sched_ext: Fix rq->core_pick corruption under core scheduling
Core scheduling's pick_next_task() picks what to run on every SMT sibling of the core in a single pass under the shared core-wide rq lock. The selection state is consistent only while the lock is held continuously, so ->pick_task() originally could not release it. However, since |
||
|
Tejun Heo
|
f2da958711 |
sched_ext: Count rq lock releases in rq->scx.lock_drop_seq
Under core scheduling, pick_next_task() selects for all SMT siblings under
one continuous hold of the shared core-wide rq lock, and sched_ext's
dispatch can release that lock from inside the pick. In preparation for
making the core-sched pick detect the releases and retry, add
rq->scx.lock_drop_seq and bump it at every site that can release an rq lock
while a dispatch may be in flight. The counter is only maintained while core
scheduling is enabled. No functional changes.
Fixes:
|
||
|
Tejun Heo
|
3dd52416e4 |
sched_ext: Fix this_rq() assumptions in dispatch kfuncs
Under core scheduling, dispatch runs from within the core-wide pick and can
target a sibling rq, so ops.dispatch() may execute on a CPU different from
the dispatched rq's. Several kfunc paths assumed the two always coincide:
- scx_dsq_move() decided whether an rq lock is held by testing this_rq()'s
rq flags and lock-danced accordingly. A dispatch for a sibling took the
unlocked-context branch and acquired the source rq lock on top of the
already held dispatched rq lock which could deadlock.
- scx_bpf_sub_dispatch() dispatched this_rq() with its stashed
sub_dispatch_prev, which is NULL when dispatching for a sibling.
- finish_dispatch(), scx_bpf_dsq_reenq() and scx_bpf_dsq_nr_queued()
resolved SCX_DSQ_LOCAL to this CPU's local DSQ rather than the dispatched
rq's. The latter two are callable from other rq-locked operations too,
where SCX_DSQ_LOCAL now likewise resolves to the op's rq. This changes
behavior also without core scheduling, e.g. for ops.enqueue() running a
remote wakeup on the waking CPU, and is intended: which CPU happens to
execute an operation is incidental, the op's rq is what it is operating
on, and the resolution now matches the insert side where SCX_DSQ_LOCAL
dispatches land on the task's rq.
Use the rq tracked by scx_locked_rq(), which is set to the dispatched rq
around ops invocations and NULL in unlocked contexts.
Fixes:
|
||
|
Tejun Heo
|
ffaab58d21 |
sched_ext: Replace SCX_RQ_BAL_KEEP with a dispatch verdict return
SCX_RQ_BAL_KEEP tells the pick to keep running the previous task, a leftover
from when balancing and picking were separate operations. An rq-level flag
only works while dispatches and picks pair up one to one, which core
scheduling breaks: selections interleave through dispatch's lock drops and a
pick can consume a stale flag, keeping a task that has since been dequeued.
Fixing core scheduling support requires the decision to travel with the
dispatch that made it. Make scx_dispatch_sched() and balance_one() return an
explicit verdict instead and drop the flag's plumbing from the tools autogen
enum headers.
Also factor the pick-side invocation, its follow-up queueing and the
post-dispatch checks out of do_pick_task_scx() into dispatch_pick(). No
functional changes intended.
v2: Drop the SCX_RQ_BAL_KEEP plumbing from the tools autogen enum headers
as well (Andrea).
Fixes:
|
||
|
Tejun Heo
|
f3629c63a4 |
sched/core: Make core-sched flips wait for in-flight selections
Core scheduling's pick_next_task() operates on all sibling rqs under one
acquisition of the shared core-wide lock. A ->pick_task() that releases the
rq lock leaves every sibling __lock momentarily free, letting
__sched_core_flip(false) complete mid-selection and rebind rq_lockp() under
it. The selection resumes on the split locks, touching sibling state it no
longer protects, and __schedule() finally releases a lock that was never
taken while leaking the one that was.
Count in-flight core-wide selections in the leader's rq->core_pick_in_flight
and make __sched_core_flip() wait for the count to drain. The count only
changes under the shared lock, which the flip holds while sampling, so no
other ordering is needed. The wait can repeat while selections overlap, but
the flip backs off between samples and flips are rare cookie-lifetime
events.
sched_core_cpu_deactivate() moves the count to the new leader - a stale copy
left behind would bias it forever if that CPU later returns as its own
leader.
Fixes:
|
||
|
Tejun Heo
|
c10b216a07 |
sched/core: Handle pick_task() releasing the rq lock
Core scheduling's pick_next_task() breaks when a ->pick_task()
implementation can release the rq lock. The selection state derived on entry
is only valid while the lock is held continuously. Once a pick can drop the
lock, an interleaving selection can invalidate all of it: the single-CPU
fast path can commit an uncookied pick although the core went cookied during
the release, and forceidle committed by the interleaving selection skews the
restarted pass's accounting.
Fix it by restarting the whole selection when a pick returns RETRY_TASK
after releasing the lock: a single restart point above the state derivation
replaces the per-loop restart labels, so a retry picks up state committed by
interleaving selections and accounts and resets forceidle like a fresh
selection would.
need_sync and fi_before latch across retries. Clock validity can't be
re-derived - there is no program-ordered way to tell whether the own and
core rq clocks are still updated after the lock was released, as other
lockers' pin cycles may or may not have invalidated them. When restarting,
clear core_clock_updated so that the sibling loop re-updates the core rq,
and update the own rq clock if invalidated.
Fixes:
|
||
|
Tejun Heo
|
872a8f6b08 |
Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next into for-7.3-arena-args
Pull bpf-next
|
||
|
Boqun Feng
|
3b0e2a22d4 |
preempt: Introduce HAS_SEPARATE_PREEMPT_RESCHED_BITS
With the changes that enable preempt count to track IRQ disabling
nesting, we don't have enough bits in 32-bit preempt count
implementation, as a result we move NMI nesting bits out of the 32-bit
preempt count. However on the architectures that can support 64-bit
preempt count implementation, we can keep the NMI nesting bits in the
32-bit preempt count and avoid maintaining NMI nesting bits outside of
the same cache line.
Therefore HAS_SEPARATE_PREEMPT_RESCHED_BITS is introduced to allow
architectures to select this. Note that under this Kconfig, preempt
count is maintained in a 64-bit word however preempt_count() still
remains as an int because all the effective bits still fit in
(previously we mask out NEED_RESCHED bit in preempt_count()). This
should make no functional changes for existing preempt_count() users.
Enable this for x86_64 along with the introduction of the Kconfig.
[boqun: Undo the __preempt_count_{add,sub}() optimization in 32-bit
preempt count since it may introduce {over,under}flow]
Originally-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260804161447.84806-11-boqun@kernel.org
|
||
|
Boqun Feng
|
560fcaa92e |
sched: Avoid signed comparison of preempt_count() in __cant_migrate()
Currently preempt_count() is always a non-negative int on all archs (PREEMPT_NEED_RESCHED archs will mask out the MSB when returning preempt_count()), hence the checking in __cant_migrate() is in fact just checking whether preempt_count() is 0 or not. In a future change, we are going to use all the 32 bits of preempt_count(), which would make negative int values possible from preempt_count(). Therefore convert the "> 0" comparison into a zero check to prepare for the future change. No functional changes are intended. Signed-off-by: Boqun Feng <boqun@kernel.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://patch.msgid.link/20260804161447.84806-10-boqun@kernel.org |