After commit f90fff1e15 ("posix-cpu-timers: fix race between
handle_posix_cpu_timers() and posix_cpu_timer_del()"), tsk->sighand is
stable in handle_posix_cpu_timers(), so it can use the plain
spin_lock_irqsave(&tsk->sighand->siglock).
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Link: https://patch.msgid.link/akjp8AGpY8eJG5I1@redhat.com
Wongi and Jungwoo decoded and reported a non-leader exec() related race
which can result in an UAF:
sys_timer_delete() exec()
posix_cpu_timer_del()
// Observes old leader
p = pid_task(pid, pid_type); de_thread()
switch_leader();
release_task(old_leader)
__exit_signal(old_leader)
sighand = lock(old_leader, sighand);
posix_cpu_timers*_exit();
sighand = lock_task_sighand(p) unhash_task(old_leader);
sh = lock(p, sighand) old_leader->sighand = NULL;
unlock(sighand);
(p->sighand == NULL)
unlock(sh)
return NULL;
// Returns without action
if(!sighand)
return 0;
free_posix_timer();
This is "harmless" unless the deleted timer was armed and enqueued in
p->signal because on exec() a TGID targeted timer is inherited.
As sys_timer_delete() freed the underlying posix timer object
run_posix_cpu_timers() or any timerqueue related add/delete operations on
other timers will access the freed object's timerqueue node, which results
in an UAF.
There is a similar problem vs. posix_cpu_timer_set(). For regular posix
timers it just transiently returns -ESRCH to user space, but for the use
case in do_cpu_nanosleep() it's the same UAF just that the k_itimer is
allocated on the stack.
Also posix_cpu_timer_rearm() fails to rearm the timer, which means it stops
to expire.
While debating solutions Frederic pointed out another problem:
posix_cpu_timer_del(tmr)
__exit_signal(p)
posix_cpu_timers*_exit(p);
unhash_task(p);
p->sighand = NULL;
sh = lock_task_sighand(p)
sighand = p->sighand;
if (!sighand)
return NULL;
lock(sighand);
if (!sh)
WARN_ON_ONCE(timer_queued(tmr));
On weakly ordered architectures it is not guaranteed that
posix_cpu_timer_del() will observe the stores in posix_cpu_timers*_exit()
when p->sighand is observed as NULL, which means the WARN() can be a false
positive.
Solve these issues by:
1) Changing the store in __exit_signal() to smp_store_release().
2) Adding a smp_acquire__after_ctrl_dep() into the !sighand path
of lock_task_sighand().
3) Creating a helper function for looking up the task and locking sighand
which does not return when sighand == NULL. Instead it retries the
task lookup and only if that fails it gives up.
4) Using that helper in the three affected functions.
#1/#2 ensures that the reader side which observes sighand == NULL also
observes all preceeding stores, i.e. the stores in posix_cpu_timers*_exit()
and the ones in unhash_task().
#3 ensures that the above described non-leader exec() situation is handled
gracefully. When the task lookup returns the old leader, but sighand ==
NULL then it retries. In the non-leader exec() case the subsequent task
lookup will observe the new leader due to #1/#2. In normal exit() scenarios
the subsequent lookup fails.
When the task lookup fails, the function also checks whether the timer is
still enqueued and issues a warning if that's the case. Unfortunately there
is nothing which can be done about it, but as the task is already not
longer visible the timer should not be accessed anymore. This check also
requires memory ordering, which is not provided when the first lookup
fails. To achieve that the check is preceeded by a smp_rmb() which pairs
with the smp_wmb() in write_seqlock() in __exit_signal(). That ensures that
the stores in posix_cpu_timers*_exit() are visible.
The history of the non-leader exec() issue goes back to the early days of
posix CPU timers, which stored a pointer to the group leader task in the
timer. That obviously fails when a non-leader exec() switches the leader.
commit e0a7021710 ("posix-cpu-timers: workaround to suppress the problems
with mt exec") added a temporary workaround for that in 2010 which survived
about 10 years. The fix for the workaround changed the task pointer to a
pid pointer, but failed to see the subtle race described above. So the
Fixes tag picks that commit, which seems to be halfways accurate.
Thanks to Frederic Weissbecker, Oleg Nesterov and Peter Zijlstra for
review, feedback and suggestions and to Wongi and Jungwoo for the excellent
bug report and analysis!
Fixes: 55e8c8eb2c ("posix-cpu-timers: Store a reference to a pid not a task")
Reported-by: Wongi Lee <qw3rtyp0@gmail.com>
Reported-by: Jungwoo Lee <jwlee2217@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Cc: stable@vger.kernel.org
clock_base_next_timer_safe() was added by commit a64ad57e41 ("hrtimer:
Simplify run_hrtimer_queues()") but has never had a caller; the queue
iteration in __hrtimer_run_queues() uses clock_base_next_timer() instead.
The two are functionally equivalent: struct hrtimer embeds the timerqueue
node at offset 0, so container_of() of a NULL node yields NULL. Thus
clock_base_next_timer() already returns NULL on an empty queue and the
explicit NULL check in the _safe variant is redundant.
Being a static __always_inline function it does not trigger
-Wunused-function, so the dead code has gone unnoticed. Remove it.
No functional change.
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260625140901.929554-1-zhanxusheng@xiaomi.com
The compat version of settimeofday() uses '>' instead of '>=' when
validating tv_usec against USEC_PER_SEC, allowing the value 1000000 to pass
the check. After the subsequent conversion to nanoseconds (tv_nsec *=
NSEC_PER_USEC), this results in tv_nsec == NSEC_PER_SEC, which violates the
timespec invariant that tv_nsec must be strictly less than NSEC_PER_SEC.
The native settimeofday() was already fixed in commit ce4abda5e1 ("time:
Fix off-by-one in settimeofday() usec validation"), but the compat
counterpart was missed.
Fix it by using '>=' to reject tv_usec values outside the valid range [0,
USEC_PER_SEC - 1].
Fixes: 5e0fb1b57b ("y2038: time: avoid timespec usage in settimeofday()")
Signed-off-by: Wang Yan <wangyan01@kylinos.cn>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260622103348.120255-1-wangyan01@kylinos.cn
A comment in kernel/time/hrtimer.c incorrectly refers to
CONFIG_NOHZ_COMMON instead of CONFIG_NO_HZ_COMMON. Correct it.
Discovered while searching for CONFIG_* symbols referenced in code but
not defined in any Kconfig file.
Signed-off-by: Ethan Nelson-Moore <enelsonmoore@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260609034314.25029-1-enelsonmoore@gmail.com
update_rlimit_cpu() converts the RLIMIT_CPU value to nanoseconds with
u64 nsecs = rlim_new * NSEC_PER_SEC;
On 32-bit kernels both rlim_new (unsigned long) and NSEC_PER_SEC
(1000000000L) are 32-bit, so the multiplication is performed in unsigned
long and truncated for rlim_new > 4 seconds before being widened to u64.
The same file already casts to u64 for the matching computation in
check_process_timers():
u64 softns = (u64)soft * NSEC_PER_SEC;
As a result, the truncated value is installed into the CPUCLOCK_PROF
expiry cache (nextevt), causing the process CPU timer to be programmed
to fire prematurely for any RLIMIT_CPU soft limit >= 5 seconds. The
actual SIGXCPU/SIGKILL decision in check_process_timers() already casts
to u64 and is therefore correct, so limit enforcement is not broken;
only the expiry-cache programming is wrong. Apply the same cast here so
both paths convert rlim_cur identically.
64-bit kernels are unaffected.
Fixes: 858cf3a8c5 ("timers/itimer: Convert internal cputime_t units to nsec")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260616112017.1681372-1-zhanxusheng@xiaomi.com
Commit f24df84cbe ("time/jiffies: Register jiffies clocksource before
usage") moved the jiffies clocksource registration into
clocksource_default_clock(), so that it is registered lazily on the first
call. __clocksource_register() acquires clocksource_mutex, but the first
caller is timekeeping_init(), which invokes clocksource_default_clock()
while holding tk_core.lock, a raw spinlock.
Acquiring a sleeping mutex while holding a raw spinlock is invalid.
The default clocksource only has to be registered before
tk_setup_internals() consumes its mult/shift/maxadj. Neither
clocksource_default_clock(), the ->enable() callback, nor the registration
itself need tk_core.lock, so fetch and enable the clock before acquiring
the lock. This preserves the "register before usage" ordering while
keeping clocksource_mutex out of the raw spinlock section.
clocksource_default_clock() has a second caller,
clocksource_done_booting(), which invokes it with clocksource_mutex already
held. That path avoids a recursive lock because timekeeping_init() has
already run and set cs_jiffies_registered, so the registration is skipped
there. This change does not alter that; it only fixes the invalid wait
context in timekeeping_init().
Fixes: f24df84cbe ("time/jiffies: Register jiffies clocksource before usage")
Signed-off-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reported-by: Breno Leitao <leitao@debian.org>
Reported-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Breno Leitao <leitao@debian.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260616070914.65818-1-mikhail.v.gavrilov@gmail.com
- Remove the redundant CONFIG_GENERIC_TIME_VSYSCALL after converting the
remaining users over.
- Rework and sanitize the MIPS VDSO handling, so it does not handle the
time related VDSO if there is no VDSO capable clocksource available.
Also stop mapping VDSO data pages unconditionally even if there is no
usage possible.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotog4QHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoYD5D/41UOOboDxkv3Ei0gS12RfFL/pHWPotWMvA
eUS4Ohko2pYubZB+iv87QQD5HC6TBN9yXbUjpvS0wN9jVSlXgQOdJ0JkNYYbPfjH
nsBt7g/ksCA0Tv0KxnUUtECCTu6frtXkSi5iDU03DVNvxLMDmRaXg+D10TKGiCfb
q1g2IoenA6YWz8pUkwOSI5wInepN8Dt9s5VXRsBQS9J9Uin/ZWAIwghIEhfFPNZu
braNUSyJ+A7xISipOjWD1kEfBjcUzr24AuMZ9XHpxnykdWoa57qNEoTnjm+bVWpU
hK0ED4nXSM8qxioV1Kye0h+hUi0T7amNtq6OSfAL+oXWHINvUe2PybqK+sZ1pPFb
0MSP/J4mV9CIvriPA33fTXw/VxigI/puCMtGuSLLXHwh2xW8hadz7wGU9lqF4E7E
f+X9NYqZSmx/82z0KKPFtIQyfrozquRBzy2ZiUloh7ek7GKF5tvuN8lds3Ghuf/6
9Hqr/YquG1Zuu+j95ozVIm7fhDjmMktxAwyAbg5WkNGqOEw7MyhoUGXjKKG29uLa
PRitT9DklL5vndzjizkLc1C2DuC++OhpnJCqRWOjJ9/TbvBC6Rn6aKwYyVTFX9xM
4VauNLQg0q9uG0TyW6o3aakaFk9OIKT+vl7bWO0G4g2YZaKDOs7s+b9sdNNNQq5R
lTL5F0KUCQ==
=Z90m
-----END PGP SIGNATURE-----
Merge tag 'timers-vdso-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull vdso updates from Thomas Gleixner:
- Remove the redundant CONFIG_GENERIC_TIME_VSYSCALL after converting
the remaining users over.
- Rework and sanitize the MIPS VDSO handling, so it does not handle the
time related VDSO if there is no VDSO capable clocksource available.
Also stop mapping VDSO data pages unconditionally even if there is no
usage possible.
* tag 'timers-vdso-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
MIPS: VDSO: Fold MIPS_CLOCK_VSYSCALL into MIPS_GENERIC_GETTIMEOFDAY
MIPS: VDSO: Gate microMIPS restriction on GCC version
MIPS: VDSO: Fold MIPS_DISABLE_VDSO into MIPS_GENERIC_GETTIMEOFDAY
clocksource/drivers/mips-gic-timer: Only use VDSO_CLOCKMODE_GIC when it is a available
MIPS: csrc-r4k: Only use VDSO_CLOCKMODE_R4K when it is a available
MIPS: VDSO: Only map the data pages when the vDSO is used
MIPS: Introduce Kconfig MIPS_GENERIC_GETTIMEOFDAY
vdso/datastore: Always provide symbol declarations
MAINTAINERS: Add include/linux/vdso_datastore.h to vDSO block
vdso/gettimeofday: Rename __arch_get_vdso_u_timens_data()
vdso/treewide: Drop GENERIC_TIME_VSYSCALL
vdso/vsyscall: Gate update_vsyscall() behind CONFIG_GENERIC_GETTIMEOFDAY
riscv: vdso: Drop CONFIG_GENERIC_TIME_VSYSCALL guard around syscall fallbacks
- Expand timekeeping snapshot mechanisms
The various snapshot functions are mostly used for PTP to collect
"atomic" snapshots of various involved clocks.
They lack support for the recently introduced AUX clocks and do not
provide the underlying counter value (e.g. TSC) to user space. Exposing
the counter value snapshot allows for better control and steering.
Convert the hard wired ktime_get_snapshot() to take a clock ID, which
allows the caller to select the clock ID to be captured along with
CLOCK_MONONOTONIC_RAW. Additionally capture the underlying hardware
counter value and the clock source ID of the counter.
Expand the hardware based snapshot capture where devices provide a
mechanism to snapshot the hardware PTP clock and the system counter
(usually via PCI/PTM) to support AUX clocks and also provide the
captured counter value back to the caller and not only the clock
timestamps derived from it.
- Add a new optional read_snapshot() callback to clocksources
That is required to capture atomic snapshots from clocksources which
are derived from TSC with a scaling mechanism (e.g. Hyper-V, KVMclock).
The value pair is handed back in the snapshot structure to the callers,
so they can do the necessary correlations in a more precise way.
This touches usage sites of the affected functions and data structure all
over the tree, but stays fully backwards compatible for the existing user
space exposed interfaces. New PTP IOCTLs will provide access to the
extended functionality in later kernel versions.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotoSAQHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoYWbEACv/g3pGDxWxfzOI2h6vQgGxDvD3LwmdhPE
bzXRRaxp3/J0rZTQmCghknVGDPVjepNQgKkUMXfaFG2UZmiPHG5qVTXO6DddguS4
cQc0SUO3e422lUPCoBmTULZ+vlctb4LJsWXPQHYNKC73KqMJtWte7T2HBiFDK5RB
O0S34DZtkvOW4tHIu0RwlwCXZ0gcO+zsjxKA8K/P6sMtKBQU1/rRkZZCx2KCvq0F
Rx3NTGoY4if/C83YBq1cEn8BvXrcQQH4ZOOWuySsLGJGRPZ1dXGP+JtfRWutk/f9
HZztlaXcEz71dJXlhBxc0Eb/86uC3POEq7ZYvQdzLbsSZ/3AbalksL9CLyxgdHtc
U964SuwOVPcYfEytd4TWb1nu7JgOR0olYK+l4AbCt4EdKst5TADCJ7rtlZV3Idp+
Yg1GN3TwJcKItUNX9Szk+7MbvB8EWOEl7Obahfm48qDK1pqFe08qhOzSCeRXu+Bb
QiupC3ndzUB1Yjf3DPV6wQl4Fl/TscrAVrPlnGCOJEKXtUKFxvcKquy/W29UD//w
NuxKO2zK05UDsbBEwnZiCrdSGGNiLBYUbHfx2UvA7M0rfrrbjmG4rCFPotxhNb54
UuqgdM8G45MkyBV3qSSh3VC0XeD7UqzQtMYgUjjhvtapLlsri69vzL2DnQUcajSG
dgjzIg9O3g==
=kBrz
-----END PGP SIGNATURE-----
Merge tag 'timers-ptp-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull timekeeping updates from Thomas Gleixner:
"Updates for NTP/timekeeping and PTP:
- Expand timekeeping snapshot mechanisms
The various snapshot functions are mostly used for PTP to collect
"atomic" snapshots of various involved clocks.
They lack support for the recently introduced AUX clocks and do not
provide the underlying counter value (e.g. TSC) to user space.
Exposing the counter value snapshot allows for better control and
steering.
Convert the hard wired ktime_get_snapshot() to take a clock ID,
which allows the caller to select the clock ID to be captured along
with CLOCK_MONONOTONIC_RAW. Additionally capture the underlying
hardware counter value and the clock source ID of the counter.
Expand the hardware based snapshot capture where devices provide a
mechanism to snapshot the hardware PTP clock and the system counter
(usually via PCI/PTM) to support AUX clocks and also provide the
captured counter value back to the caller and not only the clock
timestamps derived from it.
- Add a new optional read_snapshot() callback to clocksources
That is required to capture atomic snapshots from clocksources
which are derived from TSC with a scaling mechanism (e.g. Hyper-V,
KVMclock).
The value pair is handed back in the snapshot structure to the
callers, so they can do the necessary correlations in a more
precise way.
This touches usage sites of the affected functions and data structure
all over the tree, but stays fully backwards compatible for the
existing user space exposed interfaces. New PTP IOCTLs will provide
access to the extended functionality in later kernel versions"
* tag 'timers-ptp-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (28 commits)
ptp: vmclock: Use hw_cycles from snapshot for precise TSC pairing
x86/kvmclock: Implement read_snapshot() for kvmclock clocksource
clocksource/hyperv: Implement read_snapshot() for TSC page clocksource
timekeeping: Add clocksource read_snapshot() method and hw_cycles to snapshot
ptp: Switch to ktime_get_snapshot_id() for pre/post timestamps
timekeeping: Add support for AUX clock cross timestamping
timekeeping: Remove system_device_crosststamp::sys_realtime
ALSA: hda/common: Use system_device_crosststamp::sys_systime
wifi: iwlwifi: Use system_device_crosststamp::sys_systime
ptp: Use system_device_crosststamp::sys_systime
timekeeping: Prepare for cross timestamps on arbitrary clock IDs
timekeeping: Remove ktime_get_snapshot()
virtio_rtc: Use provided clock ID for history snapshot
net/mlx5: Use provided clock ID for history snapshot
igc: Use provided clock ID for history snapshot
ice/ptp: Use provided clock ID for history snapshot
wifi: iwlwifi: Adopt PTP cross timestamps to core changes
timekeeping: Add CLOCK ID to system_device_crosststamp
timekeeping: Add system_counterval_t to struct system_device_crosststamp
timekeeping: Add CLOCK_AUX support for ktime_get_snapshot_id()
...
- Fix a long standing TOCTOU in get_cpu_sleep_time_us()
- Make the CPU offline NOHZ handling more robust by disabling NOHZ on the
outgoing CPU early instead of creating unneeded state which needs to be
undone.
- Unify idle CPU time accounting instead of having two different
accounting mechanisms. These two different mechanisms are not really
independent, but the different properties can in the worst case cause
that gloabl idle time can be observed going backwards.
- Consolidate the idle/iowait time retrieval interfaces instead of
converting back and forth between them.
- Make idle interrupt time accounting more robust. The original code
assumes that interrupt time accouting is enabled and therefore stops
elapsing idle time while an interrupt is handled in NOHZ dyntick
state. That assumption is not correct as interrupt time accounting can
be disabled at compile and runtime.
- Fix an accounting error between dyntick idle time and dyntick idle
steal time. The stolen time is not accounted and therefore idle time
becomes inaccurate. The stolen time is now accounted after the fact as
there is no way to predict the steal time upfront.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotm7YQHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoWEOEADG/RzW9l7fwUcsnubRyHER7GoIEZGP/VO/
nLQnI+M1LwdVyK8Oq8WntSnqOGevVdUKQlLFYMXdwovS8TWCi7PWcQeMIsfJnwT/
6ugwo4E3mSBcseMjN8eHkEYH+1YmN6JSYQ+5eXT0JUXUxJlMFgtv3ZnTbQOF6Y3k
Nkz1THojzdgTMn6WU/01AOXDR8Nhb2gOOQLDF/1ItZWnsDhbrDE2l99OaAELfeIo
8BZFQBPYRfuR8HaGgd8m2OPGnbw+cXMHUrTscMMQvmYv2wJVcNMB4AmVaUmGMRTR
c3xs8QG2nUmIm72ENfe2pPDPyxy9JUJ54ro6/rLtcNQZ2wVCznhNahm6wRUa8f/0
7gnQE5nS2SNtdpL5StDVzk2+AZl6SrU8+ss51f1owPcNwKs+tt9XE+GVyL5wZpC8
IeD2SzVQLIIc0+ZvIkt9n76lLJkiLZkmt6UAnvhiz5NNn8XQlgC/4Uadk9CwC5vx
t6swpE06D7vx7lm+ycfYUl6VPje5g/YvkkDFJqJulIlEDNpvb85+W6lMTyO7rwVl
ln5TzOGPp7lJDwu6dAfg0QtHcUmUIFq05FuOIRNC4ZNxkxL2XDMzoe5IHxcWzU9O
CHFGBUNZf+30vxvKeh+rkjZalt7/vgQaxb5somGmqv8g/6SJtJyB2qlq+gqH/+lQ
MeG8tFGtVQ==
=+cVw
-----END PGP SIGNATURE-----
Merge tag 'timers-nohz-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull NOHZ updates from Thomas Gleixner:
- Fix a long standing TOCTOU in get_cpu_sleep_time_us()
- Make the CPU offline NOHZ handling more robust by disabling NOHZ on
the outgoing CPU early instead of creating unneeded state which needs
to be undone.
- Unify idle CPU time accounting instead of having two different
accounting mechanisms. These two different mechanisms are not really
independent, but the different properties can in the worst case cause
that gloabl idle time can be observed going backwards.
- Consolidate the idle/iowait time retrieval interfaces instead of
converting back and forth between them.
- Make idle interrupt time accounting more robust. The original code
assumes that interrupt time accouting is enabled and therefore stops
elapsing idle time while an interrupt is handled in NOHZ dyntick
state. That assumption is not correct as interrupt time accounting
can be disabled at compile and runtime.
- Fix an accounting error between dyntick idle time and dyntick idle
steal time. The stolen time is not accounted and therefore idle time
becomes inaccurate. The stolen time is now accounted after the fact
as there is no way to predict the steal time upfront.
* tag 'timers-nohz-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
sched/cputime: Handle dyntick-idle steal time correctly
sched/cputime: Handle idle irqtime gracefully
sched/cputime: Provide get_cpu_[idle|iowait]_time_us() off-case
tick/sched: Consolidate idle time fetching APIs
tick/sched: Account tickless idle cputime only when tick is stopped
tick/sched: Remove unused fields
tick/sched: Move dyntick-idle cputime accounting to cputime code
tick/sched: Remove nohz disabled special case in cputime fetch
tick/sched: Unify idle cputime accounting
s390/time: Prepare to stop elapsing in dynticks-idle
powerpc/time: Prepare to stop elapsing in dynticks-idle
sched/cputime: Correctly support generic vtime idle time
sched/cputime: Remove superfluous and error prone kcpustat_field() parameter
sched/idle: Handle offlining first in idle loop
tick/sched: Fix TOCTOU in nohz idle time fetch
- Harden the user space controllable hrtimer interfaces further to
protect against unpriviledged DoS attempts by arming timers in the past.
- Add per-capacity hierarchies to the timer migration code to prevent
timer migration accross different capacity domains. This code has been
disabled last minute as there is a pathological problem with SoCs which
advertise a larger number of capacity domains. The problem is under
investigation and the code won't be active before v7.3, but that turned
out to be less intrusive than a full revert as it preserves the
preparatory steps and allows people to work on the final resolution
- Export time namespace functionality as a recent user can be built as a
module.
- Initialize the jiffies clocksource before using it. The recent
hardening against time moving backward requires that the related
members of struct clocksource have been initialized, otherwise it
clamps the readout to 0, which makes time stand sill and causes boot
delays.
- Fix a more than twenty year old PID reference count leak in an error
path of the POSIX CPU timer code.
- The usual small fixes, improvements and cleanups all over the place.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotj9MQHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoZgMEADHAbRUevdga3L6BkPy5X/4HwKp5YpwIzAO
NXrIugAWjObQnhiPoVV9IYrBSj7MKX1hXTOIt/4G8vj/LK1Q7f/YtLIhp5mi7Qhe
EJXS5gRvjbHwlOLko+IRyRNJIy0PcL5VRCSEqmS/wbAblfmuGsVXP6Z4mlbWdG3i
hux6w3sbbVy1rYo/hYEM+sA4IATH6cepzSNbT3jStyqZGWYDRuGlimcMGlIKe0hG
mczpQx+MoBTXL3PsZJrTu8cLACSqZRNkiNaX1yh3xKGLQdrchbydSmL55ucUvQQb
HWWotjt0PWHJgWNQVZpwFA+V2otV+XT7nVoxQlOQHwQxe1qroa781Sg6ZPXNH6Av
KH88cvUuu7f6vVRtijUmiar9IVBy4MGZ7CpZLqCEBeAxMhwATidHOzxYZIGKOxJF
k9Im1nIWkem3eMqGdwrmAuDHNQA9KCDgencDpT+6jJ/6rMuezIteVJjlJFsR4KGa
1bXwX6NqlLvs4dWfiihP8mLsn73t1CeN0HJSJkdjib6tHS0AQpVCV35z8QgRmeMs
bSiFfVFutqAwu1KUC3ESKtzGDVirIjI6FAhbUOzE5SUq0APu1gXBih8142jPuQGz
9hyWBdVVsUIfQdEqY9xhSnXhUXcekPOe5rnDGECNP8MHxP9bNASTBHeOY3ELeKxp
7+Bjk34eqg==
=KRcs
-----END PGP SIGNATURE-----
Merge tag 'timers-core-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull timer core updates from Thomas Gleixner:
"Updates for the time/timer core subsystem:
- Harden the user space controllable hrtimer interfaces further to
protect against unpriviledged DoS attempts by arming timers in the
past.
- Add per-capacity hierarchies to the timer migration code to prevent
timer migration accross different capacity domains. This code has
been disabled last minute as there is a pathological problem with
SoCs which advertise a larger number of capacity domains. The
problem is under investigation and the code won't be active before
v7.3, but that turned out to be less intrusive than a full revert
as it preserves the preparatory steps and allows people to work on
the final resolution
- Export time namespace functionality as a recent user can be built
as a module.
- Initialize the jiffies clocksource before using it. The recent
hardening against time moving backward requires that the related
members of struct clocksource have been initialized, otherwise it
clamps the readout to 0, which makes time stand sill and causes
boot delays.
- Fix a more than twenty year old PID reference count leak in an
error path of the POSIX CPU timer code.
- The usual small fixes, improvements and cleanups all over the
place"
* tag 'timers-core-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (31 commits)
posix-cpu-timers: Fix pid refcount leak in do_cpu_nanosleep() error path
time/jiffies: Register jiffies clocksource before usage
timers/migration: Temporarily disable per capacity hierarchies
timers/migration: Turn tmigr_hierarchy level_list into a flexible array
timers/migration: Deactivate per-capacity hierarchies under nohz_full
timers/migration: Fix hotplug migrator selection target on asymetric capacity machines
ntsync: Honour caller's time namespace for absolute MONOTONIC timeouts
time/namespace: Export init_time_ns and do_timens_ktime_to_host()
timers/migration: Update stale @online doc to @available
timers: Fix flseep() typo in kernel-doc comment
hrtimer: Fix the bogus return type of __hrtimer_start_range_ns()
hrtimer: Return ktime_t from hrtimer_get_next_event()/hrtimer_next_event_without()
clocksource: Clean up clocksource_update_freq() functions
alarmtimer: Remove stale return description from alarm_handle_timer()
selftests/posix_timers: Use CLOCK_THREAD_CPUTIME_ID for ITIMER_PROF measurements
scripts/timers: Add timer_migration_tree.py
timers/migration: Handle capacity in connect tracepoints
timers/migration: Split per-capacity hierarchies
timers/migration: Track CPUs in a hierarchy
timers/migration: Abstract out hierarchy to prepare for CPU capacity awareness
...
- Add devm helpers for clocksources, which allows to simplify driver
teardown and probe failure handling.
- More module conversion work
- Update the support for the ARM EL2 virtual timer including the required
ACPI changes.
- Add clockevent and clocksource support for the TI Dual Mode Timer
- Fix the support for multiple watchdog instances in the TEGRA186 driver
- Add D1 timer support to the SUN5I driver
- The usual devicetree updates, cleanups and small fixes all over the place
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmotjBUQHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYoaXND/0Z+wx27W4g4n9OTc//dfG+iV4eg05On9Fg
LhlnyanjvIiZwWEBR0AdLDFHgxCqQV0fHpgldsz/Qlk/5uXcpsrGIhqwAiWynppU
SLcZyW5XrZp4syTQfIxaAwAqMzGfW3nHWEkaRYCo/Q8oTDfvrXzY0AWYoK+bDkL3
NmnIuNGdLD+VoD4E3tn0/Q/MTtA4+DI9Cq98S5lOOchkO01ZOxx9OTalhAGFEvOo
zk6YSW2qeUtvc3stahPSvAOaC6EcAalp7wNjWDeHV/c2kUmoKlGpsgvb8Mt+0vPD
exNdctuTFGU2zrS1s0IB22n8RkSCzloxiTBs4Ec88ecn+cLIwq3oeuqchNiH/Quw
YzQWuYhG1ML5i5/g3sAnYu+Hj5DOvB/0IBiYlCxKH4rf49RtBn1AQWt4OzdKzuQS
/fnYqw+4N5XCodHzQbFv7Kbs+21lXXYubt9RLvSl6QYc4xfJRfWCS7rJdXUExw3D
5visGs+tUozi8C23t+BCWJ7jJsd0uNk51gmvOPUI3ybgL5YFwz1YdY4OwAAQT57i
MGiDbC4nMz7nlyWkl40ZVjOvl8bNvCGozq30tRux+CPmpYUi/87ToraKZy8ulSra
+26/ekATKXSfcyUh+TinA10AnnmCdxUyfA1WpZ6z/bCVdZqqHXK6HBf0OY8+u3um
zhrXH2BJBQ==
=4vQE
-----END PGP SIGNATURE-----
Merge tag 'timers-clocksource-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull clocksource updates from Thomas Gleixner:
"Updates for clocksource/clockevent drivers:
- Add devm helpers for clocksources, which allows to simplify driver
teardown and probe failure handling.
- More module conversion work
- Update the support for the ARM EL2 virtual timer including the
required ACPI changes.
- Add clockevent and clocksource support for the TI Dual Mode Timer
- Fix the support for multiple watchdog instances in the TEGRA186
driver
- Add D1 timer support to the SUN5I driver
- The usual devicetree updates, cleanups and small fixes all over the
place"
* tag 'timers-clocksource-2026-06-13' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (24 commits)
clocksource: move NXP timer selection to drivers/clocksource
clocksource/drivers/timer-tegra186: Reserve and service a kernel watchdog
clocksource/drivers/timer-tegra186: Register all accessible watchdog timers
clocksource/drivers/timer-tegra186: Correct num_wdts for Tegra186 and Tegra234
clocksource/drivers/timer-tegra186: Fix support for multiple watchdog instances
clocksource/drivers/timer-ti-dm: Add clockevent support
clocksource/drivers/timer-ti-dm: Add clocksource support
clocksource/drivers/timer-ti-dm: Fix property name in comment
dt-bindings: timer: arm,arch_timer: Fix requirements for interrupt description
clocksource/drivers/arm_arch_timer: Default to EL2 virtual timer when running VHE
ACPI: GTDT: Parse information related to the EL2 virtual timer
ACPI: GTDT: Account for GTDTv3 size when walking the platform timer descriptors
clocksource: Add devm_clocksource_register_*() helpers
clocksource/drivers/sun5i: Add D1 hstimer support
dt-bindings: timer: allwinner,sun5i-a13-hstimer: add H616 and D1
dt-bindings: timer: Add StarFive JHB100 clint
dt-bindings: timer: renesas,rz-mtu3: document RZ/{T2H,N2H}
dt-bindings: timer: renesas,rz-mtu3: Remove TCIU8 interrupt
dt-bindings: timer: Remove sifive,fine-ctr-bits property
clocksource/drivers/timer-of: Make the code compatible with modules
...
In do_cpu_nanosleep(), posix_cpu_timer_create() takes a pid reference
via get_pid() and stores it in timer.it.cpu.pid. If the subsequent
posix_cpu_timer_set() call fails, the function returns immediately
without calling posix_cpu_timer_del() to release the pid reference,
causing a leak.
Fix it by calling posix_cpu_timer_del() before the unlock-and-return
on the error path, consistent with the other exit paths in the same
function.
Fixes: 1da177e4c3 ("Linux-2.6.12-rc2")
Signed-off-by: WenTao Liang <vulab@iscas.ac.cn>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260611161738.97043-1-vulab@iscas.ac.cn
Teddy reported that a XEN HVM has a long boot delay, which was bisected to
the recent enhancements to the negative motion detection. It turned out
that the jiffies clocksource is used in early boot before it is registered,
which leaves the max_delta_raw field at zero. That causes the read out to
be clamped to the max delta of 0, which means time is not making progress.
Cure it by ensuring that it is initialized before its first usage in
timekeeping_init().
Fixes: 76031d9536 ("clocksource: Make negative motion detection more robust")
Reported-by: Teddy Astie <teddy.astie@vates.tech>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Teddy Astie <teddy.astie@vates.tech>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/87y0gn3fve.ffs@fw13
Closes: https://lore.kernel.org/all/1780914594.8631fc262581453bbf619ec5b2062170.19ea6c8227b000701b@vates.tech
Some workloads with different CPU capacities consume more power with
timer migration than before. The recently introduced per capacity
hierarchies were supposed to alleviate this problem. However it appears
to also regress other types of workloads, especially when plenty of
capacities live together in the same machine.
Disable the feature until a reasonable solution is found.
Fixes: 098cbaad8e ("timers/migration: Split per-capacity hierarchies")
Reported-by: Christian Loehle <christian.loehle@arm.com>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260609123356.28449-1-frederic@kernel.org
Closes: https://lore.kernel.org/all/3b79338f-6cfc-4722-8062-9103db2c8ad1@arm.com
tmigr_handle_remote_cpu() skips timer_expire_remote() when cpu ==
smp_processor_id(), assuming the local softirq path already handled this
CPU's timers.
This assumption is wrong because jiffies can advance after the handling of
the CPU's global timers in run_timer_base(BASE_GLOBAL) and before
tmigr_handle_remote() evaluates the expiry times.
As a consequence a timer which expires after the CPU local timer wheel
advanced and becomes expired in the remote handling is ignored and the
callback is never invoked and removed from the timer wheel.
What's worse is that fetch_next_timer_interrupt_remote() keeps reporting it
as expired, and the event is re-queued with expires == now on each
iteration. The goto-again loop spins indefinitely.
Fix this by calling timer_expire_remote() unconditionally. That's minimal
overhead for the common case as __run_timer_base() returns immediately if
there is nothing to expire in the local wheel.
[ tglx: Amend change log and add a comment ]
Fixes: 7ee9887703 ("timers: Implement the hierarchical pull model")
Reported-by: Alon Kariv <alonka@amazon.com>
Signed-off-by: Amit Matityahu <amitmat@amazon.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260603170139.33628-1-amitmat@amazon.com
Add a read_snapshot() callback to struct clocksource which returns the
derived clocksource value while also providing the underlying hardware
counter reading and the related clocksource ID.
This allows ktime_get_snapshot_id() to populate new hw_cycles and hw_csid
fields in struct system_time_snapshot.
For clocksources that are derived from an underlying counter (e.g., Hyper-V
TSC page scales TSC to 10MHz, kvmclock scales TSC to 1GHz), this provides
atomic access to both the derived value needed for timekeeping
calculations, and the raw hardware counter needed by consumers like KVM's
master clock and the vmclock PTP driver.
[ tglx: Reworked it slightly ]
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Assisted-by: Kiro:claude-opus-4.6-1m
Link: https://patch.msgid.link/20260526230635.136914-1-dwmw2@infradead.org
Link: https://patch.msgid.link/20260529195558.202568489@kernel.org
Now that all prerequisites are in place add the final support for AUX
clocks in get_device_system_crosststamp(), which enables the PTP layer to
support hardware cross timestamps with a new IOTCL.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Arthur Kiyanovski <akiyano@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260529195558.097464513@kernel.org
PTP device system crosstime stamps support only CLOCK_REALTIME, which is
meaningless for AUX clocks. The PTP core hands in the clock ID already, so
prepare the core code to honor it.
- Add a new sys_systime field to struct system_device_crosststamp which
aliases the sys_realtime field. Once all users are converted
sys_realtime can be removed.
- Prepare get_device_system_crosststamp() and the related code for it by
switching to sys_systime and providing the initial changes to utilize
different time keepers.
No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: David Woodhouse <dwmw@amazon.co.uk>
Tested-by: Arthur Kiyanovski <akiyano@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260529195557.846634842@kernel.org
An upcoming extension to the PTP IOCTL requires to return the system counter
value and the clocksource ID to user space. get_device_system_crosststamp() has
this information already.
Extend struct system_device_crosststamp with a system_counterval_t member
and fill in the data.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: David Woodhouse <dwmw@amazon.co.uk>
Tested-by: Arthur Kiyanovski <akiyano@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260529195557.429406675@kernel.org
Now that all users are converted it's possible to enable snapshotting of
CLOCK_AUX time. The underlying clocksource is the same as for all other
CLOCK variants.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Arthur Kiyanovski <akiyano@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260529195557.380601005@kernel.org
All users are converted over to ktime_get_snapshot_id() and
system_time_snapshot::systime and ::monoraw.
Remove the leftovers.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: David Woodhouse <dwmw@amazon.co.uk>
Tested-by: Arthur Kiyanovski <akiyano@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260529195557.330029635@kernel.org
The level_list array is allocated separately right after the parent
struct. The size of the array is already known.
Move level_list to the struct tail as a flexible array member and fold the
two allocations into a single kzalloc_flex().
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Claude:Opus-4.7
Link: https://patch.msgid.link/20260522231618.41622-1-rosenp@gmail.com
NOHZ_FULL CPUs global timers are guaranteed to be handled by the timekeeper
CPU, which never stops its tick and therefore remains active in the
hierarchy.
But since the introduction of per-capacity hierarchies, this guarantee is
broken because the timekeeper may not belong to the same hierarchy as all
the NOHZ_FULL CPUs.
Fix it with simply turning off capacity awareness when NOHZ_FULL is
running and force a single hierarchy. NOHZ_FULL is not exactly optimized
powerwise anyway.
Fixes: 098cbaad8e ("timers/migration: Split per-capacity hierarchies")
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260519220926.63437-3-frederic@kernel.org
When a top-level migrator is deactivated, either at CPU down hotplug time
or when a CPU is domain isolated, a new migrator is elected among the
available CPUs and woken up to take over the migration duty.
However that election must happen at the scope of a given hierarchy and not
globally, which the introduction of per-capacity hierarchies failed to
handle.
As a result a given hierarchy may end up without migrator to handle global
timers.
Fix it by making sure that the new migrator belongs to the same hierarchy
as the outgoing CPU.
Fixes: 098cbaad8e ("timers/migration: Split per-capacity hierarchies")
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260519220926.63437-2-frederic@kernel.org
There is no real point in switching to dyntick-idle cputime accounting mode
if the tick is not actually stopped. This just adds overhead, notably
fetching the GTOD, on each idle exit and each idle IRQ entry for no reason
during short idle trips.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20260508131647.43868-12-frederic@kernel.org
Although the dynticks-idle cputime accounting is necessarily tied to the
tick subsystem, the actual related accounting code has no business residing
there and should be part of the scheduler cputime code.
Move away the relevant pieces and state machine to where they belong.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20260508131647.43868-10-frederic@kernel.org
Even when nohz is not runtime enabled, the dynticks idle cputime accounting
can run and the common idle cputime accessors are still relevant.
Remove the nohz disabled special case accordingly.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20260508131647.43868-9-frederic@kernel.org
The non-vtime dynticks-idle cputime accounting is a big mess that
accumulates within two concurrent statistics, each having their own
shortcomings:
* The accounting for online CPUs which is based on the delta between
tick_nohz_start_idle() and tick_nohz_stop_idle().
Pros:
- Works when the tick is off
- Has nsecs granularity
Cons:
- Account idle steal time but doesn't substract it from idle
cputime.
- Assumes CONFIG_IRQ_TIME_ACCOUNTING by not accounting IRQs but
the IRQ time is simply ignored when
CONFIG_IRQ_TIME_ACCOUNTING=n
- The windows between 1) idle task scheduling and the first call
to tick_nohz_start_idle() and 2) idle task between the last
tick_nohz_stop_idle() and the rest of the idle time are
blindspots wrt. cputime accounting (though mostly insignificant
amount)
- Relies on private fields outside of kernel stats, with specific
accessors.
* The accounting for offline CPUs which is based on ticks and the
jiffies delta during which the tick was stopped.
Pros:
- Handles steal time correctly
- Handle CONFIG_IRQ_TIME_ACCOUNTING=y and
CONFIG_IRQ_TIME_ACCOUNTING=n correctly.
- Handles the whole idle task
- Accounts directly to kernel stats, without midlayer accumulator.
Cons:
- Doesn't elapse when the tick is off, which doesn't make it
suitable for online CPUs.
- Has TICK_NSEC granularity (jiffies)
- Needs to track the dyntick-idle ticks that were accounted and
substract them from the total jiffies time spent while the tick
was stopped. This is an ugly workaround.
Having two different accounting for a single context is not the only
problem: since those accountings are of different natures, it is
possible to observe the global idle time going backward after a CPU goes
offline.
Clean up the situation with introducing a hybrid approach that stays
coherent and works for both online and offline CPUs:
* Tick based or native vtime accounting operate before the idle loop
is entered and resume once the idle loop prepares to exit.
* When the idle loop starts, switch to dynticks-idle accounting as is
done currently, except that the statistics accumulate directly to the
relevant kernel stat fields.
* Private dyntick cputime accounting fields are removed.
* Works on both online and offline case.
Further improvement will include:
* Only switch to dynticks-idle cputime accounting when the tick actually
goes in dynticks mode.
* Handle CONFIG_IRQ_TIME_ACCOUNTING=n correctly such that the
dynticks-idle accounting still elapses while on IRQs.
* Correctly substract idle steal cputime from idle time
Reported-by: Xin Zhao <jackzxcui1989@163.com>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20260508131647.43868-8-frederic@kernel.org
Currently whether generic vtime is running or not, the idle cputime is
fetched from the nohz accounting.
However generic vtime already does its own idle cputime accounting. Only
the kernel stat accessors are not plugged to support it.
Read the idle generic vtime cputime when it's running, this will allow to
later more clearly split nohz and vtime cputime accounting.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20260508131647.43868-5-frederic@kernel.org
When the nohz idle time is fetched, the current clock timestamp is taken
outside the seqcount, which can result in a race as reported by Sashiko:
get_cpu_sleep_time_us() tick_nohz_start_idle()
----------------------- ---------------------
now = ktime_get()
write_seqcount_begin(idle_sleeptime_seq);
idle_entrytime = ktime_get()
tick_sched_flag_set(ts, TS_FLAG_IDLE_ACTIVE);
write_seqcount_end(&ts->idle_sleeptime_seq);
read_seqcount_begin(idle_sleeptime_seq)
delta = now - idle_entrytime);
//!! But now < idle_entrytime
idle = *sleeptime + delta;
read_seqcount_retry(&ts->idle_sleeptime_seq, seq)
Here the read side fetches the timestamp before the write side and its
update. As a result the time delta computed on the read side is negative
(ktime_t is signed) and breaks the cputime monotonicity guarantee.
This could possibly be fixed with reading the current clock timestamp
inside the seqcount but the reader overhead might then increase. Also
simply checking that the current timestamp is above the idle entry time
is enough to prevent any issue of the like.
Fixes: 620a30fa0b ("timers/nohz: Protect idle/iowait sleep time under seqcount")
Reported-by: Sashiko
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260508131647.43868-2-frederic@kernel.org
The validation check uses '>' instead of '>=' when comparing tv_usec
against USEC_PER_SEC, allowing the value 1000000 through. After
conversion to nanoseconds (*= 1000), this produces tv_nsec ==
NSEC_PER_SEC, violating the timespec invariant that tv_nsec must be
less than NSEC_PER_SEC.
Use '>=' to reject tv_usec values that are not in the valid range of
0 to 999999.
Fixes: 5e0fb1b57b ("y2038: time: avoid timespec usage in settimeofday()")
Signed-off-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/4rikk44zew3s6577dugmx4jyblz7o5c57niuap6ct3td5yfm6w@gh7pcumg7qor
The stub for arch_inlined_clockevent_set_next_coupled() has 'u64 u64
cycles' in its parameter list. Since u64 is a typedef, the compiler
parses the second 'u64' as the parameter name, making 'cycles' an
unused token. Remove the duplicate so the parameter is correctly named.
Fixes: 89f951a1e8 ("clockevents: Provide support for clocksource coupled comparators")
Signed-off-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/7tostpvxzdn6tobmyow63a5rweatls5kux3scqp2vzhe7mv6uq@ecr746b4hyhf
timens_ktime_to_host() in compares the current time namespace against
init_time_ns for the fast path. It calls do_timens_ktime_to_host() for the
offset case. Both symbols are needed at link time by any caller of the
inline.
All current callers are builtin, but ntsync can be built as module, which
prevents it from using it.
Export both with EXPORT_SYMBOL_GPL.
Signed-off-by: Maoyi Xie <maoyixie.tju@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260528063311.3300393-2-maoyixie.tju@gmail.com
system_time_snapshot::systime provides the same information as
system_time_snapshot::real when the snapshot was taken with
ktime_get_snapshot_id(CLOCK_REALTIME).
Convert the history interpolation over to use 'systime' and 'monoraw' as
'real/raw' are going away once all users are converted.
As a side effect this is the first step to support CLOCK_AUX with
get_device_crosstime_stamp() and the history interpolation.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: David Woodhouse <dwmw@amazon.co.uk>
Tested-by: Arthur Kiyanovski <akiyano@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260529195557.024415766@kernel.org
ktime_get_snapshot() provides a snapshot of the underlying clocksource
counter value and the corresponding CLOCK_MONOTONIC_RAW, CLOCK_REALTIME and
CLOCK_BOOTTIME timestamps.
There is no usage of CLOCK_REALTIME and CLOCK_BOOTTIME at the same time and
CLOCK_BOOTTIME support was just added for the ARM64 KVM tracing mechanism,
which needs CLOCK_BOOTTIME and the underlying clocksource counter value.
ktime_get_snapshot() is also not suitable for usage with CLOCK_AUX, but
that's a prerequisite to support PTP hardware timestamping for CLOCK_AUX
steering.
As a first step, rename ktime_get_snapshot() to ktime_get_snapshot_id(),
which now takes a clockid argument to select the clock which needs to be
captured. The result is stored in system_time_snapshot::systime, which will
replace the system_time_snapshot::real/boot members once all usage sites
have been converted.
ktime_get_snapshot() is a simple wrapper which hands in CLOCK_REALTIME as
clockid argument for the conversion period. That means CLOCK_REALTIME is
now captured twice, but that redunancy is only temporary.
As all usage sites of struct system_time_snapshot has to be updated anyway,
rename the 'raw' member to 'monoraw' for clarity.
No functional change vs. current users of ktime_get_snapshot()
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: David Woodhouse <dwmw@amazon.co.uk>
Tested-by: Arthur Kiyanovski <akiyano@amazon.com>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260529195556.971591633@kernel.org
Commit 8312cab5ff ("timers/migration: Rename 'online' bit to
'available'") renamed the 'online' field of struct tmigr_cpu to
'available'. The kernel doc comment above the struct still describes the
old field name.
Update it to reflect the actual field name and use the 'available' wording
in the description.
Fixes: 8312cab5ff ("timers/migration: Rename 'online' bit to 'available'")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260526022106.1302279-1-zhanxusheng@xiaomi.com
Introduce device-managed helpers for clocksource registration.
The clocksource framework currently provides __clocksource_register_scale()
along with convenience wrappers for Hz and kHz registration. However,
drivers must handle error paths and cleanup manually, typically by pairing
registration with an explicit clocksource_unregister() call.
Add a devm-based variant, __devm_clocksource_register_scale(), along with
devm_clocksource_register_hz() and devm_clocksource_register_khz() helpers.
These helpers register the clocksource and attach a devres action to
automatically unregister it on driver detach or probe failure.
This simplifies driver code by:
* removing explicit cleanup paths
* ensuring correct teardown ordering
* aligning with the devm-based resource management model widely used
across the kernel
While drivers can open-code devm_add_action_or_reset(), providing a
dedicated helper avoids duplication, reduces boilerplate, and ensures
consistent usage across drivers, following patterns used in other
subsystems.
This is also particularly useful for drivers built as modules, where
device-managed resource handling avoids manual cleanup in remove paths and
ensures correct teardown on module unload.
This helper is self-contained and can be adopted progressively by drivers.
No functional change.
Signed-off-by: Daniel Lezcano <daniel.lezcano@oss.qualcomm.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260506153831.605159-1-daniel.lezcano@oss.qualcomm.com
__hrtimer_start_range_ns() has a bool return type, but returns actually
three different values, which are checked at the call site.
Make the return type int.
Fixes: bd5956166d ("hrtimer: Provide hrtimer_start_range_ns_user()")
Reported-by: Dan Carpenter <error27@gmail.com
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Remove the unused functions __clocksource_update_freq_hz() and
__clocksource_update_freq_khz().
Then make __clocksource_update_freq_scale() static as it is not used
from external callers anymore. Also clean up the comment accordingly.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260504-clocksource-update_freq-v2-1-3e696fb01776@linutronix.de
alarm_handle_timer() was converted from returning enum alarmtimer_restart
to void, but the kernel-doc "Return:" line was not removed. Remove the
stale description.
Fixes: 2634303f87 ("alarmtimers: Remove return value from alarm functions")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260429080635.166790-1-zhanxusheng@xiaomi.com
Systems with heterogeneous CPU capacities, such as big.LITTLE, have
reported power issues since the introduction of the new timer migration
code.
Timers migrate from small capacity CPUs to big ones, degrading their
target residency and thus overall power consumption.
Solve this with splitting hierarchies per CPU capacity. For example in
a big.LITTLE machine, split a single hierarchy in two: one for big
capacity CPUs and another one for small capacity CPUs. This way global
timers only migrate across CPUs of the same capacity.
For simplicity purpose, split hierarchies keep the same number of
possible levels as if there were a single hierarchy, even though the
CPUs are distributed between multiple hierarchies. This could be a
problem on NUMA systems with heterogeneous CPU capacities (provided that
ever exists yet) where useless intermediate nodes may be created.
Solving this properly will imply on boot to know in advance how many
capacities are available and the number of CPUs for each of them.
Reported-by: Sehee Jeong <sehee1.jeong@samsung.com>
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260423165354.95152-5-frederic@kernel.org
When a new root is created, the old root is connected to it and
propagates up its own assumed to be active state, since the hotplug
control CPU is itself active and part of the old root.
However with per-capacity hierarchies, this assumption won't be true
anymore because the hotplug control CPU calling the timer migration
prepare callback may not belong to the same hierarchy as the booting
CPU.
To solve this, track the available CPUs per hierarchies so that the
root connection can be offlined to safe CPUs.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260423165354.95152-4-frederic@kernel.org
In order to prepare for separating out CPUs from different capacities in
distinct hierarchies, create a hierarchy structure that group setup
must rely upon.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260423165354.95152-3-frederic@kernel.org
The hotplug control CPU is assumed to be active in the hierarchy but
that doesn't imply that the root is active. If the current CPU is not
the one that activated the current hierarchy, and the CPU performing
this duty is still halfway through the tree, the root may still be
observed inactive. And this can break the activation of a new root as in
the following scenario:
1) Initially, the whole system has 64 CPUs and only CPU 63 is awake.
[GRP1:0]
active
/ | \
/ | \
[GRP0:0] [...] [GRP0:7]
idle idle active
/ | \ |
CPU 0 CPU 1 ... CPU 63
idle idle active
2) CPU 63 goes idle _but_ due to a #VMEXIT it hasn't yet reached the
[GRP1:0]->parent dereference (that would be NULL and stop the walk)
in __walk_groups_from().
[GRP1:0]
idle
/ | \
/ | \
[GRP0:0] [...] [GRP0:7]
idle idle idle
/ | \ |
CPU 0 CPU 1 ... CPU 63
idle idle idle
3) CPU 1 wakes up, activates GRP0:0 but didn't yet manage to propagate
up to GRP1:0 due to yet another #VMEXIT.
[GRP1:0]
idle
/ | \
/ | \
[GRP0:0] [...] [GRP0:7]
active idle idle
/ | \ |
CPU 0 CPU 1 ... CPU 63
idle active idle
3) CPU 0 wakes up and doesn't need to walk above GRP0:0 as it's CPU 1
role.
[GRP1:0]
idle
/ | \
/ | \
[GRP0:0] [...] [GRP0:7]
active idle idle
/ | \ |
CPU 0 CPU 1 ... CPU 63
active active idle
4) CPU 0 boots CPU 64. It creates a new root for it.
[GRP2:0]
idle
/ \
/ \
[GRP1:0] [GRP1:1]
idle idle
/ | \ \
/ | \ \
[GRP0:0] [...] [GRP0:7] [GRP0:8]
active idle idle idle
/ | \ | |
CPU 0 CPU 1 ... CPU 63 CPU 64
active active idle offline
5) CPU 0 activates the new root, but note that GRP1:0 is still idle,
waiting for CPU 1 to resume from #VMEXIT and activate it.
[GRP2:0]
active
/ \
/ \
[GRP1:0] [GRP1:1]
idle idle
/ | \ \
/ | \ \
[GRP0:0] [...] [GRP0:7] [GRP0:8]
active idle idle idle
/ | \ | |
CPU 0 CPU 1 ... CPU 63 CPU 64
active active idle offline
6) CPU 63 resumes after #VMEXIT and sees the new GRP1:0 parent.
Therefore it propagates the stale inactive state of GRP1:0 up to
GRP2:0.
[GRP2:0]
idle
/ \
/ \
[GRP1:0] [GRP1:1]
idle idle
/ | \ \
/ | \ \
[GRP0:0] [...] [GRP0:7] [GRP0:8]
active idle idle idle
/ | \ | |
CPU 0 CPU 1 ... CPU 63 CPU 64
active active idle offline
7) CPU 1 resumes after #VMEXIT and finally activates GRP1:0. But it
doesn't observe its parent link because no ordering enforced that.
Therefore GRP2:0 is spuriously left idle.
[GRP2:0]
idle
/ \
/ \
[GRP1:0] [GRP1:1]
active idle
/ | \ \
/ | \ \
[GRP0:0] [...] [GRP0:7] [GRP0:8]
active idle idle idle
/ | \ | |
CPU 0 CPU 1 ... CPU 63 CPU 64
active active idle offline
Such races are highly theoretical and the problem would solve itself
once the old root ever becomes idle again. But it still leaves a taste
of discomfort.
Fix it with enforcing a fully ordered atomic read of the old root state
before propagating the activate state up to the new root. It has a two
directions ordering effect:
* Acquire + release of the latest old root state: If the hotplug control
CPU is not the one that woke up the old root, make sure to acquire its
active state and propagate it upwards through the ordered chain of
activation (the acquire pairs with the cmpxchg() in tmigr_active_up()
and subsequent releases will pair with atomic_read_acquire() and
smp_mb__after_atomic() in tmigr_inactive_up()).
* Release: If the hotplug control CPU is not the one that must wake up
the old root, but the CPU covering that is lagging behind its duty,
publish the links from the old root to the new parents. This way the
lagging CPU will propagate the active state itself.
Fixes: 7ee9887703 ("timers: Implement the hierarchical pull model")
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260423165354.95152-2-frederic@kernel.org
All alarmtimer users are converted to alarm_start_timer(). Remove the now
unused interfaces.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260408114952.670899355@kernel.org
Use the new alarm_start_timer() for arming and rearming posix interval
timers and for clock_nanosleep() so that already expired timers do not go
through the full timer interrupt cycle.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260408114952.400451460@kernel.org
Alarm timers utilize hrtimers for normal operation and only switch to the
RTC on suspend. In order to catch already expired timers early and without
going through a timer interrupt cycle, provide a new start function which
internally uses hrtimer_start_range_ns_user().
If hrtimer_start_range_ns_user() detects an already expired timer, it does
not queue it. In that case remove the timer from the alarm base as well.
Return the status queued or not back to the caller to handle the early
expiry.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260408114952.332822525@kernel.org
Switch the arm and rearm callbacks for hrtimer based posix timers over to
hrtimer_start_expires_user() so that already expired timers are not
queued. Hand the result back to the caller, which then queues the signal.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260408114952.266001916@kernel.org
The [re]arm callbacks will return true when the timer was queued and false
if it was already expired at enqueue time.
In both cases the call sites can trivially queue the signal right there,
when the timer was already expired. That avoids a full round trip through
the hrtimer interrupt.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260408114952.198028466@kernel.org
In order to catch expiry times which are already in the past the
timer_arm() and timer_rearm() callbacks need to be able to report back to
the caller whether the timer has been queued or not.
Change the function signature and let all implementations return true for
now. While at it simplify posix_cpu_timer_rearm().
No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260408114952.130222296@kernel.org
Most hrtimer sleepers are user controlled and user space can hand arbitrary
expiry values in as long as they are valid timespecs. If the expiry value
is in the past then this requires a full loop through reprogramming the
clock event device, taking the hrtimer interrupt, waking the task and
reprogram again.
Use hrtimer_start_expires_user() which avoids the full round trip by
checking the timer for expiry on enqueue.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Calvin Owens <calvin@wbinvd.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260408114952.062400833@kernel.org
Calvin reported an odd NMI watchdog lockup which claims that the CPU locked
up in user space. He provided a reproducer, which set's up a timerfd based
timer and then rearms it in a loop with an absolute expiry time of 1ns.
As the expiry time is in the past, the timer ends up as the first expiring
timer in the per CPU hrtimer base and the clockevent device is programmed
with the minimum delta value. If the machine is fast enough, this ends up
in a endless loop of programming the delta value to the minimum value
defined by the clock event device, before the timer interrupt can fire,
which starves the interrupt and consequently triggers the lockup detector
because the hrtimer callback of the lockup mechanism is never invoked.
The clockevents code already has a last resort mechanism to prevent that,
but it's sensible to catch such issues before trying to reprogram the clock
event device.
Provide a variant of hrtimer_start_range_ns(), which sanity checks the
timer after queueing it. It does not so before because the timer might be
armed and therefore needs to be dequeued. also we optimize for the latest
possible point to check, so that the clock event prevention is avoided as
much as possible.
If the timer is already expired _before_ the clock event is reprogrammed,
remove the timer from the queue and signal to the caller that the operation
failed by returning false.
That allows the caller to take immediate action without going through the
loops and hoops of the hrtimer interrupt.
The queueing code can't invoke the timer callback as the caller might hold
a lock which is taken in the callback.
Add a tracepoint which allows to analyze the expired at start situation.
Reported-by: Calvin Owens <calvin@wbinvd.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Calvin Owens <calvin@wbinvd.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260408114951.995031895@kernel.org
The prevention mechanism against timer interrupt starvation missed to reset
the next_event_forced flag in a couple of places:
- When the clock event state changes. That can cause the flag to be
stale over a shutdown/startup sequence
- When a non-forced event is armed, which then prevents rearming before
that event. If that event is far out in the future this will cause
missed timer interrupts.
- In the suspend wakeup handler.
That led to stalls which have been reported by several people.
Add the missing resets, which fixes the problems for the reporters.
Fixes: d6e152d905 ("clockevents: Prevent timer interrupt starvation")
Reported-by: Hanabishi <i.r.e.c.c.a.k.u.n+kernel.org@gmail.com>
Reported-by: Eric Naim <dnaim@cachyos.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Hanabishi <i.r.e.c.c.a.k.u.n+kernel.org@gmail.com>
Tested-by: Eric Naim <dnaim@cachyos.org>
Cc: stable@vger.kernel.org
Closes: https://lore.kernel.org/68d1e9ac-2780-4be3-8ee3-0788062dd3a4@gmail.com
Link: https://patch.msgid.link/87340xfeje.ffs@tglx
Everything:
Total patches: 368
Reviews/patch: 1.56
Reviewed rate: 74%
Excluding DAMON:
Total patches: 316
Reviews/patch: 1.77
Reviewed rate: 81%
Excluding DAMON and zram:
Total patches: 306
Reviews/patch: 1.81
Reviewed rate: 82%
Excluding DAMON, zram and maple_tree:
Total patches: 276
Reviews/patch: 2.01
Reviewed rate: 91%
Significant patch series in this merge:
- The 30 patch series "maple_tree: Replace big node with maple copy"
from Liam Howlett is mainly prepararatory work for ongoing development
but it does reduce stack usage and is an improvement.
- The 12 patch series "mm, swap: swap table phase III: remove swap_map"
from Kairui Song offers memory savings by removing the static swap_map.
It also yields some CPU savings and implements several cleanups.
- The 2 patch series "mm: memfd_luo: preserve file seals" from Pratyush
Yadav adds file seal preservation to LUO's memfd code.
- The 2 patch series "mm: zswap: add per-memcg stat for incompressible
pages" from Jiayuan Chen adds additional userspace stats reportng to
zswap.
- The 4 patch series "arch, mm: consolidate empty_zero_page" from Mike
Rapoport implements some cleanups for our handling of ZERO_PAGE() and
zero_pfn.
- The 2 patch series "mm/kmemleak: Improve scan_should_stop()
implementation" from Zhongqiu Han provides an robustness improvement and
some cleanups in the kmemleak code.
- The 4 patch series "Improve khugepaged scan logic" from Vernon Yang
"improves the khugepaged scan logic and reduces CPU consumption by
prioritizing scanning tasks that access memory frequently".
- The 2 patch series "Make KHO Stateless" from Jason Miu simplifies
Kexec Handover by "transitioning KHO from an xarray-based metadata
tracking system with serialization to a radix tree data structure that
can be passed directly to the next kernel"
- The 3 patch series "mm: vmscan: add PID and cgroup ID to vmscan
tracepoints" from Thomas Ballasi and Steven Rostedt enhances vmscan's
tracepointing.
- The 5 patch series "mm: arch/shstk: Common shadow stack mapping helper
and VM_NOHUGEPAGE" from Catalin Marinas is a cleanup for the shadow
stack code: remove per-arch code in favour of a generic implementation.
- The 2 patch series "Fix KASAN support for KHO restored vmalloc
regions" from Pasha Tatashin fixes a WARN() which can be emitted the KHO
restores a vmalloc area.
- The 4 patch series "mm: Remove stray references to pagevec" from Tal
Zussman provides several cleanups, mainly udpating references to "struct
pagevec", which became folio_batch three years ago.
- The 17 patch series "mm: Eliminate fake head pages from vmemmap
optimization" from Kiryl Shutsemau simplifies the HugeTLB vmemmap
optimization (HVO) by changing how tail pages encode their relationship
to the head page.
- The 2 patch series "mm/damon/core: improve DAMOS quota efficiency for
core layer filters" from SeongJae Park improves two problematic
behaviors of DAMOS that makes it less efficient when core layer filters
are used.
- The 3 patch series "mm/damon: strictly respect min_nr_regions" from
SeongJae Park improves DAMON usability by extending the treatment of the
min_nr_regions user-settable parameter.
- The 3 patch series "mm/page_alloc: pcp locking cleanup" from Vlastimil
Babka is a proper fix for a previously hotfixed SMP=n issue. Code
simplifications and cleanups ennsed.
- The 16 patch series "mm: cleanups around unmapping / zapping" from
David Hildenbrand implements "a bunch of cleanups around unmapping and
zapping. Mostly simplifications, code movements, documentation and
renaming of zapping functions".
- The 6 patch series "support batched checking of the young flag for
MGLRU" from Baolin Wang supports batched checking of the young flag for
MGLRU. It's part cleanups; one benchmark shows large performance
benefits for arm64.
- The 5 patch series "memcg: obj stock and slab stat caching cleanups"
from Johannes Weiner provides memcg cleanup and robustness improvements.
- The 5 patch series "Allow order zero pages in page reporting" from
Yuvraj Sakshith enhances page_reporting's free page reporting - it is
presently and undesirably order-0 pages when reporting free memory.
- The 6 patch series "mm: vma flag tweaks" from Lorenzo Stoakes is
cleanup work following from the recent conversion of the VMA flags to a
bitmap.
- The 10 patch series "mm/damon: add optional debugging-purpose sanity
checks" from SeongJae Park adds some more developer-facing debug checks
into DAMON core.
- The 2 patch series "mm/damon: test and document power-of-2
min_region_sz requirement" from SeongJae Park adds an additional DAMON
kunit test and makes some adjustments to the addr_unit parameter
handling.
- The 3 patch series "mm/damon/core: make passed_sample_intervals
comparisons overflow-safe" from SeongJae Park fixes a hard-to-hit time
overflow issue in DAMON core.
- The 7 patch series "mm/damon: improve/fixup/update ratio calculation,
test and documentation" from SeongJae Park is a "batch of misc/minor
improvements and fixups" for DAMON.
- The 4 patch series "mm: move vma_(kernel|mmu)_pagesize() out of
hugetlb.c" from David Hildenbrand fixes a possible issue with dax-device
when CONFIG_HUGETLB=n. Some code movement was required.
- The 6 patch series "zram: recompression cleanups and tweaks" from
Sergey Senozhatsky provides "a somewhat random mix of fixups,
recompression cleanups and improvements" in the zram code.
- The 11 patch series "mm/damon: support multiple goal-based quota
tuning algorithms" from SeongJae Park extend DAMOS quotas goal
auto-tuning to support multiple tuning algorithms that users can select.
- The 4 patch series "mm: thp: reduce unnecessary
start_stop_khugepaged()" from Breno Leitao fixes the khugpaged sysfs
handling so we no longer spam the logs with reams of junk when
starting/stopping khugepaged.
- The 3 patch series "mm: improve map count checks" from Lorenzo Stoakes
provides some cleanups and slight fixes in the mremap, mmap and vma
code.
- The 5 patch series "mm/damon: support addr_unit on default monitoring
targets for modules" from SeongJae Park extends the use of DAMON core's
addr_unit tunable.
- The 5 patch series "mm: khugepaged cleanups and mTHP prerequisites"
from Nico Pache provides cleanups in the khugepaged and is a base for
Nico's planned khugepaged mTHP support.
- The 15 patch series "mm: memory hot(un)plug and SPARSEMEM cleanups"
from David Hildenbrand implements code movement and cleanups in the
memhotplug and sparsemem code.
- The 2 patch series "mm: remove CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE and
cleanup CONFIG_MIGRATION" from David Hildenbrand rationalizes some
memhotplug Kconfig support.
- The 6 patch series "change young flag check functions to return bool"
from Baolin Wang is "a cleanup patchset to change all young flag check
functions to return bool".
- The 3 patch series "mm/damon/sysfs: fix memory leak and NULL
dereference issues" from Josh Law and SeongJae Park fixes a few
potential DAMON bugs.
- The 25 patch series "mm/vma: convert vm_flags_t to vma_flags_t in vma
code" from "converts a lot of the existing use of the legacy vm_flags_t
data type to the new vma_flags_t type which replaces it". Mainly in the
vma code.
- The 21 patch series "mm: expand mmap_prepare functionality and usage"
from Lorenzo Stoakes "expands the mmap_prepare functionality, which is
intended to replace the deprecated f_op->mmap hook which has been the
source of bugs and security issues for some time". Cleanups,
documentation, extension of mmap_prepare into filesystem drivers.
- The 13 patch series "mm/huge_memory: refactor zap_huge_pmd()" from
Lorenzo Stoakes simplifies and cleans up zap_huge_pmd(). Additional
cleanups around vm_normal_folio_pmd() and the softleaf functionality are
performed.
-----BEGIN PGP SIGNATURE-----
iHUEABYIAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCad3HDQAKCRDdBJ7gKXxA
jrUQAPwNhPk5nPSxnyxjAeQtOBHqgCdnICeEismLajPKd9aYRgEA0s2XAu3tSUYi
GrBnWImHG3s4ePQxVcPCegWTsOUrXgQ=
=1Q7o
-----END PGP SIGNATURE-----
Merge tag 'mm-stable-2026-04-13-21-45' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM updates from Andrew Morton:
- "maple_tree: Replace big node with maple copy" (Liam Howlett)
Mainly prepararatory work for ongoing development but it does reduce
stack usage and is an improvement.
- "mm, swap: swap table phase III: remove swap_map" (Kairui Song)
Offers memory savings by removing the static swap_map. It also yields
some CPU savings and implements several cleanups.
- "mm: memfd_luo: preserve file seals" (Pratyush Yadav)
File seal preservation to LUO's memfd code
- "mm: zswap: add per-memcg stat for incompressible pages" (Jiayuan
Chen)
Additional userspace stats reportng to zswap
- "arch, mm: consolidate empty_zero_page" (Mike Rapoport)
Some cleanups for our handling of ZERO_PAGE() and zero_pfn
- "mm/kmemleak: Improve scan_should_stop() implementation" (Zhongqiu
Han)
A robustness improvement and some cleanups in the kmemleak code
- "Improve khugepaged scan logic" (Vernon Yang)
Improve khugepaged scan logic and reduce CPU consumption by
prioritizing scanning tasks that access memory frequently
- "Make KHO Stateless" (Jason Miu)
Simplify Kexec Handover by transitioning KHO from an xarray-based
metadata tracking system with serialization to a radix tree data
structure that can be passed directly to the next kernel
- "mm: vmscan: add PID and cgroup ID to vmscan tracepoints" (Thomas
Ballasi and Steven Rostedt)
Enhance vmscan's tracepointing
- "mm: arch/shstk: Common shadow stack mapping helper and
VM_NOHUGEPAGE" (Catalin Marinas)
Cleanup for the shadow stack code: remove per-arch code in favour of
a generic implementation
- "Fix KASAN support for KHO restored vmalloc regions" (Pasha Tatashin)
Fix a WARN() which can be emitted the KHO restores a vmalloc area
- "mm: Remove stray references to pagevec" (Tal Zussman)
Several cleanups, mainly udpating references to "struct pagevec",
which became folio_batch three years ago
- "mm: Eliminate fake head pages from vmemmap optimization" (Kiryl
Shutsemau)
Simplify the HugeTLB vmemmap optimization (HVO) by changing how tail
pages encode their relationship to the head page
- "mm/damon/core: improve DAMOS quota efficiency for core layer
filters" (SeongJae Park)
Improve two problematic behaviors of DAMOS that makes it less
efficient when core layer filters are used
- "mm/damon: strictly respect min_nr_regions" (SeongJae Park)
Improve DAMON usability by extending the treatment of the
min_nr_regions user-settable parameter
- "mm/page_alloc: pcp locking cleanup" (Vlastimil Babka)
The proper fix for a previously hotfixed SMP=n issue. Code
simplifications and cleanups ensued
- "mm: cleanups around unmapping / zapping" (David Hildenbrand)
A bunch of cleanups around unmapping and zapping. Mostly
simplifications, code movements, documentation and renaming of
zapping functions
- "support batched checking of the young flag for MGLRU" (Baolin Wang)
Batched checking of the young flag for MGLRU. It's part cleanups; one
benchmark shows large performance benefits for arm64
- "memcg: obj stock and slab stat caching cleanups" (Johannes Weiner)
memcg cleanup and robustness improvements
- "Allow order zero pages in page reporting" (Yuvraj Sakshith)
Enhance free page reporting - it is presently and undesirably order-0
pages when reporting free memory.
- "mm: vma flag tweaks" (Lorenzo Stoakes)
Cleanup work following from the recent conversion of the VMA flags to
a bitmap
- "mm/damon: add optional debugging-purpose sanity checks" (SeongJae
Park)
Add some more developer-facing debug checks into DAMON core
- "mm/damon: test and document power-of-2 min_region_sz requirement"
(SeongJae Park)
An additional DAMON kunit test and makes some adjustments to the
addr_unit parameter handling
- "mm/damon/core: make passed_sample_intervals comparisons
overflow-safe" (SeongJae Park)
Fix a hard-to-hit time overflow issue in DAMON core
- "mm/damon: improve/fixup/update ratio calculation, test and
documentation" (SeongJae Park)
A batch of misc/minor improvements and fixups for DAMON
- "mm: move vma_(kernel|mmu)_pagesize() out of hugetlb.c" (David
Hildenbrand)
Fix a possible issue with dax-device when CONFIG_HUGETLB=n. Some code
movement was required.
- "zram: recompression cleanups and tweaks" (Sergey Senozhatsky)
A somewhat random mix of fixups, recompression cleanups and
improvements in the zram code
- "mm/damon: support multiple goal-based quota tuning algorithms"
(SeongJae Park)
Extend DAMOS quotas goal auto-tuning to support multiple tuning
algorithms that users can select
- "mm: thp: reduce unnecessary start_stop_khugepaged()" (Breno Leitao)
Fix the khugpaged sysfs handling so we no longer spam the logs with
reams of junk when starting/stopping khugepaged
- "mm: improve map count checks" (Lorenzo Stoakes)
Provide some cleanups and slight fixes in the mremap, mmap and vma
code
- "mm/damon: support addr_unit on default monitoring targets for
modules" (SeongJae Park)
Extend the use of DAMON core's addr_unit tunable
- "mm: khugepaged cleanups and mTHP prerequisites" (Nico Pache)
Cleanups to khugepaged and is a base for Nico's planned khugepaged
mTHP support
- "mm: memory hot(un)plug and SPARSEMEM cleanups" (David Hildenbrand)
Code movement and cleanups in the memhotplug and sparsemem code
- "mm: remove CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE and cleanup
CONFIG_MIGRATION" (David Hildenbrand)
Rationalize some memhotplug Kconfig support
- "change young flag check functions to return bool" (Baolin Wang)
Cleanups to change all young flag check functions to return bool
- "mm/damon/sysfs: fix memory leak and NULL dereference issues" (Josh
Law and SeongJae Park)
Fix a few potential DAMON bugs
- "mm/vma: convert vm_flags_t to vma_flags_t in vma code" (Lorenzo
Stoakes)
Convert a lot of the existing use of the legacy vm_flags_t data type
to the new vma_flags_t type which replaces it. Mainly in the vma
code.
- "mm: expand mmap_prepare functionality and usage" (Lorenzo Stoakes)
Expand the mmap_prepare functionality, which is intended to replace
the deprecated f_op->mmap hook which has been the source of bugs and
security issues for some time. Cleanups, documentation, extension of
mmap_prepare into filesystem drivers
- "mm/huge_memory: refactor zap_huge_pmd()" (Lorenzo Stoakes)
Simplify and clean up zap_huge_pmd(). Additional cleanups around
vm_normal_folio_pmd() and the softleaf functionality are performed.
* tag 'mm-stable-2026-04-13-21-45' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (369 commits)
mm: fix deferred split queue races during migration
mm/khugepaged: fix issue with tracking lock
mm/huge_memory: add and use has_deposited_pgtable()
mm/huge_memory: add and use normal_or_softleaf_folio_pmd()
mm: add softleaf_is_valid_pmd_entry(), pmd_to_softleaf_folio()
mm/huge_memory: separate out the folio part of zap_huge_pmd()
mm/huge_memory: use mm instead of tlb->mm
mm/huge_memory: remove unnecessary sanity checks
mm/huge_memory: deduplicate zap deposited table call
mm/huge_memory: remove unnecessary VM_BUG_ON_PAGE()
mm/huge_memory: add a common exit path to zap_huge_pmd()
mm/huge_memory: handle buggy PMD entry in zap_huge_pmd()
mm/huge_memory: have zap_huge_pmd return a boolean, add kdoc
mm/huge: avoid big else branch in zap_huge_pmd()
mm/huge_memory: simplify vma_is_specal_huge()
mm: on remap assert that input range within the proposed VMA
mm: add mmap_action_map_kernel_pages[_full]()
uio: replace deprecated mmap hook with mmap_prepare in uio_info
drivers: hv: vmbus: replace deprecated mmap hook with mmap_prepare
mm: allow handling of stacked mmap_prepare hooks in more drivers
...
- Make the handling of compat functions consistent and more robust
- Rework the underlying data store so that it is dynamically
allocated, which allows the conversion of the last holdout SPARC64
to the generic VDSO implementation
- Rework the SPARC64 VDSO to utilize the generic implementation
- Mop up the left overs of the non-generic VDSO support in the core
code.
- Expand the VDSO selftest and make them more robust
- Allow time namespaces to be enabled independently of the generic
VDSO support, which was not possible before due to SPARC64 not
using it.
- Various cleanups and improvements in the related code.
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmnb0v8QHHRnbHhAa2Vy
bmVsLm9yZwAKCRCmGPVMDXSYocfqD/9ywgnvwRH6B612mY4PI3qCbLHs6n9f78aH
YwyXmmfBZ5vt1ZtptHD+BAxiIMm9GC+/exdj5zhcOWucnBVhorcloE6evxhkJAMn
RhTQFKkEmcA/UV2Yfct9r+33kgZRyu4IIul4J7hgn2o5T1BqwZbOil0W/O5adr5P
MDLxjT1OLV80ZZWI9qbWcR/aR7W7sHcdwfVPPqjhombRY7f391Mo3dZeM5C2y55x
8TXCEqVpN1RJzFinWEgQN7QpP4OmF0rRuXSrDQpkH6pk/+RSqNlT/QGG7MJtmCQR
E6CeBjNRUn318KiroaGyTKlM9xsL3gNoiCY24ZTwzZxx3g5gSAR3KTCTJhQU0hpu
Svxj+ksqEAyW7fAOIsbce6W8fUPKC2KM+juXgPKcqZ5hjE2fALD+eEYMlq00jSiu
sj71007cM9tZKOXPdWs3Fv7AY2Yj7iiRiRz9gv1wqS1z7ybxiaFjxjLYYakej0tr
rmwBDEGhNow7msZZttr01BRZk9hDUWfIiJtL+0BrgRLNzst2A7WoagtZ2s0Z7Psl
RjtWgYNBDJ878xK0J+Djqb9TyLraGWZShIIna9uYCAJX9i954xfKJ//NOnUkZhcl
jslDLHhdttyJ+TmgIsc1ntUGvYvHqH5ywQpyDfWepMKyIYdaJLHOr2K6bwFnGHdw
uocXvLrkXw==
=8ixX
-----END PGP SIGNATURE-----
Merge tag 'timers-vdso-2026-04-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull vdso updates from Thomas Gleixner:
- Make the handling of compat functions consistent and more robust
- Rework the underlying data store so that it is dynamically allocated,
which allows the conversion of the last holdout SPARC64 to the
generic VDSO implementation
- Rework the SPARC64 VDSO to utilize the generic implementation
- Mop up the left overs of the non-generic VDSO support in the core
code
- Expand the VDSO selftest and make them more robust
- Allow time namespaces to be enabled independently of the generic VDSO
support, which was not possible before due to SPARC64 not using it
- Various cleanups and improvements in the related code
* tag 'timers-vdso-2026-04-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (51 commits)
timens: Use task_lock guard in timens_get*()
timens: Use mutex guard in proc_timens_set_offset()
timens: Simplify some calls to put_time_ns()
timens: Add a __free() wrapper for put_time_ns()
timens: Remove dependency on the vDSO
vdso/timens: Move functions to new file
selftests: vDSO: vdso_test_correctness: Add a test for time()
selftests: vDSO: vdso_test_correctness: Use facilities from parse_vdso.c
selftests: vDSO: vdso_test_correctness: Handle different tv_usec types
selftests: vDSO: vdso_test_correctness: Drop SYS_getcpu fallbacks
selftests: vDSO: vdso_test_gettimeofday: Remove nolibc checks
Revert "selftests: vDSO: parse_vdso: Use UAPI headers instead of libc headers"
random: vDSO: Remove ifdeffery
random: vDSO: Trim vDSO includes
vdso/datapage: Trim down unnecessary includes
vdso/datapage: Remove inclusion of gettimeofday.h
vdso/helpers: Explicitly include vdso/processor.h
vdso/gettimeofday: Add explicit includes
random: vDSO: Add explicit includes
MIPS: vdso: Explicitly include asm/vdso/vdso.h
...
Calvin reported an odd NMI watchdog lockup which claims that the CPU locked
up in user space. He provided a reproducer, which sets up a timerfd based
timer and then rearms it in a loop with an absolute expiry time of 1ns.
As the expiry time is in the past, the timer ends up as the first expiring
timer in the per CPU hrtimer base and the clockevent device is programmed
with the minimum delta value. If the machine is fast enough, this ends up
in a endless loop of programming the delta value to the minimum value
defined by the clock event device, before the timer interrupt can fire,
which starves the interrupt and consequently triggers the lockup detector
because the hrtimer callback of the lockup mechanism is never invoked.
As a first step to prevent this, avoid reprogramming the clock event device
when:
- a forced minimum delta event is pending
- the new expiry delta is less then or equal to the minimum delta
Thanks to Calvin for providing the reproducer and to Borislav for testing
and providing data from his Zen5 machine.
The problem is not limited to Zen5, but depending on the underlying
clock event device (e.g. TSC deadline timer on Intel) and the CPU speed
not necessarily observable.
This change serves only as the last resort and further changes will be made
to prevent this scenario earlier in the call chain as far as possible.
[ tglx: Updated to restore the old behaviour vs. !force and delta <= 0 and
fixed up the tick-broadcast handlers as pointed out by Borislav ]
Fixes: d316c57ff6 ("[PATCH] clockevents: add core functionality")
Reported-by: Calvin Owens <calvin@wbinvd.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Calvin Owens <calvin@wbinvd.org>
Tested-by: Borislav Petkov <bp@alien8.de>
Link: https://lore.kernel.org/lkml/acMe-QZUel-bBYUh@mozart.vkv.me/
Link: https://patch.msgid.link/20260407083247.562657657@kernel.org
In alarmtimer_suspend(), timerqueue_getnext() is called under
base->lock, but next->expires is read after the lock is released.
This is safe because suspend freezes all relevant task contexts,
but reading the node while holding the lock makes the code easier
to reason about and not worry about a theoretical UAF.
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260407143627.19405-1-zhanxusheng@xiaomi.com
Commit 56534673ce ("tick/nohz: Optimize check_tick_dependency() with
early return") added a fast path that returns !val when the tick_stop
tracepoint is disabled.
This is inverted: the slow path returns true when a dependency IS found
(val != 0), but !val returns true when val is zero (no dependency). The
result is that can_stop_full_tick() sees "dependency found" when there are
none, and the tick never stops on nohz_full CPUs.
Fix this by returning !!val instead of !val, matching the slow-path semantics.
Fixes: 56534673ce ("tick/nohz: Optimize check_tick_dependency() with early return")
Signed-off-by: Josh Snyder <josh@code406.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: Claude:claude-opus-4-6
Link: https://patch.msgid.link/20260402-fix-idle-tick2-v1-1-eecb589649d3@code406.com
The #endif comment says "BITS_PER_LONG >= 64", but the corresponding #if
guard is "BITS_PER_LONG < 64".
The comment was originally correct when the block had a three-way
#if/#else/#endif structure, where the #else branch provided a 64-bit inline
version. Commit 79bf2bb335 ("[PATCH] tick-management: dyntick / highres
functionality") removed the #else branch but did not update the #endif
comment, leaving it inconsistent with the remaining #if condition.
Fix the comment to match the preprocessor guard.
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260331074811.26147-1-zhanxusheng@xiaomi.com
The comment in exit_itimers() still refers to itimer_delete(),
which was replaced by posix_timer_delete(). Update the comment
accordingly.
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260326142210.98632-1-zhanxusheng@xiaomi.com
Previously, missing time namespace support in the vDSO meant that time
namespaces needed to be disabled globally. This was expressed in a hard
dependency on the generic vDSO library. This also meant that architectures
without any vDSO or only a stub vDSO could not enable time namespaces.
Now that all architectures using a real vDSO are using the generic library,
that dependency is not necessary anymore.
Remove the dependency and let all architectures enable time namespaces.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260326-vdso-timens-decoupling-v2-2-c82693a7775f@linutronix.de
As a preparation of the untangling of time namespaces and the vDSO, move
the glue functions between those subsystems into a new file.
While at it, switch the mutex lock and mmap_read_lock() in the vDSO
namespace code to guard().
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260326-vdso-timens-decoupling-v2-1-c82693a7775f@linutronix.de
Calling smp_processor_id() on:
- In CONFIG_DEBUG_PREEMPT=y, if preemption/irq is disabled, then it does
not print any warning.
- In CONFIG_DEBUG_PREEMPT=n, it doesn't do anything apart from getting
__smp_processor_id
So with both CONFIG_DEBUG_PREEMPT=y/n, in preemption disabled section it is
better to cache the value. It saves a few cycles. Though tiny, repeated
adds up.
timer_clear_idle() is called with interrupts disabled. So cache the value
once.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Link: https://patch.msgid.link/20260323193630.640311-5-sshegde@linux.ibm.com
alarm_timer_forward() passes arguments to alarm_forward() in the wrong
order:
alarm_forward(alarm, timr->it_interval, now);
However, alarm_forward() is defined as:
u64 alarm_forward(struct alarm *alarm, ktime_t now, ktime_t interval);
and uses the second argument as the current time:
delta = ktime_sub(now, alarm->node.expires);
Passing the interval as "now" results in incorrect delta computation,
which can lead to missed expirations or incorrect overrun accounting.
This issue has been present since the introduction of
alarm_timer_forward().
Fix this by swapping the arguments.
Fixes: e7561f1633 ("alarmtimer: Implement forward callback")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260323061130.29991-1-zhanxusheng@xiaomi.com
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmm3G/UeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGZJUH/R0vQ3Vha48QDEic
1NREwaHxAoTFi0i3y7OPPklqrP2V09D1qg4Q6fExYQVTQgV6F2DRjVbyPKrmr4ay
BA6aHrUdnFngYHpDlI1b1r7rJiAIN4WFHl7StO70bS+EB+UPsP9cfP3CKXUfKfqT
kyHXzUrd5QnjYmlb9rQw1E6rzsRamNtGUtZf7TwDidJYjtm3sPeDHUkjyRy4xkYd
UouIu6W7UXoicl38bJAgaWBY5BiYtjN6ktnY4/gcqDeqYd7mTM3Eb1B+OSXgFfip
F0OYfJhfWn+63WnPA+1I5jXWC1UrdVXTMK/NTYjhmGlfdmkLcWDlNGtu+qKZbpwj
fmF3Kyo=
=6nX1
-----END PGP SIGNATURE-----
Merge tag 'v7.0-rc4' into timers/core, to resolve conflict
Resolve conflict between this change in the upstream kernel:
4c652a4772 ("rseq: Mark rseq_arm_slice_extension_timer() __always_inline")
... and this pending change in timers/core:
0e98eb1481 ("entry: Prepare for deferred hrtimer rearming")
Signed-off-by: Ingo Molnar <mingo@kernel.org>
The clocksource watchdog code has over time reached the state of an
impenetrable maze of duct tape and staples. The original design, which was
made in the context of systems far smaller than today, is based on the
assumption that the to be monitored clocksource (TSC) can be trivially
compared against a known to be stable clocksource (HPET/ACPI-PM timer).
Over the years it turned out that this approach has major flaws:
- Long delays between watchdog invocations can result in wrap arounds
of the reference clocksource
- Scalability of the reference clocksource readout can degrade on large
multi-socket systems due to interconnect congestion
This was addressed with various heuristics which degraded the accuracy of
the watchdog to the point that it fails to detect actual TSC problems on
older hardware which exposes slow inter CPU drifts due to firmware
manipulating the TSC to hide SMI time.
To address this and bring back sanity to the watchdog, rewrite the code
completely with a different approach:
1) Restrict the validation against a reference clocksource to the boot
CPU, which is usually the CPU/Socket closest to the legacy block which
contains the reference source (HPET/ACPI-PM timer). Validate that the
reference readout is within a bound latency so that the actual
comparison against the TSC stays within 500ppm as long as the clocks
are stable.
2) Compare the TSCs of the other CPUs in a round robin fashion against
the boot CPU in the same way the TSC synchronization on CPU hotplug
works. This still can suffer from delayed reaction of the remote CPU
to the SMP function call and the latency of the control variable cache
line. But this latency is not affecting correctness. It only affects
the accuracy. With low contention the readout latency is in the low
nanoseconds range, which detects even slight skews between CPUs. Under
high contention this becomes obviously less accurate, but still
detects slow skews reliably as it solely relies on subsequent readouts
being monotonically increasing. It just can take slightly longer to
detect the issue.
3) Rewrite the watchdog test so it tests the various mechanisms one by
one and validating the result against the expectation.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Borislav Petkov (AMD) <bp@alien8.de>
Tested-by: Daniel J Blueman <daniel@quora.org>
Reviewed-by: Jiri Wiesner <jwiesner@suse.de>
Reviewed-by: Daniel J Blueman <daniel@quora.org>
Link: https://patch.msgid.link/20260123231521.926490888@kernel.org
Link: https://patch.msgid.link/87h5qeomm5.ffs@tglx
Using a non-continuous aka untrusted clocksource as a watchdog for another
untrusted clocksource is equivalent to putting the fox in charge of the
henhouse.
That's especially true with the jiffies clocksource which depends on
interrupt delivery based on a periodic timer. Neither the frequency of that
timer is trustworthy nor the kernel's ability to react on it in a timely
manner and rearm it if it is not self rearming.
Just don't bother to deal with this. It's not worth the trouble and only
relevant to museum piece hardware.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260123231521.858743259@kernel.org
The container_of() call is open-coded multiple times.
Add a helper macro.
Use container_of_const() to preserve constness.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260311-hrtimer-cleanups-v1-12-095357392669@linutronix.de
This pointer indirection is a remnant from when ktime_t was a struct,
today it is pointless.
Drop the pointer indirection.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260311-hrtimer-cleanups-v1-9-095357392669@linutronix.de
The value will be assigned to before any usage.
No other function in hrtimer.c does such a zero-initialization.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260311-hrtimer-cleanups-v1-7-095357392669@linutronix.de
Neither the array nor the offsets it is pointing to are meant to be
changed through the array.
Mark both the array and the values it points to as const.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260311-hrtimer-cleanups-v1-5-095357392669@linutronix.de
In aux_clock_enable() the clocksource from tkr_raw is used to call
tk_setup_internals(). Do the same in tk_aux_update_clocksource(). While
the clocksources will be the same in any case, this is less confusing.
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260311-hrtimer-cleanups-v1-4-095357392669@linutronix.de
When the hrtimer_interrupt needs to restart more than 3 times and still has
expired timers, the interrupt is considered hung. To give the system a
little time to recover, the hardware timer is programmed a little into the
future.
Prior to commit 2889243848 ("hrtimer: Re-arrange hrtimer_interrupt()"),
this was relative to the amount of time spend serving the interrupt with a
max of 100 msec.
However, in order to simplify, and because this condition 'should' not
happen, the timeout was unconditionally set to 100 msec.
'Obviously' there is a benchmark that hits this hard, by programming a
ton of very short timers :-/
Since reprogramming is decoupled from the interrupt handling, the actual
execution time is lost, however the code does track max_hang_time. Using
that, rather than the 100 ms max restores performance.
stress-ng --timeout 60 --times --verify --metrics --no-rand-seed --timermix 64
bogo ops/s
288924384856^1: 23715979.93
288924384856: 11550049.77
patched: 23361116.78
Additionally, Thomas noted that cpu_base->hang_detected should not be
cleared until the next interrupt, such that __hrtimer_reprogram() won't
undo the extra delay.
Fixes: 2889243848 ("hrtimer: Re-arrange hrtimer_interrupt()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260311121500.GF652779@noisy.programming.kicks-ass.net
Closes: https://lore.kernel.org/oe-lkp/202603102229.74b9dee4-lkp@intel.com
The trace_clock_jiffies() function that handles the "uptime" clock for
tracing calls jiffies_64_to_clock_t(). This causes the function tracer to
constantly recurse when the tracing clock is set to "uptime". Mark it
notrace to prevent unnecessary recursion when using the "uptime" clock.
Fixes: 58d4e21e50 ("tracing: Fix wraparound problems in "uptime" trace clock")
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260306212403.72270bb2@robin
After sparc64, there are no remaining users of ARCH_CLOCKSOURCE_DATA
and it can just be removed.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Andreas Larsson <andreas@gaisler.com>
Reviewed-by: Andreas Larsson <andreas@gaisler.com>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260304-vdso-sparc64-generic-2-v6-14-d8eb3b0e1410@linutronix.de
[Thomas: drop sparc64 bits from the patch]
permissive for auxiliary clocks, to not reject syscalls
based on the status field that do not try to modify the
status field. This makes ABI behavior in clock_adjtime()
consistent with CLOCK_REALTIME.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmmsxzkRHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1hq8g//fRTp9p2pVfmRWUoxWELrT/bMK1r+D6F3
6BYkwp68peRhchVrFxkI/Y37rjAIC8CXZSPuvkubqIROrH3gA7SCCQYCcZKdss+t
i3lbpQF8IbagPIS5btpOAN2KRCu2S7aqjDdH0rWb9VhQdlW7fI71Z72Uz07YEA+q
TWpy3gE531P/dgAqcvIAyMHnFZDCb1S6z8wZvT3SV4r4GkczfXpTFyNHHtETSu0V
7isuOBfloM4HpDU50oUotlqBiwigH27J2Ad6aIrnCA7iaQPrzREysG+8E96ShhaB
g6+qaQS5gTgFryA1bggA6LzGveLOI8bjy2kZ2SnZWuFPj46OReGIuwK4kyY07jz2
xk0sd37alN16ETKhGVLfAgjmzVGoKVNnp4ak9J3VmMbxWEmXeObuOC8SmF9VImc1
4bRaG9+Tlfd4DtOOz2+E4VcPE1D9A2tMw4esgUaXRrrp4GlEcKOJ5PRlWj0uGvrh
xLPLbL0XIiWsjMsHdVs4Gq9Z0MvfRHc4VLOviIqLFtHox2DscZypPkyjKAv5inp0
/VWyUYJkkr07RMQQ3nqHnP+lzAfO2aSeZ72D9NnHStL3RPbGC4jYvpoi8dnH0/TT
PKJgj2jb7u3h+1cxKBi1RM0JbxUYD5+4N8zfJISa9uqkHZ3XY3VyuuT+2RHO6CQp
d1BdX0V4oDA=
=zjov
-----END PGP SIGNATURE-----
Merge tag 'timers-urgent-2026-03-08' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fix from Ingo Molnar:
"Make clock_adjtime() syscall timex validation slightly more permissive
for auxiliary clocks, to not reject syscalls based on the status field
that do not try to modify the status field.
This makes the ABI behavior in clock_adjtime() consistent with
CLOCK_REALTIME"
* tag 'timers-urgent-2026-03-08' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
timekeeping: Fix timex status validation for auxiliary clocks
Borislav reported a division by zero in the timekeeping code and random
hangs with the new coupled clocksource/clockevent functionality.
It turned out that the TSC clocksource is not always updating the
freq_khz field of the clocksource on registration. The coupled mode
conversion calculation requires the frequency and as it's not
initialized the resulting factor is zero or a random value. As a
consequence this causes a division by zero or random boot hangs.
Instead of chasing down all clocksources which fail to update that
member, fill it in at registration time where the caller has to supply
the frequency anyway. Except for special clocksources like jiffies which
never can have coupled mode.
To make this more robust put a check into the registration function to
validate that the caller supplied a frequency if the coupled mode
feature bit is set. If not, emit a warning and clear the feature bit.
Fixes: cd38bdb8e6 ("timekeeping: Provide infrastructure for coupled clockevents")
Reported-by: Borislav Petkov <bp@alien8.de>
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Borislav Petkov <bp@alien8.de>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Link: https://patch.msgid.link/87cy1jsa4m.ffs@tglx
Closes: https://lore.kernel.org/20260303213027.GA2168957@ax162
Nathan reported a boot failure after the coupled clocksource/event support
was enabled for the TSC deadline timer. It turns out that on the affected
test systems the TSC frequency is not refined against HPET, so it is
registered with the same frequency as the TSC-early clocksource.
As a consequence the update function which checks for a change of the
shift/mult pair of the clocksource fails to compute the conversion
limit, which is zero initialized. This check is there to avoid pointless
computations on every timekeeping update cycle (tick).
So the actual clockevent conversion function limits the delta expiry to
zero, which means the timer is always programmed to expire in the
past. This obviously results in a spectacular timer interrupt storm,
which goes unnoticed because the per CPU interrupts on x86 are not
exposed to the runaway detection mechanism and the NMI watchdog is not
yet functional. So the machine simply stops booting.
That did not show up in testing. All test machines refine the TSC frequency
so TSC has a differrent shift/mult pair than TSC-early and the conversion
limit is properly initialized.
Cure that by setting the conversion limit right at the point where the new
clocksource is installed.
Fixes: cd38bdb8e6 ("timekeeping: Provide infrastructure for coupled clockevents")
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/87bjh4zies.ffs@tglx
Closes: https://lore.kernel.org/20260303012905.GA978396@ax162
The timekeeping_validate_timex() function validates the timex status
of an auxiliary system clock even when the status is not to be changed,
which causes unexpected errors for applications that make read-only
clock_adjtime() calls, or set some other timex fields, but without
clearing the status field.
Do the AUX-specific status validation only when the modes field contains
ADJ_STATUS, i.e. the application is actually trying to change the
status. This makes the AUX-specific clock_adjtime() behavior consistent
with CLOCK_REALTIME.
Fixes: 4eca49d0b6 ("timekeeping: Prepare do_adtimex() for auxiliary clocks")
Signed-off-by: Miroslav Lichvar <mlichvar@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260225085231.276751-1-mlichvar@redhat.com
* Fix error when reporting jiffies converted values back to user space
Return the converted value instead of "Invalid argument" error.
* Testing
Spent around a week in linux-next -enough for this small fix-
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEErkcJVyXmMSXOyyeQupfNUreWQU8FAmmoNL0ACgkQupfNUreW
QU/dMQv/YL6Lpv76iFGOL8gP1tU3oebKWKGJYDcQtqPZfLnlFmWu+XliHCldqJ7J
Ur9u2KleA0jM/Szq/v4FOyq2L7992dpKSkzM6ZsMyEfrz0e21WCZus40pcpE0L2j
kMNo4Vf3bAP+18KNsxh6zUc9WeYJ3suySmme+je2WNkab/io9XNUxYv7LhnKWze7
3iCXYZj/HtF3G9/xk0v3Ihlw6rNRVxNPfC3DpGXlvtnTSchlj9S9IK4pczcAmdw8
CNTEGCi+yzZYCcyI310IoeH0d3L5k39daJqtSC0BlVp607kr57nt5Hygf08WdnG8
2U+lvoWKp7odyu9/D1nqcpoQVY+9IzRkW+RM1bnYOmNYAiFrhiKTNCpZOhhqWn6P
3f3zvRq3Wt9zuA8upGjT6adxTrPMkpiqQD4POExgzSvoqkZ31Lw1/A6INtdWng82
+rFdL4PqdElrghVl07zydX5UWz/+fZKsQMz/j1cKROhKRQsaLXWIYHbo6OSlp6AC
JLONWmgW
=F0SK
-----END PGP SIGNATURE-----
Merge tag 'sysctl-7.00-fixes-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl
Pull sysctl fix from Joel Granados:
- Fix error when reporting jiffies converted values back to user space
Return the converted value instead of "Invalid argument" error
* tag 'sysctl-7.00-fixes-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl:
time/jiffies: Fix sysctl file error on configurations where USER_HZ < HZ
Commit 2dc164a48e ("sysctl: Create converter functions with two new
macros") incorrectly returns error to user space when jiffies sysctl
converter is used. The old overflow check got replaced with an
unconditional one:
+ if (USER_HZ < HZ)
+ return -EINVAL;
which will always be true on configurations with "USER_HZ < HZ".
Remove the check; it is no longer needed as clock_t_to_jiffies() returns
ULONG_MAX for the overflow case and proc_int_u2k_conv_uop() checks for
"> INT_MAX" after conversion
Fixes: 2dc164a48e ("sysctl: Create converter functions with two new macros")
Reported-by: Colm Harrington <colm.harrington@oracle.com>
Signed-off-by: Gerd Rausch <gerd.rausch@oracle.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
- Fix circular locking dependency in cpuset partition code by deferring
housekeeping_update() calls to a workqueue instead of calling them
directly under cpus_read_lock.
- Fix null-ptr-deref in rebuild_sched_domains_cpuslocked() when
generate_sched_domains() returns NULL due to kmalloc failure.
- Fix incorrect cpuset behavior for effective_xcpus in
partition_xcpus_del() and cpuset_update_tasks_cpumask() in
update_cpumasks_hier().
- Fix race between task migration and cgroup iteration.
-----BEGIN PGP SIGNATURE-----
iIQEABYKACwWIQTfIjM1kS57o3GsC/uxYfJx3gVYGQUCaadVVQ4cdGpAa2VybmVs
Lm9yZwAKCRCxYfJx3gVYGef0AQDLuJE3vzc2VeCBc4rGcj7ZSRmc3tc28lOqHRzi
XEx1iwD+PeFcb9wt1CTqA5hAiIY1LGR/5iO1kTH7paRd16DBRAc=
=S8WE
-----END PGP SIGNATURE-----
Merge tag 'cgroup-for-7.0-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
- Fix circular locking dependency in cpuset partition code by
deferring housekeeping_update() calls to a workqueue instead
of calling them directly under cpus_read_lock
- Fix null-ptr-deref in rebuild_sched_domains_cpuslocked() when
generate_sched_domains() returns NULL due to kmalloc failure
- Fix incorrect cpuset behavior for effective_xcpus in
partition_xcpus_del() and cpuset_update_tasks_cpumask()
in update_cpumasks_hier()
- Fix race between task migration and cgroup iteration
* tag 'cgroup-for-7.0-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
cgroup/cpuset: fix null-ptr-deref in rebuild_sched_domains_cpuslocked
cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock
cgroup/cpuset: Defer housekeeping_update() calls from CPU hotplug to workqueue
cgroup/cpuset: Move housekeeping_update()/rebuild_sched_domains() together
kselftest/cgroup: Simplify test_cpuset_prs.sh by removing "S+" command
cgroup/cpuset: Set isolated_cpus_updating only if isolated_cpus is changed
cgroup/cpuset: Clarify exclusion rules for cpuset internal variables
cgroup/cpuset: Fix incorrect use of cpuset_update_tasks_cpumask() in update_cpumasks_hier()
cgroup/cpuset: Fix incorrect change to effective_xcpus in partition_xcpus_del()
cgroup: fix race between task migration and iteration
for the common HZ=100, 250 or 1000 cases, only inlining them
for odd HZ values like HZ=300.
The inlining overhead showed up in performance tests of the TCP code.
(Marked as an RFC pull request, as it's not a regression.)
Signed-off-by: Ingo Molnar <mingo@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJFBAABCgAvFiEEBpT5eoXrXCwVQwEKEnMQ0APhK1gFAmmkA94RHG1pbmdvQGtl
cm5lbC5vcmcACgkQEnMQ0APhK1ggmhAAqvNgH1vG+Ad1/r2aZ0iQNOJ3lLL9DKVU
V0Q2iXiRPKvzjXaHB+C2B2GkGgBBxyvdl1wGPpknI/OkoC8aBzK8jSchmfLFfWdx
C1MC0xoZRtaWDQaMYYQ83ZGQqdKHct4ZrwfsPu+g95NGvNg3m/W8p3cZjruknvH3
bKZmbN2wQiSx6+PB7/FkEjV7eAlaskYYdKLNqZzd/62oQ6is4ppAjEAp+X/FidJv
0lUVr7ILx6lZHjWnavUjex2iSvvZ8XqiaYVKlj5EE9cCaPhXDTpkaO8l/ELcjpHL
ZBBV6rpbDo7+Mo3UnUiz+CaYi6XAAOwgj6wZBiBXhVQUAW0PIlMaefDjccWFlDw1
oAcRCg5i3KF8cvrfQYUs4519W52eWOThqQ1fs5ql6P6ycHZ0KTsmaRAbNih811RN
A2VMTkiyX25bXOUQ9e5Y7cYOvDMGGCWiocT6C7Is9gZQMfkj92NDCKURLwRYzzBr
2XDjg46YekGXwy8OamMwXMRcdyUC5fAIaWOaq7IL1K3cgbS2qbZx55Y85+AOfngF
DvFWDIfjslYMZqzQQ+4+MaJitRQ2V6CqdOP3kQbJ3Z6DmIcwi7DkOzqH1hEglb7O
IjXCxjxosZv4iofpxr0FKJPx7KBVSzzxezjMLzeijM+zDdbF4GWFpqRD1ONh/4vm
/1tfaC6TU/E=
=duTI
-----END PGP SIGNATURE-----
Merge tag 'timers-urgent-2026-03-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fix from Ingo Molnar:
"Improve the inlining of jiffies_to_msecs() and jiffies_to_usecs(), for
the common HZ=100, 250 or 1000 cases. Only use a function call for odd
HZ values like HZ=300 that generate more code.
The function call overhead showed up in performance tests of the TCP
code"
* tag 'timers-urgent-2026-03-01' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
time/jiffies: Inline jiffies_to_msecs() and jiffies_to_usecs()
When modifying the expiry of a armed timer it is first dequeued, then the
expiry value is updated and then it is queued again.
This can be avoided when the new expiry value is within the range of the
previous and the next timer as that does not change the position in the RB
tree.
The linked timerqueue allows to peak ahead to the neighbours and check
whether the new expiry time is within the range of the previous and next
timer. If so just modify the timer in place and spare the enqueue and
requeue effort, which might end up rotating the RB tree twice for nothing.
This speeds up the handling of frequently rearmed hrtimers, like the hrtick
scheduler timer significantly.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260224163431.873359816@kernel.org
To prepare for optimizing the rearming of enqueued timers, switch to the
linked timerqueue. That allows to check whether the new expiry time changes
the position of the timer in the RB tree or not, by checking the new expiry
time against the previous and the next timers expiry.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260224163431.806643179@kernel.org
Replace the open coded container_of() orgy with a trivial
clock_base_next_timer() helper.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260224163431.532927977@kernel.org
The per clock base cached expiry time allows to do a more efficient
evaluation of the next expiry on a CPU.
Separate the reprogramming evaluation from the NOHZ idle evaluation which
needs to exclude the NOHZ timer to keep the reprogramming path lean and
clean.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260224163431.468186893@kernel.org
