github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-10-22 12:52:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Updates for the core timekeeping and timers subsystem:

Browse Source 
- Futher cleanups for the POSIX timer exit/exec related code to simplify
     the code flow and to make it more robust against future changes.
 
   - Prevent tearing of timekeeper::ktime_sec which can be accessed without the
     sequence count protection.
 
     The timekeeper is updated by memcpy() with an uneven sequence count to
     force readers into the retry path. On 64-bit machine the ktime_sec
     member can be accessed atomically without the sequence count
     protection. That is only correct when the update side memcpy()
     guarantees atomicity at the 64-bit granularity. That's not guaranteed
     due to debug options, compilers and instruction availability. If the
     memcpy() does not provide the atomicity then the unprotected readers
     can observe store tearing.
 
     To prevent this write the ktime_sec member before the memcpy() so that
     a non-atomic memcpy() will write the same data. A similar fix has been
     done for the xtime_sec member, but the ktime_sec equivalent was missed.
 
   - Prevent time64_to_tm() day truncation on 32-bit kernels
 
     The function receives a 64-bit seconds value but stores the quotient of
     the computation in a long, which truncates the result for large input
     values. Cure this by using div_s64_rem() which stores the quotient in
     s64 and only leaves the seconds-within-day reminder in a long, which is
     guaranteed to fit. This fix comes with a corresponding kunit test to
     prevent further mishaps.
 
   - Mark data-racy accesses to the hrtimer_sleeper::task member.
 
     The recent updates to KCSAN flagged these benign races. Annotate them
     with new helper functions, convert all existing usage sites and mark
     the task member private after that so that future users which fail to
     use the helpers can be detected automatically.
 
   - Address similar benign KCSAN flagged issues in the hrtimer, timer and
     timer migration code.
 
   - Fixes for a few selftest issues
 -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCgAuFiEEQp8+kY+LLUocC4bMphj1TA10mKEFAmrX3OwQHHRnbHhAa2Vy
 bmVsLm9yZwAKCRCmGPVMDXSYoXDHD/94X2ix8PjV3YibEd9u86LHLtT9OolBtQgP
 Sum+LB5ARHy4aJGp8N1zGTA6S/zHm7fejrbdTS8xQAdK0BERP7BBIz9XmOq4+147
 S2TViWdL9gH6at3D5EYBuXmdtUqamesg9qB0u2x2pAtJsI5bgIsInpHBKOzmvkAM
 RSOHoQBrmRcSLe9M8PKMze4qS4Y8ckt2QKVXhZ89ncQEjeWC9piQh2yA8kLeFc9L
 6dvRxZTEOtedJM8+5CUHjrUaCac1mT7QbAVpHSjVVPMjvILeUx5bWK/ryTbvnr7e
 askWx+EpZR4+ZQOWDZ/sO99r+WcZNpGu+UGw9ChNrRxhqV6r9OvDFDslokvvn6RN
 ldjkRbbSkqXthEac/jtE3M6E1w6Vh/tLzi1Ow6TstJhwbF6E1WBgafkqjxmWun03
 PZO1M9UOL2uA7od7mJrMKtQskC3/AgviPwUr79owBxkgAUnmtragO+NQrns3ud8O
 921q+dj6WfdTt+FK1wXYjzLOlqV9zUADl1uAUNkf57gxp7ehD5KUJJRPVKrztKKA
 iJo393McWfaAHsb6WiBQfC7z4/akTmCe5S5L7add6u44jXTR+ZkVQwpziGRagXby
 3+7Y/p4HOvZmaEmsUc6I6+/TsIrjttMJY/4i020U/5rKCxZ3ufAy5o1Ept/EteCk
 GeEVNP8f6Q==
 =L9JF
 -----END PGP SIGNATURE-----

Merge tag 'timers-core-2026-10-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull timer core updates from Thomas Gleixner:
 "Updates for the core timekeeping and timers subsystem:

   - Futher cleanups for the POSIX timer exit/exec related code to
     simplify the code flow and to make it more robust against future
     changes.

   - Prevent tearing of timekeeper::ktime_sec which can be accessed
     without the sequence count protection.

     The timekeeper is updated by memcpy() with an uneven sequence count
     to force readers into the retry path. On 64-bit machine the
     ktime_sec member can be accessed atomically without the sequence
     count protection. That is only correct when the update side
     memcpy() guarantees atomicity at the 64-bit granularity. That's not
     guaranteed due to debug options, compilers and instruction
     availability. If the memcpy() does not provide the atomicity then
     the unprotected readers can observe store tearing.

     To prevent this write the ktime_sec member before the memcpy() so
     that a non-atomic memcpy() will write the same data. A similar fix
     has been done for the xtime_sec member, but the ktime_sec
     equivalent was missed.

   - Prevent time64_to_tm() day truncation on 32-bit kernels

     The function receives a 64-bit seconds value but stores the
     quotient of the computation in a long, which truncates the result
     for large input values. Cure this by using div_s64_rem() which
     stores the quotient in s64 and only leaves the seconds-within-day
     reminder in a long, which is guaranteed to fit. This fix comes with
     a corresponding kunit test to prevent further mishaps.

   - Mark data-racy accesses to the hrtimer_sleeper::task member.

     The recent updates to KCSAN flagged these benign races. Annotate
     them with new helper functions, convert all existing usage sites
     and mark the task member private after that so that future users
     which fail to use the helpers can be detected automatically.

   - Address similar benign KCSAN flagged issues in the hrtimer, timer
     and timer migration code.

   - Fixes for a few selftest issues"

* tag 'timers-core-2026-10-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (26 commits)
  selftests/timers: clocksource-switch: Fix unchecked open()/read()
  selftests: timers: Measure the CPU timers on the clock they count
  selftests: timers: Count what tick is worth in the drift estimate
  time/kunit: Add time64_to_tm() case beyond 32-bit day count
  time: Prevent time64_to_tm() day truncation on 32-bit
  timekeeping: Use READ_ONCE/WRITE_ONCE() for ktime_sec to prevent tearing
  timers/migration: Mark racy updates to tmigr_event::ignore field
  timers: Mark racy updates to hlist_node::pprev field
  hrtimer: Apply READ_ONCE() to lockless base->running loads
  hrtimer: Mark the hrtimer_sleeper structure's ->task field __private
  hrtimer: Update hrtimer_resolution only if value changes
  futex: Use accessor for hrtimer_sleeper ->task field in requeue
  rtmutex: Use accessor for hrtimer_sleeper ->task field
  net: pktgen: Use accessor for hrtimer_sleeper ->task field
  timers: Use accessor for hrtimer_sleeper ->task field in sleep_timeout.c
  futex: Use accessor for hrtimer_sleeper ->task field in waitwake.c
  io-uring/rw: Use accessor for hrtimer_sleeper ->task field
  wait: Use accessor for hrtimer_sleeper ->task field
  aio: Use accessor for hrtimer_sleeper ->task field
  hrtimer: Mark data-racy accesses to hrtimer_sleeper ->task field
  ...
This commit is contained in:
Linus Torvalds 2026-10-21 11:20:38 -10:00
commit aaf3d47e53
26 changed files with 227 additions and 112 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
fs/aio.c
View File

@ -1402,7 +1402,7 @@ static long read_events(struct kioctx *ctx, long min_nr, long nr,
w.min_nr = min_nr - ret;
ret2 = prepare_to_wait_event(&ctx->wait, &w.w, TASK_INTERRUPTIBLE);
if (!ret2 && !t.task)
if (!ret2 && !hrtimer_sleeper_task_get(&t))
ret2 = -ETIME;
if (aio_read_events(ctx, min_nr, nr, event, &ret) || ret2)

13
fs/exec.c
View File

@ -1116,17 +1116,6 @@ static struct file *bprm_identity_file(const struct linux_binprm *bprm)
return bprm->file;
}
static void posixtimer_exec(struct task_struct *me)
{
#ifdef CONFIG_POSIX_TIMERS
spin_lock_irq(&me->sighand->siglock);
posix_cpu_timers_exit(me);
spin_unlock_irq(&me->sighand->siglock);
exit_itimers(me);
flush_itimer_signals();
#endif
}
/*
* Calling this is the point of no return. None of the failures will be
* seen by userspace since either the process is already taking a fatal
@ -1180,7 +1169,7 @@ int begin_new_exec(struct linux_binprm * bprm)
* timer would not remove an enqueued timer because the TID lookup
* of the old TID fails.
*/
posixtimer_exec(me);
posixtimer_exec();
/* see the comment in check_unsafe_exec() */
current->fs->in_exec = 0;

10
include/linux/hrtimer.h
View File

@ -72,7 +72,7 @@ enum hrtimer_mode {
*/
struct hrtimer_sleeper {
struct hrtimer timer;
struct task_struct *task;
struct task_struct *__private task;
};
static inline void hrtimer_set_expires(struct hrtimer *timer, ktime_t time)
@ -320,6 +320,14 @@ extern int schedule_hrtimeout_range_clock(ktime_t *expires,
const enum hrtimer_mode mode,
clockid_t clock_id);
extern int schedule_hrtimeout(ktime_t *expires, const enum hrtimer_mode mode);
static inline struct task_struct *hrtimer_sleeper_task_get(struct hrtimer_sleeper *sl)
{
return READ_ONCE(ACCESS_PRIVATE(sl, task));
}
static inline void hrtimer_sleeper_task_set(struct hrtimer_sleeper *sl, struct task_struct *t)
{
WRITE_ONCE(ACCESS_PRIVATE(sl, task), t);
}
/* Soft interrupt function to run the hrtimer queues: */
extern void hrtimer_run_queues(void);

6
include/linux/list.h
View File

@ -1172,7 +1172,7 @@ static inline void hlist_move_list(struct hlist_head *old,
{
new->first = old->first;
if (new->first)
new->first->pprev = &new->first;
WRITE_ONCE(new->first->pprev, &new->first);
old->first = NULL;
}
@ -1189,10 +1189,10 @@ static inline void hlist_splice_init(struct hlist_head *from,
struct hlist_head *to)
{
if (to->first)
to->first->pprev = &last->next;
WRITE_ONCE(to->first->pprev, &last->next);
last->next = to->first;
to->first = from->first;
from->first->pprev = &to->first;
WRITE_ONCE(from->first->pprev, &to->first);
from->first = NULL;
}

39
include/linux/posix-timers.h
View File

@ -66,38 +66,6 @@ struct cpu_timer {
struct task_struct __rcu *handling;
};
static inline bool cpu_timer_enqueue(struct timerqueue_head *head,
struct cpu_timer *ctmr)
{
ctmr->head = head;
return timerqueue_add(head, &ctmr->node);
}
static inline bool cpu_timer_queued(struct cpu_timer *ctmr)
{
return !!ctmr->head;
}
static inline bool cpu_timer_dequeue(struct cpu_timer *ctmr)
{
if (cpu_timer_queued(ctmr)) {
timerqueue_del(ctmr->head, &ctmr->node);
ctmr->head = NULL;
return true;
}
return false;
}
static inline u64 cpu_timer_getexpires(struct cpu_timer *ctmr)
{
return ctmr->node.expires;
}
static inline void cpu_timer_setexpires(struct cpu_timer *ctmr, u64 exp)
{
ctmr->node.expires = exp;
}
static inline void posix_cputimers_init(struct posix_cputimers *pct)
{
memset(pct, 0, sizeof(*pct));
@ -224,14 +192,15 @@ struct k_itimer {
} ____cacheline_aligned_in_smp;
void run_posix_cpu_timers(void);
void posix_cpu_timers_exit(struct task_struct *task);
void posix_cpu_timers_exit_group(struct task_struct *task);
void set_process_cpu_timer(struct task_struct *task, unsigned int clock_idx,
u64 *newval, u64 *oldval);
int update_rlimit_cpu(struct task_struct *task, unsigned long rlim_new);
#ifdef CONFIG_POSIX_TIMERS
void posixtimer_exec(void);
void posixtimer_exit(bool group_dead);
static inline void posixtimer_putref(struct k_itimer *tmr)
{
if (rcuref_put(&tmr->rcuref))
@ -259,6 +228,8 @@ static inline bool posixtimer_valid(const struct k_itimer *timer)
return !(val & 0x1UL);
}
#else /* CONFIG_POSIX_TIMERS */
static inline void posixtimer_exec(void) { }
static inline void posixtimer_exit(bool group_dead) { }
static inline void posixtimer_sigqueue_getref(struct sigqueue *q) { }
static inline void posixtimer_sigqueue_putref(struct sigqueue *q) { }
#endif /* !CONFIG_POSIX_TIMERS */

1
include/linux/sched/task.h
View File

@ -94,7 +94,6 @@ static inline void exit_thread(struct task_struct *tsk)
extern __noreturn void do_group_exit(int);
extern void exit_files(struct task_struct *);
extern void exit_itimers(struct task_struct *);
extern pid_t kernel_clone(struct kernel_clone_args *kargs);
struct task_struct *copy_process(struct pid *pid, int trace, int node,

2
include/linux/wait.h
View File

@ -556,7 +556,7 @@ do { \
} \
\
__ret = ___wait_event(wq_head, condition, state, 0, 0, \
if (!__t.task) { \
if (!hrtimer_sleeper_task_get(&__t)) { \
__ret = -ETIME; \
break; \
} \

2
io_uring/rw.c
View File

@ -1295,7 +1295,7 @@ static u64 io_hybrid_iopoll_delay(struct io_ring_ctx *ctx, struct io_kiocb *req)
set_current_state(TASK_INTERRUPTIBLE);
hrtimer_sleeper_start_expires(&timer, mode);
if (timer.task)
if (hrtimer_sleeper_task_get(&timer))
io_schedule();
hrtimer_cancel(&timer.timer);

13
kernel/exit.c
View File

@ -168,12 +168,6 @@ static void __exit_signal(struct release_task_post *post, struct task_struct *ts
lockdep_tasklist_lock_is_held());
spin_lock(&sighand->siglock);
#ifdef CONFIG_POSIX_TIMERS
posix_cpu_timers_exit(tsk);
if (group_dead)
posix_cpu_timers_exit_group(tsk);
#endif
if (group_dead) {
tty = sig->tty;
sig->tty = NULL;
@ -940,13 +934,12 @@ void __noreturn do_exit(long code)
panic("Attempted to kill init! exitcode=0x%08x\n",
tsk->signal->group_exit_code ?: (int)code);
#ifdef CONFIG_POSIX_TIMERS
hrtimer_cancel(&tsk->signal->real_timer);
exit_itimers(tsk);
#endif
if (tsk->mm)
setmax_mm_hiwater_rss(&tsk->signal->maxrss, tsk->mm);
}
posixtimer_exit(group_dead);
acct_collect(code, group_dead);
if (group_dead)
tty_audit_exit();

2
kernel/futex/requeue.c
View File

@ -744,7 +744,7 @@ int handle_early_requeue_pi_wakeup(struct futex_hash_bucket *hb,
/* Handle spurious wakeups gracefully */
ret = -EWOULDBLOCK;
if (timeout && !timeout->task)
if (timeout && !hrtimer_sleeper_task_get(timeout))
ret = -ETIMEDOUT;
else if (signal_pending(current))
ret = -ERESTARTNOINTR;

8
kernel/futex/waitwake.c
View File

@ -383,7 +383,7 @@ void futex_do_wait(struct futex_q *q, struct hrtimer_sleeper *timeout)
* flagged for rescheduling. Only call schedule if there
* is no timeout, or if it has yet to expire.
*/
if (!timeout || timeout->task)
if (!timeout || hrtimer_sleeper_task_get(timeout))
schedule();
}
__set_current_state(TASK_RUNNING);
@ -539,7 +539,7 @@ int futex_wait_multiple_setup(struct futex_vector *vs, int count, int *woken)
static void futex_sleep_multiple(struct futex_vector *vs, unsigned int count,
struct hrtimer_sleeper *to)
{
if (to && !to->task)
if (to && !hrtimer_sleeper_task_get(to))
return;
for (; count; count--, vs++) {
@ -590,7 +590,7 @@ int futex_wait_multiple(struct futex_vector *vs, unsigned int count,
if (ret >= 0)
return ret;
if (to && !to->task)
if (to && !hrtimer_sleeper_task_get(to))
return -ETIMEDOUT;
else if (signal_pending(current))
return -ERESTARTSYS;
@ -725,7 +725,7 @@ int __futex_wait(u32 __user *uaddr, unsigned int flags, u32 val,
if (!futex_unqueue(&q))
return 0;
if (to && !to->task)
if (to && !hrtimer_sleeper_task_get(to))
return -ETIMEDOUT;
/*

2
kernel/locking/rtmutex.c
View File

@ -1644,7 +1644,7 @@ static int __sched rt_mutex_slowlock_block(struct rt_mutex_base *lock,
break;
}
if (timeout && !timeout->task) {
if (timeout && !hrtimer_sleeper_task_get(timeout)) {
ret = -ETIMEDOUT;
break;
}

23
kernel/time/hrtimer.c
View File

@ -780,7 +780,8 @@ static void hrtimer_switch_to_hres(void)
return;
}
base->hres_active = true;
hrtimer_resolution = HIGH_RES_NSEC;
if (hrtimer_resolution != HIGH_RES_NSEC)
hrtimer_resolution = HIGH_RES_NSEC;
tick_setup_sched_timer(true);
/* "Retrigger" the interrupt to get things going */
@ -2003,7 +2004,7 @@ bool hrtimer_active(const struct hrtimer *timer)
base = READ_ONCE(timer->base);
seq = raw_read_seqcount_begin(&base->seq);
if (timer->is_queued || base->running == timer)
if (timer->is_queued || READ_ONCE(base->running) == timer)
return true;
} while (read_seqcount_retry(&base->seq, seq) || base != READ_ONCE(timer->base));
@ -2040,7 +2041,7 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base, struct hrtimer_cloc
lockdep_assert_held(&cpu_base->lock);
debug_hrtimer_deactivate(timer);
base->running = timer;
WRITE_ONCE(base->running, timer);
/*
* Separate the ->running assignment from the ->is_queued assignment.
@ -2099,7 +2100,7 @@ static void __run_hrtimer(struct hrtimer_cpu_base *cpu_base, struct hrtimer_cloc
raw_write_seqcount_barrier(&base->seq);
WARN_ON_ONCE(base->running != timer);
base->running = NULL;
WRITE_ONCE(base->running, NULL);
}
static void __hrtimer_run_queues(struct hrtimer_cpu_base *cpu_base, ktime_t now,
@ -2323,9 +2324,9 @@ void hrtimer_run_queues(void)
static enum hrtimer_restart hrtimer_wakeup(struct hrtimer *timer)
{
struct hrtimer_sleeper *t = container_of(timer, struct hrtimer_sleeper, timer);
struct task_struct *task = t->task;
struct task_struct *task = hrtimer_sleeper_task_get(t);
t->task = NULL;
hrtimer_sleeper_task_set(t, NULL);
if (task)
wake_up_process(task);
@ -2354,7 +2355,7 @@ void hrtimer_sleeper_start_expires(struct hrtimer_sleeper *sl, enum hrtimer_mode
/* If already expired, clear the task pointer and set current state to running */
if (!hrtimer_start_expires_user(&sl->timer, mode)) {
sl->task = NULL;
hrtimer_sleeper_task_set(sl, NULL);
__set_current_state(TASK_RUNNING);
}
}
@ -2388,7 +2389,7 @@ static void __hrtimer_setup_sleeper(struct hrtimer_sleeper *sl, clockid_t clock_
}
__hrtimer_setup(&sl->timer, hrtimer_wakeup, clock_id, mode);
sl->task = current;
hrtimer_sleeper_task_set(sl, current);
}
/**
@ -2432,17 +2433,17 @@ static int __sched do_nanosleep(struct hrtimer_sleeper *t, enum hrtimer_mode mod
set_current_state(TASK_INTERRUPTIBLE|TASK_FREEZABLE);
hrtimer_sleeper_start_expires(t, mode);
if (likely(t->task))
if (likely(hrtimer_sleeper_task_get(t)))
schedule();
hrtimer_cancel(&t->timer);
mode = HRTIMER_MODE_ABS;
} while (t->task && !signal_pending(current));
} while (hrtimer_sleeper_task_get(t) && !signal_pending(current));
__set_current_state(TASK_RUNNING);
if (!t->task)
if (!hrtimer_sleeper_task_get(t))
return 0;
restart = &current->restart_block;

103
kernel/time/posix-cpu-timers.c
View File

@ -439,6 +439,38 @@ static void trigger_base_recalc_expires(struct k_itimer *timer,
base->nextevt = 0;
}
static inline bool cpu_timer_enqueue(struct timerqueue_head *head,
struct cpu_timer *ctmr)
{
ctmr->head = head;
return timerqueue_add(head, &ctmr->node);
}
static inline bool cpu_timer_queued(struct cpu_timer *ctmr)
{
return !!ctmr->head;
}
static inline bool cpu_timer_dequeue(struct cpu_timer *ctmr)
{
if (cpu_timer_queued(ctmr)) {
timerqueue_del(ctmr->head, &ctmr->node);
ctmr->head = NULL;
return true;
}
return false;
}
static inline u64 cpu_timer_getexpires(struct cpu_timer *ctmr)
{
return ctmr->node.expires;
}
static inline void cpu_timer_setexpires(struct cpu_timer *ctmr, u64 exp)
{
ctmr->node.expires = exp;
}
/*
* Dequeue the timer and reset the base if it was its earliest expiration.
* It makes sure the next tick recalculates the base next expiration so we
@ -607,6 +639,7 @@ static int posix_cpu_timer_del(struct k_itimer *timer)
}
if (!ret) {
WARN_ON_ONCE(cpu_timer_queued(&timer->it.cpu));
put_pid(timer->it.cpu.pid);
timer->it_status = POSIX_TIMER_DISARMED;
}
@ -639,18 +672,50 @@ static void cleanup_timers(struct posix_cputimers *pct)
cleanup_timerqueue(&pct->bases[CPUCLOCK_SCHED].tqhead);
}
static inline void posix_cpu_timers_exit_work(void);
/*
* These are both called with the siglock held, when the current thread
* is being reaped. When the final (leader) thread in the group is reaped,
* posix_cpu_timers_exit_group will be called after posix_cpu_timers_exit.
* Invoked from posixtimer_exit_task() after PF_EXITING was set in tsk::flags or
* from posixtimer_exec_cleanup().
*/
void posix_cpu_timers_exit(struct task_struct *tsk)
void posix_cpu_timers_exit_task(void)
{
cleanup_timers(&tsk->posix_cputimers);
posix_cpu_timers_exit_work();
guard(spinlock_irq)(&current->sighand->siglock);
cleanup_timers(&current->posix_cputimers);
}
void posix_cpu_timers_exit_group(struct task_struct *tsk)
/*
* Invoked from posixtimer_exit_group() after PF_EXITING was set in tsk::flags.
*/
void posix_cpu_timers_exit_group(void)
{
cleanup_timers(&tsk->signal->posix_cputimers);
posix_cpu_timers_exit_task();
guard(spinlock_irq)(&current->sighand->siglock);
cleanup_timers(&current->signal->posix_cputimers);
}
/*
* This function validates that POSIX CPU timers can be safely enqueued on the
* target task.
*
* Enqueue is allowed when PF_EXITING is not set. If set then it is only allowed
* for process shared timers (type = PIDTYPE_TGID) as long as tsk::signal::flags
* does not have SIGNAL_GROUP_EXIT set. PIDTYPE_PID targets are not allowed at
* all when the task has PF_EXITING set.
*
* This guarantees that after the POSIX timer cleanup in posixtimer_exit() no
* POSIX CPU timers are queued on the task or in case of a group exit on the
* process.
*/
static inline bool task_can_enqueue_timer(struct task_struct *tsk, enum pid_type type)
{
if (likely(!(tsk->flags & PF_EXITING)))
return true;
return type == PIDTYPE_TGID && !(tsk->signal->flags & SIGNAL_GROUP_EXIT);
}
/*
@ -663,7 +728,13 @@ static void arm_timer(struct k_itimer *timer, struct task_struct *p)
struct cpu_timer *ctmr = &timer->it.cpu;
u64 newexp = cpu_timer_getexpires(ctmr);
lockdep_assert_held(&p->sighand->siglock);
timer->it_status = POSIX_TIMER_ARMED;
if (unlikely(!task_can_enqueue_timer(p, clock_pid_type(timer->it_clock))))
return;
if (!cpu_timer_enqueue(&base->tqhead, ctmr))
return;
@ -1201,6 +1272,20 @@ static void posix_cpu_timers_work(struct callback_head *work)
mutex_unlock(&cw->mutex);
}
static inline void posix_cpu_timers_exit_work(void)
{
/* Canceling the work is only valid for exit() but not for exec() */
if (!(current->flags & PF_EXITING))
return;
/*
* current->flags has PF_EXITING set so this can be done lockless and
* with interrupts enabled as PF_EXITING prevents the interrupt from
* scheduling the work.
*/
if (current->posix_cputimers_work.scheduled)
task_work_cancel(current, &current->posix_cputimers_work.work);
}
/*
* Invoked from the posix-timer core when a cancel operation failed because
* the timer is marked firing. The caller holds rcu_read_lock(), which
@ -1331,6 +1416,8 @@ static inline void __run_posix_cpu_timers(struct task_struct *tsk)
lockdep_posixtimer_exit();
}
static inline void posix_cpu_timers_exit_work(void) { }
static void posix_cpu_timer_wait_running(struct k_itimer *timr)
{
cpu_relax();
@ -1477,7 +1564,7 @@ void run_posix_cpu_timers(void)
* posix_cpu_timer_del() may fail to lock_task_sighand(tsk) and
* miss timer->it.cpu.firing != 0.
*/
if (tsk->exit_state)
if (tsk->flags & PF_EXITING)
return;
/*

26
kernel/time/posix-timers.c
View File

@ -1077,13 +1077,9 @@ SYSCALL_DEFINE1(timer_delete, timer_t, timer_id)
return 0;
}
/*
* Invoked from do_exit() when the last thread of a thread group exits.
* At that point no other task can access the timers of the dying
* task anymore.
*/
void exit_itimers(struct task_struct *tsk)
static void posixtimer_delete_timers(void)
{
struct task_struct *tsk = current;
struct hlist_head timers;
struct hlist_node *next;
struct k_itimer *timer;
@ -1120,6 +1116,24 @@ void exit_itimers(struct task_struct *tsk)
}
}
void posixtimer_exit(bool group_dead)
{
if (group_dead) {
hrtimer_cancel(&current->signal->real_timer);
posix_cpu_timers_exit_group();
posixtimer_delete_timers();
} else {
posix_cpu_timers_exit_task();
}
}
void posixtimer_exec(void)
{
posix_cpu_timers_exit_task();
posixtimer_delete_timers();
flush_itimer_signals();
}
SYSCALL_DEFINE2(clock_settime, const clockid_t, which_clock,
const struct __kernel_timespec __user *, tp)
{

3
kernel/time/posix-timers.h
View File

@ -51,3 +51,6 @@ int common_timer_set(struct k_itimer *timr, int flags,
struct itimerspec64 *old_setting);
void posix_timer_set_common(struct k_itimer *timer, struct itimerspec64 *new_setting);
int common_timer_del(struct k_itimer *timer);
void posix_cpu_timers_exit_task(void);
void posix_cpu_timers_exit_group(void);

4
kernel/time/sleep_timeout.c
View File

@ -212,7 +212,7 @@ int __sched schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
hrtimer_set_expires_range_ns(&t.timer, *expires, delta);
hrtimer_sleeper_start_expires(&t, mode);
if (likely(t.task))
if (likely(hrtimer_sleeper_task_get(&t)))
schedule();
hrtimer_cancel(&t.timer);
@ -220,7 +220,7 @@ int __sched schedule_hrtimeout_range_clock(ktime_t *expires, u64 delta,
__set_current_state(TASK_RUNNING);
return !t.task ? 0 : -EINTR;
return !hrtimer_sleeper_task_get(&t) ? 0 : -EINTR;
}
EXPORT_SYMBOL_GPL(schedule_hrtimeout_range_clock);

16
kernel/time/time_test.c
View File

@ -87,8 +87,24 @@ static void time64_to_tm_test_date_range(struct kunit *test)
}
}
static void time64_to_tm_test_wide_day_count(struct kunit *test)
{
/* 2^31 days: the first count that does not fit in a 32-bit long. */
time64_t timestamp = (1LL << 31) * 86400;
struct tm result;
time64_to_tm(timestamp, 0, &result);
KUNIT_EXPECT_EQ(test, result.tm_year, 5879680);
KUNIT_EXPECT_EQ(test, result.tm_mon, 6);
KUNIT_EXPECT_EQ(test, result.tm_mday, 12);
KUNIT_EXPECT_EQ(test, result.tm_yday, 193);
KUNIT_EXPECT_EQ(test, result.tm_wday, 6);
}
static struct kunit_case time_test_cases[] = {
KUNIT_CASE_SLOW(time64_to_tm_test_date_range),
KUNIT_CASE(time64_to_tm_test_wide_day_count),
{}
};

6
kernel/time/timeconv.c
View File

@ -49,8 +49,9 @@ void time64_to_tm(time64_t totalsecs, int offset, struct tm *result)
u32 u32tmp, day_of_century, year_of_century, day_of_year, month, day;
u64 u64tmp, udays, century, year;
bool is_Jan_or_Feb, is_leap_year;
long days, rem;
int remainder;
long rem;
s64 days;
days = div_s64_rem(totalsecs, SECS_PER_DAY, &remainder);
rem = remainder;
@ -70,7 +71,8 @@ void time64_to_tm(time64_t totalsecs, int offset, struct tm *result)
result->tm_sec = rem % 60;
/* January 1, 1970 was a Thursday. */
result->tm_wday = (4 + days) % 7;
div_s64_rem(days + 4, 7, &remainder);
result->tm_wday = remainder;
if (result->tm_wday < 0)
result->tm_wday += 7;

4
kernel/time/timekeeping.c
View File

@ -861,8 +861,10 @@ static void timekeeping_update_from_shadow(struct tk_data *tkd, unsigned int act
*
* Write xtime_sec first so that even if the memcpy() tears the store
* data integrity is provided for ktime_get_real_seconds().
* The same goes for ktime_sec and ktime_get_seconds().
*/
WRITE_ONCE(tkd->timekeeper.xtime_sec, tk->xtime_sec);
WRITE_ONCE(tkd->timekeeper.ktime_sec, tk->ktime_sec);
memcpy(&tkd->timekeeper, tk, sizeof(*tk));
write_seqcount_end(&tkd->seq);
}
@ -1169,7 +1171,7 @@ time64_t ktime_get_seconds(void)
struct timekeeper *tk = &tk_core.timekeeper;
WARN_ON(timekeeping_suspended);
return tk->ktime_sec;
return READ_ONCE(tk->ktime_sec);
}
EXPORT_SYMBOL_GPL(ktime_get_seconds);

2
kernel/time/timer.c
View File

@ -890,7 +890,7 @@ static inline void detach_timer(struct timer_list *timer, bool clear_pending)
__hlist_del(entry);
if (clear_pending)
entry->pprev = NULL;
WRITE_ONCE(entry->pprev, NULL);
entry->next = LIST_POISON2;
}

6
kernel/time/timer_migration.c
View File

@ -715,7 +715,7 @@ static void __tmigr_cpu_activate(struct tmigr_cpu *tmc)
trace_tmigr_cpu_active(tmc);
tmc->cpuevt.ignore = true;
WRITE_ONCE(tmc->cpuevt.ignore, true);
WRITE_ONCE(tmc->wakeup, KTIME_MAX);
walk_groups(&tmigr_active_up, &data, tmc);
@ -1258,7 +1258,7 @@ u64 tmigr_cpu_new_timer(u64 nextexp)
ret = READ_ONCE(tmc->wakeup);
if (nextexp != KTIME_MAX) {
if (nextexp != tmc->cpuevt.nextevt.expires ||
tmc->cpuevt.ignore) {
READ_ONCE(tmc->cpuevt.ignore)) {
ret = tmigr_new_timer(tmc, nextexp);
/*
* Make sure the reevaluation of timers in idle path
@ -1362,7 +1362,7 @@ static u64 __tmigr_cpu_deactivate(struct tmigr_cpu *tmc, u64 nextexp)
* or CPU goes offline.
*/
if (nextexp != KTIME_MAX)
tmc->cpuevt.ignore = false;
WRITE_ONCE(tmc->cpuevt.ignore, false);
walk_groups(&tmigr_inactive_up, &data, tmc);
return data.firstexp;

4
net/core/pktgen.c
View File

@ -2346,11 +2346,11 @@ static void spin(struct pktgen_dev *pkt_dev, ktime_t spin_until)
set_current_state(TASK_INTERRUPTIBLE);
hrtimer_sleeper_start_expires(&t, HRTIMER_MODE_ABS);
if (likely(t.task))
if (likely(hrtimer_sleeper_task_get(&t)))
schedule();
hrtimer_cancel(&t.timer);
} while (t.task && pkt_dev->running && !signal_pending(current));
} while (hrtimer_sleeper_task_get(&t) && pkt_dev->running && !signal_pending(current));
__set_current_state(TASK_RUNNING);
end_time = ktime_get();
}

23
tools/testing/selftests/timers/clocksource-switch.c
View File

@ -40,16 +40,23 @@
int get_clocksources(char list[][30])
{
int fd, i;
size_t size;
ssize_t size;
char buf[512];
char *head, *tmp;
fd = open("/sys/devices/system/clocksource/clocksource0/available_clocksource", O_RDONLY);
if (fd < 0)
return 0;
size = read(fd, buf, 512);
size = read(fd, buf, sizeof(buf) - 1);
close(fd);
if (size <= 0)
return 0;
buf[size] = '\0';
for (i = 0; i < 10; i++)
list[i][0] = '\0';
@ -74,11 +81,21 @@ int get_clocksources(char list[][30])
int get_cur_clocksource(char *buf, size_t size)
{
ssize_t len;
int fd;
fd = open("/sys/devices/system/clocksource/clocksource0/current_clocksource", O_RDONLY);
if (fd < 0)
return -1;
size = read(fd, buf, size);
len = read(fd, buf, size - 1);
close(fd);
if (len <= 0)
return -1;
buf[len] = '\0';
return 0;
}

7
tools/testing/selftests/timers/posix_timers.c
View File

@ -113,9 +113,10 @@ static void check_itimer(int which, const char *name)
done = 0;
if (which == ITIMER_VIRTUAL)
if (which == ITIMER_VIRTUAL) {
clock_id = CLOCK_THREAD_CPUTIME_ID;
signal(SIGVTALRM, sig_handler);
else if (which == ITIMER_PROF) {
} else if (which == ITIMER_PROF) {
clock_id = CLOCK_THREAD_CPUTIME_ID;
signal(SIGPROF, sig_handler);
}
@ -148,7 +149,7 @@ static void check_timer_create(int which)
struct itimerspec val = {
.it_value.tv_sec = DELAY,
};
int clock_id = CLOCK_REALTIME;
int clock_id = which;
timer_t id;
done = 0;

12
tools/testing/selftests/timers/raw_skew.c
View File

@ -91,6 +91,7 @@ int main(int argc, char **argv)
{
struct timespec mon, raw, start, end;
long long delta1, delta2, interval, eppm, ppm;
long tick_nom;
struct timex tx1, tx2;
setbuf(stdout, NULL);
@ -129,6 +130,17 @@ int main(int argc, char **argv)
/* Avg the two actual freq samples adjtimex gave us */
ppm = (long long)(tx1.freq + tx2.freq) * 1000 / 2;
ppm = shift_right(ppm, 16);
/*
* The tick value holds the coarse part of the adjustment, a
* microsecond of the tick length per unit, and time sync daemons
* do put part of their correction there. What it is worth has to
* be counted in as well, or a clock disciplined through it looks
* off by a hundred ppm a unit against what the two clocks show.
*/
tick_nom = USEC_PER_SEC / sysconf(_SC_CLK_TCK);
ppm += (long long)(tx1.tick + tx2.tick - 2 * tick_nom) *
(USEC_PER_SEC / tick_nom) * 1000 / 2;
printf(" %lld.%i(act)", ppm/1000, abs((int)(ppm%1000)));
if (llabs(eppm - ppm) > 1000) {