Tianchen Ding e6c228b950 sched: Fix the check of nr_running at queue wakelist ... [ Upstream commit 28156108fe ] The commit 2ebb177175 ("sched/core: Offload wakee task activation if it the wakee is descheduling") checked rq->nr_running <= 1 to avoid task stacking when WF_ON_CPU. Per the ordering of writes to p->on_rq and p->on_cpu, observing p->on_cpu (WF_ON_CPU) in ttwu_queue_cond() implies !p->on_rq, IOW p has gone through the deactivate_task() in __schedule(), thus p has been accounted out of rq->nr_running. As such, the task being the only runnable task on the rq implies reading rq->nr_running == 0 at that point. The benchmark result is in [1]. [1] https://lore.kernel.org/all/e34de686-4e85-bde1-9f3c-9bbc86b38627@linux.alibaba.com/ Suggested-by: Valentin Schneider <vschneid@redhat.com> Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Valentin Schneider <vschneid@redhat.com> Link: https://lore.kernel.org/r/20220608233412.327341-2-dtcccc@linux.alibaba.com Signed-off-by: Sasha Levin <sashal@kernel.org>		2022-08-21 15:16:13 +02:00
..
bpf	bpf: Fix subprog names in stack traces.	2022-08-21 15:15:47 +02:00
cgroup	sched, cpuset: Fix dl_cpu_busy() panic due to empty cs->cpus_allowed	2022-08-21 15:16:12 +02:00
configs
debug	lockdown: also lock down previous kgdb use	2022-05-30 09:33:22 +02:00
dma	dma-direct: don't over-decrypt memory	2022-06-22 14:13:20 +02:00
entry	KVM: rseq: Update rseq when processing NOTIFY_RESUME on xfer to KVM guest	2021-10-06 15:55:49 +02:00
events	perf/core: Fix data race between perf_event_set_output() and perf_mmap_close()	2022-07-29 17:19:11 +02:00
gcov
irq	genirq: GENERIC_IRQ_IPI depends on SMP	2022-08-21 15:15:28 +02:00
kcsan
livepatch	livepatch: Fix build failure on 32 bits processors	2022-04-08 14:40:15 +02:00
locking	locking/lockdep: Fix lockdep_init_map_*() confusion	2022-08-21 15:15:33 +02:00
power	PM: hibernate: defer device probing when resuming from hibernation	2022-08-21 15:15:31 +02:00
printk	printk: fix return value of printk.devkmsg __setup handler	2022-04-08 14:40:08 +02:00
rcu	rcu: Make TASKS_RUDE_RCU select IRQ_WORK	2022-06-09 10:20:51 +02:00
sched	sched: Fix the check of nr_running at queue wakelist	2022-08-21 15:16:13 +02:00
time	wireguard: ratelimiter: use hrtimer in selftest	2022-08-21 15:15:52 +02:00
trace	blktrace: Trace remapped requests correctly	2022-08-21 15:15:36 +02:00
.gitignore
acct.c
async.c	Revert "module, async: async_synchronize_full() on module init iff async is used"	2022-02-23 12:01:00 +01:00
audit_fsnotify.c
audit_tree.c	audit: move put_tree() to avoid trim_trees refcount underflow and UAF	2021-09-03 10:09:31 +02:00
audit_watch.c
audit.c	audit: improve audit queue handling when "audit=1" on cmdline	2022-02-08 18:30:34 +01:00
audit.h	audit: log AUDIT_TIME_* records only from rules	2022-04-08 14:40:00 +02:00
auditfilter.c
auditsc.c	audit: log AUDIT_TIME_* records only from rules	2022-04-08 14:40:00 +02:00
backtracetest.c
bounds.c
capability.c
compat.c
configs.c
context_tracking.c
cpu_pm.c	PM: cpu: Make notifier chain use a raw_spinlock_t	2021-09-15 09:50:40 +02:00
cpu.c	random: clear fast pool, crng, and batches in cpuhp bring up	2022-05-30 09:33:36 +02:00
crash_core.c
crash_dump.c
cred.c	Revert "Add a reference to ucounts for each cred"	2021-09-08 08:49:00 +02:00
delayacct.c
dma.c
exec_domain.c
exit.c	fix race between exit_itimers() and /proc/pid/timers	2022-07-21 21:19:59 +02:00
extable.c
fail_function.c
fork.c	copy_process(): Move fd_install() out of sighand->siglock critical section	2022-02-23 12:01:08 +01:00
freezer.c
futex.c
gen_kheaders.sh
groups.c
hung_task.c
iomem.c
irq_work.c
jump_label.c
kallsyms.c
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kexec_core.c
kexec_elf.c
kexec_file.c	ima: force signature verification when CONFIG_KEXEC_SIG is configured	2022-07-21 21:20:11 +02:00
kexec_internal.h
kexec.c
kheaders.c
kmod.c
kprobes.c	kprobes: Forbid probing on trampoline and BPF code areas	2022-08-21 15:16:11 +02:00
ksysfs.c
kthread.c	kthread: Fix PF_KTHREAD vs to_kthread() race	2021-09-03 10:09:31 +02:00
latencytop.c
Makefile
module_signature.c
module_signing.c
module-internal.h
module.c	module: check for exit sections in layout_sections() instead of module_init_section()	2022-05-25 09:18:02 +02:00
notifier.c
nsproxy.c
padata.c
panic.c
params.c
pid_namespace.c	memcg: enable accounting for pids in nested pid namespaces	2021-09-18 13:40:36 +02:00
pid.c
profile.c	profiling: fix shift too large makes kernel panic	2022-08-21 15:16:05 +02:00
ptrace.c	ptrace: Reimplement PTRACE_KILL by always sending SIGKILL	2022-06-09 10:20:49 +02:00
range.c
reboot.c
regset.c
relay.c
resource.c
rseq.c	rseq: Remove broken uapi field layout on 32-bit little endian	2022-04-08 14:40:03 +02:00
scftorture.c	scftorture: Fix distribution of short handler delays	2022-06-09 10:21:01 +02:00
scs.c
seccomp.c	seccomp: Fix setting loaded filter count during TSYNC	2021-08-18 08:59:06 +02:00
signal.c	signal handling: don't use BUG_ON() for debugging	2022-07-21 21:20:18 +02:00
smp.c	smp: Fix offline cpu check in flush_smp_call_function_queue()	2022-04-20 09:23:29 +02:00
smpboot.c
smpboot.h
softirq.c
stackleak.c	gcc-plugins/stackleak: Use noinstr in favor of notrace	2022-02-23 12:01:00 +01:00
stacktrace.c
static_call.c	static_call: Fix unused variable warn w/o MODULE	2021-09-08 08:49:00 +02:00
stop_machine.c
sys_ni.c
sys.c	prctl: allow to setup brk for et_dyn executables	2021-09-26 14:08:57 +02:00
sysctl-test.c
sysctl.c	mm: sysctl: fix missing numa_stat when !CONFIG_HUGETLB_PAGE	2022-07-21 21:20:13 +02:00
task_work.c
taskstats.c
test_kprobes.c
torture.c
tracepoint.c	tracepoint: Use rcu get state and cond sync for static call updates	2021-09-03 10:09:30 +02:00
tsacct.c	taskstats: Cleanup the use of task->exit_code	2022-01-27 10:54:33 +01:00
ucount.c	Revert "Add a reference to ucounts for each cred"	2021-09-08 08:49:00 +02:00
uid16.c
uid16.h
umh.c
up.c
user_namespace.c	Revert "Add a reference to ucounts for each cred"	2021-09-08 08:49:00 +02:00
user-return-notifier.c
user.c
usermode_driver.c
utsname_sysctl.c
utsname.c
watch_queue.c	watch_queue: Fix missing locking in add_watch_to_object()	2022-08-03 12:00:44 +02:00
watchdog_hld.c
watchdog.c
workqueue_internal.h
workqueue.c	workqueue: Fix unbind_workers() VS wq_worker_running() race	2022-01-16 09:14:22 +01:00