Currently, we make certain decisions based on capacities of individual
CPUs. However, we may be interested in grouping certain cpus of
different capacities together. In this case, we will need to use their
cluster IDs to differentiate between two CPUs.
Change-Id: I20d9c473a98e55dfdd658b96fc9285f0a0f51652
Signed-off-by: Shaleen Agrawal <quic_shalagra@quicinc.com>
Update variable type to ensure all the valid values can be
accomodated in variable.
Change-Id: Ieaf0301d6ad524bfb87ff019aec41063ebc4c4c4
Signed-off-by: Ashay Jaiswal <quic_ashayj@quicinc.com>
walt_rt_energy_aware_wake_cpu could return -1, which makes
walt_select_task_rq_rt use the task's prev cpu (the backup cpu).
However, the backup_cpu could be halted.
Address this by replacing the chosen cpu with the lowest order non-halted
cpu in the system.
Change-Id: I366bd2e9093b945a093ae4542a77983b0bc57229
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
When the android vendor hook is not registered yet, there
is already tasks. And if those kind of tasks being killed
without any wakeup after vendor hook registered, it will
be a NULL referenced data.
Change-Id: Ied2eb3805dd01740a01707e2016c24a4648b1b25
Signed-off-by: Maria Yu <quic_aiquny@quicinc.com>
The sliding window logic in update_history is unnecessary complicated
and involves moving each window entry into a new position. Instead, we
can simply replace the older entries in place.
Change-Id: I49a80184241393219f421421f07514e01b8a674e
Signed-off-by: Shaleen Agrawal <quic_shalagra@quicinc.com>
After reviewing, it was determind that the can_halt_mask is no
longer needed. Remove it and cleanup code as appropriate.
Change-Id: I5430e6f3eda505e0f8fe1cedb9fa7b20425f6900
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
cpu_active mask is a super-set of cpu_online. Any cpus that are
active are by definition online, so it is redundant to check both.
Cleanup the usage of cpu_active and cpu_online, to eliminate
redundancies and ensure usage of the super set (active) when
needed.
Change-Id: I085cb94a609819b6786c81d8abc784aae8b4ff54
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
If a task is enqueued on a CPU for which it is not allowed
as dictated by the cpus_ptr of the task then report the issue.
Change-Id: Ib049e9ae07be1830d9ed6166c47aa78aff7e1b12
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Minor cleanup including renaming of the drain functions and
commenting the update_halt_cpus function and usage.
Change-Id: I44502cde850af8fc6124e7b8ae014c562b21d0d9
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
The number history samples should be raised to a higher value so that an
active history of 100ms is considered in accounting task demand. Raise
the history size from 5 to 8 given the default windows size is 8ms. This
will capture 64ms of history for 8ms windows, and 128ms of history for
16ms windows, granting a compromise to prevent further code bloat.
Change-Id: I63ad5b2a20c9a32d3a65448a78d6aec98afe6ad5
Signed-off-by: Shaleen Agrawal <quic_shalagra@quicinc.com>
Adapt the min/max capacity cpu helper functions to use cluster IDs to
guide decisions rather than capacities.
Change-Id: Ic204f286c8eb95742ab64d366ce30d5ee780c340
Signed-off-by: Shaleen Agrawal <quic_shalagra@quicinc.com>
This change is for general scheduler improvement.
Change-Id: I6235ca8fce3d0c2ab3bcab1ecee0d97c6d0a9942
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
In walt_find_energy_efficient_cpu, the trace for compute energy
was being called just prior to an update to best energy cpu. The
goal of the trace is to print the selected state of the routine,
not an intermediate state.
Change-Id: I8132b42e65b9b3bada9316c0ba738fa3baac9f49
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
This change is for general scheduler improvement.
Change-Id: I997827d143a370e49e43ecba8494ee8e69652b57
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
select_task_rq_fair invokes the select_task_rq_fair hook, which
ultimately invokes walt_find_energy_aware_cpu for cfs tasks. This
can return a cpu that is halted.
Update the one remaining case that can cause feec to return a
halted cpu.
Change-Id: Ic97345764ad8be1060868414869dc0e455f4bc54
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
select_task_rq_rt() and push_rt_task() need to select
a cpu, and ultimately attempt to get a good match on
a cpu through find_lowest_rq. Walt hooks into this
routine to make an adjustment such that the best energy
aware cpu is found.
It is possible that walt_rt_energy_aware_wake_cpu will be
unable to find a non-halted cpu. find_lowest_rq at that
point will use the lowest_mask, initialized from the
task cpus_ptr, to find a CPU. At this point, find_lowest_rq
might return a halted CPU.
Instead, in the case where walt cannot find a best cpu to
choose, force the lowest_mask to no longer include halted
cpus, which will push find_lowest_rq to no longer choose a
halted cpu. In the unlikely event that find_lowest_rq still
fails to find a cpu it will pick anything from the
lowest_mask. In the unlikely event that lowest_mask is empty
after removing all of the halted cpus, find_lowest_rq will
fail and the cpu chosen will be unconstrained by halt.
Change-Id: I398bc804eb474396f6f06e2c3d4308fea71dafe8
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
rto_next_cpu can select any rd->rto_mask cpu, which may be
halted. restrict rto_next_cpu to unhalted CPUs.
Change-Id: I1c41c4c72b630d00cca080ed73609f6fdfb7143d
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
When set_cpus_allowed_ptr_locked runs, it chooses a new destination
cpu for the task based upon the new mask for the task. Unfortunately
this means that halted cpus can be included.
Register for a trace hook such that dest_cpu can be recomputed, and
a dest_cpu that is not halted, chosen.
Change-Id: I0ab072eb395bece8c0ab896d7a1008f9cf55f468
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
rebalance_domains will call load_balance, and ultimately will
call detach_tasks, walt_migrated_queued_task, and set_task_cpu, choosing
a cpu that is halted.
can_migrate_task will be used to validate whether a particular migration
is allowed to happen. Reject migrations for tasks that are being
migrated to a halted cpu.
Change-Id: I0b2f5d3857a2b1db9a2fa5017d5238f143a85ed9
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
The halted flag must be checked when performing a wakeup of
an rt task, to prevent tasks from running on halted CPUs.
Change-Id: Ib4a5cff48611abb0d1f71fc10be45439d2b7dd64
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Currently walt_find_best_target can return a halted cpu.
Prevent the PREV_CPU_FASTPATH for cpus that are currently
halted, and reject them in general from being selected.
Change-Id: Iced61bf9b1355703c6403d1dc9f90d6e64aefdc6
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
If a halted CPU goes idle, don't try to pull tasks into
the CPU.
Change-Id: I50383f736ae03992b8bb43f5aa12cc2803068f6f
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Make the minimum change for core_ctl to work properly with halt
instead of pause.
Change-Id: I7d2a2b50daaffc96846c86b8fd9dedf7f604b0b6
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
halt/start tracepoints need a little enhancement, to ensure
a measurement can be made from the start of an operation.
Also to debug RT task placements, add the lowest_mask
cpu mask which contains the cpus available for the
RT task placement algorithm to trace_sched_cpu_util, and
update the trace to show the halted state.
Change-Id: Idd5ab03d8ca5af21d42415dce790993c8c5df1d5
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
References to pause have been removed and replaced with
halt. Remove walt pause functionality from the build.
Change-Id: I07c412cfba4d9f13aff298aed43a0e208db9f046
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Provide apis to the system in support of halting
cpus, as a replacement for pause/isolation.
Track the requests to halt a cpu, and only
start the cpu again when all requests have been
recanted.
Perform a migration of tasks off of a halted cpu,
if that cpu is online and not idle.
Change-Id: I99340b12e01ff53499b4107d843b7d48e6d6dff1
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Optimize walt_irq_work to reduce locking for migrations.
Reduce scheduler overheads by reducing the locks needed
for the walt irq work migrations. Precisely track the
cpus locked, and unlock them at conclusion of work function.
Change-Id: I6ea6d24dc69e9876387253b489e91f82e45ed6d0
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
When a task yields, it relinquishes the cpu and scheduler
is tasked to find another task.
However he MVP implementation could return the same task
leading to a loop where the yielded task gets to run back.
To fix this, drop the MVP status of tasks that yield
themselves. The MVP status is restored the next time they
actually get enqueued - -likely because of a wakeup from
sleep or task migration.
Change-Id: Ia2a2074f18c4de56e0695ee2c5f1daf8a4eb5980
Signed-off-by: Tengfei Fan <tengfeif@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
As a side effect of commit f64fafbec582dd46169fba528cad2341c0c2d906
the check to skip frequency update if the same frequency as
the previous frequency is being applied, got removed at the point of
update.
Restore this check at the appropriate place.
Change-Id: I7aee7f84f0a64747b01ac31c2fdcbb4440908deb
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Restore obj file in Makefile that was removed in
commit 3908a213bc56 ("pause: port the users of pause").
Change-Id: I2014b049ca7cb5e4188b07087cc22fa9ee40cf85
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
This change is for general scheduler improvement.
Change-Id: Iffd4ae221581aaa4aeb244a0cddd40a8b6aac74d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvements.
Change-Id: I37d6cb75ca8b08d9ca155b86b7d71ff369f46e14
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
This change is for general scheduler improvements.
Change-Id: Ia2854ae8701151761fe0780b6451133ab09a050b
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
When CPU trying to move a task to other cpu in active load balance or
by other means, then the other helping cpu marked as reserved to avoid
it for other scheduler decisions. Once the task moved successfully,
the reservation will be cleared enables for other scheduler decisions.
The reserved flag is been analogously protected with busy cpu’s
rq->active_balance, which is protected with runqueue locks. So whenever
rq->active_balance is set for busy cpu, then reserved flag would set for
helping cpu.
Sometimes, it is observed that, cpu is marked as reserved with no cpu's
rq->active_balance set. There are some unlikely possible corner cases
may cause this behavior:
- On active load balance path, cpu stop machine returns queued status
of active_balance work on cpu_stopper, which is not checked on active
balance path. so when stop machine is not able to queue ( unlikely),
then reserved flag wouldn't be cleared.
So, catch the return value and on failure, clear reserved flag for cpu.
- Clear_walt_request() called on the cpu to clear any pending walt works,
it may possible that, push_task might have changed or cleared, then
the reserved cpu would be left uncleared.
So clear the push_cpu independent of push_task.
Change-Id: I75d032bf399cb3da8e807186b1bc903114168a4e
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
There is a bad cast from unsigned long to unsigned int that is taking
place in sched_cpu_util tracepoint, and this is causing bad values to
appear in the traces.
Change-Id: I51561ac582b616e3319c8f43648cf5a4bfc632b6
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: If9c4859601bdcb1ab92bf6d7125bcb74c9ec3f7b
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
push_task assignment is guarded by corresponding runqueue's lock. Read
it only when lock is acquired.
Change-Id: I74668df66b4e18286b919b136d912a3f8afda11c
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
get_pid_task internally increments the reference counter, which could
cause unbalanced reference counter (always high). Only call
get_task_struct when pid is 0.
Change-Id: I409b477d0714ce35a0dc980782ee5878106ecffd
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
To determine why a particular long preemption or long irqs off
period has happened, without crashing the device, the tracepoints
need improvement. The callers should be included.
Change-Id: If147969717517563e56cd20d3c036f51835a5c73
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
This change is for general scheduler improvemnts
Change-Id: I0a0b065ff2d1a563bccdd161dad890fae26f11c3
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Upstream has done away with get_online_cpus and put_online_cpus wrappers
around cpu_read_lock() and cpu_read_unlock().
Reflect these changes downstream.
Change-Id: Ie7ae59b6faba93ea43ce1ba3a4b043ae87e1337f
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Upstream does not carry pause feature. Remove related references.
Change-Id: I7ac0352a761fa78af3918a5b3a61424df0d22a2f
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Aandoned upstream, reflecting it in kernel/msm-5.10.
Change-Id: I0c1ebd96372803519b180641707562034996479b
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
This change is for general scheduler improvement.
Change-Id: Id758a4a0a836fe30b9f63b8f00c017247cf3b84c
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Change the FIE to use frequency scaling information provided by
cpufreq driver as opposed to the higher priority arch specific counters.
This is done by resetting the struct scale_freq_data for each CPU which
holds the callback information specific to the HW counters, thus forcing
the FIE to revert to default, the cpufreq driver.
Change-Id: I23f508f9859be6a35155e5d000caf1c0b19f5584
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Port the users of pause from the 5.10 kernel to ensure code is up to date.
Change-Id: Ibe52cc37e0dca425f2f472b9cf254a2453bd9c47
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
num_sched_clusters has to be less than MAX_CLUSTERS, complain otherwise.
Change-Id: I0648585139d31dc948cf73aae7d093757a5762cc
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The rq lock variable in struct rq is changed upstream.
Reflect changes downstream.
Change-Id: I8711d42dbe10f769b8568895993b92a6bcbcd69e
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
'task_of' function is defined in sched.h
Remove redefinition in walt_cfs.c
Change-Id: I07dd511fae0a74a03ebf2ae47f2b227404866972
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
