It is possible that pause/resume of CPUs can fail, but
walt_pause/resume_cpus() always returns zero which turns
even failures into success. This would leave clients go
into bad state in case of failures. To avoid such issues
separate out ref count updates and clearing pause/resume
masks (when the CPUs are already paused/resumed cases) and
rearrange the code appropriately to make sure ref counting
is properly handled in case of errors.
Change-Id: I284752bf9372cada30a2b648f1c9c54446923117
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I306ff3e6fb571efd7ab6cfc36edcbb4dfa264c3d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The dequeue boolean flag is used to calculate the
busy hysteresis time which helps delay lpms. Fix the logic
to detect whether its an enqueue or a dequeue.
Change-Id: Ie7177bba80aa3d164a27587290194c0d13929384
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Use IS_ENABLED macro to check for a config flag instead of using
ifdef.
Change-Id: I1c703f9abddf1f66e5b2795f4cf9a1b230f8da4f
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Currently, we boost frequencies by 25% across the board. However, due to
region 3 silver CPUs energy concerns, we don't want to apply this boost
provided that the utilization on silver CPUs is high enough.
Additionally, there is the caveat that if the contribution from RT task
is high enough (currently hard coded to 25% of silver capacity), we
continue to apply the boost.
To manipulate the silver threshold (set to 1000 by default, therefore
this is disabled), use:
echo x > /proc/sys/walt/sched_silver_thres .
Change-Id: If689d0935f0c60f7ccc19e63780286c815eb801a
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add AMU based cycle and instruction counts to sched_switch_with_ctrs
tracepoint. This will help get cycle and instruction counts if the
AMUs are supported by hardware.
Change-Id: Ia014b6b5afb143f4f790350843f37ae479075ae7
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
The current code in force_compatible_cpus_allowed_ptr() holds just
the cpus_read_lock, i.e. serializes with the second step of pause.
However it doesn't serialize with the first step. IOW, a CPUX chosen
in __set_cpus_allowed_ptr_locked() may end up getting deactivated in
the pause first and stopper ends up returning early in __migrate_tasks()
leaving the 32bit task on an invalid CPU.
A simple fix is to serialize force_compatible_cpus_allowed_ptr() with
pause first step. Add a cpu_maps_update_begin/done() in
force_compatible_cpus_allowed_ptr().
Change-Id: I24e7838ee9813b4336bfcebc823e498c3606c793
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Under FBT, we currently do not check if an active candidate is actually
valid before setting it as part of the candidates mask. This can lead to
page faults.
Change-Id: I5aedb3bd1e076736270aa03b8889c67606c2a25d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
In order to debug tasks not being added to top-app group correctly,
introduce a tracepoint to aid with such scenarios.
Change-Id: Ibdfaf00f65661dc2700f30c951ab8be711cda2b2
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
When the system is suspending, the console itself can be locked
up for a period of time. Unfortunately, pr_err will wait until
the console is unlocked, causing significant delays. This causes
a defect that seems to point to walt_pause.
Fix this by ensuring that all pr_err statements are eliminated
by using printk_deferred throughout walt_pause. This will ensure
that the operation is not locked on console access.
Change-Id: Id2e93a33d5c272fe0ee9b6f18bef5eb8ad4de07d
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
The current code in force_compatible_cpus_allowed_ptr() holds just
the cpus_read_lock, i.e. serializes with the second step of pause.
However it doesn't serialize with the first step. IOW, a CPUX chosen
in __set_cpus_allowed_ptr_locked() may end up getting deactivated in
the pause first and stopper ends up returning early in __migrate_tasks()
leaving the 32bit task on an invalid CPU.
A simple fix is to serialize force_compatible_cpus_allowed_ptr() with
pause first step. Add a cpu_maps_update_begin/done() in
force_compatible_cpus_allowed_ptr().
Change-Id: If118aabf9e1252e58dcfe52609d59520c029bc41
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
walt_init() is a function that could sleep and it should not be called
in an atomic context.
Introduce a work to do walt init.
Change-Id: Ief3e9500d7675b998220693ac1908ac74e876f50
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
To catch early errors, it's important to ensure that the walt debug
module is insmodded early.
Change-Id: I1418c16a30041469aa04025018a1f0a79065c390
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
When calculating max_util for energy evaluations, we adjust the util on
the basis of frequency, using arch_scale_freq_capacity. However, the
return value is scaled to 1024 and not to the capacity of the CPU.
Change-Id: I214ac4b40b88105147c2e5dcc350ff1601496d4c
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Per cluster energy contribution is essential to debug energy evaluation.
Add code to print it.
Change-Id: Ifbe80d125b09facd358933ab3537260fea0584ac
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
rcu unlock is being called in multiple places in walt_find_best_target().
Consolidate these calls.
Change-Id: I86701bc975fcc7d68b6faa95b952b9dd40c2f8a9
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Currently, we are determining the L2 cache sibling of silvers at runtime
on every iteration FBT. It is better to do this setup once, and gather
this information from hardware rather than hardcoding the results.
Change-Id: I06ec2f590dc9ccc2ec9161f1a60eb47d77244153
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ieca864be1e26e39c56fd0be6be135d4085e5d8aa
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently we use ~6% (right shift by 4 is really 6.25%) as a
threshold of energy
saving to move a task to a cpu other than the prev_cpu.
Change it to 3%.
Change-Id: I0d0177bbef2c22bcf37e6da42c3ed857f986c478
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ib4c5091436e0536bcb2ee63e9a31d9b1dd51dbe9
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The proper kernel provides tracehooks to indicate when its done
initializing topology. Use that hook to simplify walt initialization.
Change-Id: Ifa7540f2f65bf469c1f8007bb15d5e1317475929
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I9e3d2f76f07c2b2f2a5d4a6d9831489251de1455
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
When a task is being placed on silver and is not a filtered task and is
not a small task, look for a gold for this task.
Change-Id: Ic6f5a047e4f25c66f4b7be1f0b48ca49a2c5aa50
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently, the code uses walt util for energy evaluation. That could be
inaccurate because walt prs signals drive cpufreq. Change to using
prs.
Change-Id: Iea073c7963bf27ab1308950b07a2aead1c73805c
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
A cluster that does not host prev cpu and or any of the candidates
doesn't affect the energy values. Don't calculate their contributions.
Change-Id: Id1b890dbf157fa27d90287202f7916c6ab102bba
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently, energy evaluation occurs only on one target cpu chosen
across all CPU clusters.
Change the algorithm to track target cpus for each cluster and
subsequently perform energy calculations on these candidates.
For a cluster, if both an idle cpu and a busy cpu is found, prefer
idle cpu. That also means, once idle if found there is no need to
visit upcoming busy cpus as we traverse the cpus in a cluster.
Change-Id: Idd832748cdd01e0995783b1959332acd66e94cd6
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently the sched_switch_ctrs trace event is dependent on
debugfs to get enabled, and not compatible with all builds.
Move the sched_switch_ctrs to dynamic trace point registration
under walt dynamic tp, as the trace event has considerable
overhead and need to enabled on need basis.
Change-Id: I6d9eb328ecf14a15e27d67dd5c1a46b0383aa860
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Some trace events might not need to be enabled always, but
needed for debug purpose. It is not optimal to register trace
event, which is used for debug purpose only. So registering such
trace events might add extra overhead in hotpaths.
So add control knob for registering such trace events to trace points,
which can be enabled whenever needed them. And can be unregister when
not needed.
To enable such event registration, write:
echo 1 > /proc/sys/walt/sched_enable_tp
And register sched_overutilized trace event with dynamic support, this
event is copied from Qais's debug change in chrome project:
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2128628
Change-Id: I8983e31d405a635b38c7c985b56c2f564855d3e6
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Disable LPM for hysteresis time if CONSERVATIVE_BOOST is enabled.
Change-Id: If88513b664c00ecb2cd48961b907bec719e822a9
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Certain unsigned int values are being specified as %lu instead of %u and
it is leading to incorrectly displayed values.
Change-Id: I26f65ef24a89c277e2da0f88c1e3d683ef545057
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
When a task is not on a rq when it should be, halt and
print some useful information.
Change-Id: Iaecbf372c712acba98879c8afb4e073d50a962f7
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Add util_busy_hist feature which looks at the total cpu_util
across all CPUs as a metric for LPM biasing on CPUs.
Change-Id: Ie148178dfdc8f2b13fd447b1a87d60176f53f4a7
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Add util_busy_hist sysctl tunables which solely
look at the sum of cpu_util across CPUs as a metric for
LPM biasing on any/all CPUs.
Change-Id: Ib981c21a58059ef02772ee797808c5f30547a688
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Printing CPUs info is useful while going through error paths.
Change-Id: If84794163b6c417d419785fdddd2eed98588f2eb
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
The rq clock gets updated in the scheduler tick. However, if the
lock is acquired remotely between the scheduler tick and the
vendor hook, the clock update flags gets reset. Do the clock
update in walt_cfs_account_mvp_runtime based on the clock update
flags.
Change-Id: Ie4254a191a18458cf1684070bc96ff63cf0103d3
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Protect MVP tasks list access in the tick path with rq lock.
Change-Id: I387aa104cbacaf445bf739b8fb6d631283bd0ada
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I3280c12b5446b8c12fd4269efc2c1dad70e4ae6a
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Change min_freq and max_freq to be policy_min_freq, policy_max_freq.
Change-Id: Ief7ee31209338e6fc25f4ad8933b60256ac5eb95
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
When experiencing frequency related issues, it will be helpful to be
aware of any potential clamping issues, as well as reasons why we might
end up not updating to a different frequency.
Change-Id: I0be24d713421357e077cd01d5c84f43138a17b24
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add sched_sync_hint_enable not to enable/disable sync.
Change-Id: I455545487d002b4e3caf96c6cf8569549c3b1bbe
Signed-off-by: Tengfei Fan <tengfeif@codeaurora.org>
lock ordering issue is observed in below scenario where preferred cluster
for the task is being set on one CPU while task is being attached to
related thread group which leads to deadlock.
CPUX:
scheduler_tick() --> acquire rq_lock()
trace_android_rvh_tick_entry()
android_rvh_tick_entry()
set_preferred_cluster() --> try to acquire RTG lock taken by CPUY
CPUY:
cpu_cgroup_attach()
trace_android_rvh_cpu_cgroup_attach()
android_rvh_cpu_cgroup_attach()
__sched_set_group_id()
add_task_to_group() --> acquire RTG lock
__task_rq_lock() --> try to acquire task's rq_lock() which is CPUX
Fix the issue by moving set_preferred_cluster() related piece of code
which is being called from scheduler_tick() to android_vh_scheduler_tick
trace hook.
Change-Id: Ib9e2d70ae45a304d21cde3de6b8a84f9049061ab
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I5f08d40d6e9f18a905d87db078fa1a70abaa9030
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The parameter to lockdep_assert_held should be a pointer to a lock, not the
lock itself.
Change-Id: I4586d86b8684b21370a2bc522862932481fef538
Signed-off-by: Elliot Berman <eberman@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I9a94b20b18bf5b78946fd26d3337f5227d735e03
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I2a10a999f94e2f21eb97761516c135aadbe305e7
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Walt placement code could return the prev_cpu for placement without
checking if it is in affinity mask.
Fix it by checking task's cpus_ptr.
Change-Id: I6edf2249bccffd1868dd546dc18b45af311cc91d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
