It is necessary to maintain debug checks in the walt code
while preventing errant crashes in released code.
Create a sysctl node that will control whether walt issues
will crash the system, and create a macro to take advantage
of it. Support the debug strings required by walt, so that
this can be easily incorporated into walt files.
Change-Id: Ie81cefe3c6d730574599f8d804d010f4a917bba7
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Validate that cumulative_runnable_avg is zero when the next
scheduled task is the idle task.
Change-Id: Ia14f9c773c48bf44dfb71080b9360dd473b73923
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Validate that the task selected is in a good state.
on_cpu must not be 1 when picking a task. on_cpu is only
set in prepare_task() just before the task is switched in.
on_rq must not be 0 or migrating when picking a task,
because it must be active. Only active tasks are
are enqueued on a runqueue, and a migrating task shouldn't
be picked as the rq it is on should be locked doing
the migration.
cpu must be equal to the rq's cpu. pick_next_task must
not be picking a task which is not enqueued in this rq's
cpu.
One exception to the above, is when the scheduler picks
a task that is the same as the previous task. In this case
the task might already be on_cpu.
Change-Id: I07c94c3b2eab0a328222f29550e0515cdcd6e7ed
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: Ic1b53470b3ebc9a8c2d34d34200c4b671af95840
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Need restricted hook so that it can sleep.
Change-Id: I2b546f57f7ee58beed3dad84f201fd075fa0d21d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
The dequeue cpu must always be the same as the enqueue cpu.
Write a check to store off the enqueue cpu in the appropriate
hook, and check the same at dequeue.
Change-Id: I16bbc8e769b2a49c72b394456dde92213c6f3370
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: I8e6d7afe12387d86f3721c6fd880812ba86d885e
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
[satyap@codeaurora.org: port to msm-5.10]
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
When the placement code has to skip energy evaluation and quickly return
the best cpu amongst the candidates, the current code chooses the max
capacity cpu within them. This leads to overcrowding on the max cpu,
as we could be operating in the skip energy eval mode for a long time.
So instead of choosing max capacity cpu, choose a cpu with most spare
capacity.
Change-Id: I13e435cbfc802c5a1002703f928b5f56ef5d8e1f
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently the code is finding the max_cap_cpu amongst candidates
but is unused if we are energy evaluating. Clean up the code.
Change-Id: I317ed8328851b69534c375c69387647a94ada659
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
When need_idle flag is set & lower capacity CPU is idle, there is no need
to go for energy evaluation with higher capacity CPUs. Placement code used
to skip energy evaluation in previous Kernels, this commit matches the
need_idle functionality of previous Kernels.
Change-Id: Ib21baa2b99e882e27550894ef814ed7114d01e8c
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
It is possible that the Pause and Resume operations fail and
using pr_warn() could lead to core ctl task running for long
period when it has to push the logbuf data to console which
eventually cause RT throttling. Avoid this scenario by using
printk_deferred() instead of pr_warn().
Change-Id: I2717205216caf994a8995d6f11178499a7015201
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Debug capabilities are needed throughout walt files, while
the walt-specific debug/dumping capabilities are strictly in
walt.c and dependent upon local variables.
Support the ability to cause a crash across all walt functionality.
Change-Id: I10fffcf028e39964f92b4a5c05dfd35f0eff63fb
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Use unsigned long data type for counter values since the
pmu and amu counters can be 64 bit based on the hardware
capability.
Change-Id: I27f5e4d0741d7a0a1a7a6cc17edb9028a138708d
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
The current code does not honor sync wakeup. Make it do that.
Change-Id: I509fd704accdebd32ad2281c26a23e2044c6dc09
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Currently, when adjusting the max util with respect to current cluster
frequency, we use capacity_orig_of as the multiplier. This is incorrect,
because capacity_orig_of will change depending on the current max freq.
Change-Id: Ifb38c46a8dc68d18458f6b683f0e8e66857a0644
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
It is possible that pause/resume of CPUs can fail, but
walt_pause/resume_cpus() always returns zero which turns
even failures into success. This would leave clients go
into bad state in case of failures. To avoid such issues
separate out ref count updates and clearing pause/resume
masks (when the CPUs are already paused/resumed cases) and
rearrange the code appropriately to make sure ref counting
is properly handled in case of errors.
Change-Id: I284752bf9372cada30a2b648f1c9c54446923117
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I306ff3e6fb571efd7ab6cfc36edcbb4dfa264c3d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The dequeue boolean flag is used to calculate the
busy hysteresis time which helps delay lpms. Fix the logic
to detect whether its an enqueue or a dequeue.
Change-Id: Ie7177bba80aa3d164a27587290194c0d13929384
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Use IS_ENABLED macro to check for a config flag instead of using
ifdef.
Change-Id: I1c703f9abddf1f66e5b2795f4cf9a1b230f8da4f
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Currently, we boost frequencies by 25% across the board. However, due to
region 3 silver CPUs energy concerns, we don't want to apply this boost
provided that the utilization on silver CPUs is high enough.
Additionally, there is the caveat that if the contribution from RT task
is high enough (currently hard coded to 25% of silver capacity), we
continue to apply the boost.
To manipulate the silver threshold (set to 1000 by default, therefore
this is disabled), use:
echo x > /proc/sys/walt/sched_silver_thres .
Change-Id: If689d0935f0c60f7ccc19e63780286c815eb801a
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add AMU based cycle and instruction counts to sched_switch_with_ctrs
tracepoint. This will help get cycle and instruction counts if the
AMUs are supported by hardware.
Change-Id: Ia014b6b5afb143f4f790350843f37ae479075ae7
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
The current code in force_compatible_cpus_allowed_ptr() holds just
the cpus_read_lock, i.e. serializes with the second step of pause.
However it doesn't serialize with the first step. IOW, a CPUX chosen
in __set_cpus_allowed_ptr_locked() may end up getting deactivated in
the pause first and stopper ends up returning early in __migrate_tasks()
leaving the 32bit task on an invalid CPU.
A simple fix is to serialize force_compatible_cpus_allowed_ptr() with
pause first step. Add a cpu_maps_update_begin/done() in
force_compatible_cpus_allowed_ptr().
Change-Id: I24e7838ee9813b4336bfcebc823e498c3606c793
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Under FBT, we currently do not check if an active candidate is actually
valid before setting it as part of the candidates mask. This can lead to
page faults.
Change-Id: I5aedb3bd1e076736270aa03b8889c67606c2a25d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
In order to debug tasks not being added to top-app group correctly,
introduce a tracepoint to aid with such scenarios.
Change-Id: Ibdfaf00f65661dc2700f30c951ab8be711cda2b2
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
When the system is suspending, the console itself can be locked
up for a period of time. Unfortunately, pr_err will wait until
the console is unlocked, causing significant delays. This causes
a defect that seems to point to walt_pause.
Fix this by ensuring that all pr_err statements are eliminated
by using printk_deferred throughout walt_pause. This will ensure
that the operation is not locked on console access.
Change-Id: Id2e93a33d5c272fe0ee9b6f18bef5eb8ad4de07d
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
The current code in force_compatible_cpus_allowed_ptr() holds just
the cpus_read_lock, i.e. serializes with the second step of pause.
However it doesn't serialize with the first step. IOW, a CPUX chosen
in __set_cpus_allowed_ptr_locked() may end up getting deactivated in
the pause first and stopper ends up returning early in __migrate_tasks()
leaving the 32bit task on an invalid CPU.
A simple fix is to serialize force_compatible_cpus_allowed_ptr() with
pause first step. Add a cpu_maps_update_begin/done() in
force_compatible_cpus_allowed_ptr().
Change-Id: If118aabf9e1252e58dcfe52609d59520c029bc41
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
walt_init() is a function that could sleep and it should not be called
in an atomic context.
Introduce a work to do walt init.
Change-Id: Ief3e9500d7675b998220693ac1908ac74e876f50
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
To catch early errors, it's important to ensure that the walt debug
module is insmodded early.
Change-Id: I1418c16a30041469aa04025018a1f0a79065c390
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
When calculating max_util for energy evaluations, we adjust the util on
the basis of frequency, using arch_scale_freq_capacity. However, the
return value is scaled to 1024 and not to the capacity of the CPU.
Change-Id: I214ac4b40b88105147c2e5dcc350ff1601496d4c
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Per cluster energy contribution is essential to debug energy evaluation.
Add code to print it.
Change-Id: Ifbe80d125b09facd358933ab3537260fea0584ac
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
rcu unlock is being called in multiple places in walt_find_best_target().
Consolidate these calls.
Change-Id: I86701bc975fcc7d68b6faa95b952b9dd40c2f8a9
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Currently, we are determining the L2 cache sibling of silvers at runtime
on every iteration FBT. It is better to do this setup once, and gather
this information from hardware rather than hardcoding the results.
Change-Id: I06ec2f590dc9ccc2ec9161f1a60eb47d77244153
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ieca864be1e26e39c56fd0be6be135d4085e5d8aa
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently we use ~6% (right shift by 4 is really 6.25%) as a
threshold of energy
saving to move a task to a cpu other than the prev_cpu.
Change it to 3%.
Change-Id: I0d0177bbef2c22bcf37e6da42c3ed857f986c478
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ib4c5091436e0536bcb2ee63e9a31d9b1dd51dbe9
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The proper kernel provides tracehooks to indicate when its done
initializing topology. Use that hook to simplify walt initialization.
Change-Id: Ifa7540f2f65bf469c1f8007bb15d5e1317475929
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I9e3d2f76f07c2b2f2a5d4a6d9831489251de1455
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
When a task is being placed on silver and is not a filtered task and is
not a small task, look for a gold for this task.
Change-Id: Ic6f5a047e4f25c66f4b7be1f0b48ca49a2c5aa50
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently, the code uses walt util for energy evaluation. That could be
inaccurate because walt prs signals drive cpufreq. Change to using
prs.
Change-Id: Iea073c7963bf27ab1308950b07a2aead1c73805c
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
A cluster that does not host prev cpu and or any of the candidates
doesn't affect the energy values. Don't calculate their contributions.
Change-Id: Id1b890dbf157fa27d90287202f7916c6ab102bba
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently, energy evaluation occurs only on one target cpu chosen
across all CPU clusters.
Change the algorithm to track target cpus for each cluster and
subsequently perform energy calculations on these candidates.
For a cluster, if both an idle cpu and a busy cpu is found, prefer
idle cpu. That also means, once idle if found there is no need to
visit upcoming busy cpus as we traverse the cpus in a cluster.
Change-Id: Idd832748cdd01e0995783b1959332acd66e94cd6
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently the sched_switch_ctrs trace event is dependent on
debugfs to get enabled, and not compatible with all builds.
Move the sched_switch_ctrs to dynamic trace point registration
under walt dynamic tp, as the trace event has considerable
overhead and need to enabled on need basis.
Change-Id: I6d9eb328ecf14a15e27d67dd5c1a46b0383aa860
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Some trace events might not need to be enabled always, but
needed for debug purpose. It is not optimal to register trace
event, which is used for debug purpose only. So registering such
trace events might add extra overhead in hotpaths.
So add control knob for registering such trace events to trace points,
which can be enabled whenever needed them. And can be unregister when
not needed.
To enable such event registration, write:
echo 1 > /proc/sys/walt/sched_enable_tp
And register sched_overutilized trace event with dynamic support, this
event is copied from Qais's debug change in chrome project:
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2128628
Change-Id: I8983e31d405a635b38c7c985b56c2f564855d3e6
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Disable LPM for hysteresis time if CONSERVATIVE_BOOST is enabled.
Change-Id: If88513b664c00ecb2cd48961b907bec719e822a9
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Certain unsigned int values are being specified as %lu instead of %u and
it is leading to incorrectly displayed values.
Change-Id: I26f65ef24a89c277e2da0f88c1e3d683ef545057
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
When a task is not on a rq when it should be, halt and
print some useful information.
Change-Id: Iaecbf372c712acba98879c8afb4e073d50a962f7
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Add util_busy_hist feature which looks at the total cpu_util
across all CPUs as a metric for LPM biasing on CPUs.
Change-Id: Ie148178dfdc8f2b13fd447b1a87d60176f53f4a7
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
