Because of online cpu not make sure active, but active
make sure online, so when want to get active_cpus, make
it and cpu_active_mask.
Change-Id: Ie73b85506252edfad1f1258c9e059b0167dfe4a5
Signed-off-by: Tengfei Fan <tengfeif@codeaurora.org>
Increase the nr_threshold percentage to 40 from 15.
Change-Id: I32ce7246fc4cd32d4c8110bef63971c9a2dceb55
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Create an event to periodically dump the configuration
of the pmu counter registers. This can be used in
conjunction with the existing sched_switch_with_ctrs
event to confirm which counter is configured in each
register.
Change-Id: Iaf11279e94d4af3f8aff522282585e680acd5f4d
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
While updating cpu_rq->cpu_capacity, we are ignoring rt pressure, and
using same as cpu_capacity_orig, which might lead to unfair scheduler
decisions.
Fix it by accouting rt scale to cpu_capacity.
Change-Id: Icead0af0d8a309cc5f93438a4d4d7e7e7c93653c
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I6f953cd04ada8ccb36bbf1e2fe341375092b35c4
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Improve core_ctl tracing such that the necessary flags for
debugging eval_need are present in the trace, and so that
all returns go through the same tracepoint, unlocking, and
return functionality.
Change-Id: I61d5ab86ba7650bea77e4416b0ffa9a07869bbf3
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I42edf6c6df05f0bf72e4c57ee701916860773428
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I17cca453e5a4e6bbf4d0b55440d9f776dc2583cb
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Check if necessary to do exit and next_boost enter
oprations instead of doing it regardlessly.
Core control boost refcounts has been wrong if doing
current sched_boost exit and next_boost enter by mistake.
Change-Id: Ie40ebf0be0e1b99291d07d160ebef5964d44ba16
Signed-off-by: Maria Yu <aiquny@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: I9c8c1a9c00534700b63834384c3553145ae8c935
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I4ccf43485d46451837d3114de370b5ea9bc1f824
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I8e5fbdb142a891f79b0073b2fc0ee7e81d46c845
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: Iae112690c5cf125db5d197555d47244a63bfb32e
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
It is normal case to see hotplug become disabled and when
this happens, pause_cpus and resume_cpus will fail. To
walt_pause this is an issue as the requested action cannot
be performed, but not from a system perspective.
Reduce the severity of the walt_pause error upon failure
of pause_cpus() and resume_cpus() to debug only. Corrective
action is taken, so the warning doesn't need to be on the
console.
Change-Id: I5667a1efe46c07379e31bcfd9ff1cfeedd5be64c
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
When there is a failure with the call to pause_cpus() or
resume_cpus() core_ctl will receive that error and retry
on a window boundary. Currently this results in a
printk_deferred message going to the console very frequently
for a normal use case (hotplug disabled).
Change these console messages to pr_debug() to eliminate
the messages unless DEBUG is defined, and loglevel is increased.
Change-Id: I99bd04cd71dbc0b365df07dea3776a11afa6b7d5
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Walt debug code needs to go through the macro used to control
whether or not a crash will happen on error. Additionally,
update code so that when crashing is not supported, walt
will attempt to correct the error.
Change-Id: I2c9efb38e395db90b9fbfbf310329d2775ee7fae
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
It is necessary to maintain debug checks in the walt code
while preventing errant crashes in released code.
Create a sysctl node that will control whether walt issues
will crash the system, and create a macro to take advantage
of it. Support the debug strings required by walt, so that
this can be easily incorporated into walt files.
Change-Id: Ie81cefe3c6d730574599f8d804d010f4a917bba7
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Validate that cumulative_runnable_avg is zero when the next
scheduled task is the idle task.
Change-Id: Ia14f9c773c48bf44dfb71080b9360dd473b73923
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Validate that the task selected is in a good state.
on_cpu must not be 1 when picking a task. on_cpu is only
set in prepare_task() just before the task is switched in.
on_rq must not be 0 or migrating when picking a task,
because it must be active. Only active tasks are
are enqueued on a runqueue, and a migrating task shouldn't
be picked as the rq it is on should be locked doing
the migration.
cpu must be equal to the rq's cpu. pick_next_task must
not be picking a task which is not enqueued in this rq's
cpu.
One exception to the above, is when the scheduler picks
a task that is the same as the previous task. In this case
the task might already be on_cpu.
Change-Id: I07c94c3b2eab0a328222f29550e0515cdcd6e7ed
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: Ic1b53470b3ebc9a8c2d34d34200c4b671af95840
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Need restricted hook so that it can sleep.
Change-Id: I2b546f57f7ee58beed3dad84f201fd075fa0d21d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
The dequeue cpu must always be the same as the enqueue cpu.
Write a check to store off the enqueue cpu in the appropriate
hook, and check the same at dequeue.
Change-Id: I16bbc8e769b2a49c72b394456dde92213c6f3370
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: I8e6d7afe12387d86f3721c6fd880812ba86d885e
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
[satyap@codeaurora.org: port to msm-5.10]
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
When the placement code has to skip energy evaluation and quickly return
the best cpu amongst the candidates, the current code chooses the max
capacity cpu within them. This leads to overcrowding on the max cpu,
as we could be operating in the skip energy eval mode for a long time.
So instead of choosing max capacity cpu, choose a cpu with most spare
capacity.
Change-Id: I13e435cbfc802c5a1002703f928b5f56ef5d8e1f
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently the code is finding the max_cap_cpu amongst candidates
but is unused if we are energy evaluating. Clean up the code.
Change-Id: I317ed8328851b69534c375c69387647a94ada659
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
When need_idle flag is set & lower capacity CPU is idle, there is no need
to go for energy evaluation with higher capacity CPUs. Placement code used
to skip energy evaluation in previous Kernels, this commit matches the
need_idle functionality of previous Kernels.
Change-Id: Ib21baa2b99e882e27550894ef814ed7114d01e8c
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
It is possible that the Pause and Resume operations fail and
using pr_warn() could lead to core ctl task running for long
period when it has to push the logbuf data to console which
eventually cause RT throttling. Avoid this scenario by using
printk_deferred() instead of pr_warn().
Change-Id: I2717205216caf994a8995d6f11178499a7015201
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Debug capabilities are needed throughout walt files, while
the walt-specific debug/dumping capabilities are strictly in
walt.c and dependent upon local variables.
Support the ability to cause a crash across all walt functionality.
Change-Id: I10fffcf028e39964f92b4a5c05dfd35f0eff63fb
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Use unsigned long data type for counter values since the
pmu and amu counters can be 64 bit based on the hardware
capability.
Change-Id: I27f5e4d0741d7a0a1a7a6cc17edb9028a138708d
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
The current code does not honor sync wakeup. Make it do that.
Change-Id: I509fd704accdebd32ad2281c26a23e2044c6dc09
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Currently, when adjusting the max util with respect to current cluster
frequency, we use capacity_orig_of as the multiplier. This is incorrect,
because capacity_orig_of will change depending on the current max freq.
Change-Id: Ifb38c46a8dc68d18458f6b683f0e8e66857a0644
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
It is possible that pause/resume of CPUs can fail, but
walt_pause/resume_cpus() always returns zero which turns
even failures into success. This would leave clients go
into bad state in case of failures. To avoid such issues
separate out ref count updates and clearing pause/resume
masks (when the CPUs are already paused/resumed cases) and
rearrange the code appropriately to make sure ref counting
is properly handled in case of errors.
Change-Id: I284752bf9372cada30a2b648f1c9c54446923117
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I306ff3e6fb571efd7ab6cfc36edcbb4dfa264c3d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The dequeue boolean flag is used to calculate the
busy hysteresis time which helps delay lpms. Fix the logic
to detect whether its an enqueue or a dequeue.
Change-Id: Ie7177bba80aa3d164a27587290194c0d13929384
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Use IS_ENABLED macro to check for a config flag instead of using
ifdef.
Change-Id: I1c703f9abddf1f66e5b2795f4cf9a1b230f8da4f
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Currently, we boost frequencies by 25% across the board. However, due to
region 3 silver CPUs energy concerns, we don't want to apply this boost
provided that the utilization on silver CPUs is high enough.
Additionally, there is the caveat that if the contribution from RT task
is high enough (currently hard coded to 25% of silver capacity), we
continue to apply the boost.
To manipulate the silver threshold (set to 1000 by default, therefore
this is disabled), use:
echo x > /proc/sys/walt/sched_silver_thres .
Change-Id: If689d0935f0c60f7ccc19e63780286c815eb801a
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add AMU based cycle and instruction counts to sched_switch_with_ctrs
tracepoint. This will help get cycle and instruction counts if the
AMUs are supported by hardware.
Change-Id: Ia014b6b5afb143f4f790350843f37ae479075ae7
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
The current code in force_compatible_cpus_allowed_ptr() holds just
the cpus_read_lock, i.e. serializes with the second step of pause.
However it doesn't serialize with the first step. IOW, a CPUX chosen
in __set_cpus_allowed_ptr_locked() may end up getting deactivated in
the pause first and stopper ends up returning early in __migrate_tasks()
leaving the 32bit task on an invalid CPU.
A simple fix is to serialize force_compatible_cpus_allowed_ptr() with
pause first step. Add a cpu_maps_update_begin/done() in
force_compatible_cpus_allowed_ptr().
Change-Id: I24e7838ee9813b4336bfcebc823e498c3606c793
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Under FBT, we currently do not check if an active candidate is actually
valid before setting it as part of the candidates mask. This can lead to
page faults.
Change-Id: I5aedb3bd1e076736270aa03b8889c67606c2a25d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
In order to debug tasks not being added to top-app group correctly,
introduce a tracepoint to aid with such scenarios.
Change-Id: Ibdfaf00f65661dc2700f30c951ab8be711cda2b2
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
When the system is suspending, the console itself can be locked
up for a period of time. Unfortunately, pr_err will wait until
the console is unlocked, causing significant delays. This causes
a defect that seems to point to walt_pause.
Fix this by ensuring that all pr_err statements are eliminated
by using printk_deferred throughout walt_pause. This will ensure
that the operation is not locked on console access.
Change-Id: Id2e93a33d5c272fe0ee9b6f18bef5eb8ad4de07d
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
The current code in force_compatible_cpus_allowed_ptr() holds just
the cpus_read_lock, i.e. serializes with the second step of pause.
However it doesn't serialize with the first step. IOW, a CPUX chosen
in __set_cpus_allowed_ptr_locked() may end up getting deactivated in
the pause first and stopper ends up returning early in __migrate_tasks()
leaving the 32bit task on an invalid CPU.
A simple fix is to serialize force_compatible_cpus_allowed_ptr() with
pause first step. Add a cpu_maps_update_begin/done() in
force_compatible_cpus_allowed_ptr().
Change-Id: If118aabf9e1252e58dcfe52609d59520c029bc41
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
walt_init() is a function that could sleep and it should not be called
in an atomic context.
Introduce a work to do walt init.
Change-Id: Ief3e9500d7675b998220693ac1908ac74e876f50
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
To catch early errors, it's important to ensure that the walt debug
module is insmodded early.
Change-Id: I1418c16a30041469aa04025018a1f0a79065c390
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
When calculating max_util for energy evaluations, we adjust the util on
the basis of frequency, using arch_scale_freq_capacity. However, the
return value is scaled to 1024 and not to the capacity of the CPU.
Change-Id: I214ac4b40b88105147c2e5dcc350ff1601496d4c
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Per cluster energy contribution is essential to debug energy evaluation.
Add code to print it.
Change-Id: Ifbe80d125b09facd358933ab3537260fea0584ac
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
rcu unlock is being called in multiple places in walt_find_best_target().
Consolidate these calls.
Change-Id: I86701bc975fcc7d68b6faa95b952b9dd40c2f8a9
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Currently, we are determining the L2 cache sibling of silvers at runtime
on every iteration FBT. It is better to do this setup once, and gather
this information from hardware rather than hardcoding the results.
Change-Id: I06ec2f590dc9ccc2ec9161f1a60eb47d77244153
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ieca864be1e26e39c56fd0be6be135d4085e5d8aa
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
