This change is for general scheduler improvement.
Change-Id: I000f0d6aa486da31cf805f2aa1d21a821f6ebfdc
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Id6818d7f79e8babb316aff3568cb65397df53c6f
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ie81267e970c55da4f5f5f2a6ffb9b7f64e48503c
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ia289e3ac635e22e66ae14c7cb2d171b4ad6bbf60
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I7e2f06b0f93d6c3c2cc439f895083f4b7cb96212
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I509129b041185e8f6b83eb586b7ce542889ebe5d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I78a13f15f7b95d7b6583222fbc68b828e651b802
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Id0ed126e0d5bd18957438d4d79bce6c310791c00
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Until recently, we used to do all the dequeue task accounting business
after the call the dequeue task call was done. However, due to an
upstream change in the position of the trace hook, we modified our value
adds to account for doing this work prior to dequeue task. However,
there were unforeseen consequences as a result. As there is
now an "after" dequeue hook available, it is more intuitive to do this
work after dequeue task has taken place, and furthermore, this keeps the
work in line with the manner in which it is done on prior chipsets
ensuring stability.
Change-Id: Iaa274ed39780316fd2d462ad22d4d8b1772def45
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Energy is defined as the cost * load. Currently, the cost is derived
from prs and sum_util is also the sum of prs. This is unoptimal, as
prs denotes frequency, but not load. Therefore, it is
better to use CRA to denote the load instead.
Change-Id: I503f0895951390ad111be24daab6cc76c289821b
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Improve the naming and types of variables in eval_need,
reduce computation and adjust the tracepoint to reflect
the new names of variables.
Change-Id: Ifc19bc802bbc8382577b0e4ed4b4c9be77ffe99e
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for a general scheduler improvement.
Change-Id: I68b72c7b557edfff685b58ec78525b806f0a7142
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Because of online cpu not make sure active, but active
make sure online, so when want to get active_cpus, make
it and cpu_active_mask.
Change-Id: Ie73b85506252edfad1f1258c9e059b0167dfe4a5
Signed-off-by: Tengfei Fan <tengfeif@codeaurora.org>
Increase the nr_threshold percentage to 40 from 15.
Change-Id: I32ce7246fc4cd32d4c8110bef63971c9a2dceb55
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Create an event to periodically dump the configuration
of the pmu counter registers. This can be used in
conjunction with the existing sched_switch_with_ctrs
event to confirm which counter is configured in each
register.
Change-Id: Iaf11279e94d4af3f8aff522282585e680acd5f4d
Signed-off-by: Amir Vajid <avajid@codeaurora.org>
While updating cpu_rq->cpu_capacity, we are ignoring rt pressure, and
using same as cpu_capacity_orig, which might lead to unfair scheduler
decisions.
Fix it by accouting rt scale to cpu_capacity.
Change-Id: Icead0af0d8a309cc5f93438a4d4d7e7e7c93653c
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I6f953cd04ada8ccb36bbf1e2fe341375092b35c4
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Improve core_ctl tracing such that the necessary flags for
debugging eval_need are present in the trace, and so that
all returns go through the same tracepoint, unlocking, and
return functionality.
Change-Id: I61d5ab86ba7650bea77e4416b0ffa9a07869bbf3
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I42edf6c6df05f0bf72e4c57ee701916860773428
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I17cca453e5a4e6bbf4d0b55440d9f776dc2583cb
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Check if necessary to do exit and next_boost enter
oprations instead of doing it regardlessly.
Core control boost refcounts has been wrong if doing
current sched_boost exit and next_boost enter by mistake.
Change-Id: Ie40ebf0be0e1b99291d07d160ebef5964d44ba16
Signed-off-by: Maria Yu <aiquny@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: I9c8c1a9c00534700b63834384c3553145ae8c935
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I4ccf43485d46451837d3114de370b5ea9bc1f824
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I8e5fbdb142a891f79b0073b2fc0ee7e81d46c845
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: Iae112690c5cf125db5d197555d47244a63bfb32e
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
It is normal case to see hotplug become disabled and when
this happens, pause_cpus and resume_cpus will fail. To
walt_pause this is an issue as the requested action cannot
be performed, but not from a system perspective.
Reduce the severity of the walt_pause error upon failure
of pause_cpus() and resume_cpus() to debug only. Corrective
action is taken, so the warning doesn't need to be on the
console.
Change-Id: I5667a1efe46c07379e31bcfd9ff1cfeedd5be64c
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
When there is a failure with the call to pause_cpus() or
resume_cpus() core_ctl will receive that error and retry
on a window boundary. Currently this results in a
printk_deferred message going to the console very frequently
for a normal use case (hotplug disabled).
Change these console messages to pr_debug() to eliminate
the messages unless DEBUG is defined, and loglevel is increased.
Change-Id: I99bd04cd71dbc0b365df07dea3776a11afa6b7d5
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Walt debug code needs to go through the macro used to control
whether or not a crash will happen on error. Additionally,
update code so that when crashing is not supported, walt
will attempt to correct the error.
Change-Id: I2c9efb38e395db90b9fbfbf310329d2775ee7fae
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
It is necessary to maintain debug checks in the walt code
while preventing errant crashes in released code.
Create a sysctl node that will control whether walt issues
will crash the system, and create a macro to take advantage
of it. Support the debug strings required by walt, so that
this can be easily incorporated into walt files.
Change-Id: Ie81cefe3c6d730574599f8d804d010f4a917bba7
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Validate that cumulative_runnable_avg is zero when the next
scheduled task is the idle task.
Change-Id: Ia14f9c773c48bf44dfb71080b9360dd473b73923
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Validate that the task selected is in a good state.
on_cpu must not be 1 when picking a task. on_cpu is only
set in prepare_task() just before the task is switched in.
on_rq must not be 0 or migrating when picking a task,
because it must be active. Only active tasks are
are enqueued on a runqueue, and a migrating task shouldn't
be picked as the rq it is on should be locked doing
the migration.
cpu must be equal to the rq's cpu. pick_next_task must
not be picking a task which is not enqueued in this rq's
cpu.
One exception to the above, is when the scheduler picks
a task that is the same as the previous task. In this case
the task might already be on_cpu.
Change-Id: I07c94c3b2eab0a328222f29550e0515cdcd6e7ed
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: Ic1b53470b3ebc9a8c2d34d34200c4b671af95840
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Need restricted hook so that it can sleep.
Change-Id: I2b546f57f7ee58beed3dad84f201fd075fa0d21d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
The dequeue cpu must always be the same as the enqueue cpu.
Write a check to store off the enqueue cpu in the appropriate
hook, and check the same at dequeue.
Change-Id: I16bbc8e769b2a49c72b394456dde92213c6f3370
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvemnts.
Change-Id: I8e6d7afe12387d86f3721c6fd880812ba86d885e
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
[satyap@codeaurora.org: port to msm-5.10]
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
When the placement code has to skip energy evaluation and quickly return
the best cpu amongst the candidates, the current code chooses the max
capacity cpu within them. This leads to overcrowding on the max cpu,
as we could be operating in the skip energy eval mode for a long time.
So instead of choosing max capacity cpu, choose a cpu with most spare
capacity.
Change-Id: I13e435cbfc802c5a1002703f928b5f56ef5d8e1f
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Currently the code is finding the max_cap_cpu amongst candidates
but is unused if we are energy evaluating. Clean up the code.
Change-Id: I317ed8328851b69534c375c69387647a94ada659
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
When need_idle flag is set & lower capacity CPU is idle, there is no need
to go for energy evaluation with higher capacity CPUs. Placement code used
to skip energy evaluation in previous Kernels, this commit matches the
need_idle functionality of previous Kernels.
Change-Id: Ib21baa2b99e882e27550894ef814ed7114d01e8c
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
It is possible that the Pause and Resume operations fail and
using pr_warn() could lead to core ctl task running for long
period when it has to push the logbuf data to console which
eventually cause RT throttling. Avoid this scenario by using
printk_deferred() instead of pr_warn().
Change-Id: I2717205216caf994a8995d6f11178499a7015201
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Debug capabilities are needed throughout walt files, while
the walt-specific debug/dumping capabilities are strictly in
walt.c and dependent upon local variables.
Support the ability to cause a crash across all walt functionality.
Change-Id: I10fffcf028e39964f92b4a5c05dfd35f0eff63fb
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Use unsigned long data type for counter values since the
pmu and amu counters can be 64 bit based on the hardware
capability.
Change-Id: I27f5e4d0741d7a0a1a7a6cc17edb9028a138708d
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
The current code does not honor sync wakeup. Make it do that.
Change-Id: I509fd704accdebd32ad2281c26a23e2044c6dc09
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Currently, when adjusting the max util with respect to current cluster
frequency, we use capacity_orig_of as the multiplier. This is incorrect,
because capacity_orig_of will change depending on the current max freq.
Change-Id: Ifb38c46a8dc68d18458f6b683f0e8e66857a0644
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
It is possible that pause/resume of CPUs can fail, but
walt_pause/resume_cpus() always returns zero which turns
even failures into success. This would leave clients go
into bad state in case of failures. To avoid such issues
separate out ref count updates and clearing pause/resume
masks (when the CPUs are already paused/resumed cases) and
rearrange the code appropriately to make sure ref counting
is properly handled in case of errors.
Change-Id: I284752bf9372cada30a2b648f1c9c54446923117
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I306ff3e6fb571efd7ab6cfc36edcbb4dfa264c3d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The dequeue boolean flag is used to calculate the
busy hysteresis time which helps delay lpms. Fix the logic
to detect whether its an enqueue or a dequeue.
Change-Id: Ie7177bba80aa3d164a27587290194c0d13929384
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Use IS_ENABLED macro to check for a config flag instead of using
ifdef.
Change-Id: I1c703f9abddf1f66e5b2795f4cf9a1b230f8da4f
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Currently, we boost frequencies by 25% across the board. However, due to
region 3 silver CPUs energy concerns, we don't want to apply this boost
provided that the utilization on silver CPUs is high enough.
Additionally, there is the caveat that if the contribution from RT task
is high enough (currently hard coded to 25% of silver capacity), we
continue to apply the boost.
To manipulate the silver threshold (set to 1000 by default, therefore
this is disabled), use:
echo x > /proc/sys/walt/sched_silver_thres .
Change-Id: If689d0935f0c60f7ccc19e63780286c815eb801a
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add AMU based cycle and instruction counts to sched_switch_with_ctrs
tracepoint. This will help get cycle and instruction counts if the
AMUs are supported by hardware.
Change-Id: Ia014b6b5afb143f4f790350843f37ae479075ae7
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
The current code in force_compatible_cpus_allowed_ptr() holds just
the cpus_read_lock, i.e. serializes with the second step of pause.
However it doesn't serialize with the first step. IOW, a CPUX chosen
in __set_cpus_allowed_ptr_locked() may end up getting deactivated in
the pause first and stopper ends up returning early in __migrate_tasks()
leaving the 32bit task on an invalid CPU.
A simple fix is to serialize force_compatible_cpus_allowed_ptr() with
pause first step. Add a cpu_maps_update_begin/done() in
force_compatible_cpus_allowed_ptr().
Change-Id: I24e7838ee9813b4336bfcebc823e498c3606c793
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
