Add util_busy_hist sysctl tunables which solely
look at the sum of cpu_util across CPUs as a metric for
LPM biasing on any/all CPUs.
Change-Id: Ib981c21a58059ef02772ee797808c5f30547a688
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Printing CPUs info is useful while going through error paths.
Change-Id: If84794163b6c417d419785fdddd2eed98588f2eb
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
The rq clock gets updated in the scheduler tick. However, if the
lock is acquired remotely between the scheduler tick and the
vendor hook, the clock update flags gets reset. Do the clock
update in walt_cfs_account_mvp_runtime based on the clock update
flags.
Change-Id: Ie4254a191a18458cf1684070bc96ff63cf0103d3
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Protect MVP tasks list access in the tick path with rq lock.
Change-Id: I387aa104cbacaf445bf739b8fb6d631283bd0ada
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I3280c12b5446b8c12fd4269efc2c1dad70e4ae6a
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Change min_freq and max_freq to be policy_min_freq, policy_max_freq.
Change-Id: Ief7ee31209338e6fc25f4ad8933b60256ac5eb95
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
When experiencing frequency related issues, it will be helpful to be
aware of any potential clamping issues, as well as reasons why we might
end up not updating to a different frequency.
Change-Id: I0be24d713421357e077cd01d5c84f43138a17b24
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add sched_sync_hint_enable not to enable/disable sync.
Change-Id: I455545487d002b4e3caf96c6cf8569549c3b1bbe
Signed-off-by: Tengfei Fan <tengfeif@codeaurora.org>
lock ordering issue is observed in below scenario where preferred cluster
for the task is being set on one CPU while task is being attached to
related thread group which leads to deadlock.
CPUX:
scheduler_tick() --> acquire rq_lock()
trace_android_rvh_tick_entry()
android_rvh_tick_entry()
set_preferred_cluster() --> try to acquire RTG lock taken by CPUY
CPUY:
cpu_cgroup_attach()
trace_android_rvh_cpu_cgroup_attach()
android_rvh_cpu_cgroup_attach()
__sched_set_group_id()
add_task_to_group() --> acquire RTG lock
__task_rq_lock() --> try to acquire task's rq_lock() which is CPUX
Fix the issue by moving set_preferred_cluster() related piece of code
which is being called from scheduler_tick() to android_vh_scheduler_tick
trace hook.
Change-Id: Ib9e2d70ae45a304d21cde3de6b8a84f9049061ab
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I5f08d40d6e9f18a905d87db078fa1a70abaa9030
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The parameter to lockdep_assert_held should be a pointer to a lock, not the
lock itself.
Change-Id: I4586d86b8684b21370a2bc522862932481fef538
Signed-off-by: Elliot Berman <eberman@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I9a94b20b18bf5b78946fd26d3337f5227d735e03
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I2a10a999f94e2f21eb97761516c135aadbe305e7
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Walt placement code could return the prev_cpu for placement without
checking if it is in affinity mask.
Fix it by checking task's cpus_ptr.
Change-Id: I6edf2249bccffd1868dd546dc18b45af311cc91d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
In scheduler code printk_deferred should be used instead
of pr_emerg, as this will avoid taking any locks and defer
output to a safer time.
Change-Id: I06b9060754d97f4ca141c561eb80216530e0e25e
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
There is a redundant check taking place in set_task_cpu hook. The cpu
will never be negative from core kernel side, so remove it.
Change-Id: I6d8ab62573f401d6aaeda9c5ddb1183c69e8c7d6
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Currently, small tasks get placed on mid-cap CPUs if uclamp is active
on them. This causes power numbers to increase.
Introduce filtering such that small tasks stay on silver.
Change-Id: I7e31d8a5ffd4c4f64d7bd3668970216550d23d24
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
When a user space task move across cpuset cgroups, the affinity
settings are lost. Fix this issue by caching the affinity of
a user space task during affinity change system call and restore
it during cpuset cgroup change when possible. The cpuset part
is implemented in the code but affinity system call part is
not implemented. This patch implements that.
Change-Id: I9322400eb275aeeb35fb623210417df4730cc6ad
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
attempt to catch double enqueue/dequeue of a task, which
can impact walt accounting.
Change-Id: I2ff83e75d65f083c8d2896dd10313c193a16fa7b
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Catch when the rq is not the same as this task's rq.
Catch a double enqueue or dequeue of the same task.
Change-Id: I322588f4c8ad59ee48c6d14bd87c5db6d842e6f6
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I2435c4ecd4c6a978af188a6385d306691376bb0a
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Introduce information into the debugging of
fixup_cumulative_runnable_avg by ensuring that the data from
the task of interest is dumped, and that the runqueue in
question is locked.
Change-Id: Idb6e3bd1b4ba67512fa6663555c22e933c9d0a6e
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ife8de1eee2b7aa862979073c075466c97ba980e3
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
The current code does the task group initialization from a sysctl
handler invoked via post-boot. But some tasks could have been
added to the cgroups and walt would miss to act upon them.
Depending on user space to notify when to populate task group
pointers is racy. This patch moves away from this approach and use
the cpu controller cgroup online vendor hook to initialize the task
group pointers. During module load, iterate over all the existing
cgroups if any and do the initialization.
Change-Id: I133099a49f94de836b95403c5f1126587de84919
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
We are no longer using the boosted member in the fbt_env struct. Clean
it up.
Change-Id: If8d0a68b8fb090a889c59667eb0fd8534c62af65
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
We are using the terms "prev_delta" and "best_delta" to represent energy
consumption values. Change to "prev_energy" and "best_energy" to reflect
the same.
Change-Id: Ie63e3029c3be6ee5e8cc12867d690b0fc708416f
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
We check if the current task running on the CPU has the highest per task
boost to avoid placing task on said CPU. However, if the CPU is idle,
then this check becomes unnecessary.
Change-Id: I257df862ee2b344c0c602f36f254b38f61b78e55
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
spare_cap is being calculated prior to idle_cpu evaluation. If an idle
cpu is found, then we have no need to go through this operation. As
such, move the computation to after idle cpu evaluation.
Change-Id: I86ff39909f2f0cb096ac39d02b6171838c257357
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
In FBT, we are making an additional call to uclamp_task_util while we
have already cached that value in a local variable. Use the cached value
instead of calling the function again.
Change-Id: I05828f02570056309256907aa7fee1396134706a
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add rq->rt.rt_time to debug output to better track the accumulated
time that exceeded runtime. This will allow us to have a better
understanding of the decision made to throttle RT threads.
Change-Id: Icd8bc43583bddd61019f0d7f73a2315a8103d94a
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
It is possible that wrq->push_task is null but
walt_lb_active_migration assumes it is non-null. Local variable
push_task is used after copying the value, and can be dereferenced.
Update walt_lb_active_migration to safely check push_task, and
ensure that the active migration flag is cleared.
Change-Id: I3b555430f1ba940053756fcbf2cca5c7d19de9fb
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
core_ctl is launching a kthread which uses a spinlock, prior
to initializing the spinlock. This creates a race, and a lockup.
Change-Id: I170a77addc6e4c2cc90b6683b50723008760f2f4
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
When sched_ravg_window_nr_ticks tunable is read and write concurrently,
the read can return the previous value though the write is already
happened. Fix this by reading the sysctl_sched_ravg_window_nr_ticks
value under mutex.
Change-Id: Ie1146c47ac1a184fc8bf4a2824bf6db28fd7156d
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
sched_dynamic_ravg_window_enable tunable is not currently used. If 0
is written to this tunable, the dynamic window size feature is disabled
in the kernel which results in bad performance. So remove this unused
tunable.
Change-Id: I64282146e526f98d049be25502ef638f538b0496
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
As part of reducing the arguments to sched_task_util trace point,
remove is_rtg argument as well. It can be retrieved from task_struct.
The recent update to this trace point made walt_get_rtg_status()
an extern function. Move it to a header file and restore the inline
functionality.
Change-Id: If447600fd9835e3b78724eb7a94b87c79090fe7f
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Reduce redundant parameters passed to sched_task_util tracepoint
by invoking relevant functions that return these values during
runtime.
Change-Id: Ie146a5a7a2ac6e4879115f49d4ccc6f7dcadcb52
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
preemptirq_long_init() return value is incorrectly being checked which
is causing few of the trace hooks to be not even registered. Update the
check to make sure return value is checked properly.
Change-Id: If68a23d585c9e460ec1e0f4902a4ed9c9cae4c68
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
dequeue_task() hook invocation is being changed from post dequeue to
pre dequeue. Consider the task being dequeued is out of the scheduling
class managed list.
Change-Id: Ib95ec7f8a873587493487a3c31c3f53ea78b0d95
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
The cpufreq constraints must be reflected in the capacity of
a CPU. Add a hook for cpufreq limits to implement this.
Change-Id: I779cb16bd66ea99c16e55faea7383e034d22440a
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
With refcounting, a cpu will get stuck in a paused state
if client software doesn't account properly for paused CPUs.
To properly track which CPUs it has made a request to be paused,
the driver needs to record the success or failure of an operation
and only record a change in CPU state, in the case of success.
Change-Id: I7bab17026d67ac1733c77083eae97af51c30dd37
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
During test cases where multiple entities (thermal, core_ctl, hotplug)
attempting to pause or offline CPUs, error conditions caused core_ctl
to get stuck, unable to pause or resume CPUs.
Improve the core_ctl implementation such that an error to pause or
resume is carefully tracked, such that core_ctl will retry those
operations at a later time. Simplify paused_by_us to eliminate
redundant information.
Change-Id: Id156e6644da5264f5552daea7d3092916c44c434
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
Client code can miss the actual paused state of a cpu, if ref-counting
is performed when the pause/resume operation is incomplete, or if
an additional pause/resume operation executes prior to updating ref-counts.
This becomes a problem with the online work handler, which can run
at the same time as any walt_pause/walt_resume call.
Protect the calls to pause/resume CPUs, such that the actual state of the
CPUS is known and stored, prior to beginning a new operation. Ensure
that the work function has accurate information before performing its
resume operation.
Change-Id: Iac279ac731f7c97b289547d36cce7de6e2b91c3a
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
With refcounting, the walt wrapper for pause/resume can keep
track of whether a CPU should be paused or not, where this
information used to be distributed to the clients using the api.
Simplify the design, such that a CPU is automatically paused
as it is coming online, affecting the active/inactive state of
the CPU, but not adjusting the ref-counts.
Going forward, client software using walt_pause/walt_resume
will not be required to keep track of the online state of a
cpu.
Change-Id: I83e3cc8743942c3a482eb588546b6ea9bb7bf2e3
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ib1ec2f655193a71d401d7e51f7890537d4f75aef
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Remove the TODO comment about frequency in-variance in walt governor
as the code is completed. Also remove a FIXME comment from WALT IRQ
work as the code looks correct there.
Change-Id: Ib5447009ab009d7cf2991c0f22d04a766de59f3a
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
There is a revised member to track the wake_q length upstream.
Change-Id: Ie1b9619be3616d4749f5fa2da8584f5fd4b45a58
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: If7c9fd9d76d54da94acec15018c82e0d66d4162c
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
