The utilization return by cpu_util() is bounded by the original
capacity of the CPU. Load balancer uses the utilization to
break the ties. Use unbounded cpu utilization so that the most
loaded CPU gets help.
Change-Id: Id16cea205e6eb07a42a1e86fda533cd825df6908
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Active migration of running tasks is allowed only onto idle CPUs.
Add a check for it in walt_lb_find_busiest_lower_cap_cpu().
Change-Id: I195aadc8b5822d4f231edd2f57d9df5ef1a6e350
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Currently we are using task_fits_capacity() for misfit task
accounting. This does not consider boost scenarios. We also
end up accounting 100% tasks running on the highest capacity
CPU as misfit. A task on the highest capacity CPU should never
be treated as misfit.
rq->misfit_task_load should use task_util() not task_load(). The
later is task demand scaled to sched_ravg_window in nano seconds.
Change-Id: I1f923e33530d2d04d7bfca9dd42920f799794864
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
walt_lb_find_busiest_lower_cap_cpu() is meant to be called when
the destination CPU has more capacity than the source CPU group.
However, the condition for calling this function is inverted.
Change-Id: I03dc62f42795479d21f676f8e27459955229dc82
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Implement select_task_rq_rt trace hook to be able to find suitable
CPU for the RT tasks from wake-up path.
Change-Id: Id1e8a4529d7391bf39a2a069ae37c146fb8a55c9
Signed-off-by: Satya Durga Srinivasu Prabhala <satyap@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Currently there is no trace support for indicating
whether an application is 32 or 64 bit compatible.
Since SOCs going forward will have a mix of support
for 32 bit vs 64 bit applications, this is needed
information.
Update the trace_sched_enq_deq output such that the
task's compatibility (32 or 64) is printed.
Change-Id: I780b6b290a77a8405dd268b45a1e7affd035036b
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
The core kernel side has race conditions which lead to memory faults.
Disable this feature temporarily.
Change-Id: I0013753d571f1e7265ce385a1130f2671c574ab0
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
The clean up including removing unused variables and functions,
converting global variables to static.
Change-Id: Ib5126ac9df701fdde354e0e5d759f0177db5a28b
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
sched-walt-debug module does not get compiled unless SCHED_WALT
config is turned on. However the config corresponding to the
debug module shows in menuconfig even without enabling SCHED_WALT.
Fix this by specifying the dependency in the Kconfig.
Change-Id: Ie466c0ca11bbcdec43af2624c3631ddff5c77024
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
The walt_find_best_target() has a special case for the active task
migration attempt from tick path. It tries hard to find an idle
CPU by disabling most_spare_cap_cpu selection. However this is
all unnecessary if we can check for the idle CPU condition at
the caller site. This helps the overall execution time of wfbt().
Change-Id: I89de44824dca8729b8c62898f9e75dc8e0e1c544
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
walt_find_best_target() which selects the best CPU has a special
handling for running tasks. The idle CPUs are only considered as
migrating a running task to a busy CPU will result in wait time.
A running task is detected via task->state == TASK_RUNNING check,
which is true for tick path active migration tasks. It is also
true for newly created tasks. These tasks are getting placed on
idle CPUs of the higher capacity clusters though the lower capacity
CPUs are lightly loaded. Detect the running task via task->on_rq to
fix this issue.
Change-Id: I154cd6c6f934061526bce30fe596d9ad36c9e07f
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
core_ctl is part of WALT. There is no separate config for
core_ctl. So it makes sense to declare all the WALT related
public interfaces in a single header file.
Change-Id: I0ae5585c0037295eee4268705f0e553d14c60fd9
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
wake_up_idle related functions are meant for other modules
usage. Make them public by moving to appropriate header.
Change-Id: I888b511e656d6c160c98b5da44bc873573a9beed
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
The walt_task_struct structure is used by WALT module. WALT
type cast the task_struct's android vendor data to
walt_task_struct and use it for the scheduler needs. However
there can be other users who wants to cache information per task.
Currently walt_task_struct is defined in WALT's private header and
can't be included by other modules. Move the walt_task_struct and
dependent structures to appropriate header file so that is becomes
public.
This movement also allows wake_up_idle related funcitons to define
in the public header and keep the inline functionality instead of
exporting them for the other modules usage.
Change-Id: If8c94c7d6cd709bbc56aa08eb964d135a492a530
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
set_task_boost() API is meant for other modules, so export
this function. Also move the function declaration to
appropriate header.
Change-Id: I7213b137a0763140c631bea30653321674fef272
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Re-introduce the earlier sched_load_boost feature as
waltgov's boost feature. A tunable called "boost" is
exposed per cpufreq policy which can be used to boost
(either positive or negative) the utilization, thus
the selected frequency.
Change-Id: I3a41a985d449e5c753b5afcbc375749e67036089
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
The sched_load_boost feature will be introduced in a different
form in the subsequent patch.
Change-Id: Ib528ccd0fcb78f85685399264cb58212752ca26b
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
cpu_frequency trace events are coming twice when a frequency
switch happens. This is due to commit 08d8c65e84 ("cpufreq:
Move traces and update to policy->cur to cpufreq core") in the
upstream which takes care of cpu_frequency trace event and
updates to policy->cur when cpufreq_driver_fast_switch() is
called.
So remove the redundant code from waltgov cpufreq governor to
avoid duplicate cpu_frequency trave events.
Change-Id: I490e766abcc09ed2a580e5f09930a54d76367999
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Schedutil flags are no longer used in WALT as we moved to
waltgov cpufreq governor. So remove them.
Change-Id: Iedc68215ed0e994b8e051b282a575ac699ebf7e0
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
The core_ctl has to know how many CPUs are active in each cluster.
If the current need is less than the number of active CPUs, it tries
to pause the CPUs provided tunables are in agreement. On the other
hand, if the current need is more than the number of active CPUs,
it tries to resume the CPUs, provided those CPUs are paused by us
in the first place.
To determine the number of active CPUs, we should consider only
active CPUs. The others could be in pause or hotplugged out. Since
we only use cpu_active_mask for both, we don't need wrappers to
tell this to us. All we need is to maintain how many CPUs that
are going to be paused/resumed in the current iteration and check
against the need/limits.
Change-Id: I0a29da869b52eff35ff1dfa3094c51d6ece4e49f
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
static_key_disable_cpuslocked() API is called to disable TTWU_QUEUE
feature. This API expects read side of cpu_hotplug_lock lock is taken.
Since we are not acquiring that lock, use static_key_disable() API which
internally takes care of the locking.
Change-Id: Ic8c7023235dd2e03bfe93d3f87a6a68e4f5b9b0d
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
There are references to kernel command line parameters like sched_predl
and sched_ravg_window. These are not supported any more. So remove the
left over code.
Change-Id: I865f44fafee3a1d8f3d0d6f8f7e2341aff9ca964
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
LPM governor can query the scheduler preference about the CPU's
c-state via sched_lpm_disallowed_time() API. It returns 0 when
shallowest c-state is preferred and associated timeout is set.
Change-Id: Ie8246aa9ec4638730d3442468aa9dc1969a90369
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
We export necessary symbols as msm_perf driver needs to use core_ctl and
expand the core_ctl header file to add non WALT stubs.
Change-Id: I2fa65d6e5b00cfc88b97579df7074e8f82cb23ac
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
should_we_pause() always return true. Update the conditional
check at the caller and remove this function.
Change-Id: I085bc7737888fcafc74350a7d22176ae27bdece2
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
The core_ctl thread is woken up only after evaluating the need.
so another evaluation in core_ctl thread context is not needed.
Change-Id: I28162b758c18cda4ff05f1b1a4b1a5553c895e7e
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
We have 1 core_ctl thread for all clusters. Evaluate all clusters
and then wakeup the core_ctl if required.
Change-Id: I8131ae97280d8c478fd061e0aac06f1e56c5b6ea
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
cpuhp_pause trace point is in place. so remove the sched_pause
trace point from WALT module.
Change-Id: Ife250254207321a9033142a34319f04cdf3a2dbe
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
sysctl_task_read_pid sysctl needs to be set to the desired task's
pid before reading any of the per-task sysctl. So use the same
mutex for both of these sysctl handlers to prevent data tearing.
While at it, fix an incorrect goto in sched_task_handler()
function and restrict the range of sysctl_task_read_pid.
Change-Id: I544010f51a28ff6e5c4f117d68816de147ef3d0f
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I8f521dfd31ff1c8cc1f16805e4e0ac78228d9383
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
WALT is using padding avaialble in task_struct, rq and task_group
structures. Add a build time test to make sure that we are not
using more than the avaialble padding.
Change-Id: Icd7e23b4536075bc53a975eeff838eedd58b29cc
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
In WALT task placement algorithm, we select the root domain to be the
root domain corresponding to the current cpu, however, this could be
null if the cpu is inactive. This, in fact, resulted in undesired
behaviour when evaluating a single old threaded usecase, since desired
cpu was inactive, thread was being placed on a fallback cpu.
Change-Id: I68eef10dc84db15796a0d43af7215f4dbbdc059e
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
The TODO comments in walt_lb.c are not applicable anymore.
So remove them.
Change-Id: I3cd780ad3f05e974944d6aa60f0a3da155c7b1be
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
walt_update_cluster_topology requires the availability
of the cpufreq policies. These are not available
immediately after insmoding walt, and this causes a
crash through SCHED_BUG_ON.
Register for the cpufreq notifier for POLICY creation,
and init the second half of walt init after the
policy creation for all cpus.
Change-Id: If1af62a4991f3226bd0b661f5a5e419ceea3817a
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Id19f38a2b9a67b130876cb8df1e88c6a56aa6aad
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I30b3899e191d880e741f922323d0826b5c292498
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I616acbe5bfdb0e1bf88e923d98b4a3d54d26d942
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I064002ae2c7d8ffd444975ad74524f5fa8de5465
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Optimize the new idle load balance entry conditions to improve
the performance/power.
Change-Id: Ia859bc499e320e36e3ff32be8d690c53152a26cc
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
WALT improves the scheduler performance from a power/perf and thermal
perspective. Bring it in as a module.
Change-Id: Ibeb6c0480796e8d8fcd81e1bdda7a117ae02c980
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
