Currently walt_find_best_target can return a halted cpu.
Prevent the PREV_CPU_FASTPATH for cpus that are currently
halted, and reject them in general from being selected.
Change-Id: Iced61bf9b1355703c6403d1dc9f90d6e64aefdc6
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
If a halted CPU goes idle, don't try to pull tasks into
the CPU.
Change-Id: I50383f736ae03992b8bb43f5aa12cc2803068f6f
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Make the minimum change for core_ctl to work properly with halt
instead of pause.
Change-Id: I7d2a2b50daaffc96846c86b8fd9dedf7f604b0b6
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
halt/start tracepoints need a little enhancement, to ensure
a measurement can be made from the start of an operation.
Also to debug RT task placements, add the lowest_mask
cpu mask which contains the cpus available for the
RT task placement algorithm to trace_sched_cpu_util, and
update the trace to show the halted state.
Change-Id: Idd5ab03d8ca5af21d42415dce790993c8c5df1d5
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
References to pause have been removed and replaced with
halt. Remove walt pause functionality from the build.
Change-Id: I07c412cfba4d9f13aff298aed43a0e208db9f046
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Provide apis to the system in support of halting
cpus, as a replacement for pause/isolation.
Track the requests to halt a cpu, and only
start the cpu again when all requests have been
recanted.
Perform a migration of tasks off of a halted cpu,
if that cpu is online and not idle.
Change-Id: I99340b12e01ff53499b4107d843b7d48e6d6dff1
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Optimize walt_irq_work to reduce locking for migrations.
Reduce scheduler overheads by reducing the locks needed
for the walt irq work migrations. Precisely track the
cpus locked, and unlock them at conclusion of work function.
Change-Id: I6ea6d24dc69e9876387253b489e91f82e45ed6d0
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
When a task yields, it relinquishes the cpu and scheduler
is tasked to find another task.
However he MVP implementation could return the same task
leading to a loop where the yielded task gets to run back.
To fix this, drop the MVP status of tasks that yield
themselves. The MVP status is restored the next time they
actually get enqueued - -likely because of a wakeup from
sleep or task migration.
Change-Id: Ia2a2074f18c4de56e0695ee2c5f1daf8a4eb5980
Signed-off-by: Tengfei Fan <tengfeif@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
As a side effect of commit f64fafbec582dd46169fba528cad2341c0c2d906
the check to skip frequency update if the same frequency as
the previous frequency is being applied, got removed at the point of
update.
Restore this check at the appropriate place.
Change-Id: I7aee7f84f0a64747b01ac31c2fdcbb4440908deb
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Restore obj file in Makefile that was removed in
commit 3908a213bc56 ("pause: port the users of pause").
Change-Id: I2014b049ca7cb5e4188b07087cc22fa9ee40cf85
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
This change is for general scheduler improvement.
Change-Id: Iffd4ae221581aaa4aeb244a0cddd40a8b6aac74d
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvements.
Change-Id: I37d6cb75ca8b08d9ca155b86b7d71ff369f46e14
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
This change is for general scheduler improvements.
Change-Id: Ia2854ae8701151761fe0780b6451133ab09a050b
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
When CPU trying to move a task to other cpu in active load balance or
by other means, then the other helping cpu marked as reserved to avoid
it for other scheduler decisions. Once the task moved successfully,
the reservation will be cleared enables for other scheduler decisions.
The reserved flag is been analogously protected with busy cpu’s
rq->active_balance, which is protected with runqueue locks. So whenever
rq->active_balance is set for busy cpu, then reserved flag would set for
helping cpu.
Sometimes, it is observed that, cpu is marked as reserved with no cpu's
rq->active_balance set. There are some unlikely possible corner cases
may cause this behavior:
- On active load balance path, cpu stop machine returns queued status
of active_balance work on cpu_stopper, which is not checked on active
balance path. so when stop machine is not able to queue ( unlikely),
then reserved flag wouldn't be cleared.
So, catch the return value and on failure, clear reserved flag for cpu.
- Clear_walt_request() called on the cpu to clear any pending walt works,
it may possible that, push_task might have changed or cleared, then
the reserved cpu would be left uncleared.
So clear the push_cpu independent of push_task.
Change-Id: I75d032bf399cb3da8e807186b1bc903114168a4e
Signed-off-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
There is a bad cast from unsigned long to unsigned int that is taking
place in sched_cpu_util tracepoint, and this is causing bad values to
appear in the traces.
Change-Id: I51561ac582b616e3319c8f43648cf5a4bfc632b6
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: If9c4859601bdcb1ab92bf6d7125bcb74c9ec3f7b
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
push_task assignment is guarded by corresponding runqueue's lock. Read
it only when lock is acquired.
Change-Id: I74668df66b4e18286b919b136d912a3f8afda11c
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
get_pid_task internally increments the reference counter, which could
cause unbalanced reference counter (always high). Only call
get_task_struct when pid is 0.
Change-Id: I409b477d0714ce35a0dc980782ee5878106ecffd
Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
To determine why a particular long preemption or long irqs off
period has happened, without crashing the device, the tracepoints
need improvement. The callers should be included.
Change-Id: If147969717517563e56cd20d3c036f51835a5c73
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
This change is for general scheduler improvemnts
Change-Id: I0a0b065ff2d1a563bccdd161dad890fae26f11c3
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Upstream has done away with get_online_cpus and put_online_cpus wrappers
around cpu_read_lock() and cpu_read_unlock().
Reflect these changes downstream.
Change-Id: Ie7ae59b6faba93ea43ce1ba3a4b043ae87e1337f
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Upstream does not carry pause feature. Remove related references.
Change-Id: I7ac0352a761fa78af3918a5b3a61424df0d22a2f
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
Aandoned upstream, reflecting it in kernel/msm-5.10.
Change-Id: I0c1ebd96372803519b180641707562034996479b
Signed-off-by: Sai Harshini Nimmala <quic_snimmala@quicinc.com>
This change is for general scheduler improvement.
Change-Id: Id758a4a0a836fe30b9f63b8f00c017247cf3b84c
Signed-off-by: Stephen Dickey <quic_dickey@quicinc.com>
Change the FIE to use frequency scaling information provided by
cpufreq driver as opposed to the higher priority arch specific counters.
This is done by resetting the struct scale_freq_data for each CPU which
holds the callback information specific to the HW counters, thus forcing
the FIE to revert to default, the cpufreq driver.
Change-Id: I23f508f9859be6a35155e5d000caf1c0b19f5584
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Port the users of pause from the 5.10 kernel to ensure code is up to date.
Change-Id: Ibe52cc37e0dca425f2f472b9cf254a2453bd9c47
Signed-off-by: Stephen Dickey <dickey@codeaurora.org>
num_sched_clusters has to be less than MAX_CLUSTERS, complain otherwise.
Change-Id: I0648585139d31dc948cf73aae7d093757a5762cc
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
The rq lock variable in struct rq is changed upstream.
Reflect changes downstream.
Change-Id: I8711d42dbe10f769b8568895993b92a6bcbcd69e
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
'task_of' function is defined in sched.h
Remove redefinition in walt_cfs.c
Change-Id: I07dd511fae0a74a03ebf2ae47f2b227404866972
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Upstream has done away with the MAX_USER_RT_PRIO macro.
Reflect these changes downstream.
Change-Id: I836bcae569af2e98db5cfc1330d04c80c5e68563
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
Upstream changed task's state variable in struct task_struct
to non-volatile.
Reflect these changes downstream.
Change-Id: Icc9736e6b5f6fb5b7d02fde6d3117a9131d98b50
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ic5929726768f6a9ca2a5823e2795eeb5e172d93a
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I9cd7715f390d52163b7280ebb6e9f04bdfc38d4c
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I2873e2f0cb5a8e1d2349644c796bf188275882b6
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I94d790387a8d528fdfd954a8db2c8783eb090644
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I28ccf141ccd52c0b8afc4548f83dd4e69b954670
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
Add target load threshold and target load shift tunables for each
policy. This allows us to scale cpu frequency in a configurable
way for each cluster.
Change-Id: I2c41ce6ef8b5a91f0b55be6af075db70695efd98
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Raw frequency was being cached earlier than the down rate limit check
causing the calculated frequency to be cached even in a situation where
the up/down rate limit check failed a frequency update. The following
changes have been made wrt to the above observation:
1. The up/down rate limit check has been moved further up
the frequency update path, where frequency is resolved and
cached_raw_freq is utilized to ensure that all possible frequency
checks are consolidated to one location.
2. Reset cached_raw_freq in the event of down rate limit check failing,
thus forcing the frequency update path to resolve a new frequency
value instead of using an invalid cached_raw_frequency and
ensuring that the latest frequency is applied.
Change-Id: I449ca6552aeb5f30be9d29ac472c0ba421f25da4
Signed-off-by: Sai Harshini Nimmala <snimmala@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ia9be6164f683a5fa29aa5d1ce13633e1cc6016ba
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
When crashing is not enabled, let WALT attempt to correct behavior.
Change-Id: Ica1c6a8aff465c5747db9acd0e16d99a7c2c4128
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I6ba6408bd9934bde681452280c3be56cd8bc96e8
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ib0003f97269920cb29f125da2ced34417a5a2229
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I000f0d6aa486da31cf805f2aa1d21a821f6ebfdc
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Id6818d7f79e8babb316aff3568cb65397df53c6f
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ie81267e970c55da4f5f5f2a6ffb9b7f64e48503c
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: Ia289e3ac635e22e66ae14c7cb2d171b4ad6bbf60
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I7e2f06b0f93d6c3c2cc439f895083f4b7cb96212
Signed-off-by: Abhijeet Dharmapurikar <adharmap@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I509129b041185e8f6b83eb586b7ce542889ebe5d
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
This change is for general scheduler improvement.
Change-Id: I78a13f15f7b95d7b6583222fbc68b828e651b802
Signed-off-by: Lingutla Chandrasekhar <clingutla@codeaurora.org>
Signed-off-by: Shaleen Agrawal <shalagra@codeaurora.org>
