rpmh_rsc_debug() can race with tcs_tx_done() interrupt handler and
may not see any tcs busy and hence won't do bug_on(). Later it may
go ahead and free() the request which interrupt handler is processing.
Lets always bug_on() upon timeout.
Change-Id: I238a6a3639077850df158cb1f0190656e014bb57
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
rpm_msgs address is calculated from memory allocated during write_batch.
Correct the offset from where rpm_msgs actually starts and assign req
pointer to point to correct address.
Also add list_del before freeing memory in rpmh_invalidate.
Change-Id: Idca43fe73bd238e90911158b08b3889c001e1123
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
RSC major version 3.0 has different register offsets than previous
versions. Add support to read the current version during probe and
use register offsets accordingly.
Change-Id: I8174516ecd2ab7a370e1bb6592d4329c4ff1f81d
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
In addition to logging using FTRACE. let's also log in the IPC logging
for all RPMH transactions.
Change-Id: I2b19006bbc776053f63eee55b447428fdb023ba4
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
The RSC interrupt is issued only after the request is complete. For
fire-n-forget requests, the irq-done interrupt is sent after issuing the
RPMH request and for response-required request, the interrupt is
triggered only after all the requests are complete.
These unnecessary checks in the interrupt handler issues AHB reads from
a critical path. Let's remove them and clean up error handling in
rpmh_request data structures.
Change-Id: Ie3a3569603f001d6cdb59310c3f2752cdcd7a84d
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
When we crash the system and display the TCS controller state, let's
also dump the fast-path TCS information if there was an active
transaction going on at that time.
Change-Id: Iabc7b8af22dd148fca617e571f1392ba3d9a93ab
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
Add fast-path interface for making a lock-less RPMH request. The
interface provides APIs that can be called from a interrupt/scheduler
context. The _init_ call shall be used to initialize the dedicated TCS
with {addr, data} and subsequent calls shall _update_ the data and
trigger the request immediately.
Only active state requests shall be sent through these interfaces.
Requests to remove the vote during sleep and re-apply the vote during
wake must use the regular rpmh_write/_async variants APIs (locking).
Change-Id: I2505fc487ddca1a32185b116c0d8680818dfd2b8
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
In order to support lockless RPMh request, let's dedicate a TCS for this
purpose. This TCS shall be defined as FAST_PATH_TCS in the TCS
configuration for the RSC.
The fast-path TCS will have the following properties -
- A single TCS shall be defined as a fast-path TCS.
- Fast-path TCS will not use interrupt for signalling request
completion.
- Fast path will poll controller status for idle before sending request.
- DRV properties are not updated during the fast-path use.
- Fast path requests must be first initialized with {addr, data}.
- Subsequent updates will only update the data in the TCS and trigger.
- Requests must be fire-n-forget only (no completion).
Change-Id: I235ea60159e40affa9ce1e4069e11eca3925a423
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
Add a new definition for fast-path TCS configuration.
Change-Id: I610b14dcd893885e5b1c32c434caa7aa7102fd4c
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
In OS-Initiated mode of PSCI, a serialization is established in Linux
between the last CPU powering down the cluster / domain and the CPU
powering up back. The serialization is established by the genpd
framework and therefore it doesn't help to lock around the cache.
Refactor out the core of the flush functionality from the locking
wrappers. Calling the core functionality avoids a spin trylock, which
the RCU framework might consider suspicious under the circumstances.
Change-Id: I242fb4827b71152bda36ea5870393054b8703309
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
RSC is part the CPU subsystem and powers off the CPU domains when all
the CPUs and no RPMH transactions are pending from any of the drivers.
The RSC needs to flush the 'sleep' and 'wake' votes that are critical
for saving power when all the CPUs are in idle. Doing this every time a
CPU powers down, increases latency to enter idle state. So, we want to
do this, only when it makes sense - when the last CPU is powering down.
Let's make RSC part of the CPU PM domains, by attaching it to the
cluster PD (defined in DT). Registering for PM domain notifications, RSC
driver can be notified that the last CPU is powering down. When the last
CPU is powering down the domain, let's flush the 'sleep' and 'wake'
votes that are stored in the data buffers into the hardware.
Change-Id: Id748413f0f86a4ac73c37e0780b2b22a143a1e61
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
lockdep_assert_irqs_disabled() was added to check rpmh_flush() can
only be invoked when irqs are disabled, this is true for APPS RSC
as the last CPU going to deepest low power mode is writing sleep and
wake TCSes.
However for Display RSC, drivers can invoke rpmh_write_sleep_and_wake()
to immediately write cached sleep and wake sets to TCSes from any CPU.
Conditionally check if RSC controller supports solver mode then do not
check for irqs disabled as such RSC can write sleep and wake TCSes at any
point.
Change-Id: I8ad05cd81af15a0d57353b55d8098cf0e47f79f7
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
Let clients call rpmh_write_sleep_and_wake() to immediately
write cached sleep and wake data to the TCSes.
Change-Id: I04a94ee622a39a0aa0e829f4c40190a870fb2fb1
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
Allow RPMH requests to succeed even when the always-On subsystem is not
available. In standalone mode, requests are never sent to the TCS
mailbox driver. The command DB provides the information to make the
decision to run in standalone mode or not.
Change-Id: Ia2bc5024da0d7e6607347b1a481b2feb9584059a
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
Controllers may be in 'solver' state, where they could be in autonomous
mode executing low power modes for their hardware and as such are not
available for sending active votes. Device driver may notify RPMH API
that the controller is in solver mode and when in such mode, disallow
requests from platform drivers for state change using the RSC.
Change-Id: Ibe777d23fd58489cfb8132c0168af8e29284084d
Signed-off-by: Lina Iyer <ilina@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
Output the TCS state when the debug api is invoked. The state of the
TCS, the contents and the IRQ status is presented. Additionally, crash
the system if any TCS is busy to help with the debug.
Change-Id: I7ef5480cc6da7208d5f82351d863561fcc1abf68
Signed-off-by: Raju P.L.S.S.S.N" <rplsssn@codeaurora.org>
Signed-off-by: Maulik Shah <quic_mkshah@quicinc.com>
Add symbols currently present on kalama as of msm-5.15
'commit 1701298d2205 ("Merge "defconfig:kalama_consolidate: Enable
CONFIG_UFS_DBG"").
Change-Id: Ie9570f48bbcef1c32e49f629e079cf7af0f572a9
Signed-off-by: Guru Das Srinagesh <quic_gurus@quicinc.com>
Place initial votes for BCMs marked as keepalive during initialization
such that we force the BCMs on even in the absence of explicit client
votes.
Change-Id: I0cf1b2f2a76d5fc094329520ce2a9f1691a3f681
Signed-off-by: Mike Tipton <quic_mdtipton@quicinc.com>
Some BCMs only need to be kept alive early in the boot process. Add the
keepalive_early flag, which only prevents them from turning off before
the sync_state callback.
Change-Id: I0699fb0420d1d9f9473e28ede57380fd5b08ca53
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Currently the initial floor is only voted in AMC/WAKE, which means the
floor is removed when entering the sleep set. However, some use cases
require bus BW even when the CPU is idle. If all clients haven't voted
yet, then assume we need to keep these votes around permanently.
Change-Id: I2277e62e0a4e1ea949f1c7fbad052e76cf451c68
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Some QoS enable urgent forwarding by default, but we don't always want
this. So don't assume the register already has zero and write it
unconditionally.
Change-Id: I257e099a588870f8d6e6d68b7fd794e22781450e
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Add prio_fwd_disable flag support to stop forwarding QoS information
from the master. So respective noc will ignore master's claimed priority
and send out the default priortiy.
Change-Id: I21ca11b44f200453c70c02f195779ef583a90e8c
Signed-off-by: Vivek Aknurwar <quic_viveka@quicinc.com>
Skip bcm init and implement stubs for icc apis when stub property
is defined.
Change-Id: Ic56a75578b6fa4fe35bebaf103d41205e444efed
Signed-off-by: Vivek Aknurwar <viveka@codeaurora.org>
Some QoS blocks require voting for BCMs before their registers can be
accessed, since they require clocks and regulators controlled by BCM.
Hence add qos_proxy flag to indicate the BCM needs to be voted for
QoS configuration.
Change-Id: Idc618e7a3d28d31d38d85f5b21e6b9f12e86ebe2
Signed-off-by: Veera Vegivada <vvegivad@codeaurora.org>
Add support for QCOM_ICC_TAG_PERF_MODE on RPMh architectures. When
QCOM_ICC_TAG_PERF_MODE is specified using icc_set_tag(), each node in
the requested path will use latency-optimized settings if supported by
the node. The improved latency comes at the cost of higher power.
Change-Id: I83c52e5677f8842b4c17987053d566971b84086f
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Using devm_ioremap_resource will check the resource boundaries.
In legacy hardwares we have memory overlaps between the noc devices.
Hence use devm_ioremap instead of devm_ioremap_resource.
Change-Id: Ifc8a1bb60183058a7c3845c231edfa53115d817e
Signed-off-by: Veera Vegivada <quic_vvegivad@quicinc.com>
Add binding for QCOM_ICC_TAG_PERF_MODE, which can be used with
icc_set_tag() to indicate that the path should use latency-optimized
settings for each node in the path if supported.
Change-Id: I748df7bdaa79784ade4ad9a135e4f91b92b97513
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Add skip-qos node property to skip qos settings which happens on
each provider probe. Early in presil this property is useful to skip
qos settings as not all needed clocks are voted/set on and device ends
up unclocked access. This property will help to deliver interconnect
earlier without qos settings which can be enabled when things are ready
before bring up.
Change-Id: I9619f651f40f49ec03d9ea0dc7c45b3b13ffae6c
Signed-off-by: Vivek Aknurwar <viveka@codeaurora.org>
Signed-off-by: Mike Tipton <quic_mdtipton@quicinc.com>
Some BCMs aren't directly associated with the data path (i.e. ACV) and
therefore don't communicate using BW. Instead, they are simply
enabled/disabled with a simple bit mask. Add support for these.
Change-Id: I9b73e0ce05a09338d22571ff311c5b058fb8e1b2
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Some QoS blocks require voting for BCMs before their registers can be
accessed, since they require clocks and regulators controlled by BCM.
Vote for these BCMs before configuring QoS during probe.
Change-Id: Ic93d224c7dc56c8e56d527179a0c2d1e888e3822
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Some BCMs that aren't directly associated with NOC ports (i.e. ACV)
publish their width as 0 in cmd_db. The client votes are multiplied by
this width when adjusting for node/BCM width differences. This means a
zero width also zeroes the client votes, making it impossible to vote on
these zero-width BCMs. Set a width floor of 1 to allow aggregation on
these nodes.
Also set a floor on the BCM units. They should never by 0 in cmd_db, but
if they are it would cause a div-by-zero.
Change-Id: I14ba89411ec02077db6d2c27b6de932c5b7dfe1e
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Currently avg and peak bandwidth are voted as allowed max on kernel boot.
Add stub icc_get_bw() callback that return default avg, peak bandwidth.
Change-Id: I5c42f933d770e2c63c8bbc2d8e7afe5b9748ba1d
Signed-off-by: Vivek Aknurwar <viveka@codeaurora.org>
Add a debug library that provides QCOM-specific interconnect debug
features. This includes support for printing all enabled interconnect
votes to console when entering suspend.
Change-Id: I61302f95c94176f833bd1c88ddc76b88121b3f79
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
Add stub icc_set() and icc_aggregate() callbacks that return success
without doing anything. These can be used early in development before
the rest of the system is ready to service RPMh requests. Additionally,
they can be used during probe to avoid unnecessary set/aggregate() calls
each time icc_node_add() is called.
Change-Id: If495ffc8f9839c3d2651c166c62a7936927e3c02
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
AMC requests made while the RSC is in solver mode result in a -EBUSY
response from the rpmh driver. This is expected and is a non-issue since
in this case we just need the request to apply to the WAKE set instead.
It's only possible to see this for secondary RSCs like the Display RSC,
since the primary Apps RSC will only be in solver during the power
collapse/restore sequences while SW isn't running. Ignore -EBUSY so we
don't return an error and short-circuit before sending the necessary
WAKE/SLEEP requests.
Change-Id: Id34b91c6e46d282be2522434d3c2b4837291b3b7
Signed-off-by: Mike Tipton <mdtipton@codeaurora.org>
