Using the DMA in cyclic mode modifies the hardware buffer descriptor
chain in xilinx_dma_prep_dma_cyclic so that the last descriptor used by
the cyclic transfer points back to the first descriptor, but it never
restores the original descriptor ring. This breaks using non-cyclic mode
after cyclic mode with an error like:
xilinx-vdma 86000000.dma: Channel 00000000354d5c8d has errors 100, cdr 6de40000 tdr 6de40400
The only way to get out of this error state is to rebuild the hardware
buffer descriptor ring by releasing and re-acquiring the channel.
Fix using non-cyclic mode after cyclic mode by always restoring the
original buffer descriptor ring in the same manner as it is set up by
xilinx_dma_alloc_chan_resources().
Fixes: 23059408b6 ("dmaengine: xilinx_dma: Fix race condition in the driver for multiple descriptor scenario")
Signed-off-by: Alex Bereza <alex@bereza.email>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Link: https://patch.msgid.link/20260817-fix-hw-buf-desc-after-cyclic-mode-v1-1-1fe47e701d6c@bereza.email
Link: https://patch.msgid.link/20260818-fix-hw-buf-desc-after-cyclic-mode-v2-1-530ff44c6a81@bereza.email
Signed-off-by: Vinod Koul <vkoul@kernel.org>
xilinx_dma_alloc_chan_resources() builds a static ring of hardware
buffer descriptors once and the driver uses this ring throughout the
lifetime of a channel. This requires the allocation order of hardware
buffer descriptors from chan->free_seg_list to stay in sync with the
hardware buffer descriptor ring built at channel allocation time by
returning oldest descriptors to chan->free_seg_list first.
When chan->pending_list is not empty e.g. during
xilinx_dma_terminate_all() the chan->free_seg_list and the order of the
static hardware buffer descriptor ring get out of sync. Descriptors age
in this order: pending -> active -> done. So freeing pending_list first
returns the newest buffer descriptors to the chan->free_seg_list first
and thus breaks the order required by the static hardware buffer
descriptor ring. Then when the channel is reused, after a wrap around of
the free_seg_list the DMA will find a hardware buffer descriptor with a
length field that is still zeroed and stop with something like this:
xilinx-vdma 86000000.dma: Channel 000000003a21d7b8 has errors 10, cdr 6de4c000 tdr 6de4c000
After this no more descriptors are completed and a consumer potentially
blocks and waits forever. The only way to get out of this error state is
to rebuild the static hardware buffer descriptor ring and the
free_seg_list by releasing and re-acquiring the channel.
Fix the order in which hardware buffer descriptors are returned to
free_seg_list to ensure the mentioned requirement holds.
Fixes: 23059408b6 ("dmaengine: xilinx_dma: Fix race condition in the driver for multiple descriptor scenario")
Signed-off-by: Alex Bereza <alex@bereza.email>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Link: https://patch.msgid.link/20260817-fix-hw-buf-desc-reuse-v1-1-d79827a844c7@bereza.email
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Core:
- New API to combine configuration and preparation and users
New Support:
- Mediatek MT8189 SoC uart dma support
Updates:
- Designware dma driver flatten desc structures and simplify code,
interrupt-path groundwork changes, first part of PCI EP DMA support
- Updates to zynqmp_dma with runtime PM and device removal improvments
- Xilinx dma optimizations for AXIDMA and MCDMA channel management
-----BEGIN PGP SIGNATURE-----
iQIyBAABCgAdFiEE+vs47OPLdNbVcHzyfBQHDyUjg0cFAmqMgZEACgkQfBQHDyUj
g0e82w/4oLwj2QUjUO3QEO2l4E/RVS2/7UyCYKoJDAh66aM8Z3QjQ+lUvy+4wHHE
H8AtTpRzGEPpqqU+BckHEySf8mNhlA4ULhX17aPc+NKqYNlW8p2L63kPFYggI1ja
Z6/zVFrJDLwWTI0bSxAyJ6OoaObSXY0P6qpMcXKcqoHc981+EHMGrt/iVJM4Et/c
Q6nFMxIMgxYPI1VCMSrJqOBA4ZvymyWBisJPZf6SK0iH3PnaBnGWWWilpT8eUJxm
/MbVf2aUMetdK1+QVx1nNQUdb1eyAw50zFgbeDFDTXQ85Mfbvwh8XxxuT4wXJS5b
8MFwIEUq6iRgyjZYlykmFYBXw1A3SZWuWMUuMic4l13lw7dxk92wR31j99pIRaV3
g2iykz+gzQNQ5RNPuo/9j0s26FAmMNxbFW1RPAMnoVc33vZakV4ueaYtIy7s7mMJ
eDrE6YNDW24DWfWrSxkQGCW1qFGBUY20dXZ0sBmhUxpTLB0ubPZrWLvKKZe/qTUJ
wmrGNtTBLAYJMsGBm5VMybXwrQXa2Hvqa6KKFR2TdjpdcJwyWVHzRPpR+sJ7GfxZ
Lz59ODW3x1MnukRQS/T+RXGT42Tzk+qQQ+xK/yCrXq0VkxE3010n7SglTEEygGYP
Zk2eos4fARpXOUNqAxoKTVRyvQBxrIrHuJuQjAys2RhLHARuCQ==
=lUcr
-----END PGP SIGNATURE-----
Merge tag 'dmaengine-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"Core:
- New API to combine configuration and preparation and users
New hardware support:
- Mediatek MT8189 SoC uart dma support
Updates:
- Designware dma driver flatten desc structures and simplify code,
interrupt-path groundwork changes, first part of PCI EP DMA support
- Updates to zynqmp_dma with runtime PM and device removal
improvments
- Xilinx dma optimizations for AXIDMA and MCDMA channel management"
* tag 'dmaengine-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (73 commits)
dmaengine: dw-edma: Mark emulated IRQ as level-triggered
dmaengine: idxd: assign all engines to group 0 in IAA defaults
dmaengine: qcom_hidma: remove conditional return with no effect
dmaengine: qcom-bam-dma: fix autosuspend cleanup during removal
dmaengine: fsl-edma: tracing: no ptr dereference during log output
dmaengine: dw-edma: Program endpoint function numbers
dmaengine: dw-edma-pcie: Add chip flags to match data
dmaengine: dw-edma-pcie: Handle optional data blocks
dmaengine: dw-edma-pcie: Factor out descriptor block address lookup
dmaengine: dw-edma-pcie: Add register offset match flag
dmaengine: dw-edma-pcie: Add platform ops to match data
dmaengine: dw-edma-pcie: Rename vsec_data to dma_data
dmaengine: dw-edma-pcie: Add capability match data
dmaengine: dw-edma-pcie: Track non-LL mode in DMA data
dmaengine: dw-edma: Add partial channel ownership mode
dmaengine: dw-edma: Initialize IRQ data before requesting IRQs
dmaengine: dw-edma: Add core quiesce operations
dmaengine: dw-edma: Add per-channel interrupt routing control
dmaengine: dw-edma: Factor out HDMA interrupt setup helper
dmaengine: dw-edma: Defer channel IRQ handling to workqueue
...
'struct dma_descriptor_metadata_ops' in not modified in these drivers.
Constifying these structures moves some data to a read-only section, so
increases overall security, especially when the structure holds some
function pointers.
On a x86_64, with allmodconfig, as an example:
Before:
======
text data bss dec hex filename
120635 21584 64 142283 22bcb drivers/dma/xilinx/xilinx_dma.o
After:
=====
text data bss dec hex filename
120699 21520 64 142283 22bcb drivers/dma/xilinx/xilinx_dma.o
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Sai Sree Kartheek Adivi <s-adivi@ti.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/b0a22171f3ed68e156a2fa84383e99c23ec6b2ff.1784037977.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Replace the #include of <linux/mod_devicetable.h> by the more specific
<linux/device-id/*.h> where applicable. For most cases the include
can be dropped completely, only a few drivers need one or two headers
added.
Acked-by: Danilo Krummrich <dakr@kernel.org>
Acked-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://patch.msgid.link/1a3f2007c5c5dcf555c09a4035ce3ae8ef1b6c49.1782808461.git.u.kleine-koenig@baylibre.com
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
Rename XILINX_DMA_LOOP_COUNT to XILINX_DMA_POLL_TIMEOUT_US because it is
a timeout value, not a loop count for polling register in microseconds.
No functional changes.
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Signed-off-by: Alex Bereza <alex@bereza.email>
Link: https://patch.msgid.link/20260402-fix-atomic-poll-timeout-regression-v4-2-f30d6a6c13cb@bereza.email
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Currently when calling xilinx_dma_poll_timeout with delay_us=0 and a
condition that is never fulfilled, the CPU busy-waits for prolonged time
and the timeout triggers only with a massive delay causing a CPU stall.
This happens due to a huge underestimation of wall clock time in
poll_timeout_us_atomic. Commit 7349a69cf3 ("iopoll: Do not use
timekeeping in read_poll_timeout_atomic()") changed the behavior to no
longer use ktime_get at the expense of underestimation of wall clock
time which appears to be very large for delay_us=0. Instead of timing
out after approximately XILINX_DMA_LOOP_COUNT microseconds, the timeout
takes XILINX_DMA_LOOP_COUNT * 1000 * (time that the overhead of the for
loop in poll_timeout_us_atomic takes) which is in the range of several
minutes for XILINX_DMA_LOOP_COUNT=1000000. Fix this by using a non-zero
value for delay_us. Use delay_us=10 to keep the delay in the hot path of
starting DMA transfers minimal but still avoid CPU stalls in case of
unexpected hardware failures.
One-off measurement with delay_us=0 causes the cpu to busy wait around 7
minutes in the timeout case. After applying this patch with delay_us=10
the measured timeout was 1053428 microseconds which is roughly
equivalent to the expected 1000000 microseconds specified in
XILINX_DMA_LOOP_COUNT.
Add a constant XILINX_DMA_POLL_DELAY_US for delay_us value.
Fixes: 9495f26482 ("dmaengine: xilinx_vdma: Use readl_poll_timeout instead of do while loop's")
Fixes: 7349a69cf3 ("iopoll: Do not use timekeeping in read_poll_timeout_atomic()")
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Alex Bereza <alex@bereza.email>
Link: https://patch.msgid.link/20260402-fix-atomic-poll-timeout-regression-v4-1-f30d6a6c13cb@bereza.email
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The zynqmp_dma_remove() function was converted from returning int to
void, but the kernel doc comment was not updated to reflect this change.
Remove the stale "Return: Always '0'" documentation that no longer
applies to the void function.
Fixes: b1c50ac254 ("dmaengine: xilinx: zynqmp_dma: Convert to platform remove callback returning void")
Signed-off-by: Golla Nagendra <nagendra.golla@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260630064844.705173-3-nagendra.golla@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
In zynqmp_dma_remove(), runtime PM was disabled only after checking
state and doing a manual suspend. This can race with runtime PM in the
remove/unbind (rmmod) path.
Disable runtime PM first, then suspend only if the device is not already
suspended. To prevent any further runtime PM transitions.
Fixes: 72dd8b2914 ("dmaengine: zynqmp_dma: Add shutdown operation support")
Co-developed-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
Signed-off-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
Signed-off-by: Golla Nagendra <nagendra.golla@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260630064844.705173-2-nagendra.golla@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Optimize AXI DMA control register programming by consolidating
coalesce count and delay configuration into a single register write.
Previously, the coalesce count was written separately from the delay
configuration, resulting in two register writes. Combine these into
one write operation to reduce bus overhead.
Additionally, avoid redundant channel starts in xilinx_dma_start_transfer()
and xilinx_mcdma_start_transfer() by only calling xilinx_dma_start() when
the channel is actually idle.
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Signed-off-by: Suraj Gupta <suraj.gupta2@amd.com>
Co-developed-by: Srinivas Neeli <srinivas.neeli@amd.com>
Signed-off-by: Srinivas Neeli <srinivas.neeli@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260626092656.1563871-4-suraj.gupta2@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Relax the idle check in xilinx_dma_start_transfer() and
xilinx_mcdma_start_transfer() that prevented new transfers from being
queued when the channel was busy, so scatter-gather transfers can be
chained onto an in-flight transfer.
In scatter-gather mode, only update the CURDESC register when the active
list is empty to avoid interfering with transfers already in progress.
When the active list contains transfers, the hardware tail pointer
extension mechanism handles chaining automatically via the descriptor
next pointer chain, which is set up at channel allocation and preserved
across descriptor recycling.
Direct (non-SG) mode has no descriptor queue: writing the BTT register
launches a transfer immediately, so a new transfer must not be programmed
while one is in flight. Keep those transfers serialized by retaining the
idle check on the non-SG path. MCDMA always operates in scatter-gather
mode, so it is unaffected.
Signed-off-by: Suraj Gupta <suraj.gupta2@amd.com>
Co-developed-by: Srinivas Neeli <srinivas.neeli@amd.com>
Signed-off-by: Srinivas Neeli <srinivas.neeli@amd.com>
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260626092656.1563871-3-suraj.gupta2@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Fix a race condition in AXIDMA and MCDMA irq handlers where the channel
could be incorrectly marked as idle and attempt spurious transfers when
descriptors are still being processed.
The issue occurs when:
1. Multiple descriptors are queued and active.
2. An interrupt fires after completing some descriptors.
3. xilinx_dma_complete_descriptor() moves completed descriptors to
done_list.
4. Channel is marked idle and start_transfer() is called even though
active_list still contains unprocessed descriptors.
5. This leads to premature transfer attempts and potential descriptor
corruption or missed completions.
Only mark the channel as idle and start new transfers when the active list
is actually empty, ensuring proper channel state management and avoiding
spurious transfer attempts.
Fixes: c0bba3a99f ("dmaengine: vdma: Add Support for Xilinx AXI Direct Memory Access Engine")
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Signed-off-by: Suraj Gupta <suraj.gupta2@amd.com>
Co-developed-by: Srinivas Neeli <srinivas.neeli@amd.com>
Signed-off-by: Srinivas Neeli <srinivas.neeli@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260626092656.1563871-2-suraj.gupta2@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
A single AXIDMA controller can have one or two channels. When it has two
channels, the reset for both are tied together: resetting one channel
resets the other as well. This creates a problem where resetting one
channel will reset the registers for both channels, including clearing
interrupt enable bits for the other channel, which can then lead to
timeouts as the driver is waiting for an interrupt which never comes.
The driver currently has a probe-time work around for this: when a
channel is created, the driver also resets and enables the
interrupts. With two channels the reset for the second channel will
clear the interrupt enables for the first one. The work around in the
driver is just to manually enable the interrupts again in
xilinx_dma_alloc_chan_resources().
This workaround only addresses the probe-time issue. When channels are
reset at runtime (e.g., in xilinx_dma_terminate_all() or during error
recovery), there's no corresponding mechanism to restore the other
channel's interrupt enables. This leads to one channel having its
interrupts disabled while the driver expects them to work, causing
timeouts and DMA failures.
A proper fix is a complicated matter, as we should not reset the other
channel when it's operating normally. So, perhaps, there should be some
kind of synchronization for a common reset, which is not trivial to
implement. To add to the complexity, the driver also supports other DMA
types, like VDMA, CDMA and MCDMA, which don't have a shared reset.
However, when the two-channel AXIDMA is used in the (assumably) normal
use case, providing DMA for a single memory-to-memory device, the common
reset is a bit smaller issue: when something bad happens on one channel,
or when one channel is terminated, the assumption is that we also want
to terminate the other channel. And thus resetting both at the same time
is "ok".
With that line of thinking we can implement a bit better work around
than just the current probe time work around: let's enable the
AXIDMA interrupts at xilinx_dma_start_transfer() instead.
This ensures interrupts are enabled whenever a transfer starts,
regardless of any prior resets that may have cleared them.
This approach is also more logical: enable interrupts only when needed
for a transfer, rather than at resource allocation time, and, I think,
all the other DMA types should also use this model, but I'm reluctant to
do such changes as I cannot test them.
The reset function still enables interrupts even though it's not needed
for AXIDMA anymore, but it's common code for all DMA types (VDMA, CDMA,
MCDMA), so leave it unchanged to avoid affecting other variants.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Fixes: c0bba3a99f ("dmaengine: vdma: Add Support for Xilinx AXI Direct Memory Access Engine")
Link: https://patch.msgid.link/20260311-xilinx-dma-fix-v2-1-a725abb66e3c@ideasonboard.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The segment .control and .status fields both contain top bits which are
not part of the buffer size, the buffer size is located only in the bottom
max_buffer_len bits. To avoid interference from those top bits, mask out
the size using max_buffer_len first, and only then subtract the values.
Fixes: a575d0b4e6 ("dmaengine: xilinx_dma: Introduce xilinx_dma_get_residue")
Signed-off-by: Marek Vasut <marex@nabladev.com>
Link: https://patch.msgid.link/20260316222530.163815-1-marex@nabladev.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The cyclic DMA calculation is currently entirely broken and reports
residue only for the first segment. The problem is twofold.
First, when the first descriptor finishes, it is moved from active_list
to done_list, but it is never returned back into the active_list. The
xilinx_dma_tx_status() expects the descriptor to be in the active_list
to report any meaningful residue information, which never happens after
the first descriptor finishes. Fix this up in xilinx_dma_start_transfer()
and if the descriptor is cyclic, lift it from done_list and place it back
into active_list list.
Second, the segment .status fields of the descriptor remain dirty. Once
the DMA did one pass on the descriptor, the .status fields are populated
with data by the DMA, but the .status fields are not cleared before reuse
during the next cyclic DMA round. The xilinx_dma_get_residue() recognizes
that as if the descriptor was complete and had 0 residue, which is bogus.
Reinitialize the status field before placing the descriptor back into the
active_list.
Fixes: c0bba3a99f ("dmaengine: vdma: Add Support for Xilinx AXI Direct Memory Access Engine")
Signed-off-by: Marek Vasut <marex@nabladev.com>
Link: https://patch.msgid.link/20260316221943.160375-1-marex@nabladev.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Unlike chan->direction , struct dma_device .directions field is a
bitfield. Turn chan->direction into a bitfield to make it compatible
with struct dma_device .directions .
Fixes: 7e01511443 ("dmaengine: xilinx_dma: Set dma_device directions")
Signed-off-by: Marek Vasut <marex@nabladev.com>
Link: https://patch.msgid.link/20260316221728.160139-1-marex@nabladev.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Two members of struct xdma_desc_block are not descibed leading to
warnings, document them.
Warning: drivers/dma/xilinx/xdma.c:75 struct member 'last_interrupt' not described in 'xdma_chan'
Warning: drivers/dma/xilinx/xdma.c:75 struct member 'stop_requested' not described in 'xdma_chan'
Link: https://patch.msgid.link/20260227022905.233721-1-vkoul@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
devm_regmap_init_mmio returns an ERR_PTR() upon error, not NULL.
Fix the error check and also fix the error message. Use the error code
from ERR_PTR() instead of the wrong value in ret.
Fixes: 17ce252266 ("dmaengine: xilinx: xdma: Add xilinx xdma driver")
Signed-off-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20251014061309.283468-1-alexander.stein@ew.tq-group.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
This converts some of the visually simpler cases that have been split
over multiple lines. I only did the ones that are easy to verify the
resulting diff by having just that final GFP_KERNEL argument on the next
line.
Somebody should probably do a proper coccinelle script for this, but for
me the trivial script actually resulted in an assertion failure in the
middle of the script. I probably had made it a bit _too_ trivial.
So after fighting that far a while I decided to just do some of the
syntactically simpler cases with variations of the previous 'sed'
scripts.
The more syntactically complex multi-line cases would mostly really want
whitespace cleanup anyway.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
Core:
- Add Frank Li as susbstem reviewer to help with reviews
New Support:
- Mediatek support for Dimensity 6300 and 9200 controller
- Qualcomm Kaanapali and Glymur GPI DMA engine support
- Synopsis DW AXI Agilex5 support
- Renesas RZ/V2N SoC support
- Atmel microchip lan9691-dma support
- Tegra ADMA tegra264 support
Updates:
- sg_nents_for_dma() helper use in subsystem
- pm_runtime_mark_last_busy() redundant call update for subsystem
- Residue support for xilinx AXIDMA driver
- Intel Max SGL Size Support and capabilities for DSA3.0
- AXI dma larger than 32bits address support
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE+vs47OPLdNbVcHzyfBQHDyUjg0cFAmmUkGgACgkQfBQHDyUj
g0d6uRAAnVfM6GxVt4PuRd1t+i6qeNhqZrq+8001YtFgOJp0hxPX7k9PP1F42kjp
1zrICvvdqH8gw8+8AaT2JpIZp4vENmjGdnkCo+HU6FgGPCUmkkpehPk58Y2K3r0a
LbHNjjj7V7SDGs9SzT2It+d7KfKnv1adushBhuO7xv524hwSuetw1q8CnLPoxaPx
KNWToovCp5tlHCucWQAdmd4bPsUv1mFMvlJxK4a26WKeL7lU6BeDS06rLTNq5PNZ
51sYdSvyBOSCUcFGToeebJFsKCQukryZTXTtsKMsmLvmHaTMahu2TwNzQ+PRSBSr
kZ9GpS51tet67txGzGzJRFGDY9quKFrajQ60Om6dr9aYm2xW7gEZFa0NKTlz9q7w
kbwsPgd87sYI8MDWpinAuvwUS2OXnihjqdYVp0QouJd8eRiH1pWagtjubGRVYekC
eEZjyxpz8ZD+LT2G3I0uy2FnqCkcEjSfchBCtuPcxSSSkHRXVf4tgUAI833YIdek
gtd7h+/jepcVOcVAeaMVvVYnNIhVkHQkQC1/HmZCqNoyoY/oK8JcUF3UskzP7BPW
gvEJhtFD0RBInu5UM0rS31zF+8Q9EMbDXKY2PiCCyxtsjAe5yWyeNNsUx9DN8ixv
XyclsF7javUOZSoxzH3mCLy+x51p84Mq2KGQGL9H7PK/hWDMmoo=
=nrcD
-----END PGP SIGNATURE-----
Merge tag 'dmaengine-7.0-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"Core:
- Add Frank Li as susbstem reviewer to help with reviews
New Support:
- Mediatek support for Dimensity 6300 and 9200 controller
- Qualcomm Kaanapali and Glymur GPI DMA engine
- Synopsis DW AXI Agilex5
- Renesas RZ/V2N SoC
- Atmel microchip lan9691-dma
- Tegra ADMA tegra264
Updates:
- sg_nents_for_dma() helper use in subsystem
- pm_runtime_mark_last_busy() redundant call update for subsystem
- Residue support for xilinx AXIDMA driver
- Intel Max SGL Size Support and capabilities for DSA3.0
- AXI dma larger than 32bits address support"
* tag 'dmaengine-7.0-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (64 commits)
dmaengine: add Frank Li as reviewer
dt-bindings: dma: qcom,gpi: Update max interrupts lines to 16
dmaengine: fsl-edma: don't explicitly disable clocks in .remove()
dmaengine: xilinx: xdma: use sg_nents_for_dma() helper
dmaengine: sh: use sg_nents_for_dma() helper
dmaengine: sa11x0: use sg_nents_for_dma() helper
dmaengine: qcom: bam_dma: use sg_nents_for_dma() helper
dmaengine: qcom: adm: use sg_nents_for_dma() helper
dmaengine: pxa-dma: use sg_nents_for_dma() helper
dmaengine: lgm: use sg_nents_for_dma() helper
dmaengine: k3dma: use sg_nents_for_dma() helper
dmaengine: dw-axi-dmac: use sg_nents_for_dma() helper
dmaengine: bcm2835-dma: use sg_nents_for_dma() helper
dmaengine: axi-dmac: use sg_nents_for_dma() helper
dmaengine: altera-msgdma: use sg_nents_for_dma() helper
scatterlist: introduce sg_nents_for_dma() helper
dmaengine: idxd: Add Max SGL Size Support for DSA3.0
dmaengine: idxd: Expose DSA3.0 capabilities through sysfs
dmaengine: sh: rz-dmac: Make channel irq local
dmaengine: pl08x: Fix comment stating the difference between PL080 and PL081
...
AXIDMA IP supports reporting the amount of bytes transferred on the S2MM
channel in direct mode (i.e. non-SG), but the driver does not. Thus the
driver always reports that all of the buffer was filled.
Add xilinx_dma_get_residue_axidma_direct_s2mm() which gets the residue
amount for direct AXIDMA for S2MM direction.
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Link: https://patch.msgid.link/20251218-xilinx-dma-residue-fix-v1-1-7cd221d69d6b@ideasonboard.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
pm_runtime_put_autosuspend(), pm_runtime_put_sync_autosuspend(),
pm_runtime_autosuspend() and pm_request_autosuspend() now include a call
to pm_runtime_mark_last_busy(). Remove the now-reduntant explicit call to
pm_runtime_mark_last_busy().
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Abin Joseph <abin.joseph@amd.com>
Link: https://patch.msgid.link/20251027133232.392898-6-sakari.ailus@linux.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
When device tree lacks optional "xlnx,addrwidth" property, the addr_width
variable remained uninitialized with garbage values, causing incorrect
DMA mask configuration and subsequent probe failure. The fix ensures a
fallback to the default 32-bit address width when this property is missing.
Signed-off-by: Suraj Gupta <suraj.gupta2@amd.com>
Fixes: b72db4005f ("dmaengine: vdma: Add 64 bit addressing support to the driver")
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Reviewed-by: Folker Schwesinger <dev@folker-schwesinger.de>
Link: https://patch.msgid.link/20251021183006.3434495-1-suraj.gupta2@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The max_register field is assigned the size of the register memory
region instead of the offset of the last register.
The result is that reading from the regmap via debugfs can cause
a segmentation fault:
tail /sys/kernel/debug/regmap/xdma.1.auto/registers
Unable to handle kernel paging request at virtual address ffff800082f70000
Mem abort info:
ESR = 0x0000000096000007
EC = 0x25: DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
FSC = 0x07: level 3 translation fault
[...]
Call trace:
regmap_mmio_read32le+0x10/0x30
_regmap_bus_reg_read+0x74/0xc0
_regmap_read+0x68/0x198
regmap_read+0x54/0x88
regmap_read_debugfs+0x140/0x380
regmap_map_read_file+0x30/0x48
full_proxy_read+0x68/0xc8
vfs_read+0xcc/0x310
ksys_read+0x7c/0x120
__arm64_sys_read+0x24/0x40
invoke_syscall.constprop.0+0x64/0x108
do_el0_svc+0xb0/0xd8
el0_svc+0x38/0x130
el0t_64_sync_handler+0x120/0x138
el0t_64_sync+0x194/0x198
Code: aa1e03e9 d503201f f9400000 8b214000 (b9400000)
---[ end trace 0000000000000000 ]---
note: tail[1217] exited with irqs disabled
note: tail[1217] exited with preempt_count 1
Segmentation fault
Fixes: 17ce252266 ("dmaengine: xilinx: xdma: Add xilinx xdma driver")
Reviewed-by: Lizhi Hou <lizhi.hou@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Reviewed-by: Alexander Stein <alexander.stein@ew.tq-group.com>
Signed-off-by: Anthony Brandon <anthony@amarulasolutions.com>
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The DMAEngine provides an interface for obtaining DMA transaction
descriptors from an array of scatter gather buffers represented by
struct dma_vec. This interface is used in the DMABUF API of the IIO
framework [1][2].
To enable DMABUF support through the IIO framework for the Xilinx DMA,
implement callback .device_prep_peripheral_dma_vec() of struct
dma_device in the driver.
[1]: 7a86d46998 ("iio: buffer-dmaengine: Support new DMABUF based userspace API")
[2]: 5878853fc9 ("dmaengine: Add API function dmaengine_prep_peripheral_dma_vec()")
Signed-off-by: Folker Schwesinger <dev@folker-schwesinger.de>
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://lore.kernel.org/r/DCCKQLKOZC06.2H6LJ8RJQJNV2@folker-schwesinger.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add shutdown callback to ensure that DMA operations are properly stopped
and resources are released during system shutdown or kexec operations.
Fix incorrect PM state handling in the remove function that was causing
clock disable warnings during the shutdown operations, which was not
implemented earlier. The original logic used pm_runtime_enabled() check
after calling the pm_runtime_disable(), would always evaluate to true
after the disable call, which leads to unconditionally calling the
runtime_suspend regardless of the device's actual power state.
During shutdown, the device may already be suspended with clock disabled
from the autosuspend timer, causing the clock framework to warn about
the double-disable attempt. The pm_runtime_active() function checks the
actual device power state rather than the PM subsystem's enabled/disabled
status. ensuring the runtime_suspend is only called when the device is in
active power state. This prevents clock warnings during shutdown while
maintaining proper cleanup during normal remove operations.
Signed-off-by: Abin Joseph <abin.joseph@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://lore.kernel.org/r/20250722070255.28944-1-abin.joseph@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Coalesce the direction bits from the enabled TX and/or RX channels into
the directions bit mask of dma_device. Without this mask set,
dma_get_slave_caps() in the DMAEngine fails, which prevents the driver
from being used with an IIO DMAEngine buffer.
Signed-off-by: Thomas Gessler <thomas.gessler@brueckmann-gmbh.de>
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Link: https://lore.kernel.org/r/20250507182101.909010-1-thomas.gessler@brueckmann-gmbh.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Set the maximumg DMA segment size from the actual core configuration
value. Without this setting, the default value of 64 KiB is reported,
and larger sizes cannot be used for IIO DMAEngine buffers.
Signed-off-by: Thomas Gessler <thomas.gessler@brueckmann-gmbh.de>
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Link: https://lore.kernel.org/r/20250314134717.703287-1-thomas.gessler@brueckmann-gmbh.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Replace ternary (condition ? "enable" : "disable") syntax with helpers
from string_choices.h because:
1. Simple function call with one argument is easier to read. Ternary
operator has three arguments and with wrapping might lead to quite
long code.
2. Is slightly shorter thus also easier to read.
3. It brings uniformity in the text - same string.
4. Allows deduping by the linker, which results in a smaller binary
file.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> #dw-edma
Reviewed-by: Jernej Skrabec <jernej.skrabec@gmail.com>
Link: https://lore.kernel.org/r/20250114191021.854080-1-krzysztof.kozlowski@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The VDMA can work in two modes, parking or circular. Do not program the
parking mode registers in case circular mode is used, it is useless.
Signed-off-by: Marek Vasut <marex@denx.de>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://lore.kernel.org/r/20241031171132.56861-1-marex@denx.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The variable ret is being checked for an error and returning ret
and the following statement returns ret too. The if check is
redundant, and remove it. Just return the value returned from
the call to regmap_write.
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Link: https://lore.kernel.org/r/20241107114656.17611-1-colin.i.king@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Correctly spelled comments make it easier for the reader to understand
the code.
Replace 'enngine' with 'engine' in the comment &
replace 'trascatioin' with 'transaction' in the comment &
replace 'descripter' with 'descriptor' in the comment &
replace 'descritpor' with 'descriptor' in the comment &
replace 'rgisters' with 'registers' in the comment.
Signed-off-by: Yan Zhen <yanzhen@vivo.com>
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
Link: https://lore.kernel.org/r/20240918034114.860132-1-yanzhen@vivo.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
After commit 0edb555a65 ("platform: Make platform_driver::remove()
return void") .remove() is (again) the right callback to implement for
platform drivers.
Convert all platform drivers below drivers/dma after the previous
conversion commits apart from the wireless drivers to use .remove(),
with the eventual goal to drop struct platform_driver::remove_new(). As
.remove() and .remove_new() have the same prototypes, conversion is done
by just changing the structure member name in the driver initializer.
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@baylibre.com>
Link: https://lore.kernel.org/r/20241004062227.187726-2-u.kleine-koenig@baylibre.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
ZynqMP DMA IP and AMD Versal Gen 2 DMA IP are similar but have different
interrupt register offset. Create a dedicated compatible string to
support Versal Gen 2 DMA IP with Irq register offset for interrupt
Enable/Disable/Status/Mask functionality.
Signed-off-by: Abin Joseph <abin.joseph@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://lore.kernel.org/r/20240808100024.317497-3-abin.joseph@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
New support:
- Freescale i.MX8ULP edma support in edma driver
- StarFive JH8100 DMA support in Synopsis axi-dmac driver
Updates:
- Tracing support for freescale edma driver, updates to dpaa2 driver
- Remove unused QCom hidma DT support
- Support for i2c dma in imx-sdma
- Maintainers update for idxd and edma drivers
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE+vs47OPLdNbVcHzyfBQHDyUjg0cFAmZMPDAACgkQfBQHDyUj
g0f2dQ/+OSBO+fs8yZJaDjxETLBZFGWX/VoHIaARTUxJYHNhXXZApy6gluMJrPxM
/6AeTZO2XCvPdZWzWRslkS2pPLuR8alWJo+SimGB6VoKDAplQLemMNorvM0w1sof
rHS38JHkSUifjQtlJyY4AWcP21jVe08MpXaOQmcHfWQ8QaEhacPrq7V46x6OogAO
x/QXjiOizo6GMEOpS+CMpR2rrM4sq90GEnN0c7cXg+BXmoiCp586peqqkoAUSUry
acIb+vwrGn2WOXrWlYUpFFIvOGVfwiZBP8mnOAhN7KR6xcm0OXbLhDKn3pRofYbo
ZaORd2Nae4NJxsiVYUr3Ae+4NwxKhFi7xzY5zNjq9IgKrvOT/22E2fj1wwCfjYEj
7cShuB/1Qq+qcF3zRSWrFjKV3dFCFp2+jMrBTbaZq13Gvt6cEtOqcwT3XiRJ18A/
X8lXWzXEHvLt74AV819cEUg8K3LnoiAuXzfkQgAve8H/bxtlvO+HM6p8s9v164pb
mp9mHI5QUNG2A5xzCGQ3KCpBEUhODOml2k2dvkQRWjKUW1zXGl5+67rB6npkKJV7
fw6amW2l/cDYver3tzKQUFWRJgQt8rbNJ3u30o1Ea1NXFrr6gS/RNhVkAm3vkWrq
E3kpKgQ4A9RTqlMbvGf+/eel7oOv6RYqZWfc5LW39p3+EAoDWF4=
=arjK
-----END PGP SIGNATURE-----
Merge tag 'dmaengine-6.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"New HW support:
- Freescale i.MX8ULP edma support in edma driver
- StarFive JH8100 DMA support in Synopsis axi-dmac driver
Updates:
- Tracing support for freescale edma driver, updates to dpaa2 driver
- Remove unused QCom hidma DT support
- Support for i2c dma in imx-sdma
- Maintainers update for idxd and edma drivers"
* tag 'dmaengine-6.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (42 commits)
MAINTAINERS: Update role for IDXD driver
dmaengine: fsl-edma: use _Generic to handle difference type
dmaengine: fsl-edma: add trace event support
dmaengine: idxd: Avoid unnecessary destruction of file_ida
dmaengine: xilinx: xdma: fix module autoloading
dt-bindings: dma: fsl-edma: allow 'power-domains' property
dt-bindings: dma: fsl-edma: remove 'clocks' from required
dmaengine: fsl-dpaa2-qdma: Fix kernel-doc check warning
dmaengine: imx-sdma: Add i2c dma support
dmaengine: imx-sdma: utilize compiler to calculate ADDRS_ARRAY_SIZE_V<n>
dt-bindings: fsl-imx-sdma: Add I2C peripheral types ID
dt-bindings: fsl-dma: fsl-edma: clean up unused "fsl,imx8qm-adma" compatible string
dmaengine: fsl-edma: clean up unused "fsl,imx8qm-adma" compatible string
dt-bindings: dma: Drop unused QCom hidma binding
dmaengine: qcom: Drop hidma DT support
dmaengine: pl08x: Use kcalloc() instead of kzalloc()
dmaengine: fsl-dpaa2-qdma: Update DPDMAI interfaces to version 3
dmaengine: fsl-edma: fix miss mutex unlock at an error return path
dmaengine: pch_dma: remove unused function chan2parent
dmaengine: fsl-dpaa2-qdma: Add dpdmai_cmd_open
...
Add MODULE_DEVICE_TABLE(), so the module could be properly autoloaded
based on the alias from of_device_id table.
Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org>
Link: https://lore.kernel.org/r/20240410170317.248715-2-krzk@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The current xdma_synchronize method does not properly wait for the last
transfer to be done. Due to limitations of the XMDA engine, it is not
possible to stop a transfer in the middle of a descriptor. Said
otherwise, if a stop is requested at the end of descriptor "N" and the OS
is fast enough, the DMA controller will effectively stop immediately.
However, if the OS is slightly too slow to request the stop and the DMA
engine starts descriptor "N+1", the N+1 transfer will be performed until
its end. This means that after a terminate_all, the last descriptor must
remain valid and the synchronization must wait for this last descriptor to
be terminated.
Fixes: 855c2e1d18 ("dmaengine: xilinx: xdma: Rework xdma_terminate_all()")
Fixes: f5c392d106 ("dmaengine: xilinx: xdma: Add terminate_all/synchronize callbacks")
Cc: stable@vger.kernel.org
Suggested-by: Miquel Raynal <miquel.raynal@bootlin.com>
Signed-off-by: Louis Chauvet <louis.chauvet@bootlin.com>
Link: https://lore.kernel.org/r/20240327-digigram-xdma-fixes-v1-2-45f4a52c0283@bootlin.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
xilinx_dpdma_chan_done_irq and xilinx_dpdma_chan_vsync_irq are always
called with IRQs disabled from xilinx_dpdma_irq_handler. Therefore we
don't need to save/restore the IRQ flags.
Signed-off-by: Sean Anderson <sean.anderson@linux.dev>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Link: https://lore.kernel.org/r/20240308210034.3634938-3-sean.anderson@linux.dev
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The VDMA HSIZE register (corresponding to sgl[0].size) is only 16bit wide /
the VSIZE register (corresponding to numf) is only 13bit wide, so reject
requests not fitting within that rather than silently transferring too
little data.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://lore.kernel.org/r/20240105105956.1370220-1-peter@korsgaard.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
