The fsl_re_ctrl and fsl_re_chan_cfg structures describe memory-mapped
RAID Engine registers accessed only via ioread32be()/iowrite32be(), yet
the pointers to them (re_regs in struct fsl_re_drv_private, and jrregs
in struct fsl_re_chan) were not __iomem-qualified, so sparse emitted
"different address spaces" warnings for every register access.
Store both MMIO bases as a plain void __iomem * and derive jrregs with
void __iomem * arithmetic from re_regs, rather than carrying typed
register struct pointers through the driver. Each function that touches
the registers introduces a local typed pointer (struct fsl_re_ctrl
__iomem *ctrl) and uses ->field, which is the idiomatic kernel pattern
and keeps the registers' __iomem qualification intact.
Reported-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/oe-kbuild-all/202008111749.yy85rFMD%25lkp@intel.com/
Assisted-by: opencode:hy3-free
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260716202949.677290-4-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The final-frame bit is now passed as the "final" argument of
fill_cfd_frame() (as fsl_re_prep_dma_memcpy already did) and set in CPU
order before the single cpu_to_be32() store, replacing the previous
read-modify-write of the __be32 efrl32 field.
Assisted-by: opencode:hy3-free
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260716202949.677290-3-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The descriptor structs (fsl_re_cmpnd_frame / fsl_re_hw_desc) are
in-memory but their fields are __be32, because the structures are handed
to the device as big-endian. The driver stored CPU-endian u32 values
into them directly, which is both wrong (the engine would see
byte-swapped lengths/addresses) and flagged by sparse as a base-type
mismatch.
Wrap those stores in cpu_to_be32() so the values are little->big
converted.
Reported-by: kernel test robot <lkp@intel.com>
Link: https://lore.kernel.org/oe-kbuild-all/202008111749.yy85rFMD%25lkp@intel.com/
Assisted-by: opencode:hy3-free
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260716202949.677290-2-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
As per 'Designware Cores PCI Express Controller Databook',
Section 7.1 - Overview, HDMA supports 64 Read and 64 Write
channels. Current controller driver supports up to 8 read and
write channels only. In order to utilize all the channels the
controller driver need to have the channel related structs
and variables as per the number of channels supported by IP.
Following changes are made to enable 64 Read / 64 Write
channel support:
o Defined HDMA specific macros to reflect the channel count.
o The count of ll_regions and dt_regions in dw_edma_chip and
dw_edma_pcie_data shall be in accordance to number of read
and write channels.
o In dw_edma_probe() configure the channels as per the channels
of the IP used.
o Changed mask types to u64 for higher channel counts.
Signed-off-by: Devendra K Verma <devendra.verma@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713064854.4065262-1-devverma@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Enable DMANSECCTRL.INTREN_ANYCHINTR during probe so channel
interrupts are propagated when integrators wire DMA-350 channels
onto a shared IRQ line.
Signed-off-by: Jun Guo <jun.guo@cixtech.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Link: https://patch.msgid.link/20260521072924.3000282-2-jun.guo@cixtech.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The current descriptor layout is:
struct dw_edma_desc *desc
└─ chunk list
└─ burst[]
Creating a DMA descriptor requires at least two kzalloc() calls because
each chunk is allocated as a linked-list node. Since the number of bursts
is already known when the descriptor is created, this linked-list layer is
unnecessary.
Move the burst array directly into struct dw_edma_desc and remove the
struct dw_edma_chunk layer entirely.
Use start_burst and done_burst to track the current bursts, which current
are in the DMA link list.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-10-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The current descriptor layout is:
struct dw_edma_desc *desc
└─ chunk list
└─ burst list
Creating a DMA descriptor requires at least three kzalloc() calls because
each burst is allocated as a linked-list node. Since the number of bursts
is already known when the descriptor is created, a linked list is not
necessary.
Allocate a burst array when creating each chunk to simplify the code and
eliminate one kzalloc() call.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-9-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use common dw_edma_core_start() for both eDMA and HDMA. Remove .start()
callback functions at eDMA and HDMA.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-8-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add a non_ll_start() callback and move the common non-linked-list channel
handling into the EDMA core so it can be shared by both the EDMA and HDMA.
Prepare for the upcoming reorganization of the burst and chunk structures.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-7-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Introduce four new callbacks to fill link list entries in preparation for
replacing dw_(edma|hdma)_v0_core_start().
Filling link list entries is expected to become more complex, and without
this abstraction both eDMA and HDMA paths would need to duplicate the same
logic. Add fill-entry callbacks so the code can be shared cleanly between
eDMA and HDMA implementations.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-6-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Move the channel-enable logic into a new helper function,
dw_(edma|hdma)_v0_core_ch_enable(), in preparation for supporting dynamic
link entry additions.
No functional changes.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-5-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Some helper functions do not use any information from dw_edma_chunk, so
passing a dw_edma_chan pointer directly avoids an unnecessary level of
pointer dereferencing and simplifies data access.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-4-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
ll_region is identical for all chunks belonging to the same DMA channel,
so there is no need to copy it into each chunk. Move ll_region to
struct dw_edma_chan to avoid redundant copies.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-3-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Reusing ll_region.sz as the transfer size is misleading because
ll_region.sz represents the memory size of the EDMA link list, not the
amount of data to be transferred.
Add a new xfer_sz field to explicitly indicate the total transfer size
of a chunk.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-2-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The control field in a DMA link list entry must be updated as the final
step because it includes the CB bit, which indicates whether the entry is
ready. Add dma_wmb() to ensure the correct memory write ordering.
Currently the driver does not update DMA link entries while the DMA is
running, so no visible failure occurs. However, fixing the ordering now
prepares the driver for supporting link entry updates during DMA operation.
Tested-by: Koichiro Den <den@valinux.co.jp>
Tested-By: Devendra Verma <devendra.verma@amd.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260713-edma_ll-v7-1-6fb7498c901e@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Replace the #include of <linux/mod_devicetable.h> by the more specific
<linux/device-id/*.h> where applicable. For most cases the include
can be dropped completely, only a few drivers need one or two headers
added.
Acked-by: Danilo Krummrich <dakr@kernel.org>
Acked-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://patch.msgid.link/1a3f2007c5c5dcf555c09a4035ce3ae8ef1b6c49.1782808461.git.u.kleine-koenig@baylibre.com
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
init_pl330_debugfs() creates a debugfs file with struct pl330_dmac as
private data. pl330_remove() then unregisters the DMA device and frees
the PL330 channel/thread state without removing that file.
Keep the debugfs dentry and remove it before tearing down the DMAC state
used by the debugfs show callback.
Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn>
Link: https://patch.msgid.link/20260615091645.28878-1-pengpeng@iscas.ac.cn
Signed-off-by: Vinod Koul <vkoul@kernel.org>
When writing the lli->ctl_hi, this is an __le32 type so the
value being orred should be convered to __le32 by cpu_to_le32.
Fixes 1deb96c0fa ("dmaegine: dw-axi-dmac: Support device_prep_dma_cyclic()")
--
Note, the call to axi_chan_irq_clear() is passing lli->status_lo
through which is also an __le32 but it does not seem to be set
anywhere. Is this also a bug?
Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk>
Link: https://patch.msgid.link/20260617084944.705266-1-ben.dooks@codethink.co.uk
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Rename XILINX_DMA_LOOP_COUNT to XILINX_DMA_POLL_TIMEOUT_US because it is
a timeout value, not a loop count for polling register in microseconds.
No functional changes.
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Signed-off-by: Alex Bereza <alex@bereza.email>
Link: https://patch.msgid.link/20260402-fix-atomic-poll-timeout-regression-v4-2-f30d6a6c13cb@bereza.email
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Currently when calling xilinx_dma_poll_timeout with delay_us=0 and a
condition that is never fulfilled, the CPU busy-waits for prolonged time
and the timeout triggers only with a massive delay causing a CPU stall.
This happens due to a huge underestimation of wall clock time in
poll_timeout_us_atomic. Commit 7349a69cf3 ("iopoll: Do not use
timekeeping in read_poll_timeout_atomic()") changed the behavior to no
longer use ktime_get at the expense of underestimation of wall clock
time which appears to be very large for delay_us=0. Instead of timing
out after approximately XILINX_DMA_LOOP_COUNT microseconds, the timeout
takes XILINX_DMA_LOOP_COUNT * 1000 * (time that the overhead of the for
loop in poll_timeout_us_atomic takes) which is in the range of several
minutes for XILINX_DMA_LOOP_COUNT=1000000. Fix this by using a non-zero
value for delay_us. Use delay_us=10 to keep the delay in the hot path of
starting DMA transfers minimal but still avoid CPU stalls in case of
unexpected hardware failures.
One-off measurement with delay_us=0 causes the cpu to busy wait around 7
minutes in the timeout case. After applying this patch with delay_us=10
the measured timeout was 1053428 microseconds which is roughly
equivalent to the expected 1000000 microseconds specified in
XILINX_DMA_LOOP_COUNT.
Add a constant XILINX_DMA_POLL_DELAY_US for delay_us value.
Fixes: 9495f26482 ("dmaengine: xilinx_vdma: Use readl_poll_timeout instead of do while loop's")
Fixes: 7349a69cf3 ("iopoll: Do not use timekeeping in read_poll_timeout_atomic()")
Reviewed-by: Suraj Gupta <suraj.gupta2@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Alex Bereza <alex@bereza.email>
Link: https://patch.msgid.link/20260402-fix-atomic-poll-timeout-regression-v4-1-f30d6a6c13cb@bereza.email
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Out of memory situation on driver's probe is expected to be reported to
the driver's framework with a proper -ENOMEM error code.
Fixes: e9f08b6525 ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Vladimir Zapolskiy <vz@kernel.org>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260630144214.4080302-1-vz@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The zynqmp_dma_remove() function was converted from returning int to
void, but the kernel doc comment was not updated to reflect this change.
Remove the stale "Return: Always '0'" documentation that no longer
applies to the void function.
Fixes: b1c50ac254 ("dmaengine: xilinx: zynqmp_dma: Convert to platform remove callback returning void")
Signed-off-by: Golla Nagendra <nagendra.golla@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260630064844.705173-3-nagendra.golla@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
In zynqmp_dma_remove(), runtime PM was disabled only after checking
state and doing a manual suspend. This can race with runtime PM in the
remove/unbind (rmmod) path.
Disable runtime PM first, then suspend only if the device is not already
suspended. To prevent any further runtime PM transitions.
Fixes: 72dd8b2914 ("dmaengine: zynqmp_dma: Add shutdown operation support")
Co-developed-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
Signed-off-by: Prasanna Kumar T S M <ptsm@linux.microsoft.com>
Signed-off-by: Golla Nagendra <nagendra.golla@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260630064844.705173-2-nagendra.golla@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Optimize AXI DMA control register programming by consolidating
coalesce count and delay configuration into a single register write.
Previously, the coalesce count was written separately from the delay
configuration, resulting in two register writes. Combine these into
one write operation to reduce bus overhead.
Additionally, avoid redundant channel starts in xilinx_dma_start_transfer()
and xilinx_mcdma_start_transfer() by only calling xilinx_dma_start() when
the channel is actually idle.
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Signed-off-by: Suraj Gupta <suraj.gupta2@amd.com>
Co-developed-by: Srinivas Neeli <srinivas.neeli@amd.com>
Signed-off-by: Srinivas Neeli <srinivas.neeli@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260626092656.1563871-4-suraj.gupta2@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Relax the idle check in xilinx_dma_start_transfer() and
xilinx_mcdma_start_transfer() that prevented new transfers from being
queued when the channel was busy, so scatter-gather transfers can be
chained onto an in-flight transfer.
In scatter-gather mode, only update the CURDESC register when the active
list is empty to avoid interfering with transfers already in progress.
When the active list contains transfers, the hardware tail pointer
extension mechanism handles chaining automatically via the descriptor
next pointer chain, which is set up at channel allocation and preserved
across descriptor recycling.
Direct (non-SG) mode has no descriptor queue: writing the BTT register
launches a transfer immediately, so a new transfer must not be programmed
while one is in flight. Keep those transfers serialized by retaining the
idle check on the non-SG path. MCDMA always operates in scatter-gather
mode, so it is unaffected.
Signed-off-by: Suraj Gupta <suraj.gupta2@amd.com>
Co-developed-by: Srinivas Neeli <srinivas.neeli@amd.com>
Signed-off-by: Srinivas Neeli <srinivas.neeli@amd.com>
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260626092656.1563871-3-suraj.gupta2@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Fix a race condition in AXIDMA and MCDMA irq handlers where the channel
could be incorrectly marked as idle and attempt spurious transfers when
descriptors are still being processed.
The issue occurs when:
1. Multiple descriptors are queued and active.
2. An interrupt fires after completing some descriptors.
3. xilinx_dma_complete_descriptor() moves completed descriptors to
done_list.
4. Channel is marked idle and start_transfer() is called even though
active_list still contains unprocessed descriptors.
5. This leads to premature transfer attempts and potential descriptor
corruption or missed completions.
Only mark the channel as idle and start new transfers when the active list
is actually empty, ensuring proper channel state management and avoiding
spurious transfer attempts.
Fixes: c0bba3a99f ("dmaengine: vdma: Add Support for Xilinx AXI Direct Memory Access Engine")
Tested-by: Folker Schwesinger <dev@folker-schwesinger.de>
Signed-off-by: Suraj Gupta <suraj.gupta2@amd.com>
Co-developed-by: Srinivas Neeli <srinivas.neeli@amd.com>
Signed-off-by: Srinivas Neeli <srinivas.neeli@amd.com>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Link: https://patch.msgid.link/20260626092656.1563871-2-suraj.gupta2@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The failed_dev_add and failed_dev_name paths drop the file-device
reference while wq->wq_lock is still held. If put_device(fdev) drops the
last reference, idxd_file_dev_release() runs synchronously and tries to
take wq->wq_lock again, deadlocking.
Those paths also fall through into the later ctx cleanup labels even
though idxd_file_dev_release() owns that cleanup and frees ctx. This can
make idxd_xa_pasid_remove(ctx) and kfree(ctx) operate on a freed context.
Move idxd_wq_get() before file-device setup can fail, since the release
callback always calls idxd_wq_put(). Then unlock wq->wq_lock before
put_device(fdev) and return directly from the file-device setup failure
path, leaving ctx cleanup to the release callback.
Fixes: e6fd6d7e5f ("dmaengine: idxd: add a device to represent the file opened")
Signed-off-by: Yuho Choi <dbgh9129@gmail.com>
Reviewed-by: Dave Jiang <dave.jiang@intel.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Link: https://patch.msgid.link/20260525141550.1385581-1-dbgh9129@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The release callbacks for wq, engine, and group devices
(idxd_conf_wq_release, idxd_conf_engine_release,
idxd_conf_group_release) each call kfree() on the enclosing struct.
The setup error paths and cleanup functions also call kfree()
explicitly after put_device(), producing a double free whenever
put_device() drops the reference count to zero and fires the release.
In the setup functions, device_initialize() is called before
device_add(), so the reference count is exactly 1 at the error sites.
put_device() unconditionally fires the release, which frees the struct;
the subsequent explicit kfree() then operates on freed memory.
For idxd_setup_wqs(), the wq release callback also owns opcap_bmap
and wqcfg. The error unwind additionally freed those fields explicitly
before calling put_device(), causing further double frees on both.
Remove the redundant explicit kfree() calls from all setup error paths
and cleanup functions for wq, engine, and group structs, delegating
sole ownership of those allocations to the release callbacks.
Fixes: 7c5dd23e57 ("dmaengine: idxd: fix wq conf_dev 'struct device' lifetime")
Fixes: 75b9113090 ("dmaengine: idxd: fix engine conf_dev lifetime")
Fixes: defe49f960 ("dmaengine: idxd: fix group conf_dev lifetime")
Signed-off-by: Yuho Choi <dbgh9129@gmail.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260415205452.67155-1-dbgh9129@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Simpler to call devm_platform_ioremap_resource() as it returns multiple
error messages for whichever part fails.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260609212531.22044-1-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
If dynamic memory allocation in driver's probe function execution fails, it
should be reported to the driver's framework with -ENOMEM error code.
Fixes: 9135408c3a ("dmaengine: mediatek: Add MediaTek UART APDMA support")
Signed-off-by: Vladimir Zapolskiy <vz@kernel.org>
Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Matthias Brugger <matthias.bgg@gmail.com>
Link: https://patch.msgid.link/20260701200703.117929-1-vz@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
When terminating DMA transfers, active descriptors are not properly
reclaimed. Only cyclic descriptors were handled, leaving non-cyclic
descriptors and their LLI chains to be permanently leaked.
Fix by using vchan_terminate_vdesc() which handles both cyclic and
non-cyclic descriptors by adding them to desc_terminated queue for
proper cleanup.
Add pchan->desc != pchan->done check to prevent double-adding completed
descriptors, which would corrupt the list.
Fixes: 5558593087 ("dmaengine: sun6i: Add driver for the Allwinner A31 DMA controller")
Signed-off-by: Hongling Zeng <zenghongling@kylinos.cn>
Acked-by: Jernej Skrabec <jernej.skrabec@gmail.com>
Suggested-by: Frank Li <Frank.li@oss.nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260701045733.33654-1-zenghongling@kylinos.cn
Signed-off-by: Vinod Koul <vkoul@kernel.org>
FIELD_GET(SE_THRESH_MASK, thresh) extracts bits [31:23] from thresh and
right-shifts them, which is the inverse of the intended operation. Since
thresh is derived from se_buf_len / 2 (at most 255), bits [31:23] are
always zero, so the SE threshold is never actually programmed into the
register.
Use FIELD_PREP() instead to correctly left-shift thresh into bits [31:23]
of the valid_en_se register, consistent with the FIELD_PREP usage for
the perf tuner config just above.
Fixes: 30eba9df76 ("dmaengine: switchtec-dma: Implement hardware initialization and cleanup")
Signed-off-by: David Carlier <devnexen@gmail.com>
Review-by: Logan Gunthorpe <logang@deltatee.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260317083252.13224-1-devnexen@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Pass dma_slave_config to dw_edma_device_transfer() to support atomic
configuration and descriptor preparation when a non-NULL config is
provided to device_prep_config_sg().
Tested-by: Niklas Cassel <cassel@kernel.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260521-dma_prep_config-v7-5-1f73f4899883@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use the new .device_prep_config_sg() callback to combine configuration and
descriptor preparation.
No functional changes.
Tested-by: Niklas Cassel <cassel@kernel.org>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260521-dma_prep_config-v7-4-1f73f4899883@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Introduce dmaengine_prep_config_single_safe() and
dmaengine_prep_config_sg_safe() to provide a reentrant-safe way to
combine slave configuration and transfer preparation.
Drivers may implement the new device_prep_config_sg() callback to perform
both steps atomically. If the callback is not provided, the helpers fall
back to calling dmaengine_slave_config() followed by
dmaengine_prep_slave_sg() under per-channel spinlock protection.
Tested-by: Niklas Cassel <cassel@kernel.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260521-dma_prep_config-v7-2-1f73f4899883@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAmo7CrQUHGJoZWxnYWFz
QGdvb2dsZS5jb20ACgkQWYigwDrT+vxsvw/7B4rHdVs1/LERE4H0bBKKNz7mr6g/
6JOBDYc9ERgZpDgAwBKm+uuFzg8KLK0lf5PU7p2zNxjMVia+c4AMTHw/icN91yfL
NZLP013GQmZ6s1r290AMnVtUqd1hNm2R+r4nzXTm85y9B25tZIiJj3gssIXPKDTE
74rrRiNVO9+zj8JTED6kbSoAq40iv94Ux9VnumHy6kF1izsFAht7eJmkICgeOLRX
okBq86Tqivwv85XpqOQZTHudBwvS91HHsAdClW9UcXyRQo3RJAcba2tkglfZ2Ozo
RbfqxxKNuW/3TTziOflfzm/oImZuMQdlvu98udMQ+LacOmLN+SWPdyVFly9X87i+
ZGH4bKxtqcba8EYFZ+VRrG3JNuOWwcfmAPC93WhuwhywJ6evRg9Wufffss6GLUmH
g9id/KbMI0RRkYnne7w9nmmCwLge0/3klhf5dwyZZ/q9IyKPUqjyagdbjFA4Esp5
wLK1H9645pXtZtj5h8qMlZQ+rrDt/IYWqsCo15pB1Z4y3Lpb+JaYsHOS+Ye/3PDA
0MDeCoTvK4scZGtM6l10jn3fdGVy6EL+3thteQz/+FHzmMEXp53TnrQTBhKtdGc0
RfPLZEMuFCjbr+6u6bSu3thD1eH4TgQHIWY1Fl4FdczXzc6+A6blBY4BSZAqX8kU
xjNfpptAieR4BtM=
=h+xD
-----END PGP SIGNATURE-----
Merge tag 'pci-v7.2-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci
Pull pci updates from Bjorn Helgaas:
"Enumeration:
- Remove MPS/MRRS Kconfig settings (CONFIG_PCIE_BUS_*) that worked
around a WiFi device defect; use a quirk or boot-time
"pci=pcie_bus_tune_*" kernel parameter instead (Bjorn Helgaas)
- Always lift 2.5GT/s restriction in PCIe failed link retraining to
avoid clamping a link to 2.5GT/s after hot-plug changes the device
(Maciej W. Rozycki)
- Request bus reassignment when not probe-only to fix an enumeration
regression on Marvell CN106XX and possibly other DT-based systems
(Ratheesh Kannoth)
- Fix procfs race between pci_proc_init() and pci_bus_add_device()
that resulted in 'proc_dir_entry ... already registered' warnings
and pointer corruption (Krzysztof Wilczyński)
- Fix sysfs race that causes 'duplicate filename' warnings and boot
panics by converting PCI resource files to static attributes
(Krzysztof Wilczyński)
- Expose sysfs 'resourceN_resize' attributes only on platforms with
PCI mmap (Krzysztof Wilczyński)
- Require CAP_SYS_ADMIN to write to sysfs 'resourceN_resize'
attributes (Krzysztof Wilczyński)
- Add security_locked_down(LOCKDOWN_PCI_ACCESS) to alpha PCI resource
mmap path to match the generic path (Krzysztof Wilczyński)
- Use kstrtobool() to parse the 'rom' attribute input to avoid the
unexpected behavior of enabling the ROM when writing '0' with no
trailing newline (Krzysztof Wilczyński)
Resource management:
- Improve resource claim logging for debuggability (Ilpo Järvinen)
- Clean up several uses of const parameters (Ilpo Järvinen)
- Check option ROM header signatures and lengths before accessing to
avoid page faults and alignment faults (Guixin Liu)
ASPM:
- Don't reconfigure ASPM when entering low-power D-state; only do it
when returning back to D0 (Carlos Bilbao)
Power management:
- During suspend, set power state to 'unknown' for all devices, not
just those with drivers (Lukas Wunner)
- Skip restoring Resizable BARs and VF Resizable BARs if device
doesn't respond to config reads, to avoid invalid array accesses
(Marco Nenciarini)
- Add pci_suspend_retains_context() so drivers can tell whether
devices retain internal state across suspend/resume, since some
platforms reset devices on suspend; use this in nvme to avoid
issues on Qcom RCs (Manivannan Sadhasivam)
Power control:
- Only to power on/off devices that actually support power control to
avoid poking at incompatible devices mentioned in DT (Manivannan
Sadhasivam)
Virtualization and resets:
- Log device readiness timeouts as errors, not warnings, because the
device is likely unusable in this case (Bjorn Helgaas)
- Wait for device readiness after soft reset (D3hot ->
D0uninitialized transition), when the device may respond with
Request Retry Status (RRS) if it needs more time to initialize
(Bjorn Helgaas)
- Drop unnecessary retries when restoring BARs because resets should
now already include all required delays (Lukas Wunner)
- Avoid FLR for MediaTek MT7925 WiFi, where FLR fails after a VM
terminates uncleanly (Jose Ignacio Tornos Martinez)
- Avoid SBR for Qualcomm WCN6855/WCN7850 WiFi, SDX62/SDX65 modems,
which seem not to support it correctly (Jose Ignacio Tornos
Martinez)
Peer-to-peer DMA:
- Prevent P2PDMA as well as CPU access to non-mappable BARs, e.g.,
s390 ISM BARs (Matt Evans)
- Add Intel QAT, DSA, IAA devices to whitelist (Lukas Wunner)
Endpoint framework:
- Add endpoint controller APIs for use by function drivers to
discover auxiliary blocks like DMA engines (Koichiro Den)
- Remember DesignWare eDMA engine base/size and expose them via the
EPC aux-resource API (Koichiro Den)
- Add endpoint embedded doorbell fallback, used if MSI allocation
fails (Koichiro Den)
- Validate BAR index and remove dead BAR read in endpoint doorbell
test (Carlos Bilbao)
- Unwind MSI/MSI-X vectors if NTB initialization fails part-way
through (Koichiro Den)
- Cache sleepable pci_irq_vector() value at ISR setup to avoid
calling it from hardirq context (Koichiro Den)
- Call sleepable pci_epc_raise_irq() from a work item instead of
atomic context, e.g., when setting bits in NTB peer doorbells in
the ntb_peer_db_set() path (Koichiro Den)
- Report 0-based vNTB doorbell vector to account for link event 0 and
historically skipped slot 1 (Koichiro Den)
- Prevent configfs writes to vNTB db_count and other values that are
already in use after EPC attach (Koichiro Den)
- Account for vNTB db_valid reserved slots (link event 0 and
historically skipped slot 1) so they don't appear as valid
doorbells (Koichiro Den)
- Implement vNTB .db_vector_count()/mask() for doorbells so clients
can use multiple vectors and avoid thundering herds (Koichiro Den)
- Report 0-based NTB doorbell vector to account for link event 0 and
historically skipped slot 1 (Koichiro Den)
- Fix doorbell bitmask and IRQ vector handling to clear only
specified bits, use the correct vector for non-contiguous Linux IRQ
numbers, and validate incoming vectors (Koichiro Den)
- Implement NTB .db_vector_count()/mask() for doorbells so clients
can use multiple vectors (Koichiro Den)
Native PCIe controller infrastructure:
- Add pci_host_common_link_train_delay() for the mandatory delay
after > 5GT/s Link training completes and use it for cadence HPA,
j721e, LGA; dwc; aardvark, mediatek-gen3, rzg3s (Hans Zhang)
- Protect root bus removal with rescan lock in altera, brcmstb,
cadence, dwc, iproc, mediatek, plda, rockchip to prevent
use-after-free or crashes when racing with sysfs rescan or hotplug
(Hans Zhang)
- Add pci_host_common_parse_ports() for use by any native driver to
parse Root Port properties (per-Link features like width, speed,
PHY, power and reset control, etc should be described in Root Port
stanzas, not the host bridge; currently only reset GPIOs
implemented) (Sherry Sun)
New native PCIe controller drivers:
- Add DT binding and driver for UltraRISC DP1000 PCIe controller
(Xincheng Zhang, Jia Wang)
Altera PCIe controller driver:
- Do not dispose of the parent IRQ mapping, which belongs to the
parent interrupt controller (Mahesh Vaidya)
- Fix chained IRQ handler ordering issue and resource leaks on probe
failure (Mahesh Vaidya)
AMD MDB PCIe controller driver:
- Assert PERST# on shutdown so any connected Endpoints are held in
reset during shutdown (Sai Krishna Musham)
Amlogic Meson PCIe controller driver:
- Propagate devm_add_action_or_reset() failure to fix probe error
path (Shuvam Pandey)
- Add .remove() callback to deinitialize the host bridge and power
off the PHY (Shuvam Pandey)
Broadcom iProc PCIe controller driver:
- Restore .map_irq() assignment; its removal broke INTx on the iproc
platform bus driver (Mark Tomlinson)
Broadcom STB PCIe controller driver:
- No change, but products using certain WiFi devices may be affected
by removal of CONFIG_PCIE_BUS_* (see above)
Freescale i.MX6 PCIe controller driver:
- Move IMX6SX_GPR12_PCIE_TEST_POWERDOWN handling into the core reset
functions (Richard Zhu)
- Assert PERST# before enabling regulators to ensure that even if
power is enabled, endpoint stays inactive until REFCLK is stable
(Sherry Sun)
- Parse reset properties in Root Port nodes (falling back to host
bridge) to help support Key E connectors and the pwrctrl framework
(Sherry Sun)
- Configure i.MX95 REF_USE_PAD before PHY reset (Richard Zhu)
- Assert i.MX95 ref_clk_en after reference clock stabilizes (Richard
Zhu)
- Integrate new pwrctrl API for DTs with Root Port-level power
supplies (Sherry Sun)
Intel Gateway PCIe controller driver:
- Enable clock before PHY init for correct ordering (Florian Eckert)
- Add .start_link() callback so the driver works again (Florian
Eckert)
- Stop overwriting the ATU base address discovered by
dw_pcie_get_resources() (Florian Eckert)
- Add DT 'atu' region since this is hardware-specific, and fall back
to driver default if lacking (Florian Eckert)
Loongson PCIe controller driver:
- Ignore downstream devices only on internal bridges to avoid
Loongson hardware issue (Rong Zhang)
- Quirk old Loongson-3C6000 bridges that advertise incorrect
supported link speeds (Ziyao Li)
Marvell MVEBU PCIe controller driver:
- Use fixed-width interrupt masks to avoid truncation in 64-bit
builds (Rosen Penev)
MediaTek PCIe controller driver:
- Use FIELD_PREP() to fix incorrect operator precedence in
PCIE_FTS_NUM_L0 (Li RongQing)
- Fix IRQ domain leak when port fails to enable (Manivannan
Sadhasivam)
- Use actual physical address for MSI message address instead of
virt_to_phys() (Manivannan Sadhasivam)
- Add EcoNet EN7528 to DT binding (Caleb James DeLisle)
MediaTek PCIe Gen3 controller driver:
- Deassert PCIE_PHY_RSTB so REFCLK is stable for at least 100ms
(PCIE_T_PVPERL_MS) before deasserting PERST# (Jian Yang)
- Add .shutdown() to assert PERST# before powering down device (Jian
Yang)
- Do full device power down on removal, including asserting PERST#,
when removing driver (Chen-Yu Tsai)
- Fix a 'failed to create pwrctrl devices' error message that was
inadvertently skipped (Chen-Yu Tsai)
NVIDIA Tegra194 PCIe controller driver:
- Program the DesignWare PORT_AFR L1 entrance latency based on the
'aspm-l1-entry-delay-ns' DT property (Manikanta Maddireddy)
Qualcomm PCIe controller driver:
- Add Eliza SoC compatible in DT binding (Krishna Chaitanya Chundru)
- Set max OPP during resume so DBI register accesses don't fail with
NoC errors (Qiang Yu)
- Add pci_host_common_d3cold_possible() to determine whether
downstream devices are already in D3hot and wakeup-enabled devices
are capable of generating PME from D3cold (Krishna Chaitanya
Chundru)
- Add .get_ltssm() callback to get the LTSSM status without DBI,
since DBI may be inaccessible after PME_Turn_Off (Krishna Chaitanya
Chundru)
- Power down PHY via PARF_PHY_CTRL before disabling rails/clocks to
avoid power leakage (Krishna Chaitanya Chundru)
- Decide whether suspend should put the link in L2 and power down
using pci_host_common_d3cold_possible() instead of checking whether
ASPM L1 is enabled (Krishna Chaitanya Chundru)
- Add qcom D3cold support to tear down interconnect bandwidth and OPP
votes (Krishna Chaitanya Chundru)
- Handle unsupported mixed PERST#/PHY DT configurations, e.g., PHY in
RP node while PERST# is in the RC node, but warn about the DT issue
(Qiang Yu)
- Program T_POWER_ON based on DT 't-power-on-us' property in case
hardware advertises incorrect values (Krishna Chaitanya Chundru)
- Disable ASPM L0s for SA8775P (Shawn Guo)
- Initialize DWC MSI lock for firmware-managed ECAM hosts, which
don't use the dw_pcie_host_init() path that initializes the lock
(Yadu M G)
Renesas RZ/G3S PCIe controller driver:
- Add RZ/V2N DT support (Lad Prabhakar)
SOPHGO PCIe controller driver:
- Add 'dma-coherent' DT property for sg2042-pcie driver (Han Gao)
Synopsys DesignWare PCIe controller driver:
- Apply ECRC TLP Digest workaround for all DesignWare cores prior to
5.10a, not just 4.90a and 5.00a (Manikanta Maddireddy)
- Use common struct dw_pcie 'mode' rather than duplicating it in
artpec6, dra7xx, dwc-pcie, and keembay driver structs (Hans Zhang)
- Use DEFINE_SHOW_ATTRIBUTE for ltssm_status debugfs to reduce
boilerplate and fix a seq_file memory leak by including a
.release() callback (Hans Zhang)
- Fix a signedness bug in fault injection test code (Dan Carpenter)
- Avoid NULL pointer dereference when tearing down debugfs for
controller that lacks RAS DES capability (Shuvam Pandey)
MicroSemi Switchtec management driver:
- Add Gen6 Device IDs (Ben Reed)
Miscellaneous:
- Remove unused gpio.h include from amd-mdb, designware-plat, fu740,
visconti drivers (Andy Shevchenko)
- Fix typos in documentation (josh ziegler)
- Use FIELD_MODIFY() instead of open-coding it (Hans Zhang)"
* tag 'pci-v7.2-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/pci/pci: (168 commits)
PCI/sysfs: Use kstrtobool() to parse the ROM attribute input
PCI/sysfs: Limit BAR resize attribute scope to platforms with PCI mmap
PCI/sysfs: Remove pci_create_legacy_files() and pci_sysfs_init()
PCI/sysfs: Convert legacy I/O and memory attributes to static definitions
PCI/sysfs: Add __weak pci_legacy_has_sparse() helper
alpha/PCI: Compute legacy size in pci_mmap_legacy_page_range()
PCI: Add macros for legacy I/O and memory address space sizes
PCI/sysfs: Remove pci_{create,remove}_sysfs_dev_files()
alpha/PCI: Convert resource files to static attributes
alpha/PCI: Add static PCI resource attribute macros
alpha/PCI: Remove WARN from __pci_mmap_fits() and __legacy_mmap_fits()
alpha/PCI: Fix __pci_mmap_fits() overflow for zero-length BARs
alpha/PCI: Use PCI resource accessor macros
alpha/PCI: Use BAR index in sysfs attr->private instead of resource pointer
alpha/PCI: Add security_locked_down() check to pci_mmap_resource()
PCI/sysfs: Limit pci_sysfs_init() late_initcall compile scope
PCI/sysfs: Add stubs for pci_{create,remove}_sysfs_dev_files()
PCI/sysfs: Warn about BAR resize failure in __resource_resize_store()
PCI/sysfs: Convert PCI resource files to static attributes
PCI/proc: Fix race between pci_proc_init() and pci_bus_add_device()
...
Everything:
Total patches: 108
Reviews/patch: 0.84
Reviewed rate: 75%
Patch series in this merge:
- The 2 patch series "taskstats: fix TGID dead-thread stat retention"
from Yiyang Chen fixes a taskstats TGID aggregation bug where fields
added in the TGID query path were not preserved after thread exit, and
adds a kselftest covering the regression.
- The 2 patch series "lib/tests: string_helpers: Slight improvements"
from Andy Shevchenko improves lib/tests/string_helpers_kunit.c a little.
- The 2 patch series "lib/base64: decode fixes" from Josh Law addreesses
minor issues in lib/base64.c.
- The 3 patch series "selftests/filelock: Make output more kselftestish"
from Mark Brown makes the output from the ofdlocks test a bit easier for
tooling to work with, and also ignores the generated file.
- The 3 patch series "uaccess: unify inline vs outline
copy_{from,to}_user() selection" from Yury Norov simplifies the usercopy
code by removing the selectability of inlining copy_{from,to}_user().
- The 5 patch series "ocfs2: validate inline xattr header consumers"
from ZhengYuan Huang fixes a number of possible issues in the ocfs2
xattr code.
- The 8 patch series "lib and lib/cmdline enhancements" from Dmitry
Antipov provides additional robustness checking in the cmdline handling
code and its in-kernel testing and selftests.
- The 18 patch series "cleanup the RAID6 P/Q library" from Christoph
Hellwig cleans up the RAID6 P/Q library to match the recent updates to
the RAID 5 XOR library and other CRC/crypto libraries.
- The 3 patch series "ocfs2: harden inode validators against forged
metadata" from Michael Bommarito adds three structural checks to OCFS2
dinode validation so malformed on-disk fields are rejected before
ocfs2_populate_inode() copies them into the in-core inode.
- The 2 patch series "lib/raid: replace __get_free_pages() call with
kmalloc()" from Mike Rapoport cleans up the lib/raid code by using
kmalloc() in more places.
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCajgeIQAKCRDdBJ7gKXxA
jmlWAQCLJVDZNJMFaXy4a+YHdu3tfemLpSy83A0Le61tOZUdBQD/Sf/7rhgeaM74
32yp53TZLA8xHImCGEin/1ddPJ8DbgY=
=GW2I
-----END PGP SIGNATURE-----
Merge tag 'mm-nonmm-stable-2026-06-21-10-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull non-MM updates from Andrew Morton:
- "taskstats: fix TGID dead-thread stat retention" (Yiyang Chen)
Fix a taskstats TGID aggregation bug where fields added in the TGID
query path were not preserved after thread exit, and adds a kselftest
covering the regression.
- "lib/tests: string_helpers: Slight improvements" (Andy Shevchenko)
Improve lib/tests/string_helpers_kunit.c a little
- "lib/base64: decode fixes" (Josh Law)
Address minor issues in lib/base64.c
- "selftests/filelock: Make output more kselftestish" (Mark Brown)
Make the output from the ofdlocks test a bit easier for tooling to
work with. Also ignore the generated file
- "uaccess: unify inline vs outline copy_{from,to}_user() selection"
(Yury Norov)
Simplify the usercopy code by removing the selectability of inlining
copy_{from,to}_user().
- "ocfs2: validate inline xattr header consumers" (ZhengYuan Huang)
Fix a number of possible issues in the ocfs2 xattr code
- "lib and lib/cmdline enhancements" (Dmitry Antipov)
Provide additional robustness checking in the cmdline handling code
and its in-kernel testing and selftests
- "cleanup the RAID6 P/Q library" (Christoph Hellwig)
Clean up the RAID6 P/Q library to match the recent updates to the
RAID 5 XOR library and other CRC/crypto libraries
- "ocfs2: harden inode validators against forged metadata" (Michael
Bommarito)
Add three structural checks to OCFS2 dinode validation so malformed
on-disk fields are rejected before ocfs2_populate_inode() copies them
into the in-core inode
- "lib/raid: replace __get_free_pages() call with kmalloc()" (Mike
Rapoport)
Clean up the lib/raid code by using kmalloc() in more places
* tag 'mm-nonmm-stable-2026-06-21-10-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (108 commits)
ocfs2: fix circular locking dependency in ocfs2_dio_end_io_write
ocfs2: fix NULL h_transaction deref in ocfs2_assure_trans_credits
lib: interval_tree_test: validate benchmark parameters
ocfs2: avoid moving extents to occupied clusters
treewide: fix transposed "sign" typos and update spelling.txt
ocfs2: fix UBSAN array-index-out-of-bounds in ocfs2_sum_rightmost_rec
fat: reject BPB volumes whose data area starts beyond total sectors
selftests/uevent: increase __UEVENT_BUFFER_SIZE to avoid ENOBUFS on busy systems
lib/test_firmware: allocate the configured into_buf size
fs: efs: remove unneeded debug prints
checkpatch: cuppress warnings when Reported-by: is followed by Link:
MAINTAINERS: add Alexander as a kcov reviewer
mailmap: update Alexander Sverdlin's Email addresses
fs: fat: inode: replace sprintf() with scnprintf()
ocfs2: fix out-of-bounds write in ocfs2_remove_refcount_extent
ocfs2: fix race between ocfs2_control_install_private() and ocfs2_control_release()
ocfs2/dlm: require a ref for locking_state debugfs open
ocfs2: reject FITRIM ranges shorter than a cluster
ocfs2: validate fast symlink target during inode read
ocfs2: add journal NULL check in ocfs2_checkpoint_inode()
...
Various names for Qualcomm as a company are used in user-visible config
options: QCOM, Qualcomm and Qualcomm Technologies. Switch to unified
"Qualcomm" so it will be easier for users to identify the options when
for example running menuconfig.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Link: https://patch.msgid.link/20260423173602.92503-2-krzysztof.kozlowski@oss.qualcomm.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Replace sprintf() and strlen() patterns in sysfs show callbacks
with sysfs_emit().
sysfs_emit() is the preferred helper for formatting sysfs output
and simplifies the implementation.
Signed-off-by: Hungyu Lin <dennylin0707@gmail.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Link: https://patch.msgid.link/20260607163119.78717-1-dennylin0707@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The driver only had runtime PM callbacks. If a channel stayed allocated
across system suspend/resume, the runtime usage count could remain
non-zero while hardware state (DMAC_CFG, clocks) was lost, and
axi_dma_runtime_resume() would not run to restore it.
Add system-sleep PM ops that use pm_runtime_force_suspend() and
pm_runtime_force_resume() so suspend/resume reuses the existing
axi_dma_suspend() and axi_dma_resume() paths.
Replace pm_runtime_get() with pm_runtime_resume_and_get() in
dma_chan_alloc_chan_resources() so clocks are enabled before a client
can immediately submit a transfer and touch MMIO.
Signed-off-by: Tze Yee Ng <tze.yee.ng@altera.com>
Link: https://patch.msgid.link/18bf778a3a1cc2f377ef8eb0d1508d8ac6371896.1779688569.git.tze.yee.ng@altera.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
axi_chan_block_xfer_start() runs after the controller is already enabled,
so calling axi_dma_enable() again is unnecessary. Remove the redundant
enable call to keep the transfer start path clean and avoid repeated no-op
programming.
Signed-off-by: Niravkumar L Rabara <niravkumarlaxmidas.rabara@altera.com>
Signed-off-by: Tze Yee Ng <tze.yee.ng@altera.com>
Link: https://patch.msgid.link/060733464e19298f670cd269d4849f2092644923.1779688569.git.tze.yee.ng@altera.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The descriptor FIFO requires that all words of a descriptor are written
in order, with the control word written last to flush it into the DMA
engine. Using memcpy() with __force to __iomem is not the correct API
and does not guarantee appropriate MMIO access on all architectures.
Replace the descriptor body copy with memcpy_toio(), using
offsetof(struct msgdma_extended_desc, control) to exclude the control
word. This matches the previous sizeof(desc->hw_desc) - sizeof(u32)
length only when control is the last struct member; add a static_assert
to enforce that layout so a future field after control cannot silently
break FIFO ordering.
Keep writing the control word separately with write barriers, so it
remains the final word pushed into the FIFO.
Signed-off-by: Adrian Ng Ho Yin <adrianhoyin.ng@altera.com>
Signed-off-by: Tze Yee Ng <tze.yee.ng@altera.com>
Link: https://patch.msgid.link/f6f3b4a2e2eb0eb1a51976de3f5d1ef5bab9bd76.1779697226.git.tze.yee.ng@altera.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The first device on a PCI root bus determines whether the host bridge is
whitelisted for P2PDMA. All Intel Xeon chips since Ice Lake (ICX, 2021)
expose a device with ID 0x09a2 as first device. It is loosely associated
with the IOMMU. All these Xeon chips support P2PDMA, so since the addition
of the device with commit feaea1fe8b ("PCI/P2PDMA: Add Intel 3rd Gen
Intel Xeon Scalable Processors to whitelist"), P2PDMA has been allowed on
all new Xeons without the need to amend the whitelist:
Xeons with Performance Cores:
Sapphire Rapids (SPR, 2023)
Emerald Rapids (EMR, 2023)
Granite Rapids (GNR, 2024)
Diamond Rapids (DMR, 2026)
Xeons with Efficiency Cores:
Sierra Forest (SRF, 2024)
Clearwater Forest (CWF, 2026)
However these Xeons also expose accelerators as first device on a root bus
of its own:
QuickAssist Technology (QAT, crypto & compression accelerator)
Data Streaming Accelerator (DSA, dma engine)
In-Memory Analytics Accelerator (IAA, compression accelerator)
Whitelist them for P2PDMA as well. Move their Device ID macros from the
accelerator drivers to <linux/pci_ids.h> for reuse by P2PDMA code.
Unfortunately the Device IDs vary across Xeon generations as additional
features were added to the accelerators. This currently necessitates an
amendment for each new Xeon chip.
For future chips, this need shall be avoided by an ongoing effort to extend
ACPI HMAT with PCIe P2PDMA characteristics (latency, bandwidth, ordering
constraints). The PCI core will be able look up in this BIOS-provided ACPI
table whether P2PDMA is supported, instead of relying on a whitelist that
needs to be amended continuously.
Signed-off-by: Lukas Wunner <lukas@wunner.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com> # QAT
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/6aac4922b5fe7070b11874427a9285e42ddd05a4.1780585518.git.lukas@wunner.de
Currently, the Tegra GPC DMA hardware requires the transfer length to
be a multiple of the max burst size configured for the channel. When a
client requests a transfer where the length is not evenly divisible by
the configured max burst size, the DMA hangs with partial burst at
the end.
Fix this by reducing the burst size to the largest power-of-2 value
that evenly divides the transfer length. For example, a 40-byte
transfer with a 16-byte max burst will now use an 8-byte burst
(40 / 8 = 5 complete bursts) instead of causing a hang.
This issue was observed with the PL011 UART driver where TX DMA
transfers of arbitrary lengths were stuck.
Fixes: ee17028009 ("dmaengine: tegra: Add tegra gpcdma driver")
Cc: stable@vger.kernel.org
Signed-off-by: Kartik Rajput <kkartik@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260422064134.1323610-1-kkartik@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
For architectures like Microblaze or arm64 (where this IP is used),
DMA_DIRECT_REMAP is set which means that dma_alloc_coherent() might
remap (and hence vmalloc()) some memory. This became visible in a design
where dma_direct_use_pool() is not possible.
With the above, when calling dma_free_coherent(), vunmap() would be
called from softirq context and thus leading to a BUG().
To fix it, use a dma pool that is allocated in
.device_alloc_chan_resources() and allocate blocks from it. The key
point is that now dma_pool_free() is used in axi_dmac_free_desc() to
free the blocks and that just frees the blocks from the pool in the
sense they can be used again. In other words, no actual call to
dma_free_coherent() happens. That only happens when destroying the pool
in axi_dmac_free_chan_resources() which does not happen in any interrupt
context.
Fixes: 3f8fd25936 ("dmaengine: axi-dmac: Allocate hardware descriptors")
Signed-off-by: Nuno Sá <nuno.sa@analog.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260424-dma-dmac-handle-vunmap-v4-4-90f43412fdc0@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
There's no reason to keep struct clk in struct axi_dmac. Hence, use a
local clk variable in .probe() and drop it from struct axi_dmac.
Signed-off-by: Nuno Sá <nuno.sa@analog.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260424-dma-dmac-handle-vunmap-v4-3-90f43412fdc0@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use axi_dmac_free_desc() to free fully the descriptor at fail path when
call axi_dmac_alloc_desc() in axi_dmac_prep_peripheral_dma_vec().
Fixes: 74609e5686 ("dmaengine: dma-axi-dmac: Implement device_prep_peripheral_dma_vec")
Signed-off-by: Nuno Sá <nuno.sa@analog.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260424-dma-dmac-handle-vunmap-v4-2-90f43412fdc0@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
In dma_release_channel(), check chan->device->privatecnt after call
dma_chan_put(). However, dma_chan_put() call dma_device_put() which could
release the last reference of the device if the DMA provider is already
gone and hence free it.
Fixes it by moving dma_chan_put() after the check.
Fixes: 0f571515c3 ("dmaengine: Add privatecnt to revert DMA_PRIVATE property")
Signed-off-by: Nuno Sá <nuno.sa@analog.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260424-dma-dmac-handle-vunmap-v4-1-90f43412fdc0@analog.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The DONE_INT_MASK and ABORT_INT_MASK registers are shared by all DMA
channels, and modifying them requires a read-modify-write sequence.
Because this operation is not atomic, concurrent calls to
dw_edma_v0_core_start() can introduce race conditions if two channels
update these registers simultaneously.
Add a spinlock to serialize access to these registers and prevent race
conditions.
Fixes: 7e4b8a4fbe ("dmaengine: Add Synopsys eDMA IP version 0 support")
Cc: stable@vger.kernel.org
Signed-off-by: Frank Li <Frank.Li@nxp.com>
[den: update dw_edma.lock comment]
Link: https://lore.kernel.org/dmaengine/20260109-edma_ll-v2-1-5c0b27b2c664@nxp.com/
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Link: https://patch.msgid.link/20260521142153.2957432-5-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dw_edma_pcie_probe() treats the PCI device ID driver_data as the
template for the controller layout and copies it unconditionally. A
device bound dynamically via sysfs can match the driver without that
data, which leads to a NULL pointer dereference.
Reject such matches before enabling the device.
Fixes: 41aaff2a2a ("dmaengine: Add Synopsys eDMA IP PCIe glue-logic")
Cc: stable@vger.kernel.org
Signed-off-by: Koichiro Den <den@valinux.co.jp>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260521142153.2957432-3-den@valinux.co.jp
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Some peripherals on RZ/G3E SoCs (SSIU, SPDIF, SCU/SRC, DVC, PFC) require
explicit ACK signal routing through the ICU for level-based DMA handshaking.
Rather than extending the DT binding with an optional second #dma-cells
(which would require all DMA consumers to supply two cells even when ACK
routing is not needed), derive the ACK signal number directly from the
MID/RID request number using the linear mapping defined in RZ/G3E hardware
manual Table 4.6-28:
PFC external DMA pins (DREQ0..DREQ4):
req_no 0x000-0x004 -> ACK No. 84-88
SSIU BUSIFs (ssip00..ssip93):
req_no 0x161-0x198 -> ACK No. 28-83
SPDIF (CH0..CH2) + SCU SRC (sr0..sr9) + DVC (cmd0..cmd1):
req_no 0x199-0x1b4 -> ACK No. 0-27
ACK routing is programmed when a channel is prepared for transfer and
cleared when the channel is released or the transfer times out, following
the same pattern as MID/RID request routing.
Signed-off-by: John Madieu <john.madieu.xa@bp.renesas.com>
Link: https://patch.msgid.link/20260525110750.4020112-3-john.madieu.xa@bp.renesas.com
[fixes subsystem name tag]
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Function dw_edma_add_irq_mask() sets the mask of the
interrupts alloted to read / write channels in a variable.
The mask set for read / write channels is niether used nor
this function is called else where, making it redundant.
The redundant function can be removed safely as it is
not affecting anything.
Signed-off-by: Devendra K Verma <devendra.verma@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260526053111.3244488-1-devverma@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The dma-nbpf driver only probes devices from device tree and fails to
probe devices relying on the traditional platform device probe path. So
the platform_device_id array is unused apart from providing misleading
module meta data.
Drop it.
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
Link: https://patch.msgid.link/5f7380828873e2375e319ef091178d11a277a0ac.1779965563.git.u.kleine-koenig@baylibre.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Since commit 2e7f55ce43 ("dmaengine: cirrus: Convert to DT for Cirrus
EP93xx") the driver cannot probe devices using the traditional platform
device way any more. Thus the driver's .id_table serves no purpose any
more and can be dropped.
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
Link: https://patch.msgid.link/c3830cb95b0bb939f9cc9543dfa3047e41532c47.1779976024.git.ukleinek@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Combining the struct definition with its variable initializer confuses the
kernel-doc parser because __MUTEX_INITIALIZER() expands to contain braces,
breaking brace counting and causing:
Warning: drivers/dma/dmatest.c:152 struct member '' not described in 'dmatest_info'
Split into separate struct definition and variable declaration, which is
the standard kernel pattern.
Assisted-by: Opencode:Big-pickle
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Link: https://patch.msgid.link/20260530200322.7584-1-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The GPI DMA controller is only responsible for QUP peripherals, and
cannot work as a general-purpose DMA accelerator.
Set DMA_PRIVATE capability for it.
This fixes error messages about GPI being shown when an async-tx
consumer is loaded.
Fixes: 5d0c3533a1 ("dmaengine: qcom: Add GPI dma driver")
Signed-off-by: Icenowy Zheng <zhengxingda@iscas.ac.cn>
Reviewed-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
Link: https://patch.msgid.link/20260602070344.3707256-1-zhengxingda@iscas.ac.cn
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Convert the separately-offset phy_chans pointer to a C99 flexible array
member at the end of struct d40_base, and switch the allocation to
struct_size(). The log_chans and memcpy_chans slots continue to live
in the same allocation immediately after phy_chans, indexed via
base->log_chans. This removes the hand-rolled pointer fixup that
recomputed phy_chans from base + ALIGN(sizeof(struct d40_base), 4).
The ALIGN(sizeof(struct d40_base), 4) requirement is met implicitly by the
C compiler when using a flexible array member. With struct d40_chan
phy_chans[] as the last member, the C standard guarantees
sizeof(struct d40_base) includes trailing padding to satisfy the alignment
of the flexible array element type (struct d40_chan). Since struct d40_chan
contains members like spinlock_t, pointers, and struct dma_chan — all with
alignment ≥ 4 — the compiler ensures sizeof(struct d40_base) is already a
multiple of _Alignof(struct d40_chan) >= 4. The struct_size() macro then
computes sizeof(struct d40_base) + sizeof(struct d40_chan) * num_phy_chans,
so phy_chans[0] lands at a properly aligned offset without needing the manual
ALIGN.
Assisted-by: Claude:Opus-4.7
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Linus Walleij <linusw@kernel.org>
Link: https://patch.msgid.link/20260531020843.594892-1-rosenp@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add dev_err/dev_err_probe logging across failure paths to improve
debuggability of DMA errors during runtime and probe.
Use return dev_err_probe() pattern where no cleanup is required in the
probe function. On error paths that need explicit unwind, store the
dev_err_probe() return value in ret before jumping to the cleanup label.
Also convert existing dev_err calls in probe to dev_err_probe for
consistency, and use dev_err in non-probe functions.
Keep explicit runtime PM and DMA registration unwind instead of managed or
scoped cleanup. The scoped runtime PM guard releases the usage count with
pm_runtime_put(), while this probe error path needs pm_runtime_put_sync()
before pm_runtime_disable(). The OF DMA registration failure path also
needs to unregister the DMA engine before dropping the runtime PM reference.
Signed-off-by: Sheetal <sheetal@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260517163045.363444-1-sheetal@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add Device ID for AMD (Xilinx) CPM6 DMA IP. This IP enables
64 Read and 64 Write Channels.
Adding the relevant dw_edma_pcie_data to use 8 Read and 8 Write
channels for initial commit.
Signed-off-by: Devendra K Verma <devendra.verma@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260605112829.679697-1-devendra.verma@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
On an RZ/G2L-based system, it has been observed that when the DMA channels
for all enabled IPs are active (TX and RX for one serial IP, TX and RX for
one audio IP, and TX and RX for one SPI IP), shortly after all of them are
started, the system can become irrecoverably blocked. In one debug session
the system did not block, and the DMA HW registers were inspected. It was
found that the DER (Descriptor Error) bit in the CHSTAT register for one of
the SPI DMA channels was set.
According to the RZ/G2L HW Manual, Rev. 1.30, chapter 14.4.7 Channel
Status Register n/nS (CHSTAT_n/nS), description of the DER bit, the DER
bit is set when the LV (Link Valid) value loaded with a descriptor in link
mode is 0. This means that the DMA engine has loaded an invalid
descriptor (as defined in Table 14.14, Header Area, of the same manual).
The same chapter states that when a descriptor error occurs, the transfer
is stopped, but no DMA error interrupt is generated.
Set the LE bit on the last descriptor of a transfer. This informs the DMA
engine that this is the final descriptor for the transfer.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-19-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The Renesas RZ/G3S SoC supports a power saving mode in which power to most
of the SoC components is turned off, including the DMA IP. Add suspend to
RAM support to save and restore the DMA IP registers.
Cyclic DMA channels require special handling. Since they can be paused and
resumed during system suspend/resume, the driver restores additional
registers for these channels during the system resume phase. If a channel
was not explicitly paused during suspend, the driver ensures that it is
paused and resumed as part of the system suspend/resume flow.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-16-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Protect the driver exposed APIs with runtime PM suspend/resume calls
before accessing HW registers. As the current driver leaves runtime PM
enabled in probe, the purpose of the changes in this patch is to avoid
accessing HW registers after a failed system suspend leaves the runtime
PM state of the device improperly reinitialized.
In that case, the driver remains bound to the device, the APIs are still
exposed, and any access to HW registers without runtime resuming the
device may lead to synchronous aborts.
To avoid leaking resources in case of runtime PM failures, save the error
code returned by PM_RUNTIME_ACQUIRE_ERR() in rz_dmac_terminate_all() and
return it only at the end of the function to allow the cleanup code to
run. A similar approach is used in rz_dmac_free_chan_resources().
Because some exposed APIs (e.g. ->device_terminate_all()) may be called
from atomic context according to the documentation, mark the DMA device as
pm_runtime_irq_safe().
This patch prepares the driver for suspend-to-RAM support.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-15-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Adjust rz_dmac_chan_get_residue() to return error codes on failure and
provide the residue to callers through the residue parameter. This
prepares the code for the addition of runtime PM support.
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-14-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add cyclic DMA support to the RZ DMAC driver. A per-channel status bit is
introduced to mark cyclic channels and is set during the DMA prepare
callback. The IRQ handler checks this status bit and calls
vchan_cyclic_callback() accordingly.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-13-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The CHCTRL_SETEN bit is explicitly set in rz_dmac_enable_hw(). Updating
struct rz_dmac_chan::chctrl with this bit in
rz_dmac_prepare_desc_for_memcpy() and rz_dmac_prepare_descs_for_slave_sg()
is unnecessary in the current code base. Moreover, it conflicts with the
configuration sequence that will be used for cyclic DMA channels during
suspend to RAM. Cyclic DMA support will be introduced in subsequent
commits.
This is a preparatory commit for cyclic DMA suspend to RAM support.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-12-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Subsequent patches will add suspend/resume and cyclic DMA support to the
rz-dmac driver. This support needs to work on SoCs where power to most
components (including DMA) is turned off during system suspend. For this,
some channels (for example cyclic ones) may need to be paused and resumed
manually by the DMA driver during system suspend/resume.
Refactor the pause/resume support so the same code can be reused in the
system suspend/resume path.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-11-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The driver used a mix of virt-dma APIs and driver specific logic to
process descriptors. It maintained three internal queues: ld_free,
ld_queue, and ld_active as follows:
- ld_free: stores the descriptors pre-allocated at probe time
- ld_queue: stores descriptors after they are taken from ld_free and
prepared. At the same time, vchan_tx_prep() queues them to
vc->desc_allocated. The vc->desc_allocated list is then checked in
rz_dmac_issue_pending() and rz_dmac_irq_handler_thread() before
starting a new transfer via rz_dmac_xfer_desc(). In turn,
rz_dmac_xfer_desc() grabs the next descriptor from vc->desc_issued and
submits it for transfer
- ld_active: stores the descriptors currently being transferred
The interrupt handler moved a completed descriptor to ld_free before
invoking its completion callback. Once returned to ld_free, the
descriptor can be reused to prepare a new transfer. In theory, this
means the descriptor could be re-prepared before its completion
callback is called.
Commit fully back the driver by the virt-dma APIs. With this, only ld_free
need to be kept to track how many free descriptors are available. This
is now done as follows:
- the prepare stage removes the first descriptor from the ld_free and
prepares it
- the completion calls for it vc->desc_free() (rz_dmac_virt_desc_free())
which re-adds the descriptor at the end of ld_free
With this, the critical areas in prepare callbacks were minimized to only
getting the descriptor from the ld_free list.
Introduce struct rz_dmac_chan::desc to keep track of the currently
transferred descriptor. It is cleared in rz_dmac_terminate_all(),
referenced from rz_dmac_issue_pending() to determine whether a new transfer
can be started, and from rz_dmac_irq_handler_thread() once a descriptor has
completed. Finally, the rz_dmac_device_synchronize() was updated with
vchan_synchronize() call to ensure the terminated descriptor is freed and
the tasklet is killed.
With this, residue computation is also simplified, as it can now be
handled entirely through the virt-dma APIs.
The spin_lock/unlock operations from rz_dmac_irq_handler_thread() were
replaced by guard as the final code after rework is simpler this way.
As subsequent commits will set the Link End bit on the last descriptor
of a transfer, rz_dmac_enable_hw() is also adjusted as part of the full
conversion to virt-dma APIs. It no longer checks the channel enable
status itself; instead, its callers verify whether the channel is
enabled and whether the previous transfer has completed before starting
a new one.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-10-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add the rz_dmac_chan_is_paused() helper to check if the channel is paused.
This helper will be reused in subsequent patches.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-9-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add the rz_dmac_chan_is_enabled() helper to check if a channel is
enabled. This helper will be reused in subsequent patches.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-8-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Save the start LM descriptor to avoid starting from the beginning of the
channel's LM descriptor list in rz_dmac_calculate_residue_bytes_in_vd().
This avoids unnecessary iterations.
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-7-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add a the rz_dmac_lmdesc_addr() helper function to compute the lmdesc
address, to make the code easier to understand. The helper will be used in
subsequent patches.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-6-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use rz_dmac_disable_hw() instead of open coding it. This unifies the
code and prepares it for the addition of suspend to RAM and cyclic DMA.
The rz_dmac_disable_hw() from rz_dmac_chan_probe() was moved after
vchan_init() as it initializes the channel->vc.chan.device used in
rz_dmac_disable_hw().
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-5-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use list_first_entry_or_null() instead of open-coding it with a
list_empty() check and list_first_entry(). This simplifies the code.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-4-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
list_first_entry() does not return NULL when the list is empty,
making the existing NULL check invalid. Use list_first_entry_or_null()
instead.
Fixes: 21323b118c ("dmaengine: sh: rz-dmac: Add device_tx_status() callback")
Cc: stable@vger.kernel.org
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-3-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Once the interrupt is requested, the interrupt handler may run immediately.
Since the IRQ handler can access channel->ch_base, which is initialized
only after requesting the IRQ, this may lead to invalid memory access.
Likewise, the IRQ thread may access uninitialized data (the ld_free,
ld_queue, and ld_active lists), which may also lead to issues.
Request the interrupts only after everything is set up. To keep the error
path simpler, use dmam_alloc_coherent() instead of dma_alloc_coherent().
Fixes: 5000d37042 ("dmaengine: sh: Add DMAC driver for RZ/G2L SoC")
Cc: stable@vger.kernel.org
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Tested-by: John Madieu <john.madieu.xa@bp.renesas.com>
Signed-off-by: Claudiu Beznea <claudiu.beznea.uj@bp.renesas.com>
Tested-by: Tommaso Merciai <tommaso.merciai.xr@bp.renesas.com>
Link: https://patch.msgid.link/20260526084710.3491480-2-claudiu.beznea@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Add compatible and chip data to support GPCDMA in Tegra264, which has
differences in register layout and address bits compared to previous
versions.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-10-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use 'iommu-map', when provided, to get the stream ID to be programmed
for each channel. Iterate over the channels registered and configure
each channel device separately using of_dma_configure_id() to allow
it to use a separate IOMMU domain for the transfer. However, do this
in a second loop since the first loop populates the DMA device channels
list and async_device_register() registers the channels. Both are
prerequisites for using the channel device in the next loop.
Channels will continue to use the same global stream ID if the
'iommu-map' property is not present in the device tree.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-9-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Switch to managed registration in probe. This simplifies the error
paths in the probe and also removes the requirement of the driver
remove function.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Suggested-by: Frank Li <frank.li@nxp.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-8-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Tegra264 supports address width of 41 bits. Unlike older SoCs which use
a common high_addr register for upper address bits, Tegra264 has separate
src_high and dst_high registers to accommodate this wider address space.
Add an addr_bits property to the device data structure to specify the
number of address bits supported on each device and use that to program
the appropriate registers.
Update the sg_req struct to remove the high_addr field and use
dma_addr_t for src and dst to store the complete addresses. Extract
the high address bits only when programming the registers.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-7-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Repurpose the struct tegra_dma_channel_regs to define offsets for all the
channel registers. Previously, the struct only held the register values
for each transfer and was wrapped within tegra_dma_sg_req. Move the
values directly into tegra_dma_sg_req and use channel_regs for
storing the register offsets. Update all register reads/writes to use
the struct channel_regs. This prepares for the register offset change
in Tegra264.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-6-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
On Tegra264, reset is not available for the driver to control as
this is handled by the boot firmware. Hence make the reset control
optional and update the error message to reflect the correct error.
Signed-off-by: Akhil R <akhilrajeev@nvidia.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Acked-by: Thierry Reding <treding@nvidia.com>
Reviewed-by: Jon Hunter <jonathanh@nvidia.com>
Link: https://patch.msgid.link/20260331102303.33181-5-akhilrajeev@nvidia.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
There are multi spba-busses for i.MX8M* platforms, if only search for
the first spba-bus in DT, the found spba-bus may not the real bus of
audio devices, which cause issue for sdma p2p case, as the sdma p2p
script presently does not deal with the transactions involving two devices
connected to the AIPS bus.
Search the SDMA parent node first, which should be the AIPS bus, then
search the child node whose compatible string is spba-bus under that AIPS
bus for the above multi spba-busses case.
Fixes: 8391ecf465 ("dmaengine: imx-sdma: Add device to device support")
Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260407032755.2758049-1-shengjiu.wang@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Split out two new headers from the public pq.h:
- lib/raid/raid6/algos.h contains the algorithm lists private to
lib/raid/raid6
- include/linux/raid/pq_tables.h contains the tables also used by
async_tx providers.
The public include/linux/pq.h is now limited to the public interface for
the consumers of the RAID6 PQ API.
[hch@lst.de: remove duplicate ccflags-y line]
Link: https://lore.kernel.org/20260527074539.2292913-2-hch@lst.de
Link: https://lore.kernel.org/20260518051804.462141-10-hch@lst.de
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Ard Biesheuvel <ardb@kernel.org> # kunit only on arm64
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: "Borislav Petkov (AMD)" <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Chris Mason <clm@fb.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: David Sterba <dsterba@suse.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Li Nan <linan122@huawei.com>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Song Liu <song@kernel.org>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: WANG Xuerui <kernel@xen0n.name>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
By convention MODULE_DEVICE_TABLE() immediately follows the ID table it
exports, because this is easier to read and verify. It also makes more
sense since #ifdef for ACPI or OF could hide both of them.
Most of the drivers already have this correctly placed, so adjust
the missing ones. No functional impact.
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Link: https://patch.msgid.link/20260514060525.9253-2-krzysztof.kozlowski@oss.qualcomm.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
SpacemiT K3 reuses most of the PDMA IP design found on K1, with one
difference being the extended DRCMR base address. Add "spacemit,k3-pdma"
compatible string and define a new mmp_pdma_ops for K3 PDMA.
Signed-off-by: Guodong Xu <guodong@riscstar.com>
Signed-off-by: Troy Mitchell <troy.mitchell@linux.spacemit.com>
Link: https://patch.msgid.link/20260518-k3-pdma-v6-3-67fdf319a8f8@linux.spacemit.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Refactor the DRCMR macro into a helper function mmp_pdma_get_drcmr()
to support variable extended DRCMR base addresses across different PDMA
implementations, such as SpacemiT K3.
Signed-off-by: Guodong Xu <guodong@riscstar.com>
Signed-off-by: Troy Mitchell <troy.mitchell@linux.spacemit.com>
Link: https://patch.msgid.link/20260518-k3-pdma-v6-2-67fdf319a8f8@linux.spacemit.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
In C, bitfields are not necessarily safe to modify from multiple
threads without locking. Switch "dma_coherent" over to the "flags"
field so modifications are safe.
Cc: Christoph Hellwig <hch@lst.de>
Reviewed-by: Rafael J. Wysocki (Intel) <rafael@kernel.org>
Reviewed-by: Danilo Krummrich <dakr@kernel.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Link: https://patch.msgid.link/20260406162231.v5.7.If839f6dde98979fce177f70c6c74689a1904ee76@changeid
[ Since all DEV_FLAG_DMA_COHERENT accessors are exposed unconditionally,
also drop the CONFIG guards around dev_assign_dma_coherent() in
device_initialize() to ensure a correct default value. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
AMD MDB IP supports Linked List (LL) mode as well as non-LL mode.
The current code does not have the mechanisms to enable the
DMA transactions using the non-LL mode. The following two cases
are added with this patch:
- For the AMD (Xilinx) only, when a valid physical base address of
the device side DDR is not configured, then the IP can still be
used in non-LL mode. For all the channels DMA transactions will
be using the non-LL mode only. This, the default non-LL mode,
is not applicable for Synopsys IP with the current code addition.
- If the default mode is LL-mode, for both AMD (Xilinx) and Synosys,
and if user wants to use non-LL mode then user can do so via
configuring the peripheral_config param of dma_slave_config.
Signed-off-by: Devendra K Verma <devendra.verma@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260318070403.1634706-3-devendra.verma@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
AMD MDB PCIe endpoint support. For AMD specific support
added the following
- AMD supported PCIe Device IDs and Vendor ID (Xilinx).
- AMD MDB specific driver data
- AMD MDB specific VSEC capability to retrieve the device DDR
base address.
Signed-off-by: Devendra K Verma <devendra.verma@amd.com>
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20260318070403.1634706-2-devendra.verma@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
There are a couple of spelling mistakes, one in a comment block and
one in a module description. Fix them.
Signed-off-by: Colin Ian King <colin.i.king@gmail.com>
Link: https://patch.msgid.link/20260317204631.120332-1-colin.i.king@gmail.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
This DMA controller appears in Loongson-2K0300 and Loongson-2K3000.
It is a chain multi-channel controller that enables data transfers from
memory to memory, device to memory, and memory to device, as well as
channel prioritization configurable through the channel configuration
registers.
In addition, there are slight differences between Loongson-2K0300 and
Loongson-2K3000, such as channel register offsets and the number of
channels.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Binbin Zhou <zhoubinbin@loongson.cn>
Reviewed-by: Huacai Chen <chenhuacai@loongson.cn>
Link: https://patch.msgid.link/73bc32ba6249f1eef94fec9b349bc9efa98278ea.1772853681.git.zhoubinbin@loongson.cn
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Use guard() and scoped_guard() infrastructure instead of explicitly
acquiring and releasing spinlocks to simplify the code and ensure that
all locks are released properly.
Reviewed-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Binbin Zhou <zhoubinbin@loongson.cn>
Reviewed-by: Huacai Chen <chenhuacai@loongson.cn>
Link: https://patch.msgid.link/fb59bb25e5c4fcb84d9aa7b351285fa8d02ea8cb.1772853681.git.zhoubinbin@loongson.cn
Signed-off-by: Vinod Koul <vkoul@kernel.org>
