| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The Synopsys PCIe Endpoint IP prototype kit can be attached via any PCI
host controller, including one where the PCI bus address space is different
from the CPU address space. Therefore, we need to make sure the source and
destination addresses of the DMA slave devices are converted to the PCI bus
address space; otherwise DMA transactions may cause memory corruption.
Add a new dw_edma_pcie_address() interface to perform this translation by
using pcibios_resource_to_bus().
Link: https://lore.kernel.org/r/[email protected]
Fixes: 41aaff2a2ac0 ("dmaengine: Add Synopsys eDMA IP PCIe glue-logic")
Tested-by: Manivannan Sadhasivam <[email protected]>
Signed-off-by: Serge Semin <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Bjorn Helgaas <[email protected]>
Reviewed-by: Manivannan Sadhasivam <[email protected]>
Acked-by: Vinod Koul <[email protected]>
|
|
Since 9575632052ba ("dmaengine: make slave address physical"), the source
and destination addresses of the DMA slave device have been converted to
physical addresses in the CPU address space. It's the DMA device driver's
responsibility to convert them to the DMA bus address space. In case of the
DW eDMA device, the source or destination peripheral (slave) devices reside
in PCI bus space. Thus we need to perform the PCI Host/Endpoint windows-
based (i.e. DT "ranges" property) address translation; otherwise the eDMA
transactions won't work as expected (or can be even harmful) if the CPU and
PCI address spaces don't match.
Note 1: Even though the DMA interleaved template has both source and
destination addresses declared as dma_addr_t, only the CPU memory range
should be mapped to be seen by the DMA device since it's a subject of the
DMA getting towards the system side. The device part must not be mapped
since the slave device resides in the PCI bus space, which isn't affected
by IOMMUs or iATU translations. DW PCIe eDMA generates corresponding
MWr/MRd TLPs on its own.
Note 2: This functionality is mainly required for the remote eDMA setup
since the CPU address must be manually translated into the PCI bus space
before being written to LLI.{SAR,DAR}. If eDMA is embedded in the locally
accessible DW PCIe Root Port/Endpoint, software-based translation isn't
required since hardware will translate it via the Outbound iATU as long as
the DMA_BYPASS flag is cleared. If DMA_BYPASS is set or there is no
Outbound iATU entry that contains the SAR or DAR (for Read and Write
channel respectively), there won't be any translation performed but DMA
will proceed with the corresponding source/destination address as-is.
Link: https://lore.kernel.org/r/[email protected]
Tested-by: Manivannan Sadhasivam <[email protected]>
Signed-off-by: Serge Semin <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Bjorn Helgaas <[email protected]>
Reviewed-by: Manivannan Sadhasivam <[email protected]>
Acked-by: Vinod Koul <[email protected]>
|
|
The interleaved DMA transfer support added by 85e7518f42c8 ("dmaengine:
dw-edma: Add device_prep_interleave_dma() support") seems contradictory to
what the DMA engine defines. The next conditional statements:
if (!xfer->xfer.il->numf)
return NULL;
if (xfer->xfer.il->numf > 0 && xfer->xfer.il->frame_size > 0)
return NULL;
mean that numf can't be zero and frame_size must always be zero, otherwise
the transfer won't be executed. Furthermore, the transfer execution method
takes the frame size from the dma_interleaved_template.sgl[] array for each
frame. That array in accordance with [1] is supposed to be of
dma_interleaved_template.frame_size size, which as we discovered before the
code expects to be zero. So judging by the dw_edma_device_transfer()
implementation, the method implies the dma_interleaved_template.sgl[] array
being of dma_interleaved_template.numf size, which is wrong. Since the
dw_edma_device_transfer() method doesn't permit
dma_interleaved_template.frame_size being non-zero, the multi-chunk
interleaved transfer turns to be unsupported even though the code implies
having it supported.
Add fully functioning support of interleaved DMA transfers.
First of all, dma_interleaved_template.frame_size is supposed to be greater
or equal to one thus having at least simple linear chunked frames.
Secondly, we can create a walk-through over all the chunks and frames by
initializing the number of the eDMA burst transactions as a multiple of
dma_interleaved_template.numf and dma_interleaved_template.frame_size and
getting the frame_size-modulo of the iteration step as an index of the
dma_interleaved_template.sgl[] array.
[1] include/linux/dmaengine.h: doc struct dma_interleaved_template
Link: https://lore.kernel.org/r/[email protected]
Fixes: 85e7518f42c8 ("dmaengine: dw-edma: Add device_prep_interleave_dma() support")
Tested-by: Manivannan Sadhasivam <[email protected]>
Signed-off-by: Serge Semin <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Bjorn Helgaas <[email protected]>
Reviewed-by: Manivannan Sadhasivam <[email protected]>
Acked-by: Vinod Koul <[email protected]>
|
|
The DW eDMA controller always increments both source and destination
addresses. Permitting DMA interleaved transfers with no src_inc/dst_inc
flags set may lead to unexpected behaviour for the device users.
Terminate interleaved transfers if at least one of the
dma_interleaved_template.{src_inc,dst_inc} flag is initialized to "false".
Note that in addition, we need to increase the source and destination
addresses after each iteration.
Link: https://lore.kernel.org/r/[email protected]
Fixes: 85e7518f42c8 ("dmaengine: dw-edma: Add device_prep_interleave_dma() support")
Tested-by: Manivannan Sadhasivam <[email protected]>
Signed-off-by: Serge Semin <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Bjorn Helgaas <[email protected]>
Reviewed-by: Manivannan Sadhasivam <[email protected]>
Acked-by: Vinod Koul <[email protected]>
|
|
Interleaved DMA transfer support was added by 85e7518f42c8 ("dmaengine:
dw-edma: Add device_prep_interleave_dma() support"), but depending on the
selected channel, either source or destination address are left
uninitialized which was obviously wrong.
Initialize the destination address of the eDMA burst descriptors for
DEV_TO_MEM interleaved operations and the source address for MEM_TO_DEV
operations.
Link: https://lore.kernel.org/r/[email protected]
Fixes: 85e7518f42c8 ("dmaengine: dw-edma: Add device_prep_interleave_dma() support")
Tested-by: Manivannan Sadhasivam <[email protected]>
Signed-off-by: Serge Semin <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Bjorn Helgaas <[email protected]>
Reviewed-by: Manivannan Sadhasivam <[email protected]>
Acked-by: Vinod Koul <[email protected]>
|
|
The dw_edma_region.paddr field should be a memory base address visible by
the DW eDMA controller. If the DMA engine is embedded in the DW PCIe
Host/Endpoint controller, the address should belong to the Local CPU/
Application memory. If eDMA is remotely accessible across the PCI bus via
PCI memory IOs, the address should be part of the PCI bus memory space.
The latter case hasn't been well covered in the corresponding glue-driver.
Since pci_dev.resource[] contains resources defined in the CPU memory
space, they need to be converted to the PCI bus address space. Convert the
LL, DT and CSRs PCI memory ranges with pci_bus_address().
In addition, extend the dw_edma_region.paddr field size. The field normally
contains a memory range base address to be set in the DW eDMA Linked-List
pointer register or as a base address of the Linked-List data buffer. In
accordance with [1] the LL range is supposed to be created in the Local
CPU/Application memory, but depending on the DW eDMA utilization the memory
can be created as a part of the PCI bus address space (as in the case of
the DW PCIe Endpoint prototype kit).
In the former case dw_edma_region.paddr should be a dma_addr_t, while in
the latter one it should be a pci_bus_addr_t. Since the corresponding CSRs
are always 64 bits wide, convert dw_edma_region.paddr to be u64, and let
the client make sure it has a valid address visible by the DW eDMA
controller. For instance, the DW eDMA PCIe glue-driver initializes the
field with addresses from the PCI bus memory space.
[1] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
v.5.40a, March 2019, p.1103
Link: https://lore.kernel.org/r/[email protected]
Fixes: 41aaff2a2ac0 ("dmaengine: Add Synopsys eDMA IP PCIe glue-logic")
Tested-by: Manivannan Sadhasivam <[email protected]>
Signed-off-by: Serge Semin <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Bjorn Helgaas <[email protected]>
Reviewed-by: Manivannan Sadhasivam <[email protected]>
Acked-by: Vinod Koul <[email protected]>
|
|
If dw_edma_irq_request() fails to initialize an IRQ handler, any previously
requested IRQs will be left initialized.
Release the previously requested IRQs in the cleanup-on-error path of
dw_edma_irq_request().
Link: https://lore.kernel.org/r/[email protected]
Fixes: e63d79d1ffcd ("dmaengine: Add Synopsys eDMA IP core driver")
Tested-by: Manivannan Sadhasivam <[email protected]>
Signed-off-by: Serge Semin <[email protected]>
Signed-off-by: Lorenzo Pieralisi <[email protected]>
Signed-off-by: Bjorn Helgaas <[email protected]>
Reviewed-by: Manivannan Sadhasivam <[email protected]>
Acked-by: Vinod Koul <[email protected]>
|
|
The interrupt handler (pt_core_irq_handler()) of the ptdma
driver can be called from interrupt context. The code flow
in this function can lead down to pt_core_execute_cmd() which
will attempt to grab a mutex, which is not appropriate in
interrupt context and ultimately leads to a kernel panic.
The fix here changes this mutex to a spinlock, which has
been verified to resolve the issue.
Fixes: fa5d823b16a9 ("dmaengine: ptdma: Initial driver for the AMD PTDMA")
Signed-off-by: Eric Pilmore <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Terminate vdesc when terminating an ongoing transfer.
This will ensure that the vdesc is present in the desc_terminated list
The descriptor will be freed later in desc_free_list().
This fixes the memory leaks which can happen when terminating an
ongoing transfer.
Fixes: ee17028009d4 ("dmaengine: tegra: Add tegra gpcdma driver")
Signed-off-by: Akhil R <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
platform_get_resource() and devm_ioremap_resource() are wrapped up in the
devm_platform_ioremap_resource() helper. Use the helper and get rid of the
local variable for struct resource *. We now have a function call less.
Signed-off-by: Tudor Ambarus <[email protected]>
Acked-by: Jernej Skrabec <[email protected]>
Acked-by: Peter Ujfalusi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
for_each_child_of_node()
Since for_each_child_of_node() will increase the refcount of node, we need
to call of_node_put() manually when breaking out of the iteration.
Fixes: 9cd4360de609 ("dma: Add Xilinx AXI Video Direct Memory Access Engine driver support")
Signed-off-by: Liu Shixin <[email protected]>
Acked-by: Peter Korsgaard <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Remove empty line.
Signed-off-by: Claudiu Beznea <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Add runtime PM support which involves disabling/enabling controller's
clocks on runtime PM suspend/resume ops. The runtime suspend/resume is
done based on the work submitted to the controller: runtime resume is
happening on at_xdmac_start_xfer() and runtime suspend on
at_xdmac_tasklet().
Signed-off-by: Claudiu Beznea <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Align properly function members.
Signed-off-by: Claudiu Beznea <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
If the function sdma_load_context() fails, the sdma_desc will be
freed, but the allocated desc->bd is forgot to be freed.
We already met the sdma_load_context() failure case and the log as
below:
[ 450.699064] imx-sdma 30bd0000.dma-controller: Timeout waiting for CH0 ready
...
In this case, the desc->bd will not be freed without this change.
Signed-off-by: Hui Wang <[email protected]>
Reviewed-by: Sascha Hauer <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Follow the advice of the Documentation/filesystems/sysfs.rst and show()
should only use sysfs_emit() or sysfs_emit_at() when formatting the
value to be returned to user space.
Signed-off-by: ye xingchen <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
The sun6i DMA engine supports segment sizes up to 2^25-1 bytes. This is
explicitly stated in newer SoC documentation (H6, D1), and it is implied
in older documentation by the 25-bit width of the "bytes left in the
current segment" register field.
Exposing the real segment size limit (instead of the 64k default)
reduces the number of SG list segments needed for a transaction.
Reviewed-by: Jernej Skrabec <[email protected]>
Signed-off-by: Samuel Holland <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
The first time dma_chan_get() is called for a channel the channel
client_count is incorrectly incremented twice for public channels,
first in balance_ref_count(), and again prior to returning. This
results in an incorrect client count which will lead to the
channel resources not being freed when they should be. A simple
test of repeated module load and unload of async_tx on a Dell
Power Edge R7425 also shows this resulting in a kref underflow
warning.
[ 124.329662] async_tx: api initialized (async)
[ 129.000627] async_tx: api initialized (async)
[ 130.047839] ------------[ cut here ]------------
[ 130.052472] refcount_t: underflow; use-after-free.
[ 130.057279] WARNING: CPU: 3 PID: 19364 at lib/refcount.c:28
refcount_warn_saturate+0xba/0x110
[ 130.065811] Modules linked in: async_tx(-) rfkill intel_rapl_msr
intel_rapl_common amd64_edac edac_mce_amd ipmi_ssif kvm_amd dcdbas kvm
mgag200 drm_shmem_helper acpi_ipmi irqbypass drm_kms_helper ipmi_si
syscopyarea sysfillrect rapl pcspkr ipmi_devintf sysimgblt fb_sys_fops
k10temp i2c_piix4 ipmi_msghandler acpi_power_meter acpi_cpufreq vfat
fat drm fuse xfs libcrc32c sd_mod t10_pi sg ahci crct10dif_pclmul
libahci crc32_pclmul crc32c_intel ghash_clmulni_intel igb megaraid_sas
i40e libata i2c_algo_bit ccp sp5100_tco dca dm_mirror dm_region_hash
dm_log dm_mod [last unloaded: async_tx]
[ 130.117361] CPU: 3 PID: 19364 Comm: modprobe Kdump: loaded Not
tainted 5.14.0-185.el9.x86_64 #1
[ 130.126091] Hardware name: Dell Inc. PowerEdge R7425/02MJ3T, BIOS
1.18.0 01/17/2022
[ 130.133806] RIP: 0010:refcount_warn_saturate+0xba/0x110
[ 130.139041] Code: 01 01 e8 6d bd 55 00 0f 0b e9 72 9d 8a 00 80 3d
26 18 9c 01 00 75 85 48 c7 c7 f8 a3 03 9d c6 05 16 18 9c 01 01 e8 4a
bd 55 00 <0f> 0b e9 4f 9d 8a 00 80 3d 01 18 9c 01 00 0f 85 5e ff ff ff
48 c7
[ 130.157807] RSP: 0018:ffffbf98898afe68 EFLAGS: 00010286
[ 130.163036] RAX: 0000000000000000 RBX: ffff9da06028e598 RCX: 0000000000000000
[ 130.170172] RDX: ffff9daf9de26480 RSI: ffff9daf9de198a0 RDI: ffff9daf9de198a0
[ 130.177316] RBP: ffff9da7cddf3970 R08: 0000000000000000 R09: 00000000ffff7fff
[ 130.184459] R10: ffffbf98898afd00 R11: ffffffff9d9e8c28 R12: ffff9da7cddf1970
[ 130.191596] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 130.198739] FS: 00007f646435c740(0000) GS:ffff9daf9de00000(0000)
knlGS:0000000000000000
[ 130.206832] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 130.212586] CR2: 00007f6463b214f0 CR3: 00000008ab98c000 CR4: 00000000003506e0
[ 130.219729] Call Trace:
[ 130.222192] <TASK>
[ 130.224305] dma_chan_put+0x10d/0x110
[ 130.227988] dmaengine_put+0x7a/0xa0
[ 130.231575] __do_sys_delete_module.constprop.0+0x178/0x280
[ 130.237157] ? syscall_trace_enter.constprop.0+0x145/0x1d0
[ 130.242652] do_syscall_64+0x5c/0x90
[ 130.246240] ? exc_page_fault+0x62/0x150
[ 130.250178] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 130.255243] RIP: 0033:0x7f6463a3f5ab
[ 130.258830] Code: 73 01 c3 48 8b 0d 75 a8 1b 00 f7 d8 64 89 01 48
83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00
00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 45 a8 1b 00 f7 d8 64 89
01 48
[ 130.277591] RSP: 002b:00007fff22f972c8 EFLAGS: 00000206 ORIG_RAX:
00000000000000b0
[ 130.285164] RAX: ffffffffffffffda RBX: 000055b6786edd40 RCX: 00007f6463a3f5ab
[ 130.292303] RDX: 0000000000000000 RSI: 0000000000000800 RDI: 000055b6786edda8
[ 130.299443] RBP: 000055b6786edd40 R08: 0000000000000000 R09: 0000000000000000
[ 130.306584] R10: 00007f6463b9eac0 R11: 0000000000000206 R12: 000055b6786edda8
[ 130.313731] R13: 0000000000000000 R14: 000055b6786edda8 R15: 00007fff22f995f8
[ 130.320875] </TASK>
[ 130.323081] ---[ end trace eff7156d56b5cf25 ]---
cat /sys/class/dma/dma0chan*/in_use would get the wrong result.
2
2
2
Fixes: d2f4f99db3e9 ("dmaengine: Rework dma_chan_get")
Signed-off-by: Koba Ko <[email protected]>
Reviewed-by: Jie Hai <[email protected]>
Test-by: Jie Hai <[email protected]>
Reviewed-by: Jerry Snitselaar <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Tested-by: Joel Savitz <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
The current global interrupt clear programming register offset
was not correct. Fix the programming with right offset
Fixes: ded1f3db4cd6 ("dmaengine: tegra210-adma: prepare for supporting newer Tegra chips")
Cc: [email protected]
Signed-off-by: Mohan Kumar <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Since there is no protection for vd, a kernel panic will be
triggered here in exceptional cases.
You can refer to the processing of axi_chan_block_xfer_complete function
The triggered kernel panic is as follows:
[ 67.848444] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000060
[ 67.848447] Mem abort info:
[ 67.848449] ESR = 0x96000004
[ 67.848451] EC = 0x25: DABT (current EL), IL = 32 bits
[ 67.848454] SET = 0, FnV = 0
[ 67.848456] EA = 0, S1PTW = 0
[ 67.848458] Data abort info:
[ 67.848460] ISV = 0, ISS = 0x00000004
[ 67.848462] CM = 0, WnR = 0
[ 67.848465] user pgtable: 4k pages, 48-bit VAs, pgdp=00000800c4c0b000
[ 67.848468] [0000000000000060] pgd=0000000000000000, p4d=0000000000000000
[ 67.848472] Internal error: Oops: 96000004 [#1] SMP
[ 67.848475] Modules linked in: dmatest
[ 67.848479] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.10.100-emu_x2rc+ #11
[ 67.848483] pstate: 62000085 (nZCv daIf -PAN -UAO +TCO BTYPE=--)
[ 67.848487] pc : axi_chan_handle_err+0xc4/0x230
[ 67.848491] lr : axi_chan_handle_err+0x30/0x230
[ 67.848493] sp : ffff0803fe55ae50
[ 67.848495] x29: ffff0803fe55ae50 x28: ffff800011212200
[ 67.848500] x27: ffff0800c42c0080 x26: ffff0800c097c080
[ 67.848504] x25: ffff800010d33880 x24: ffff80001139d850
[ 67.848508] x23: ffff0800c097c168 x22: 0000000000000000
[ 67.848512] x21: 0000000000000080 x20: 0000000000002000
[ 67.848517] x19: ffff0800c097c080 x18: 0000000000000000
[ 67.848521] x17: 0000000000000000 x16: 0000000000000000
[ 67.848525] x15: 0000000000000000 x14: 0000000000000000
[ 67.848529] x13: 0000000000000000 x12: 0000000000000040
[ 67.848533] x11: ffff0800c0400248 x10: ffff0800c040024a
[ 67.848538] x9 : ffff800010576cd4 x8 : ffff0800c0400270
[ 67.848542] x7 : 0000000000000000 x6 : ffff0800c04003e0
[ 67.848546] x5 : ffff0800c0400248 x4 : ffff0800c4294480
[ 67.848550] x3 : dead000000000100 x2 : dead000000000122
[ 67.848555] x1 : 0000000000000100 x0 : ffff0800c097c168
[ 67.848559] Call trace:
[ 67.848562] axi_chan_handle_err+0xc4/0x230
[ 67.848566] dw_axi_dma_interrupt+0xf4/0x590
[ 67.848569] __handle_irq_event_percpu+0x60/0x220
[ 67.848573] handle_irq_event+0x64/0x120
[ 67.848576] handle_fasteoi_irq+0xc4/0x220
[ 67.848580] __handle_domain_irq+0x80/0xe0
[ 67.848583] gic_handle_irq+0xc0/0x138
[ 67.848585] el1_irq+0xc8/0x180
[ 67.848588] arch_cpu_idle+0x14/0x2c
[ 67.848591] default_idle_call+0x40/0x16c
[ 67.848594] do_idle+0x1f0/0x250
[ 67.848597] cpu_startup_entry+0x2c/0x60
[ 67.848600] rest_init+0xc0/0xcc
[ 67.848603] arch_call_rest_init+0x14/0x1c
[ 67.848606] start_kernel+0x4cc/0x500
[ 67.848610] Code: eb0002ff 9a9f12d6 f2fbd5a2 f2fbd5a3 (a94602c1)
[ 67.848613] ---[ end trace 585a97036f88203a ]---
Signed-off-by: Shawn.Shao <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
ldma_cfg_init() will parse DT to retrieve certain configs.
However, that is called before ldma_dma_init_vXX(), which
will make some initialization to channel configs. It will
thus incorrectly overwrite certain configs that are declared
in DT.
To fix that, we move DT parsing after initialization.
Function name is renamed to better represent what it does.
Fixes: 32d31c79a1a4 ("dmaengine: Add Intel LGM SoC DMA support.")
Signed-off-by: Peter Harliman Liem <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/afef6fc1ed20098b684e0d53737d69faf63c125f.1672887183.git.pliem@maxlinear.com
Signed-off-by: Vinod Koul <[email protected]>
|
|
Drop the non-fatal probe deferral log for getting MSI domain.
This makes the kernel log clean and we do not get recurring logs
stating: "Failed to get MSI domain".
Signed-off-by: Jayesh Choudhary <[email protected]>
Acked-by: Peter Ujfalusi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
The s3c24xx platform was removed and this driver is no longer
needed.
Reviewed-by: Krzysztof Kozlowski <[email protected]>
Signed-off-by: Arnd Bergmann <[email protected]>
|
|
The MMP_SRAM code is no longer used by the tdma driver because
the Kconfig symbol is not selected, so remove it along with its
former callsite.
Acked-By: Vinod Koul <[email protected]>
Signed-off-by: Arnd Bergmann <[email protected]>
|
|
The hdmi script already supported in sdma firmware. So add support hdmi
in sdma driver.
The design of hdmi script is different from common script such as sai.
There is no need to config buffer descriptor for HDMI. The cyclic
capability is achieved by the hdmi script. The slave config is so simple,
only config src_addr, dts_addr and direction DMA_TRANS_NONE.
Signed-off-by: Joy Zou <[email protected]>
Reviewed-by: Sascha Hauer <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
The HiSilicon DMA Engine is only present on HiSilicon SoCs. Hence add a
dependency on ARCH_HISI, to prevent asking the user about this driver
when configuring a kernel without HiSilicon SoC support.
Fixes: e9f08b65250d73ab ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Geert Uytterhoeven <[email protected]>
Link: https://lore.kernel.org/r/363a1816d36cd3cf604d88ec90f97c75f604de64.1669044190.git.geert+renesas@glider.be
Signed-off-by: Vinod Koul <[email protected]>
|
|
PSIL_EP_NATIVE endpoints may not have PEER registers for BCNT and thus
udma_decrement_byte_counters() should not try to decrement these counters.
This fixes the issue of crypto IPERF testing where the client side (EVM)
hangs without transfer of packets to the server side, seen since this
function was added.
Fixes: 7c94dcfa8fcf ("dmaengine: ti: k3-udma: Reset UDMA_CHAN_RT byte counters to prevent overflow")
Signed-off-by: Jayesh Choudhary <[email protected]>
Acked-by: Peter Ujfalusi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
On driver unload any pending descriptors are flushed and pending
DMA descriptors are explicitly completed:
idxd_dmaengine_drv_remove() ->
drv_disable_wq() ->
idxd_wq_free_irq() ->
idxd_flush_pending_descs() ->
idxd_dma_complete_txd()
With this done during driver unload any remaining descriptor is
likely stuck and can be dropped. Even so, the descriptor may still
have a callback set that could no longer be accessible. An
example of such a problem is when the dmatest fails and the dmatest
module is unloaded. The failure of dmatest leaves descriptors with
dma_async_tx_descriptor::callback pointing to code that no longer
exist. This causes a page fault as below at the time the IDXD driver
is unloaded when it attempts to run the callback:
BUG: unable to handle page fault for address: ffffffffc0665190
#PF: supervisor instruction fetch in kernel mode
#PF: error_code(0x0010) - not-present page
Fix this by clearing the callback pointers on the transmit
descriptors only when workqueue is disabled.
Fixes: 403a2e236538 ("dmaengine: idxd: change MSIX allocation based on per wq activation")
Signed-off-by: Reinette Chatre <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Fenghua Yu <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/37d06b772aa7f8863ca50f90930ea2fd80b38fc3.1670452419.git.reinette.chatre@intel.com
Signed-off-by: Vinod Koul <[email protected]>
|
|
On driver unload any pending descriptors are flushed at the
time the interrupt is freed:
idxd_dmaengine_drv_remove() ->
drv_disable_wq() ->
idxd_wq_free_irq() ->
idxd_flush_pending_descs().
If there are any descriptors present that need to be flushed this
flow triggers a "not present" page fault as below:
BUG: unable to handle page fault for address: ff391c97c70c9040
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
The address that triggers the fault is the address of the
descriptor that was freed moments earlier via:
drv_disable_wq()->idxd_wq_free_resources()
Fix the use after free by freeing the descriptors after any possible
usage. This is done after idxd_wq_reset() to ensure that the memory
remains accessible during possible completion writes by the device.
Fixes: 63c14ae6c161 ("dmaengine: idxd: refactor wq driver enable/disable operations")
Suggested-by: Dave Jiang <[email protected]>
Signed-off-by: Reinette Chatre <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Fenghua Yu <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/6c4657d9cff0a0a00501a7b928297ac966e9ec9d.1670452419.git.reinette.chatre@intel.com
Signed-off-by: Vinod Koul <[email protected]>
|
|
The workqueue is enabled when the appropriate driver is loaded and
disabled when the driver is removed. When the driver is removed it
assumes that the workqueue was enabled successfully and proceeds to
free allocations made during workqueue enabling.
Failure during workqueue enabling does not prevent the driver from
being loaded. This is because the error path within drv_enable_wq()
returns success unless a second failure is encountered
during the error path. By returning success it is possible to load
the driver even if the workqueue cannot be enabled and
allocations that do not exist are attempted to be freed during
driver remove.
Some examples of problematic flows:
(a)
idxd_dmaengine_drv_probe() -> drv_enable_wq() -> idxd_wq_request_irq():
In above flow, if idxd_wq_request_irq() fails then
idxd_wq_unmap_portal() is called on error exit path, but
drv_enable_wq() returns 0 because idxd_wq_disable() succeeds. The
driver is thus loaded successfully.
idxd_dmaengine_drv_remove()->drv_disable_wq()->idxd_wq_unmap_portal()
Above flow on driver unload triggers the WARN in devm_iounmap() because
the device resource has already been removed during error path of
drv_enable_wq().
(b)
idxd_dmaengine_drv_probe() -> drv_enable_wq() -> idxd_wq_request_irq():
In above flow, if idxd_wq_request_irq() fails then
idxd_wq_init_percpu_ref() is never called to initialize the percpu
counter, yet the driver loads successfully because drv_enable_wq()
returns 0.
idxd_dmaengine_drv_remove()->__idxd_wq_quiesce()->percpu_ref_kill():
Above flow on driver unload triggers a BUG when attempting to drop the
initial ref of the uninitialized percpu ref:
BUG: kernel NULL pointer dereference, address: 0000000000000010
Fix the drv_enable_wq() error path by returning the original error that
indicates failure of workqueue enabling. This ensures that the probe
fails when an error is encountered and the driver remove paths are only
attempted when the workqueue was enabled successfully.
Fixes: 1f2bb40337f0 ("dmaengine: idxd: move wq_enable() to device.c")
Signed-off-by: Reinette Chatre <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Reviewed-by: Fenghua Yu <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/e8d8116e5efa0fd14fadc5adae6ffd319f0e5ff1.1670452419.git.reinette.chatre@intel.com
Signed-off-by: Vinod Koul <[email protected]>
|
|
On DSA/IAX 1.0, TC-A and TC-B in GRPCFG are set as 1 to have best
performance and cannot be changed through sysfs knobs unless override
option is given.
The same values should be set on DSA 2.0 as well.
Fixes: ea7c8f598c32 ("dmaengine: idxd: restore traffic class defaults after wq reset")
Fixes: ade8a86b512c ("dmaengine: idxd: Set defaults for GRPCFG traffic class")
Signed-off-by: Fenghua Yu <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
The function set_completion_address is defined in the dma.c file, but not
called elsewhere, so remove this unused function.
drivers/dma/idxd/dma.c:66:20: warning: unused function 'set_completion_address'.
Link: https://bugzilla.openanolis.cn/show_bug.cgi?id=3416
Reported-by: Abaci Robot <[email protected]>
Signed-off-by: Jiapeng Chong <[email protected]>
Acked-by: Dave Jiang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Rx operation on SPI GSI DMA is currently not working.
As per GSI spec, link_rx bit is to be set on GO TRE on tx
channel whenever there is going to be a DMA TRE on rx
channel. This is currently set for duplex operation only.
Set the bit for rx operation as well.
This is part of changes required to bring up Rx.
Fixes: 94b8f0e58fa1 ("dmaengine: qcom: gpi: set chain and link flag for duplex")
Signed-off-by: Vijaya Krishna Nivarthi <[email protected]>
Reviewed-by: Douglas Anderson <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
BCDMA CSI RX present on AM62Ax SoC is a dedicated DMA for servicing
Camera Serial Interface (CSI) IP. Add support for the same.
Signed-off-by: Vignesh Raghavendra <[email protected]>
Acked-by: Peter Ujfalusi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
AM62A SoC has a BCDMA and PKTDMA as systems DMAs for service various
peripherals similar to AM64 SoC. Add support for the same.
Signed-off-by: Vignesh Raghavendra <[email protected]>
Acked-by: Peter Ujfalusi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Add PSIL and PDMA data for AM62Ax SoC.
Signed-off-by: Jai Luthra <[email protected]>
Signed-off-by: Vignesh Raghavendra <[email protected]>
Acked-by: Peter Ujfalusi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Reusing loop iterator fails if BCHAN is not present as iterator is
uninitialized
Signed-off-by: Vignesh Raghavendra <[email protected]>
Acked-by: Peter Ujfalusi <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine
Pull dmaengine updates from Vinod Koul:
"New support:
- Qualcomm SDM670, SM6115 and SM6375 GPI controller support
- Ingenic JZ4755 dmaengine support
- Removal of iop-adma driver
Updates:
- Tegra support for dma-channel-mask
- at_hdmac cleanup and virt-chan support for this driver"
* tag 'dmaengine-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (46 commits)
dmaengine: Revert "dmaengine: remove s3c24xx driver"
dmaengine: tegra: Add support for dma-channel-mask
dt-bindings: dmaengine: Add dma-channel-mask to Tegra GPCDMA
dmaengine: idxd: Remove linux/msi.h include
dt-bindings: dmaengine: qcom: gpi: add compatible for SM6375
dmaengine: idxd: Fix crc_val field for completion record
dmaengine: at_hdmac: Convert driver to use virt-dma
dmaengine: at_hdmac: Remove unused member of at_dma_chan
dmaengine: at_hdmac: Rename "chan_common" to "dma_chan"
dmaengine: at_hdmac: Rename "dma_common" to "dma_device"
dmaengine: at_hdmac: Use bitfield access macros
dmaengine: at_hdmac: Keep register definitions and structures private to at_hdmac.c
dmaengine: at_hdmac: Set include entries in alphabetic order
dmaengine: at_hdmac: Use pm_ptr()
dmaengine: at_hdmac: Use devm_clk_get()
dmaengine: at_hdmac: Use devm_platform_ioremap_resource
dmaengine: at_hdmac: Use devm_kzalloc() and struct_size()
dmaengine: at_hdmac: Introduce atc_get_llis_residue()
dmaengine: at_hdmac: s/atc_get_bytes_left/atc_get_residue
dmaengine: at_hdmac: Pass residue by address to avoid unnecessary implicit casts
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd
Pull iommufd implementation from Jason Gunthorpe:
"iommufd is the user API to control the IOMMU subsystem as it relates
to managing IO page tables that point at user space memory.
It takes over from drivers/vfio/vfio_iommu_type1.c (aka the VFIO
container) which is the VFIO specific interface for a similar idea.
We see a broad need for extended features, some being highly IOMMU
device specific:
- Binding iommu_domain's to PASID/SSID
- Userspace IO page tables, for ARM, x86 and S390
- Kernel bypassed invalidation of user page tables
- Re-use of the KVM page table in the IOMMU
- Dirty page tracking in the IOMMU
- Runtime Increase/Decrease of IOPTE size
- PRI support with faults resolved in userspace
Many of these HW features exist to support VM use cases - for instance
the combination of PASID, PRI and Userspace IO Page Tables allows an
implementation of DMA Shared Virtual Addressing (vSVA) within a guest.
Dirty tracking enables VM live migration with SRIOV devices and PASID
support allow creating "scalable IOV" devices, among other things.
As these features are fundamental to a VM platform they need to be
uniformly exposed to all the driver families that do DMA into VMs,
which is currently VFIO and VDPA"
For more background, see the extended explanations in Jason's pull request:
https://lore.kernel.org/lkml/[email protected]/
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd: (62 commits)
iommufd: Change the order of MSI setup
iommufd: Improve a few unclear bits of code
iommufd: Fix comment typos
vfio: Move vfio group specific code into group.c
vfio: Refactor dma APIs for emulated devices
vfio: Wrap vfio group module init/clean code into helpers
vfio: Refactor vfio_device open and close
vfio: Make vfio_device_open() truly device specific
vfio: Swap order of vfio_device_container_register() and open_device()
vfio: Set device->group in helper function
vfio: Create wrappers for group register/unregister
vfio: Move the sanity check of the group to vfio_create_group()
vfio: Simplify vfio_create_group()
iommufd: Allow iommufd to supply /dev/vfio/vfio
vfio: Make vfio_container optionally compiled
vfio: Move container related MODULE_ALIAS statements into container.c
vfio-iommufd: Support iommufd for emulated VFIO devices
vfio-iommufd: Support iommufd for physical VFIO devices
vfio-iommufd: Allow iommufd to be used in place of a container fd
vfio: Use IOMMU_CAP_ENFORCE_CACHE_COHERENCY for vfio_file_enforced_coherent()
...
|
|
Resolve conflicts in drivers/vfio/vfio_main.c by using the iommfd version.
The rc fix was done a different way when iommufd patches reworked this
code.
Signed-off-by: Jason Gunthorpe <[email protected]>
|
|
This reverts cccc46ae3623 ("dmaengine: remove s3c24xx driver") as it
causes regression due to missing header
Signed-off-by: Vinod Koul <[email protected]>
|
|
Adjust to reality and remove another layer of pointless Kconfig
indirection. CONFIG_GENERIC_MSI_IRQ is good enough to serve
all purposes.
Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
|
|
Add support for dma-channel-mask so that only the specified channels
are used. This helps to reserve some channels for the firmware.
This was initially achieved by limiting the channel number to 31 in
the driver and adjusting the register address to skip channel0 which
was reserved for a firmware. This is wrong and does not align with
the hardware.
Now, with this change, the driver can align more to the actual hardware
which has 32 channels. But this implies that there will be a break in the
ABI and the device tree need to be updated along with this change for the
driver to pickup the right interrupt corresponding to the channel
Reviewed-by: Jon Hunter <[email protected]>
Link: https://lore.kernel.org/all/Y2EFoG1H9YpfxRjs@orome/
Signed-off-by: Akhil R <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Nothing in this file needs anything from linux/msi.h
Signed-off-by: Thomas Gleixner <[email protected]>
Cc: Fenghua Yu <[email protected]>
Cc: Dave Jiang <[email protected]>
Cc: Vinod Koul <[email protected]>
Cc: [email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Convert the driver to use the core virt-dma. The driver will be easier to
maintain as it uses the list handling and the tasklet from virt-dma.
With the conversion replace the election of a new transfer in the tasklet
with the election of the new transfer in the interrupt handler. With this
we have a shorter idle window as we remove the scheduling latency of the
tasklet. I chose to do this while doing the conversion to virt-dma,
because if I made a prerequisite patch with the new transfer election in
the irq handler, I would have to duplicate some virt-dma code in the
at_hdmac driver that would end up being removed at the virt-dma conversion
anyway. So do this in a single step.
Signed-off-by: Tudor Ambarus <[email protected]>
Acked-by: Nicolas Ferre <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
The pointer to at_dma engine was never used, remove it.
Signed-off-by: Tudor Ambarus <[email protected]>
Acked-by: Nicolas Ferre <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
"chan_common" was misleading and did not suggest that's actually
a struct dma_chan underneath. Rename it so that readers can follow the
code easier. One may see some checks when running checkpatch. Those have
nothing to do with the rename and will be addressed in a further patch.
Signed-off-by: Tudor Ambarus <[email protected]>
Acked-by: Nicolas Ferre <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
"dma_common" name was misleading and did not suggest that's actually
a struct dma_device underneath. Rename it so that readers can follow the
code easier. One may see some checks and a warning when running
checkpatch. Those have nothing to do with the rename and will be addressed
in a further patch.
Signed-off-by: Tudor Ambarus <[email protected]>
Acked-by: Nicolas Ferre <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
Use the bitfield access macros in order to clean and to make the driver
easier to read. One will see some "line length exceeds 100 columns"
checkpatch warnings. I chose to not introduce new lines for regs
descriptions in order to preserve the style of the comments throughout
the definitions. Style can be fixed in a further patch.
Signed-off-by: Tudor Ambarus <[email protected]>
Acked-by: Nicolas Ferre <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|
|
at_hdmac.c
Do not expose register definitions, structures and helpers via a .h file
because there are used only by at_hdmac.c. Since there are no other users,
remove the ambiguity and move all the .h contents to the .c file.
One may notice some checkpatch warnings and errors with this move. The move
was done "as it was", checkpatch complaints can be fixed in a further
patch.
Signed-off-by: Tudor Ambarus <[email protected]>
Acked-by: Nicolas Ferre <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Vinod Koul <[email protected]>
|