aboutsummaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/amd
AgeCommit message (Collapse)AuthorFilesLines
2023-06-09drm/amdgpu: introduce new doorbell assignment table for GC 9.4.3Le Ma4-32/+33
Four basic reasons as below to do the change: 1. number of ring expand a lot on GC 9.4.3, and adjustment on old assignment cannot make each ring in a continuous doorbell space. 2. the SDMA doorbell index should not exceed 0x1FF on SDMA 4.2.2 due to regDOORBELLx_CTRL_ENTRY.BIF_DOORBELLx_RANGE_OFFSET_ENTRY field width. 3. re-design the doorbell assignment and unify the calculation as "start + ring/inst id" will make the code much concise. 4. only defining the START/END makes the table look simple v2: (Lijo) 1. replace name 2. use num_inst_per_aid/sdma_doorbell_range instead of hardcoding Signed-off-by: Le Ma <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: program GRBM_MCM_ADDR for non-AID0 GRBMLe Ma1-0/+3
Otherwise the EOP interrupt on non-AID0 cannot route to IH0. Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: convert the doorbell_index to 2 dwords offset for kiqLe Ma1-4/+3
KIQ doorbell_index is non-zero from XCC1, thus need to left-shift it like other rings. Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: set mmhub bitmask for multiple AIDsLe Ma1-1/+1
Like GFXHUB, set MMHUB0 bitmask for each AID. Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: complement the IH node_id table for multiple AIDsLe Ma3-1/+12
With different node_id, the SDMA interrupt from multiple AIDs can be distinguished by sw driver. Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: correct the vmhub reference for each XCD in gfxhub initLe Ma1-4/+8
Correct this though the value is same across different vmhub. Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: do mmhub init for multiple AIDsLe Ma1-261/+348
Mmhub on each AID needs to be initialized respectively Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: assign the doorbell index for sdma on non-AID0Le Ma2-3/+21
Allocate new sdma doorbell index for the instances only on AID1 for now. Todo: there's limitation that SDMA doorbell index on SDMA 4.4.2 needs to be less than 0x1FF, so the tail part in _AMDGPU_VEGA20_DOORBELL_ASSIGNMENT is not enough to store sdma doorbell range on maximum 4 AIDs if doorbell_range is 20. So it looks better to create a new doorbell index assignment table for 4.4.2. v2: change "(x << 1) + 2" to "(x + 1) << 1" for readability. Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: add support for SDMA on multiple AIDsLe Ma2-7/+21
Initialize SDMA instances on each AID. v2: revise coding fault in hw_fini Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: adjust some basic elements for multiple AID caseLe Ma3-3/+6
add some elements below: - num_aid - aid_id for each sdma instance - num_inst_per_aid for sdma and extend macro size below: - SDMA_MAX_INSTANCES to 16 - AMDGPU_MAX_RINGS to 96 - AMDGPU_MAX_HWIP_RINGS to 32 v2: move aid_id from amdgpu_ring to amdgpu_sdma_instance. (Lijo) Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: assign the doorbell index in 1st page to sdma page queueLe Ma3-7/+17
Previously for vega10, the sdma_doorbell_range is only enough for sdma gfx queue, thus the index on second doorbell page is allocated for sdma page queue. From vega20, the sdma_doorbell_range on 1st page is enlarged. Therefore, just leverage these index instead of allocation on 2nd page. v2: change "(x << 1) + 2" to "(x + 1) << 1" for readability and add comments. Signed-off-by: Le Ma <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Lijo Lazar <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Set XNACK per process on GC 9.4.3Amber Lin2-3/+5
Set RETRY_PERMISSION_OR_INVALID_PAGE_FAULT bit in VM_CONTEXT1_CNTL as well so XNACK can be enabled in the SQ per process. Signed-off-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Use new atomfirmware init for GC 9.4.3Lijo Lazar1-1/+2
Use the new atomfirmware initialization logic for GC 9.4.3 based ASICs also. ASIC init logic doesn't consider boot clocks during init. Signed-off-by: Lijo Lazar <[email protected]> Reviewed-by: Hawking Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Update coherence settings for svm rangesRajneesh Bhardwaj1-0/+17
Recently introduced commit "drm/amdgpu: Set cache coherency for GC 9.4.3" did not update the settings applicable for svm ranges. Add the coherence settings for svm ranges for GFX IP 9.4.3. Reviewed-by: Amber Lin <[email protected]> Signed-off-by: Rajneesh Bhardwaj <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Fix CP_HYP_XCP_CTL register programming in CPX modeMukul Joshi1-1/+1
Currently, in CPX mode, the CP_HYP_XCP_CTL register is programmed incorrectly with the number of XCCs in the partition. As a result, HIQ doesn't work in CPX mode. Fix this by programming the correct number of XCCs in a partition, which is 1, in CPX mode. Signed-off-by: Mukul Joshi <[email protected]> Reviewed-by: Le Ma <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Update SDMA queue management for GFX9.4.3Mukul Joshi5-41/+227
This patch updates SDMA queue management for multi XCC in GFX9.4.3. - Allocate/deallocate SDMA queues from the correct SDMA engines based on the partition mode. - Updates the kgd2kfd interface to fetch the correct SDMA register addresses. - It also fixes dumping correct SDMA queue info in debugfs. v2: squash in fix "drm/amdkfd: Fix XGMI SDMA user-mode queue allocation" Signed-off-by: Mukul Joshi <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Update sysfs node properties for multi XCCMukul Joshi1-2/+4
Update simd_count and array_count node properties to report values multiplied by number of XCCs in the partition. Signed-off-by: Mukul Joshi <[email protected]> Tested-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Call DQM stop during DQM uninitializeMukul Joshi1-0/+8
During DQM tear down, call DQM stop to unitialize HIQ and associated memory allocated during packet manager init. Signed-off-by: Mukul Joshi <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Fix VM fault reporting on XCC1Mukul Joshi1-3/+5
Fix VM fault reporting and clear VM fault register for XCC1. Signed-off-by: Mukul Joshi <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Update context save handling for multi XCC setup (v2)Mukul Joshi6-3/+67
Context save handling needs to be updated for a multi XCC setup: - On a multi XCC setup, KFD needs to report context save base address and size for each XCC in MQD. - Thunk will allocate a large context save area covering all XCCs which will be equal to: num_of_xccs in a partition * size of context save area for 1 XCC. However, it will report only the size of context save area for 1 XCC only in the ioctl call. - Driver then setups the MQD correctly using the size passed from Thunk and information about number of XCCs in a partition. - Update get_wave_state function to return context save area for all XCCs in the partition. v2: update the get_wave_state function for mqd manager v11 (Morris) Signed-off-by: Mukul Joshi <[email protected]> Tested-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Morris Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Add XCC inst to PASID TLB flushingMukul Joshi9-18/+25
Add XCC instance to select the correct KIQ ring when flushing TLBs on a multi-XCC setup. Signed-off-by: Mukul Joshi <[email protected]> Tested-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Add XCC instance to kgd2kfd interface (v3)Mukul Joshi17-218/+270
Gfx 9 starts to have multiple XCC instances in one device. Add instance parameter to kgd2kfd functions where XCC instance was hard coded as 0. Also, update code to pass the correct instance number when running on a multi-XCC setup. v2: introduce the XCC instance to gfx v11 (Morris) v3: rebase (Alex) Signed-off-by: Amber Lin <[email protected]> Signed-off-by: Mukul Joshi <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Tested-by: Amber Lin <[email protected]> Signed-off-by: Morris Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Add PM4 target XCCMukul Joshi4-4/+22
In a device that supports multiple XCCs, unlike AQL queues, the PM4 queue will be only processed in one XCC in the partitioning. This patch re-purposes the queue percentage variable in create queue and update queue ioctl for the user space to specify the target XCC. Signed-off-by: Amber Lin <[email protected]> Signed-off-by: Mukul Joshi <[email protected]> Tested-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Update MQD management on multi XCC setupMukul Joshi10-57/+380
Update MQD management for both HIQ and user-mode compute queues on a multi XCC setup. MQDs needs to be allocated, initialized, loaded and destroyed for each XCC in the KFD node. v2: squash in fix "drm/amdkfd: Fix SDMA+HIQ HQD allocation on GFX9.4.3" Signed-off-by: Mukul Joshi <[email protected]> Signed-off-by: Amber Lin <[email protected]> Tested-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Add spatial partitioning support in KFDMukul Joshi7-77/+208
This patch introduces multi-partition support in KFD. This patch includes: - Support for maximum 8 spatial partitions in KFD. - Initialize one HIQ per partition. - Management of VMID range depending on partition mode. - Management of doorbell aperture space between all partitions. - Each partition does its own queue management, interrupt handling, SMI event reporting. - IOMMU, if enabled with multiple partitions, will only work on first partition. - SPM is only supported on the first partition. - Currently, there is no support for resetting individual partitions. All partitions will reset together. Signed-off-by: Mukul Joshi <[email protected]> Tested-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Introduce kfd_node struct (v5)Mukul Joshi38-495/+573
Introduce a new structure, kfd_node, which will now represent a compute node. kfd_node is carved out of kfd_dev structure. kfd_dev struct now will become the parent of kfd_node, and will store common resources such as doorbells, GTT sub-alloctor etc. kfd_node struct will store all resources specific to a compute node, such as device queue manager, interrupt handling etc. This is the first step in adding compute partition support in KFD. v2: introduce kfd_node struct to gc v11 (Hawking) v3: make reference to kfd_dev struct through kfd_node (Morris) v4: use kfd_node instead for kfd isr/mqd functions (Morris) v5: rebase (Alex) Signed-off-by: Mukul Joshi <[email protected]> Tested-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Hawking Zhang <[email protected]> Signed-off-by: Morris Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Add mode2 reset logic for v13.0.6Lijo Lazar1-0/+2
Mode2 reset for v13.0.6 has similar workflow as v13.0.2 Signed-off-by: Lijo Lazar <[email protected]> Reviewed-by: Hawking Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Add some XCC programmingLijo Lazar1-0/+26
Add additional XCC programming sequences. Signed-off-by: Lijo Lazar <[email protected]> Reviewed-by: Hawking Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: add node_id to physical id conversion in EOP handlerLe Ma3-2/+29
A new field nodeid in interrupt cookie indicates the node ID. Signed-off-by: Le Ma <[email protected]> Reviewed-by: Shiwu Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: enable the ring and IB test for slave kcqShiwu Zhang3-44/+32
With the mec FW update to utilize the mqd base set by driver for kcq mapping, slave kcq ring test and IB test can be re-enabled. Signed-off-by: Shiwu Zhang <[email protected]> Reviewed-by: Le Ma <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: support gc v9_4_3 ring_test running on all xccHawking Zhang1-4/+7
Each xcc has its own sratch_reg offset Signed-off-by: Hawking Zhang <[email protected]> Reviewed-by: Le Ma <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: fix vcn doorbell range settingJames Zhu1-1/+1
Should use vcn_ring0_1 instead of doorbell index to set nbio doorbell range. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Sonny Jiang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/jpeg: enable jpeg doorbell for jpeg4.0.3James Zhu1-2/+11
Enable jpeg doorbell for jpeg4.0.3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/vcn: enable vcn doorbell for vcn4.0.3James Zhu1-1/+9
Enable vcn doorbell for vcn4.0.3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/nbio: update vcn doorbell rangeJames Zhu1-2/+2
VCN4.0.3 used up to 16 doorbells per partition. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdkfd: Set F8_MODE for gc_v9_4_3Amber Lin1-3/+5
Set F8_MODE for GC 9.4.3 as optimal/non-IEEE. Also update gc_v9_0 to gc_v9_4_3 to include more definitions such as the F8_MODE bit, and remove unused header files. v2: fix IP version check (Alex) Signed-off-by: Amber Lin <[email protected]> Reviewed-by: Felix Kuehling <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/jpeg: add multiple jpeg rings support for vcn4_0_3James Zhu2-68/+147
Add multiple jpeg rings support for vcn4_0_3 Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/jpeg: add multiple jpeg rings supportJames Zhu10-65/+69
Add multiple jpeg rings support. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/vcn: enable vcn DPG mode for VCN4_0_3James Zhu1-0/+1
Enable vcn DPG mode for VCN4_0_3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/vcn: enable vcn pg for VCN4_0_3James Zhu1-0/+1
Enable vcn pg for VCN4_0_3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/vcn: enable vcn cg for VCN4_0_3James Zhu1-0/+1
Enable vcn cg for VCN4_0_3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/jpeg: enable jpeg pg for VCN4_0_3James Zhu1-1/+2
Enable jpeg pg for VCN4_0_3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/jpeg: enable jpeg cg for VCN4_0_3James Zhu1-1/+2
Enable jpeg cg for VCN4_0_3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/vcn: add vcn support for VCN4_0_3James Zhu3-0/+1468
Add vcn support for VCN4_0_3. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/jpeg: add jpeg support for VCN4_0_3James Zhu3-1/+813
Add jpeg support for VCN4_0_3. v2: squash in delayed work typo fix (Alex) Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: add VCN4_0_3 firmwareJames Zhu1-3/+14
Add VCN4_0_3 firmware. v2: fix fw name (Alex) Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: add vcn v4_0_3 ip headersHawking Zhang2-0/+12678
Add vcn v4_0_3 register offset adn shift masks header files v2: update headers (Alex) Signed-off-by: Hawking Zhang <[email protected]> Reviewed-by: Le Ma <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu/: add more macro to support offset variantJames Zhu3-25/+42
Add more macro to support offset variant and simplify macro SOC15_WAIT_ON_RREG. Signed-off-by: James Zhu <[email protected]> Reviewed-by: Leo Liu <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Use the correct API to read registerLijo Lazar1-1/+1
Use SOC15 API so that the register offset is calculated correctly. Signed-off-by: Lijo Lazar <[email protected]> Reviewed-by: Hawking Zhang <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
2023-06-09drm/amdgpu: Add kgd2kfd for GC 9.4.3Amber Lin5-16/+206
New GC (v9.4.3) and ATHUB (v1.8.0) versions are used. Add kgd_gfx_v9_4_3_* functions if registers in use of kgd_gfx_v9_* functions are changed or have different offset. Signed-off-by: Amber Lin <[email protected]> Acked-by: Felix Kuehling <[email protected]> Reviewed-by: Mukul Joshi <[email protected]> Signed-off-by: Alex Deucher <[email protected]>
Powered by cgit, Gruvboxed by Blaster4385.