| Age | Commit message (Collapse) | Author | Files | Lines |
|
Converts IOTLB and Dev-IOTLB invalidation to a batched model. Cache tag
invalidation requests for a domain are now accumulated in a qi_batch
structure before being flushed in bulk. It replaces the previous per-
request qi_flush approach with a more efficient batching mechanism.
Co-developed-by: Lu Baolu <[email protected]>
Signed-off-by: Lu Baolu <[email protected]>
Signed-off-by: Tina Zhang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
Introduces a qi_batch structure to hold batched cache invalidation
descriptors on a per-dmar_domain basis. A fixed-size descriptor
array is used for simplicity. The qi_batch is allocated when the
first cache tag is added to the domain and freed during
iommu_free_domain().
Signed-off-by: Lu Baolu <[email protected]>
Signed-off-by: Tina Zhang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
Extracts IOTLB and Dev-IOTLB invalidation logic from cache tag flush
interfaces into dedicated helper functions. It prepares the codebase
for upcoming changes to support batched cache invalidations.
To enable direct use of qi_flush helpers in the new functions,
iommu->flush.flush_iotlb and quirk_extra_dev_tlb_flush() are opened up.
No functional changes are intended.
Co-developed-by: Lu Baolu <[email protected]>
Signed-off-by: Lu Baolu <[email protected]>
Signed-off-by: Tina Zhang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
Separate the logic for constructing IOTLB and device TLB invalidation
descriptors from the qi_flush interfaces. New helpers, qi_desc(), are
introduced to encapsulate this common functionality.
Moving descriptor composition code to new helpers enables its reuse in
the upcoming qi_batch interfaces.
No functional changes are intended.
Signed-off-by: Tina Zhang <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Lu Baolu <[email protected]>
Signed-off-by: Joerg Roedel <[email protected]>
|
|
The caching mode of an IOMMU is irrelevant to the behavior of the device
TLB. Previously, commit <304b3bde24b5> ("iommu/vt-d: Remove caching mode
check before device TLB flush") removed this redundant check in the
domain unmap path.
Checking the caching mode before flushing the device TLB after a pasid
table entry is updated is unnecessary and can lead to inconsistent
behavior.
Extends this consistency by removing the caching mode check in the pasid
table update path.
Suggested-by: Yi Liu <[email protected]>
Signed-off-by: Lu Baolu <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
Currently, PCI PASID is enabled alongside PCI ATS when an iommu domain is
attached to the device and disabled when the device transitions to block
translation mode. This approach is inappropriate as PCI PASID is a device
feature independent of the type of the attached domain.
Enable PCI PASID during the IOMMU device probe and disables it during the
release path.
Suggested-by: Yi Liu <[email protected]>
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Yi Liu <[email protected]>
Tested-by: Yi Liu <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
If qi_submit_sync() is invoked with 0 invalidation descriptors (for
instance, for DMA draining purposes), we can run into a bug where a
submitting thread fails to detect the completion of invalidation_wait.
Subsequently, this led to a soft lockup. Currently, there is no impact
by this bug on the existing users because no callers are submitting
invalidations with 0 descriptors. This fix will enable future users
(such as DMA drain) calling qi_submit_sync() with 0 count.
Suppose thread T1 invokes qi_submit_sync() with non-zero descriptors, while
concurrently, thread T2 calls qi_submit_sync() with zero descriptors. Both
threads then enter a while loop, waiting for their respective descriptors
to complete. T1 detects its completion (i.e., T1's invalidation_wait status
changes to QI_DONE by HW) and proceeds to call reclaim_free_desc() to
reclaim all descriptors, potentially including adjacent ones of other
threads that are also marked as QI_DONE.
During this time, while T2 is waiting to acquire the qi->q_lock, the IOMMU
hardware may complete the invalidation for T2, setting its status to
QI_DONE. However, if T1's execution of reclaim_free_desc() frees T2's
invalidation_wait descriptor and changes its status to QI_FREE, T2 will
not observe the QI_DONE status for its invalidation_wait and will
indefinitely remain stuck.
This soft lockup does not occur when only non-zero descriptors are
submitted.In such cases, invalidation descriptors are interspersed among
wait descriptors with the status QI_IN_USE, acting as barriers. These
barriers prevent the reclaim code from mistakenly freeing descriptors
belonging to other submitters.
Considered the following example timeline:
T1 T2
========================================
ID1
WD1
while(WD1!=QI_DONE)
unlock
lock
WD1=QI_DONE* WD2
while(WD2!=QI_DONE)
unlock
lock
WD1==QI_DONE?
ID1=QI_DONE WD2=DONE*
reclaim()
ID1=FREE
WD1=FREE
WD2=FREE
unlock
soft lockup! T2 never sees QI_DONE in WD2
Where:
ID = invalidation descriptor
WD = wait descriptor
* Written by hardware
The root of the problem is that the descriptor status QI_DONE flag is used
for two conflicting purposes:
1. signal a descriptor is ready for reclaim (to be freed)
2. signal by the hardware that a wait descriptor is complete
The solution (in this patch) is state separation by using QI_FREE flag
for #1.
Once a thread's invalidation descriptors are complete, their status would
be set to QI_FREE. The reclaim_free_desc() function would then only
free descriptors marked as QI_FREE instead of those marked as
QI_DONE. This change ensures that T2 (from the previous example) will
correctly observe the completion of its invalidation_wait (marked as
QI_DONE).
Signed-off-by: Sanjay K Kumar <[email protected]>
Signed-off-by: Jacob Pan <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Lu Baolu <[email protected]>
Signed-off-by: Joerg Roedel <[email protected]>
|
|
The static identity domain has been introduced, rendering the si_domain
obsolete. Remove si_domain and cleanup the code accordingly.
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
Software determines VT-d hardware support for passthrough translation by
inspecting the capability register. If passthrough translation is not
supported, the device is instructed to use DMA domain for its default
domain.
Add a global static identity domain with guaranteed attach semantics for
IOMMUs that support passthrough translation mode.
The global static identity domain is a dummy domain without corresponding
dmar_domain structure. Consequently, the device's info->domain will be
NULL with the identity domain is attached. Refactor the code accordingly.
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
Extract common code from domain_context_mapping_one() into new helpers,
making it reusable by other functions such as the upcoming identity domain
implementation. No intentional functional changes.
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Reviewed-by: Jerry Snitselaar <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
The has_iotlb_device flag was used to indicate if a domain had attached
devices with ATS enabled. Domains without this flag didn't require device
TLB invalidation during unmap operations, optimizing performance by
avoiding unnecessary device iteration.
With the introduction of cache tags, this flag is no longer needed. The
code to iterate over attached devices was removed by commit 06792d067989
("iommu/vt-d: Cleanup use of iommu_flush_iotlb_psi()").
Remove has_iotlb_device to avoid unnecessary code.
Suggested-by: Jason Gunthorpe <[email protected]>
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
We will use a global static identity domain. Reserve a static domain ID
for it.
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Reviewed-by: Jerry Snitselaar <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
As the driver has enforced DMA domains for devices managed by an IOMMU
hardware that doesn't support passthrough translation mode, there is no
need for static identity mappings in the si_domain. Remove the identity
mapping code to avoid dead code.
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
The iommu core defines the def_domain_type callback to query the iommu
driver about hardware capability and quirks. The iommu driver should
declare IOMMU_DOMAIN_DMA requirement for hardware lacking pass-through
capability.
Earlier VT-d hardware implementations did not support pass-through
translation mode. The iommu driver relied on a paging domain with all
physical system memory addresses identically mapped to the same IOVA
to simulate pass-through translation before the def_domain_type was
introduced and it has been kept until now. It's time to adjust it now
to make the Intel iommu driver follow the def_domain_type semantics.
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Jason Gunthorpe <[email protected]>
Reviewed-by: Kevin Tian <[email protected]>
Reviewed-by: Jerry Snitselaar <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Joerg Roedel <[email protected]>
|
|
Protect the MMIO access with safe mode.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Protect the MMIO access with safe mode.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Protect the MMIO access with safe mode.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Rather than open coding it for the queue reset.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Rather than open coding it for the queue reset.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Rather than open coding it for the queue reset.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
It's not supported under SR-IOV at the moment.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
It's not supported under SR-IOV at the moment.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
It's not supported under SR-IOV at the moment.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Implement queue reset for graphic and compute queue.
v2: use amdgpu_gfx_rlc funcs to enter/exit safe mode.
v3: use gfx_v11_0_request_gfx_index_mutex()
v4: fix mutex handling
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Jiadong Zhu <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
The reset_queue api could be used from kfd or kgd.
v2: add use_mmio parameter for mes_reset_legacy_queue.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Jiadong Zhu <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Add me/pipe/queue parameters for queue reset input.
v2: fix build (Alex)
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Jiadong Zhu <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Since the MES FW resets kernel compute queue always failed, this
may caused by the KIQ failed to process unmap KCQ. So, before MES
FW work properly that will fallback to driver executes dequeue and
resets SPI directly. Besides, rework the ring reset function and make
the busy ring type reset in each function respectively.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Add ring reset callbacks for gfx and compute.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
To match other GFX IPs.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
There is a racing condition that cp firmware modifies
MQD in reset sequence after driver updates it for
remapping. We have to wait till CP_HQD_ACTIVE becoming
false then remap the queue.
v2: fix KIQ locking (Alex)
v3: fix KIQ locking harder (Jessie)
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Jiadong Zhu <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Kiq command unmap_queues only does the dequeueing action.
We have to map the queue back with clean mqd.
v2: fix up error handling (Alex)
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Jiadong Zhu <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Add ring reset callbacks for gfx and compute.
v2: fix gfx handling
v3: wait for KIQ to complete
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
There is a racing condition that cp firmware modifies
MQD in reset sequence after driver updates it for
remapping. We have to wait till CP_HQD_ACTIVE becoming
false then remap the queue.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Jiadong Zhu <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Rename to gfx_v11_0_kgq_init_queue() to better align with
the other naming in the file.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Since the MES FW resets kernel compute queue always failed, this
may caused by the KIQ failed to process unmap KCQ. So, before MES
FW work properly that will fallback to driver executes dequeue and
resets SPI directly. Besides, rework the ring reset function and make
the busy ring type reset in each function respectively.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Prike Liang <[email protected]>
Reviewed-by: Alex Deucher <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
This version brings along the following:
- DCN35 fixes
- DML2 fixes
- IPS fixes
- ODM fixes
- Miscellaneous cleanups
- MST fixes
- SPL fixes
Acked-by: Aurabindo Pillai <[email protected]>
Signed-off-by: Aric Cyr <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Always allow un-gating. Follow legacy workaround for repeated
dppclk dto updates
Reviewed-by: Muhammad Ahmed <[email protected]>
Signed-off-by: Hansen Dsouza <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
PMO assumes that like timings can be synchronized, but DC only allows
this if the signal types match.
Reviewed-by: Austin Zheng <[email protected]>
Signed-off-by: Dillon Varone <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
[what]
Graphics hang observed with 3 displays connected to DP2.0 mst dock.
[why]
There's a mismatch in dml and dc between the assignments of hpo link
encoders.
[how]
Add a new array in dml that tracks the current mapping of HPO stream
encoders to HPO link encoders in dc.
Reviewed-by: Sung joon Kim <[email protected]>
Reviewed-by: Nicholas Kazlauskas <[email protected]>
Signed-off-by: Gabe Teeger <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
- Add interface for controlling shapness level input into DCN.
- Update SPL to support custom sharpness values.
- Add support for different sharpness values depending on YUV/RGB
content.
Reviewed-by: Samson Tam <[email protected]>
Signed-off-by: Relja Vojvodic <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
This reverts commit f0b7dcf25834afd17df316367dfe5d4c890c713c.
It is causing graphics hangs.
Reviewed-by: Martin Leung <[email protected]>
Signed-off-by: Dillon Varone <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
[Why]
Certain profiles have higher HDR multiplier than SDR boost max which
is not currently supported
[How]
Disable sharpness for these profiles
Fixes: 1b0ce903fe74 ("drm/amd/display: add improvements for text display and HDR DWM and MPO")
Reviewed-by: Martin Leung <[email protected]>
Signed-off-by: Samson Tam <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
[Why]
To provide option to dpia control power management
[How]
By adding disable_usb4_pm_support bit field in dpia_debug option to
control dpia power management
Reviewed-by: Jun Lei <[email protected]>
Signed-off-by: Meenakshikumar Somasundaram <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Add ring reset callbacks for gfx and compute.
Acked-by: Vitaly Prosyak <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
[Why]
Previous disable ODM policy due to underflow issue with sharpener.
Issue is resolved after updating sharpening policy to apply to
both windowed and fullscreen video
[How]
Remove sharpness check disabling Dynamic ODM policy
Reviewed-by: Martin Leung <[email protected]>
Signed-off-by: Samson Tam <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Backlight updates require aux and/or register access. Therefore, driver
needs to disallow IPS beforehand.
So, acquire the dc lock before calling into dc to update backlight - we
should be doing this regardless of IPS. Then, while the lock is held,
disallow IPS before calling into dc, then allow IPS afterwards (if it
was previously allowed).
Reviewed-by: Aurabindo Pillai <[email protected]>
Reviewed-by: Roman Li <[email protected]>
Signed-off-by: Leo Li <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
[Why]
Some asserts are always hit on startup/Pnp when they should only be used
to indicate when something has gone wrong.
[How]
Ignore result of getting function from bios cmd table for newer asics.
Reviewed-by: Jun Lei <[email protected]>
Signed-off-by: Daniel Sa <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
[Why]
Setting min dispclk to 50Mhz outside clock lowering function causes
unnecessary calls to SMU to lower dispclk and causes dentist hangs when
there is no stream on the pipes.
[How]
Move the set minimum dispclk logic inside the lowering dispclk if
statement.
Fixes: 234441320552 ("DCN35 set min dispclk to 50Mhz")
Reviewed-by: Nicholas Kazlauskas <[email protected]>
Signed-off-by: Nicholas Susanto <[email protected]>
Signed-off-by: Hamza Mahfooz <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
Implement the compute pipe reset, and the driver will
fallback to pipe reset when queue reset fails.
The pipe reset only deactivates the queue which is
scheduled in the pipe, and meanwhile the MEC pipe
will be reset to the firmware _start pointer. So,
it seems pipe reset will cost more cycles than the
queue reset; therefore, the driver tries to recover
by doing queue reset first.
Reviewed-by: Lijo Lazar <[email protected]>
Signed-off-by: Prike Liang <[email protected]>
Signed-off-by: Alex Deucher <[email protected]>
|
|
We have for some time the __assign_bit() API to replace
open coded
if (foo)
__set_bit(n, bar);
else
__clear_bit(n, bar);
Use this API to simplify the code. No functional change
intended.
Signed-off-by: Hongbo Li <[email protected]>
Signed-off-by: Mikulas Patocka <[email protected]>
|