github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-12 20:54:48 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

drm/xe/ggtt: invalidate engine GGTT TLBs for multi-queue GTs

Browse Source 
Engines cache GGTT translations in their own TLBs, which the
xe_tlb_inval_ggtt() does not touch.
After a GGTT unmap the engines can still hit the stale translation and
read the old page.

Issue an INVAL_FULL (intra vf) invalidation on GGTT unmap, gated on the
GT having any multi-queue engine class.

v2:
- Use xe_gt_has_multi_queue() helper and address nits (Niranjana)
v3:
- Invalidate both the primary and media GTs, similar to
  xe_ggtt_invalidate() (Matt Brost)
v5:
- Guard NULL gt and explain the multi-queue-only engine flush
  (Niranjana)

Assisted-by: Claude:claude-opus-4-8
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Tangudu Tilak Tirumalesh <tilak.tirumalesh.tangudu@intel.com>
Link: https://patch.msgid.link/20260827082011.3370672-4-tilak.tirumalesh.tangudu@intel.com
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
This commit is contained in:
Tangudu Tilak Tirumalesh 2026-08-27 13:50:11 +05:30 committed by Shuicheng Lin
parent 0f0a5aba87
commit d3f1b058a7
1 changed files with 33 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
drivers/gpu/drm/xe/xe_ggtt.c
View File

@ -451,6 +451,7 @@ int xe_ggtt_init_early(struct xe_ggtt *ggtt)
ALLOW_ERROR_INJECTION(xe_ggtt_init_early, ERRNO); /* See xe_pci_probe() */
static void xe_ggtt_invalidate(struct xe_ggtt *ggtt);
static void xe_ggtt_invalidate_engine(struct xe_ggtt *ggtt);
static void xe_ggtt_initial_clear(struct xe_ggtt *ggtt)
{
@ -482,8 +483,11 @@ static void ggtt_node_remove(struct xe_ggtt_node *node)
xe_ggtt_clear(ggtt, xe_ggtt_node_addr(node), xe_ggtt_node_size(node));
drm_mm_remove_node(&node->base);
node->base.size = 0;
if (bound && node->invalidate_on_remove)
if (bound && node->invalidate_on_remove) {
xe_ggtt_invalidate(ggtt);
/* Drain engine TLBs so a recycled range can't hit a stale entry. */
xe_ggtt_invalidate_engine(ggtt);
}
mutex_unlock(&ggtt->lock);
ggtt_node_fini(node);
@ -601,6 +605,34 @@ static void xe_ggtt_invalidate(struct xe_ggtt *ggtt)
ggtt_invalidate_gt_tlb(ggtt->tile->media_gt);
}
static void ggtt_invalidate_gt_engine_tlb(struct xe_gt *gt)
{
int err;
/*
* Multi-queue GTs miss engine GGTT TLB invalidations (GuC
* lite-restore skips the full context restore), so the engine
* flush is needed there.
*/
if (!gt || !xe_gt_has_multi_queue(gt))
return;
err = xe_tlb_inval_ggtt_full(&gt->tlb_inval);
xe_gt_WARN(gt, err, "Failed to invalidate engine GGTT TLBs (%pe)",
ERR_PTR(err));
}
/*
* Drain engine-side GGTT TLBs on teardown so a recycled range's next
* occupant can't hit a predecessor's cached translation.
*/
static void xe_ggtt_invalidate_engine(struct xe_ggtt *ggtt)
{
/* Each GT in a tile has its own engine TLBs to drain */
ggtt_invalidate_gt_engine_tlb(ggtt->tile->primary_gt);
ggtt_invalidate_gt_engine_tlb(ggtt->tile->media_gt);
}
/**
* xe_ggtt_shift_nodes() - Shift GGTT nodes to adjust for a change in usable address range.
* @ggtt: the &xe_ggtt struct instance