Even though commit 8fcb7dfb8b ("drm/xe/xe3p_lpg: Add support for
graphics IP 35.10") mentions that the support for Indirect Ring State
exists for Xe3p_LPG, it missed actually setting the feature flag in
graphics_xe3p_lpg. Fix that by adding the missing member.
Fixes: 8fcb7dfb8b ("drm/xe/xe3p_lpg: Add support for graphics IP 35.10")
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Link: https://patch.msgid.link/20260401-xe3p_lpg-indirect-ring-state-v1-1-0e4b5edf6898@intel.com
Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com>
We only need to convert to picosecond units before writing to RING_IDLEDLY.
Fixes: 7c53ff050b ("drm/xe: Apply Wa_16023105232")
Cc: Tangudu Tilak Tirumalesh <tilak.tirumalesh.tangudu@intel.com>
Acked-by: Tangudu Tilak Tirumalesh <tilak.tirumalesh.tangudu@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Link: https://patch.msgid.link/20260401012710.4165547-1-vinay.belgaumkar@intel.com
Corrects a failure on context switch. On registration of context with
indirect ring state will correct state of targeted and idle command
streamers.
This Wa requires GuC 70.53+ (uapi 1.26+). If indirect_ring_state is
enabled and Wa is unavailable, notify and disable indirect_ring_state.
Added additional XE_RTP_PASTE macros for expansion of longer rules.
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Julia Filipchuk <julia.filipchuk@intel.com>
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Link: https://patch.msgid.link/20260304003431.758201-4-julia.filipchuk@intel.com
Expose multi_lrc_engine_classes in the info debugfs output as a
useful extra piece of information for debugging.
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Signed-off-by: Xin Wang <x.wang@intel.com>
Link: https://patch.msgid.link/20260402180552.24121-3-x.wang@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Improve the readability of the info debugfs output by replacing
raw numeric engine masks with human-readable engine and class
names.
Also print per-GT engine capability data in a form that is easier
for a human to interpret directly, and as a side effect simpler
for IGT tests to use when validating engine capabilities.
Reviewed-by: Niranjana Vishwanathapura <niranjana.vishwanathapura@intel.com>
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Xin Wang <x.wang@intel.com>
Link: https://patch.msgid.link/20260402180552.24121-2-x.wang@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
There appears to have been a silent merge conflict between some commits
updating the workaround tables on Xe's -fixes and -next branches:
- Commit bc6387a2e0 ("drm/xe/xe2_hpg: Fix handling of Wa_14019988906
& Wa_14019877138") from the fixes branch moved the Xe2_HPG instance
of two workarounds touching the PSS_CHICKEN register from the
engine_was[] table to the lrc_was[] table; the equivalent
implementation for all other platforms/IPs were already properly
located on lrc_was[]. This commit on the fixes branch is a
cherry-pick of commit e04c609eed ("drm/xe/xe2_hpg: Fix handling of
Wa_14019988906 & Wa_14019877138") that already existed on the next
branch.
- Commit 55b19abb6c ("drm/xe: Consolidate workaround entries for
Wa_14019877138") and commit c2142a1a84 ("drm/xe: Consolidate
workaround entries for Wa_14019988906") consolidated the individual
entries per IP generation for each workaround into single, larger
range-based entries.
During merge conflict resolution the Xe2_HPG-specific entries (i.e.,
those with rule "GRAPHICS_VERSION_RANGE(2001, 2002)") were accidentally
resurrected, even though the table already contains the consolidated
entries that match a superset of thse ranges. These redundant entries
don't cause any build failures but do trigger a dmesg error during probe
on BMG-G21 devices:
xe 0000:03:00.0: [drm] *ERROR* Tile0: GT0: discarding save-restore reg 7044 (clear: 00000400, set: 00000400, masked: yes, mcr: yes): ret=-22
xe 0000:03:00.0: [drm] *ERROR* Tile0: GT0: discarding save-restore reg 7044 (clear: 00000020, set: 00000020, masked: yes, mcr: yes): ret=-22
Re-drop the Xe2_HPG-specific table entries to eliminate the error.
Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/7433
Fixes: 17b95278ae ("Merge tag 'drm-xe-next-2026-03-02' of https://gitlab.freedesktop.org/drm/xe/kernel into drm-next")
Cc: Dave Airlie <airlied@redhat.com>
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Reviewed-by: Shuicheng Lin <shuicheng.lin@intel.com>
Link: https://patch.msgid.link/20260401-wa_merge_conflict-v1-1-b477ab53fedc@intel.com
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
VMAs marked with VM_IO or VM_PFNMAP are not backed by struct page
objects, which GPUSVM requires in order to operate correctly. In
particular, get_pages() relies on hmm_range_fault() to resolve struct
pages for the target range.
Attempting to create an SVM range on such VMAs results in repeated
get_pages() failures and can lead to an infinite loop inside a driver’s
page‑fault handler. Prevent this by rejecting ranges on VM_IO or
VM_PFNMAP VMAs and returning -EIO.
Fixes: 99624bdff8 ("drm/gpusvm: Add support for GPU Shared Virtual Memory")
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Link: https://patch.msgid.link/20260325231608.25581-1-matthew.brost@intel.com
Add missing pad and extensions check to xe_vm_get_property_ioctl
v2:
- Combine with other check (Auld)
Fixes: 50c577eab0 ("drm/xe/xe_vm: Implement xe_vm_get_property_ioctl")
Suggested-by: Matthew Auld <matthew.auld@intel.com>
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260331181216.37775-2-jonathan.cavitt@intel.com
For Xe2 and beyond we miss printing PAT_ATS register.
Signed-off-by: Zbigniew Kempczyński <zbigniew.kempczynski@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Reviewed-by: Matthew Auld <matthew.auld@intel.com>
Link: https://patch.msgid.link/20260331134330.2535519-2-zbigniew.kempczynski@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Apply WA_14026999295 to following IPs:
Xe3p_XPC
v2:
- Move WA to "Xe3p_XPC" section at bottom of table. (Matt)
Reviewed-by: Matt Roper <matthew.d.roper@intel.com>
Signed-off-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
Link: https://patch.msgid.link/20260327102414.780515-2-mallesh.koujalagi@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
While previously applied only to both tiles GT0(Graphics) and Media(GT1)
the BMG G21(Battle image) platform via workaround Wa_14022085890,
this 1.2 GHz minimum is now the default for GT0(Graphics) tile
of BMG and CRI platforms.
Setting this frequency floor(1.2GHz) default is critical in multi GPU
environment for supporting effective Peer-to-Peer(P2P) transactions.
v2:
- Fix Indentation(Thomas)
- Add comment about power impact(Stuart)
v3:(Thomas/Ankur/Matt Roper)
- Add setting frequency to only GT0(Graphics) Tile of BMG/CRI
v4:(Stuart)
- Move WA check to pc_needs_min_freq_change function
Signed-off-by: Nareshkumar Gollakoti <naresh.kumar.g@intel.com>
Reviewed-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Link: https://patch.msgid.link/20260326063407.985568-4-naresh.kumar.g@intel.com
Signed-off-by: Matt Roper <matthew.d.roper@intel.com>
Using a ',' in place of a ';' can have unintended side effects.
Although that is not the case here, it seems best to use ';'
unless ',' is intended.
Found by inspection.
No functional change intended.
Compile tested only.
Signed-off-by: Chen Ni <nichen@iscas.ac.cn>
Reviewed-by: Shuicheng Lin <shuicheng.lin@intel.com>
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Link: https://patch.msgid.link/20260327032923.649869-1-nichen@iscas.ac.cn
XE_GGTT_FLAGS_64K is defined twice in xe_ggtt.c: once at file scope
(alongside XE_GGTT_FLAGS_ONLINE) and once inside struct xe_ggtt. The
duplicate inside the struct was originally the only definition when the
struct lived in xe_ggtt_types.h, but when commit 4f3a998a17 ("drm/xe:
Open-code GGTT MMIO access protection") added the file-scope definitions
for both FLAGS_64K and FLAGS_ONLINE, it did not remove the pre-existing
one inside the struct.
Remove the redundant definition inside struct xe_ggtt, keeping only the
file-scope pair at the top of the file.
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Zongyao Bai <zongyao.bai@intel.com>
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Link: https://patch.msgid.link/20260316222723.1722591-2-shuicheng.lin@intel.com
Enable System Controller support on CRI by setting has_sysctrl capability
flag in device descriptor and runtime device information.
This allows sysctrl subsystem and mailbox communication to be initialized
on CRI platforms.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260327131837.2192929-16-anoop.c.vijay@intel.com
Add resume handling for System Controller to support system (S3/S4) and
runtime power management transitions.
On resume, restore SoC remapper configuration and mailbox interface to
re‑establish communication with firmware.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Reviewed-by: Badal Nilawar <badal.nilawar@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260327131837.2192929-15-anoop.c.vijay@intel.com
Add mailbox communication layer required for interacting with System
Controller firmware, enabling command submission and response handling.
This patch implements command/response handling logic, including error
and timeout handling.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260327131837.2192929-14-anoop.c.vijay@intel.com
Add initialization and cleanup infrastructure for System Controller
subsystem and integrate it into xe device probe path.
During initialization, platform support is checked via has_sysctrl
capability flag and the mailbox region is configured through SoC
remapper interface.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260327131837.2192929-13-anoop.c.vijay@intel.com
Add ABI definitions, mailbox API, and command data structures required
for System Controller communication.
No functional changes. This patch introduces definitions for mailbox
communication.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260327131837.2192929-12-anoop.c.vijay@intel.com
Add register definitions for System Controller mailbox interface,
including control, data, and protocol-related fields, along with base
address and BAR configuration required by the driver.
No functional changes. This patch introduces register definitions only.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260327131837.2192929-11-anoop.c.vijay@intel.com
Add foundational type definitions for System Controller (sysctrl) support
and integrate them into the xe_device structure. Introduce a capability
flag in device descriptor and runtime information to record sysctrl
presence on supported platforms.
System Controller is a separate firmware-managed entity responsible for
selected platform-level control and coordination tasks on Intel Xe3p
discrete GPU platforms. The driver communicates with it via a mailbox
interface for delegated operations.
This commit introduces core data structures required for sysctrl support,
including MMIO region definitions, a command mutex, and state tracking
required for mailbox communication.
No functional changes. This patch provides preparatory infrastructure
for System Controller support.
Signed-off-by: Anoop Vijay <anoop.c.vijay@intel.com>
Reviewed-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Signed-off-by: Umesh Nerlige Ramappa <umesh.nerlige.ramappa@intel.com>
Link: https://patch.msgid.link/20260327131837.2192929-10-anoop.c.vijay@intel.com
In the context of porting applications to SVM, the Xe GT statistics are
used by application developers to validate expected behavior such as
proper alignment, page fault count and migrations. As those statistics
are made for kernel developers, they assume good understanding of driver
internals, which is not always the case on the application side.
Therefore, this commit documents the usage of GT statistics and clarifies
the meaning of identifiers which correspond to the values exposed via
debugfs. Documentation is close to identifiers declaration to make it
easier to maintain when adding new entries in the future.
v2: Fix page reclaim list (PRL) entries (Matthew Brost)
Assisted-by: GitHub Copilot:claude-sonnet-4.6
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://lore.kernel.org/r/20260327202749.222794-1-francois.dugast@intel.com
Signed-off-by: Francois Dugast <francois.dugast@intel.com>
xe_guc_submit_wedge() runs in the DMA-fence signaling path, where
GFP_KERNEL memory allocations are not permitted. However, registering
guc_submit_wedged_fini via drmm_add_action_or_reset() triggers such an
allocation.
Avoid this by moving the logic from guc_submit_wedged_fini() into
guc_submit_fini(), where wedged exec queue references are dropped during
normal teardown.
Fixes: 8ed9aaae39 ("drm/xe: Force wedged state and block GT reset upon any GPU hang")
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260326210116.202585-3-matthew.brost@intel.com
xe_device_declare_wedged() runs in the DMA-fence signaling path, where
GFP_KERNEL memory allocations are not allowed. However, registering
xe_device_wedged_fini via drmm_add_action_or_reset() triggers a
GFP_KERNEL allocation.
Fix this by deferring the registration of xe_device_wedged_fini until
late in the driver load sequence. Additionally, drop the wedged PM
reference only if the device is actually wedged in
xe_device_wedged_fini.
Fixes: 452bca0edb ("drm/xe: Don't suspend device upon wedge")
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260326210116.202585-2-matthew.brost@intel.com
When an SVM is closed, the garbage collector work item must be stopped
synchronously and any future queuing must be prevented. Replace
flush_work() with disable_work_sync() to ensure both conditions are
met.
Fixes: 63f6e480d1 ("drm/xe: Add SVM garbage collector")
Cc: stable@vger.kernel.org
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Link: https://patch.msgid.link/20260227015225.3081787-1-matthew.brost@intel.com
On PTL, older GSC FWs have a bug that can cause them to crash during
PXP invalidation events, which leads to a complete loss of power
management on the media GT. Therefore, we can't use PXP on FWs that
have this bug, which was fixed in PTL GSC build 1396.
Fixes: b1dcec9bd8 ("drm/xe/ptl: Enable PXP for PTL")
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Julia Filipchuk <julia.filipchuk@intel.com>
Reviewed-by: Julia Filipchuk <julia.filipchuk@intel.com>
Acked-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/20260324153718.3155504-10-daniele.ceraolospurio@intel.com
If we don't clear the flag we'll keep jumping back at the beginning of
the function once we reach the end.
Fixes: ccd3c6820a ("drm/xe/pxp: Decouple queue addition from PXP start")
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Julia Filipchuk <julia.filipchuk@intel.com>
Reviewed-by: Julia Filipchuk <julia.filipchuk@intel.com>
Link: https://patch.msgid.link/20260324153718.3155504-9-daniele.ceraolospurio@intel.com
The default case of the PXP suspend switch is incorrectly exiting
without releasing the lock. However, this case is impossible to hit
because we're switching on an enum and all the valid enum values have
their own cases. Therefore, we can just get rid of the default case
and rely on the compiler to warn us if a new enum value is added and
we forget to add it to the switch.
Fixes: 51462211f4 ("drm/xe/pxp: add PXP PM support")
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Alan Previn Teres Alexis <alan.previn.teres.alexis@intel.com>
Cc: Julia Filipchuk <julia.filipchuk@intel.com>
Reviewed-by: Julia Filipchuk <julia.filipchuk@intel.com>
Link: https://patch.msgid.link/20260324153718.3155504-8-daniele.ceraolospurio@intel.com
If the PXP HW termination fails during PXP start, the normal completion
code won't be called, so the termination will remain uncomplete. To avoid
unnecessary waits, mark the termination as completed from the error path.
Note that we already do this if the termination fails when handling a
termination irq from the HW.
Fixes: f8caa80154 ("drm/xe/pxp: Add PXP queue tracking and session start")
Signed-off-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Cc: Alan Previn Teres Alexis <alan.previn.teres.alexis@intel.com>
Cc: Julia Filipchuk <julia.filipchuk@intel.com>
Reviewed-by: Julia Filipchuk <julia.filipchuk@intel.com>
Link: https://patch.msgid.link/20260324153718.3155504-7-daniele.ceraolospurio@intel.com
Userspace passes canonical (sign-extended) GPU addresses where bits 63:48
mirror bit 47. The internal GPUVM uses non-canonical form (upper bits
zeroed), so passing raw canonical addresses into GPUVM lookups causes
mismatches for addresses above 128TiB.
Strip the sign extension with xe_device_uncanonicalize_addr() at the
top of xe_vm_madvise_ioctl(). Non-canonical addresses are unaffected.
Fixes: ada7486c56 ("drm/xe: Implement madvise ioctl for xe")
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-13-arvind.yadav@intel.com
Hook the madvise_purgeable() handler into the madvise IOCTL now that all
supporting infrastructure is complete:
- Core purge implementation (patch 3)
- BO state tracking and helpers (patches 1-2)
- Per-VMA purgeable state tracking (patch 6)
- Shrinker integration for memory reclamation (patch 10)
This final patch enables userspace to use the
DRM_XE_VMA_ATTR_PURGEABLE_STATE madvise type to mark buffers as
WILLNEED/DONTNEED and receive the retained status indicating whether
buffers were purged.
The feature was kept disabled in earlier patches to maintain
bisectability and ensure all components are in place before exposing to
userspace.
Userspace can detect kernel support for purgeable BOs by checking the
DRM_XE_QUERY_CONFIG_FLAG_HAS_PURGING_SUPPORT flag in the query_config
response.
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-12-arvind.yadav@intel.com
Encapsulate TTM purgeable flag updates and shrinker page accounting
into helper functions to prevent desynchronization between the TTM
tt->purgeable flag and the shrinker's page bucket counters.
Without these helpers, direct manipulation of xe_ttm_tt->purgeable
risks forgetting to update the corresponding shrinker counters,
leading to incorrect memory pressure calculations.
Update purgeable BO state to PURGED after successful shrinker purge
for DONTNEED BOs.
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-11-arvind.yadav@intel.com
Don't allow exporting BOs marked DONTNEED or PURGED as dma-bufs.
DONTNEED BOs can have their contents discarded at any time, making
the exported dma-buf unusable for external devices. PURGED BOs have
no backing store and are permanently invalid.
Return -EBUSY for DONTNEED BOs (temporary purgeable state) and
-EINVAL for purged BOs (permanent, no backing store).
The export path now checks the BO's purgeable state before creating
the dma-buf, preventing external devices from accessing memory that
may be purged at any time.
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-10-arvind.yadav@intel.com
Don't allow new CPU mmaps to BOs marked DONTNEED or PURGED.
DONTNEED BOs can have their contents discarded at any time, making
CPU access undefined behavior. PURGED BOs have no backing store and
are permanently invalid.
Return -EBUSY for DONTNEED BOs (temporary purgeable state) and
-EINVAL for purged BOs (permanent, no backing store).
The mmap offset ioctl now checks the BO's purgeable state before
allowing userspace to establish a new CPU mapping. This prevents
the race where userspace gets a valid offset but the BO is purged
before actual faulting begins.
Existing mmaps (established before DONTNEED) may still work until
pages are purged, at which point CPU faults fail with SIGBUS.
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-9-arvind.yadav@intel.com
Prevent marking imported or exported dma-bufs as purgeable.
External devices may be accessing these buffers without our
knowledge, making purging unsafe.
Check drm_gem_is_imported() for buffers created by other
drivers and obj->dma_buf for buffers exported to other
drivers. Silently skip these BOs during madvise processing.
This follows drm_gem_shmem's purgeable implementation and
prevents data corruption from purging actively-used shared
buffers.
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-8-arvind.yadav@intel.com
Track purgeable state per-VMA instead of using a coarse shared
BO check. This prevents purging shared BOs until all VMAs across
all VMs are marked DONTNEED.
Add xe_bo_all_vmas_dontneed() to check all VMAs before marking
a BO purgeable. Add xe_bo_recheck_purgeable_on_vma_unbind() to
handle state transitions when VMAs are destroyed - if all
remaining VMAs are DONTNEED the BO can become purgeable, or if
no VMAs remain it transitions to WILLNEED.
The per-VMA purgeable_state field stores the madvise hint for
each mapping. Shared BOs can only be purged when all VMAs
unanimously indicate DONTNEED.
This prevents the bug where unmapping the last VMA would incorrectly
flip a DONTNEED BO back to WILLNEED. The enum-based state check
preserves BO state when no VMAs remain, only updating when VMAs provide
explicit hints.
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-7-arvind.yadav@intel.com
Add purge checking to vma_lock_and_validate() to block new mapping
operations on purged BOs while allowing cleanup operations to proceed.
Purged BOs have their backing pages freed by the kernel. New
mapping operations (MAP, PREFETCH, REMAP) must be rejected with
-EINVAL to prevent GPU access to invalid memory. Cleanup
operations (UNMAP) must be allowed so applications can release
resources after detecting purge via the retained field.
REMAP operations require mixed handling - reject new prev/next
VMAs if the BO is purged, but allow the unmap portion to proceed
for cleanup.
The check_purged flag in struct xe_vma_lock_and_validate_flags
distinguishes between these cases: true for new mappings (must reject),
false for cleanup (allow).
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-6-arvind.yadav@intel.com
Block CPU page faults to buffer objects marked as purgeable (DONTNEED)
or already purged. Once a BO is marked DONTNEED, its contents can be
discarded by the kernel at any time, making access undefined behavior.
Return VM_FAULT_SIGBUS immediately to fail consistently instead of
allowing erratic behavior where access sometimes works (if not yet
purged) and sometimes fails (if purged).
For DONTNEED BOs:
- Block new CPU faults with SIGBUS to prevent undefined behavior.
- Existing CPU PTEs may still work until TLB flush, but new faults
fail immediately.
For PURGED BOs:
- Backing store has been reclaimed, making CPU access invalid.
- Without this check, accessing existing mmap mappings would trigger
xe_bo_fault_migrate() on freed backing store, causing kernel hangs
or crashes.
The purgeable check is added to both CPU fault paths:
- Fastpath (xe_bo_cpu_fault_fastpath): Returns VM_FAULT_SIGBUS
immediately under dma-resv lock, preventing attempts to
migrate/validate DONTNEED/purged pages.
- Slowpath (xe_bo_cpu_fault): Returns -EFAULT under drm_exec lock,
converted to VM_FAULT_SIGBUS.
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-5-arvind.yadav@intel.com
This allows userspace applications to provide memory usage hints to
the kernel for better memory management under pressure:
Add the core implementation for purgeable buffer objects, enabling
memory reclamation of user-designated DONTNEED buffers during eviction.
This patch implements the purge operation and state machine transitions:
Purgeable States (from xe_madv_purgeable_state):
- WILLNEED (0): BO should be retained, actively used
- DONTNEED (1): BO eligible for purging, not currently needed
- PURGED (2): BO backing store reclaimed, permanently invalid
Design Rationale:
- Async TLB invalidation via trigger_rebind (no blocking
xe_vm_invalidate_vma)
- i915 compatibility: retained field, "once purged always purged"
semantics
- Shared BO protection prevents multi-process memory corruption
- Scratch PTE reuse avoids new infrastructure, safe for fault mode
Note: The madvise_purgeable() function is implemented but not hooked
into the IOCTL handler (madvise_funcs[] entry is NULL) to maintain
bisectability. The feature will be enabled in the final patch when all
supporting infrastructure (shrinker, per-VMA tracking) is complete.
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-4-arvind.yadav@intel.com
Add infrastructure for tracking purgeable state of buffer objects.
This includes:
Introduce enum xe_madv_purgeable_state with three states:
- XE_MADV_PURGEABLE_WILLNEED (0): BO is needed and should not be
purged. This is the default state for all BOs.
- XE_MADV_PURGEABLE_DONTNEED (1): BO is not currently needed and
can be purged by the kernel under memory pressure to reclaim
resources. Only non-shared BOs can be marked as DONTNEED.
- XE_MADV_PURGEABLE_PURGED (2): BO has been purged by the kernel.
Accessing a purged BO results in error. Follows i915 semantics
where once purged, the BO remains permanently invalid ("once
purged, always purged").
Add madv_purgeable field to struct xe_bo for state tracking
of purgeable state across concurrent access paths
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-3-arvind.yadav@intel.com
Extend the DRM_XE_MADVISE ioctl to support purgeable buffer object
management by adding DRM_XE_VMA_ATTR_PURGEABLE_STATE attribute type.
This allows userspace applications to provide memory usage hints to
the kernel for better memory management under pressure:
- WILLNEED: Buffer is needed and should not be purged. If the BO was
previously purged, retained field returns 0 indicating backing store
was lost (once purged, always purged semantics matching i915).
- DONTNEED: Buffer is not currently needed and may be purged by the
kernel under memory pressure to free resources. Only applies to
non-shared BOs.
To prevent undefined behavior, the following operations are blocked
while a BO is in DONTNEED state:
- New mmap() operations return -EBUSY
- VM_BIND operations return -EBUSY
- New dma-buf exports return -EBUSY
- CPU page faults return SIGBUS
- GPU page faults fail with -EACCES
This ensures applications cannot use a BO while marked as DONTNEED,
preventing erratic behavior when the kernel purges the backing store.
The implementation includes a 'retained' output field (matching i915's
drm_i915_gem_madvise.retained) that indicates whether the BO's backing
store still exists (1) or has been purged (0).
Added DRM_XE_QUERY_CONFIG_FLAG_HAS_PURGING_SUPPORT flag to allow
userspace to detect kernel support for purgeable buffer objects
before attempting to use the feature.
Cc: Matthew Brost <matthew.brost@intel.com>
Acked-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
Signed-off-by: Arvind Yadav <arvind.yadav@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260326130843.3545241-2-arvind.yadav@intel.com
Fix include guard macros that don't match their respective file names:
- xe_gt_idle_types.h: _XE_GT_IDLE_SYSFS_TYPES_H_ -> _XE_GT_IDLE_TYPES_H_
- xe_guc_exec_queue_types.h: _XE_GUC_ENGINE_TYPES_H_ -> _XE_GUC_EXEC_QUEUE_TYPES_H_
- xe_heci_gsc.h: __XE_HECI_GSC_DEV_H__ -> _XE_HECI_GSC_H_
- xe_hw_engine_class_sysfs.h: _XE_ENGINE_CLASS_SYSFS_H_ -> _XE_HW_ENGINE_CLASS_SYSFS_H_
- xe_late_bind_fw_types.h: _XE_LATE_BIND_TYPES_H_ -> _XE_LATE_BIND_FW_TYPES_H_
- xe_platform_types.h: _XE_PLATFORM_INFO_TYPES_H_ -> _XE_PLATFORM_TYPES_H_
- xe_tile_printk.h: _xe_tile_printk_H_ -> _XE_TILE_PRINTK_H_
These guards appear to be leftovers from file renames or copy-paste
errors. Correcting them to follow the standard convention of matching
the file name prevents potential include guard collisions.
No functional change expected.
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Reviewed-by: Nitin Gote <nitin.r.gote@intel.com>
Link: https://patch.msgid.link/20260316160451.1688247-2-shuicheng.lin@intel.com
Async work (e.g., GuC queue teardowns) can call ggtt_node_remove, so the
operation must be performed under the GGTT lock to ensure the GGTT
online check remains stable. GGTT insertion and removal are heavyweight
operations (e.g., queue create/destroy), so the additional serialization
cost is negligible compared to ensuring correctness.
Fixes: 4f3a998a17 ("drm/xe: Open-code GGTT MMIO access protection")
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Dnyaneshwar Bhadane <dnyaneshwar.bhadane@intel.com>
Link: https://patch.msgid.link/20260326011207.62373-1-matthew.brost@intel.com
After setting a local variable, check that local value rather that
checking destination at which the value will be stored later.
This fixes the obvious mistake in error path; without it,
allocation fail would lead to NULL dereference during context
creation.
Fixes: 89340099c6 ("drm/xe/lrc: Refactor context init into xe_lrc_ctx_init()")
Signed-off-by: Tomasz Lis <tomasz.lis@intel.com>
Cc: Raag Jadav <raag.jadav@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260320145733.1337682-1-tomasz.lis@intel.com
Breakdown the GT stats for copy to host and copy to device per size (4K,
64K 2M) to make it easier for user space to track memory migrations.
This is helpful to verify allocation alignment is correct when porting
applications to SVM.
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260325160152.1057556-1-francois.dugast@intel.com
Add support for userspace to request a list of observed faults
from a specified VM.
v2:
- Only allow querying of failed pagefaults (Matt Brost)
v3:
- Remove unnecessary size parameter from helper function, as it
is a property of the arguments. (jcavitt)
- Remove unnecessary copy_from_user (Jainxun)
- Set address_precision to 1 (Jainxun)
- Report max size instead of dynamic size for memory allocation
purposes. Total memory usage is reported separately.
v4:
- Return int from xe_vm_get_property_size (Shuicheng)
- Fix memory leak (Shuicheng)
- Remove unnecessary size variable (jcavitt)
v5:
- Rename ioctl to xe_vm_get_faults_ioctl (jcavitt)
- Update fill_property_pfs to eliminate need for kzalloc (Jianxun)
v6:
- Repair and move fill_faults break condition (Dan Carpenter)
- Free vm after use (jcavitt)
- Combine assertions (jcavitt)
- Expand size check in xe_vm_get_faults_ioctl (jcavitt)
- Remove return mask from fill_faults, as return is already -EFAULT or 0
(jcavitt)
v7:
- Revert back to using xe_vm_get_property_ioctl
- Apply better copy_to_user logic (jcavitt)
v8:
- Fix and clean up error value handling in ioctl (jcavitt)
- Reapply return mask for fill_faults (jcavitt)
v9:
- Future-proof size logic for zero-size properties (jcavitt)
- Add access and fault types (Jianxun)
- Remove address type (Jianxun)
v10:
- Remove unnecessary switch case logic (Raag)
- Compress size get, size validation, and property fill functions into a
single helper function (jcavitt)
- Assert valid size (jcavitt)
v11:
- Remove unnecessary else condition
- Correct backwards helper function size logic (jcavitt)
v12:
- Use size_t instead of int (Raag)
v13:
- Remove engine class and instance (Ivan)
v14:
- Map access type, fault type, and fault level to user macros (Matt
Brost, Ivan)
v15:
- Remove unnecessary size assertion (jcavitt)
v16:
- Nit fixes (Matt Brost)
v17:
- Rebase and refactor (jcavitt)
v18:
- Do not copy_to_user in critical section (Matt Brost)
- Assert args->size is multiple of sizeof(struct xe_vm_fault) (Matt
Brost)
v19:
- Remove unnecessary memset (Matt Brost)
v20:
- Report canonicalized address (Jose)
- Mask out prefetch data from access type (Jose, jcavitt)
v21:
- s/uAPI/Link in the commit log links
- Align debug parameters
Link: https://github.com/intel/compute-runtime/pull/878
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Acked-by: Michal Mrozek <michal.mrozek@intel.com>
Cc: Jainxun Zhang <jianxun.zhang@intel.com>
Cc: Shuicheng Lin <shuicheng.lin@intel.com>
Cc: Raag Jadav <raag.jadav@intel.com>
Cc: Ivan Briano <ivan.briano@intel.com>
Cc: Jose Souza <jose.souza@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260324152935.72444-10-jonathan.cavitt@intel.com
Add additional information to each VM so they can report up to the first
50 seen faults. Only pagefaults are saved this way currently, though in
the future, all faults should be tracked by the VM for future reporting.
Additionally, of the pagefaults reported, only failed pagefaults are
saved this way, as successful pagefaults should recover silently and not
need to be reported to userspace.
v2:
- Free vm after use (Shuicheng)
- Compress pf copy logic (Shuicheng)
- Update fault_unsuccessful before storing (Shuicheng)
- Fix old struct name in comments (Shuicheng)
- Keep first 50 pagefaults instead of last 50 (Jianxun)
v3:
- Avoid unnecessary execution by checking MAX_PFS earlier (jcavitt)
- Fix double-locking error (jcavitt)
- Assert kmemdump is successful (Shuicheng)
v4:
- Rename xe_vm.pfs to xe_vm.faults (jcavitt)
- Store fault data and not pagefault in xe_vm faults list (jcavitt)
- Store address, address type, and address precision per fault (jcavitt)
- Store engine class and instance data per fault (Jianxun)
- Add and fix kernel docs (Michal W)
- Properly handle kzalloc error (Michal W)
- s/MAX_PFS/MAX_FAULTS_SAVED_PER_VM (Michal W)
- Store fault level per fault (Micahl M)
v5:
- Store fault and access type instead of address type (Jianxun)
v6:
- Store pagefaults in non-fault-mode VMs as well (Jianxun)
v7:
- Fix kernel docs and comments (Michal W)
v8:
- Fix double-locking issue (Jianxun)
v9:
- Do not report faults from reserved engines (Jianxun)
v10:
- Remove engine class and instance (Ivan)
v11:
- Perform kzalloc outside of lock (Auld)
v12:
- Fix xe_vm_fault_entry kernel docs (Shuicheng)
v13:
- Rebase and refactor (jcavitt)
v14:
- Correctly ignore fault mode in save_pagefault_to_vm (jcavitt)
v15:
- s/save_pagefault_to_vm/xe_pagefault_save_to_vm (Matt Brost)
- Use guard instead of spin_lock/unlock (Matt Brost)
- GT was added to xe_pagefault struct. Use xe_gt_hw_engine
instead of creating a new helper function (Matt Brost)
v16:
- Set address precision programmatically (Matt Brost)
v17:
- Set address precision to fixed value (Matt Brost)
v18:
- s/uAPI/Link in commit log links
- Use kzalloc_obj
Link: https://github.com/intel/compute-runtime/pull/878
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Cc: Shuicheng Lin <shuicheng.lin@intel.com>
Cc: Jianxun Zhang <jianxun.zhang@intel.com>
Cc: Michal Wajdeczko <Michal.Wajdeczko@intel.com>
Cc: Michal Mzorek <michal.mzorek@intel.com>
Cc: Ivan Briano <ivan.briano@intel.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260324152935.72444-9-jonathan.cavitt@intel.com
Add initial declarations for the drm_xe_vm_get_property ioctl.
v2:
- Expand kernel docs for drm_xe_vm_get_property (Jianxun)
v3:
- Remove address type external definitions (Jianxun)
- Add fault type to xe_drm_fault struct (Jianxun)
v4:
- Remove engine class and instance (Ivan)
v5:
- Add declares for fault type, access type, and fault level (Matt Brost,
Ivan)
v6:
- Fix inconsistent use of whitespace in defines
v7:
- Rebase and refactor (jcavitt)
v8:
- Rebase (jcavitt)
v9:
- Clarify address is canonical (José)
v10:
- s/uAPI/Link in the commit log links
Link: https://github.com/intel/compute-runtime/pull/878
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Reviewed-by: Shuicheng Lin <shuicheng.lin@intel.com>
Acked-by: Matthew Brost <matthew.brost@intel.com>
Acked-by: Ivan Briano <ivan.briano@intel.com>
Acked-by: José Roberto de Souza <jose.souza@intel.com>
Cc: Zhang Jianxun <jianxun.zhang@intel.com>
Cc: Ivan Briano <ivan.briano@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260324152935.72444-8-jonathan.cavitt@intel.com
The page fault handler should reject write/atomic access to read only
VMAs. Add code to handle this in xe_pagefault_service after the VMA
lookup.
v2:
- Apply max line length (Matthew)
Fixes: fb544b8445 ("drm/xe: Implement xe_pagefault_queue_work")
Signed-off-by: Jonathan Cavitt <jonathan.cavitt@intel.com>
Suggested-by: Matthew Brost <matthew.brost@intel.com>
Cc: Shuicheng Lin <shuicheng.lin@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260324152935.72444-7-jonathan.cavitt@intel.com
