mirror of
https://github.com/torvalds/linux.git
synced 2026-05-30 10:04:04 +02:00
master
34 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Francois Dugast
|
192cb1f5cb |
drm/pagemap: Enable THP support for GPU memory migration
This enables support for Transparent Huge Pages (THP) for device pages by
using MIGRATE_VMA_SELECT_COMPOUND during migration. It removes the need to
split folios and loop multiple times over all pages to perform required
operations at page level. Instead, we rely on newly introduced support for
higher orders in drm_pagemap and folio-level API.
In Xe, this drastically improves performance when using SVM. The GT stats
below collected after a 2MB page fault show overall servicing is more than
7 times faster, and thanks to reduced CPU overhead the time spent on the
actual copy goes from 23% without THP to 80% with THP:
Without THP:
svm_2M_pagefault_us: 966
svm_2M_migrate_us: 942
svm_2M_device_copy_us: 223
svm_2M_get_pages_us: 9
svm_2M_bind_us: 10
With THP:
svm_2M_pagefault_us: 132
svm_2M_migrate_us: 128
svm_2M_device_copy_us: 106
svm_2M_get_pages_us: 1
svm_2M_bind_us: 2
v2:
- Fix one occurrence of drm_pagemap_get_devmem_page() (Matthew Brost)
v3:
- Remove migrate_device_split_page() and folio_split_lock, instead rely on
free_zone_device_folio() to split folios before freeing (Matthew Brost)
- Assert folio order is HPAGE_PMD_ORDER (Matthew Brost)
- Always use folio_set_zone_device_data() in split (Matthew Brost)
v4:
- Warn on compound device page, s/continue/goto next/ (Matthew Brost)
v5:
- Revert warn on compound device page
- s/zone_device_page_init()/zone_device_folio_init() (Matthew Brost)
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Michal Mrozek <michal.mrozek@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com>
Cc: Liam R. Howlett <Liam.Howlett@oracle.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Balbir Singh <balbirs@nvidia.com>
Cc: linux-mm@kvack.org
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Francois Dugast <francois.dugast@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patch.msgid.link/20260312192126.2024853-5-francois.dugast@intel.com
|
||
|
Matthew Brost
|
139ab31aea |
drm/pagemap: Correct cpages calculation for migrate_vma_setup
cpages returned from migrate_vma_setup represents the total number of individual pages found, not the number of 4K pages. The math in drm_pagemap_migrate_to_devmem for npages is based on the number of 4K pages, so cpages != npages can fail even if the entire memory range is found in migrate_vma_setup (e.g., when a single 2M page is found). Add drm_pagemap_cpages, which converts cpages to the number of 4K pages found. Cc: Andrew Morton <akpm@linux-foundation.org> Cc: David Hildenbrand <david@kernel.org> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Cc: Liam R. Howlett <Liam.Howlett@oracle.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Zi Yan <ziy@nvidia.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Balbir Singh <balbirs@nvidia.com> Cc: linux-mm@kvack.org Reviewed-by: Francois Dugast <francois.dugast@intel.com> Signed-off-by: Francois Dugast <francois.dugast@intel.com> Reviewed-by: Balbir Singh <balbirs@nvidia.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://patch.msgid.link/20260312192126.2024853-4-francois.dugast@intel.com |
||
|
Francois Dugast
|
2e03c0c5c5 |
drm/pagemap: Add helper to access zone_device_data
This new helper helps ensure all accesses to zone_device_data use the correct API whether the page is part of a folio or not. v2: - Move to drm_pagemap.h, stick to folio_zone_device_data (Matthew Brost) - Return struct drm_pagemap_zdd * (Matthew Brost) v3: - Add stub for !CONFIG_ZONE_DEVICE (CI) Cc: Andrew Morton <akpm@linux-foundation.org> Cc: David Hildenbrand <david@kernel.org> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Cc: Liam R. Howlett <Liam.Howlett@oracle.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Zi Yan <ziy@nvidia.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Balbir Singh <balbirs@nvidia.com> Cc: linux-mm@kvack.org Suggested-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Francois Dugast <francois.dugast@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://patch.msgid.link/20260312192126.2024853-3-francois.dugast@intel.com |
||
|
Francois Dugast
|
440ec190c2 |
drm/pagemap: Unlock and put folios when possible
If the page is part of a folio, unlock and put the whole folio at once instead of individual pages one after the other. This will reduce the amount of operations once device THP are in use. Cc: Andrew Morton <akpm@linux-foundation.org> Cc: David Hildenbrand <david@kernel.org> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Cc: Liam R. Howlett <Liam.Howlett@oracle.com> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Zi Yan <ziy@nvidia.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Balbir Singh <balbirs@nvidia.com> Cc: linux-mm@kvack.org Suggested-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Francois Dugast <francois.dugast@intel.com> Reviewed-by: Balbir Singh <balbirs@nvidia.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://patch.msgid.link/20260312192126.2024853-2-francois.dugast@intel.com |
||
|
Dave Airlie
|
17b95278ae |
Merge tag 'drm-xe-next-2026-03-02' of https://gitlab.freedesktop.org/drm/xe/kernel into drm-next
UAPI Changes: - restrict multi-lrc to VCS/VECS engines (Xin Wang) - Introduce a flag to disallow vm overcommit in fault mode (Thomas) - update used tracking kernel-doc (Auld, Fixes) - Some bind queue fixes (Auld, Fixes) Cross-subsystem Changes: - Split drm_suballoc_new() into SA alloc and init helpers (Satya, Fixes) - pass pagemap_addr by reference (Arnd, Fixes) - Revert "drm/pagemap: Disable device-to-device migration" (Thomas) - Fix unbalanced unlock in drm_gpusvm_scan_mm (Maciej, Fixes) - Small GPUSVM fixes (Brost, Fixes) - Fix xe SVM configs (Thomas, Fixes) Core Changes: - Fix a hmm_range_fault() livelock / starvation problem (Thomas, Fixes) Driver Changes: - Fix leak on xa_store failure (Shuicheng, Fixes) - Correct implementation of Wa_16025250150 (Roper, Fixes) - Refactor context init into xe_lrc_ctx_init (Raag) - Fix GSC proxy cleanup on early initialization failure (Zhanjun) - Fix exec queue creation during post-migration recovery (Tomasz, Fixes) - Apply windower hardware filtering setting on Xe3 and Xe3p (Roper) - Free ctx_restore_mid_bb in release (Shuicheng, Fixes) - Drop stale MCR steering TODO comment (Roper) - dGPU memory optimizations (Brost) - Do not preempt fence signaling CS instructions (Brost, Fixes) - Revert "drm/xe/compat: Remove unused i915_reg.h from compat header" (Uma) - Don't expose display modparam if no display support (Wajdeczko) - Some VRAM flag improvements (Wajdeczko) - Misc fix for xe_guc_ct.c (Shuicheng, Fixes) - Remove unused i915_reg.h from compat header (Uma) - Workaround cleanup & simplification (Roper) - Add prefetch pagefault support for Xe3p (Varun) - Fix fs_reclaim deadlock caused by CCS save/restore (Satya, Fixes) - Cleanup partially initialized sync on parse failure (Shuicheng, Fixes) - Allow to change VFs VRAM quota using sysfs (Michal) - Increase GuC log sizes in debug builds (Tomasz) - Wa_18041344222 changes (Harish) - Add Wa_14026781792 (Niton) - Add debugfs facility to catch RTP mistakes (Roper) - Convert GT stats to per-cpu counters (Brost) - Prevent unintended VRAM channel creation (Karthik) - Privatize struct xe_ggtt (Maarten) - remove unnecessary struct dram_info forward declaration (Jani) - pagefault refactors (Brost) - Apply Wa_14024997852 (Arvind) - Redirect faults to dummy page for wedged device (Raag, Fixes) - Force EXEC_QUEUE_FLAG_KERNEL for kernel internal VMs (Piotr) - Stop applying Wa_16018737384 from Xe3 onward (Roper) - Add new XeCore fuse registers to VF runtime regs (Roper) - Update xe_device_declare_wedged() error log (Raag) - Make xe_modparam.force_vram_bar_size signed (Shuicheng, Fixes) - Avoid reading media version when media GT is disabled (Piotr, Fixes) - Fix handling of Wa_14019988906 & Wa_14019877138 (Roper, Fixes) - Basic enabling patches for Xe3p_LPG and NVL-P (Gustavo, Roper, Shekhar) - Avoid double-adjust in 64-bit reads (Shuicheng, Fixes) - Allow VF to initialize MCR tables (Wajdeczko) - Add Wa_14025883347 for GuC DMA failure on reset (Anirban) - Add bounds check on pat_index to prevent OOB kernel read in madvise (Jia, Fixes) - Fix the address range assert in ggtt_get_pte helper (Winiarski) - XeCore fuse register changes (Roper) - Add more info to powergate_info debugfs (Vinay) - Separate out GuC RC code (Vinay) - Fix g2g_test_array indexing (Pallavi) - Mutual exclusivity between CCS-mode and PF (Nareshkumar, Fixes) - Some more _types.h cleanups (Wajdeczko) - Fix sysfs initialization (Wajdeczko, Fixes) - Drop unnecessary goto in xe_device_create (Roper) - Disable D3Cold for BMG only on specific platforms (Karthik, Fixes) - Add sriov.admin_only_pf attribute (Wajdeczko) - replace old wq(s), add WQ_PERCPU to alloc_workqueue (Marco) - Make MMIO communication more robust (Wajdeczko) - Fix warning of kerneldoc (Shuicheng, Fixes) - Fix topology query pointer advance (Shuicheng, Fixes) - use entry_dump callbacks for xe2+ PAT dumps (Xin Wang) - Fix kernel-doc warning in GuC scheduler ABI header (Chaitanya, Fixes) - Fix CFI violation in debugfs access (Daniele, Fixes) - Apply WA_16028005424 to Media (Balasubramani) - Fix typo in function kernel-doc (Wajdeczko) - Protect priority against concurrent access (Niranjana) - Fix nvm aux resource cleanup (Shuicheng, Fixes) - Fix is_bound() pci_dev lifetime (Shuicheng, Fixes) - Use CLASS() for forcewake in xe_gt_enable_comp_1wcoh (Shuicheng) - Reset VF GuC state on fini (Wajdeczko) - Move _THIS_IP_ usage from xe_vm_create() to dedicated function (Nathan Chancellor, Fixes) - Unregister drm device on probe error (Shuicheng, Fixes) - Disable DCC on PTL (Vinay, Fixes) - Fix Wa_18022495364 (Tvrtko, Fixes) - Skip address copy for sync-only execs (Shuicheng, Fixes) - derive mem copy capability from graphics version (Nitin, Fixes) - Use DRM_BUDDY_CONTIGUOUS_ALLOCATION for contiguous allocations (Sanjay) - Context based TLB invalidations (Brost) - Enable multi_queue on xe3p_xpc (Brost, Niranjana) - Remove check for gt in xe_query (Nakshtra) - Reduce LRC timestamp stuck message on VFs to notice (Brost, Fixes) Signed-off-by: Dave Airlie <airlied@redhat.com> From: Matthew Brost <matthew.brost@intel.com> Link: https://patch.msgid.link/aaYR5G2MHjOEMXPW@lstrano-desk.jf.intel.com |
||
|
Linus Torvalds
|
bf4afc53b7 |
Convert 'alloc_obj' family to use the new default GFP_KERNEL argument
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
||
|
Kees Cook
|
69050f8d6d |
treewide: Replace kmalloc with kmalloc_obj for non-scalar types
This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook <kees@kernel.org> |
||
|
Linus Torvalds
|
d4a292c5f8 |
drm next fixes for 7.0-rc1
pagemap: - drm/pagemap: pass pagemap_addr by reference amdgpu: - DML 2.1 fixes - Panel replay fixes - Display writeback fixes - MES 11 old firmware compat fix - DC CRC improvements - DPIA fixes - XGMI fixes - ASPM fix - SMU feature bit handling fixes - DC LUT fixes - RAS fixes - Misc memory leak in error path fixes - SDMA queue reset fixes - PG handling fixes - 5 level GPUVM page table fix - SR-IOV fix - Queue reset fix - SMU 13.x fixes - DC resume lag fix - MPO fixes - DCN 3.6 fix - VSDB fixes - HWSS clean up - Replay fixes - DCE cursor fixes - DCN 3.5 SR DDR5 latency fixes - HPD fixes - Error path unwind fixes - SMU13/14 mode1 reset fixes - PSP 15 updates - SMU 15 updates - Sync fix in amdgpu_dma_buf_move_notify() - HAINAN fix - PSP 13.x fix - GPUVM locking fix - Fixes for DC analog support - DC FAMS fixes - DML 2.1 fixes - eDP fixes - Misc DC fixes - Fastboot fix - 3DLUT fixes - GPUVM fixes - 64bpp format fix - Fix for MacBooks with switchable gfx amdkfd: - Fix possible double deletion of validate list - Event setup fix - Device disconnect regression fix - APU GTT as VRAM fix - Fix piority inversion with MQDs - NULL check fix radeon: - HAINAN fix i915/xe display: - Regresion fix for HDR 4k displays (#15503) - Fixup for Dell XPS 13 7390 eDP rate limit - Memory leak fix on ACPI _DSM handling - Add missing slice count check during DP mode validation xe: - drm/xe: Prevent VFs from exposing the CCS mode sysfs file - SRIOV related fixes - PAT cache fix - MMIO read fix - W/a fixes - Adjust type of xe_modparam.force_vram_bar_size - Wedge mode fix - HWMon fix -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEEKbZHaGwW9KfbeusDHTzWXnEhr4FAmmYyNEACgkQDHTzWXnE hr7n0Q/+L4KscEx0ByCOBNfWJollaJKZEpdeWLQ2J9RSa/hPVOTE+k2z/TokCsYw jYIHCv8UBU/VvffWWHj0CHCRSRFcfwWo9bZm0J0C6vRSL77iXJPUDZOZ/EhP7Cy3 oMZyexBcVTwNqzI2cg8PYsGkTMPpdBUxrSNxa9d15kNJbMSm5RdAMJ4pt/9Id9V4 iCGQccsNZN8d9JdPzYU/Tay2RQOD/75l9xtywq4vb8Ktszc7gWFca+tRleYZxbtG WuXvMStqmZAUZJaTbSASbSQFQMEdyJXSsFq/T0cY1Mm5DSUqnJ27YqOzL5sqzQTE +Aeh5w5xEgq/GaLzdx9tOSxBr6mK7p251RApSunPn4nwb50iT8dBAqaYh8Zy2e0o vQtFgp3PLlnqwvdvEtqPoq6+oG/FtIsULLPLuMqlZtOe3EG7BzQsWeWASj793EtN KrTu4a9HufzZrf+t8a3ZLp2CMQJI5sCCBZBJiZ6/ImiixBEH5bJzSLPhdx7VeoMe //kVBbhYXD8oi8QMcJHdZg9ERp80D1yYJhhGWes330s1tdl87Gizy4Mu4pSH9Mds bw7u6PTS7iI7hoXvz/ITrrHkFTkxLsxVhPI2OhLAfRpIEoH9XOFgi+BCmZOEb9lT V5RhIDxrKpjQcgJ9d9dAM2arO11kTPdVTEH4tYpzQ5cHd17mls4= =SQjl -----END PGP SIGNATURE----- Merge tag 'drm-next-2026-02-21' of https://gitlab.freedesktop.org/drm/kernel Pull drm fixes from Dave Airlie: "This is the fixes and cleanups for the end of the merge window, it's nearly all amdgpu, with some amdkfd, then a pagemap core fix, i915/xe display fixes, and some xe driver fixes. Nothing seems out of the ordinary, except amdgpu is a little more volume than usual. pagemap: - drm/pagemap: pass pagemap_addr by reference amdgpu: - DML 2.1 fixes - Panel replay fixes - Display writeback fixes - MES 11 old firmware compat fix - DC CRC improvements - DPIA fixes - XGMI fixes - ASPM fix - SMU feature bit handling fixes - DC LUT fixes - RAS fixes - Misc memory leak in error path fixes - SDMA queue reset fixes - PG handling fixes - 5 level GPUVM page table fix - SR-IOV fix - Queue reset fix - SMU 13.x fixes - DC resume lag fix - MPO fixes - DCN 3.6 fix - VSDB fixes - HWSS clean up - Replay fixes - DCE cursor fixes - DCN 3.5 SR DDR5 latency fixes - HPD fixes - Error path unwind fixes - SMU13/14 mode1 reset fixes - PSP 15 updates - SMU 15 updates - Sync fix in amdgpu_dma_buf_move_notify() - HAINAN fix - PSP 13.x fix - GPUVM locking fix - Fixes for DC analog support - DC FAMS fixes - DML 2.1 fixes - eDP fixes - Misc DC fixes - Fastboot fix - 3DLUT fixes - GPUVM fixes - 64bpp format fix - Fix for MacBooks with switchable gfx amdkfd: - Fix possible double deletion of validate list - Event setup fix - Device disconnect regression fix - APU GTT as VRAM fix - Fix piority inversion with MQDs - NULL check fix radeon: - HAINAN fix i915/xe display: - Regresion fix for HDR 4k displays (#15503) - Fixup for Dell XPS 13 7390 eDP rate limit - Memory leak fix on ACPI _DSM handling - Add missing slice count check during DP mode validation xe: - drm/xe: Prevent VFs from exposing the CCS mode sysfs file - SRIOV related fixes - PAT cache fix - MMIO read fix - W/a fixes - Adjust type of xe_modparam.force_vram_bar_size - Wedge mode fix - HWMon fix * tag 'drm-next-2026-02-21' of https://gitlab.freedesktop.org/drm/kernel: (143 commits) drm/amd/display: Remove unneeded DAC link encoder register drm/amd/display: Enable DAC in DCE link encoder drm/amd/display: Set CRTC source for DAC using registers drm/amd/display: Initialize DAC in DCE link encoder using VBIOS drm/amd/display: Turn off DAC in DCE link encoder using VBIOS drm/amd/display: Don't call find_analog_engine() twice drm/amdgpu: fix 4-level paging if GMC supports 57-bit VA v2 drm/amdgpu: keep vga memory on MacBooks with switchable graphics drm/amdgpu: Set atomics to true for xgmi drm/amdkfd: Check for NULL return values drm/amd/display: Use same max plane scaling limits for all 64 bpp formats drm/amdgpu: Set vmid0 PAGE_TABLE_DEPTH for GFX12.1 drm/amdkfd: Disable MQD queue priority drm/amd/display: Remove conditional for shaper 3DLUT power-on drm/amd/display: Check return of shaper curve to HW format drm/amd/display: Correct logic check error for fastboot drm/amd/display: Skip eDP detection when no sink Revert "drm/amd/display: Add Gfx Base Case For Linear Tiling Handling" Revert "drm/amd/display: Correct hubp GfxVersion verification" Revert "drm/amd/display: Add Handling for gfxversion DcGfxBase" ... |
||
|
Arnd Bergmann
|
b61d565166
|
drm/pagemap: pass pagemap_addr by reference
Passing a structure by value into a function is sometimes problematic,
for a number of reasons. Of of these is a warning from the 32-bit arm
compiler:
drivers/gpu/drm/drm_gpusvm.c: In function '__drm_gpusvm_unmap_pages':
drivers/gpu/drm/drm_gpusvm.c:1152:33: note: parameter passing for argument of type 'struct drm_pagemap_addr' changed in GCC 9.1
1152 | dpagemap->ops->device_unmap(dpagemap,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1153 | dev, *addr);
| ~~~~~~~~~~~
This particular problem is harmless since we are not mixing compiler versions
inside of the compiler. However, passing this by reference avoids the warning
along with providing slightly better calling conventions as it avoids an
extra copy on the stack.
Fixes:
|
||
|
Arnd Bergmann
|
95162db020 |
drm/pagemap: pass pagemap_addr by reference
Passing a structure by value into a function is sometimes problematic,
for a number of reasons. Of of these is a warning from the 32-bit arm
compiler:
drivers/gpu/drm/drm_gpusvm.c: In function '__drm_gpusvm_unmap_pages':
drivers/gpu/drm/drm_gpusvm.c:1152:33: note: parameter passing for argument of type 'struct drm_pagemap_addr' changed in GCC 9.1
1152 | dpagemap->ops->device_unmap(dpagemap,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1153 | dev, *addr);
| ~~~~~~~~~~~
This particular problem is harmless since we are not mixing compiler versions
inside of the compiler. However, passing this by reference avoids the warning
along with providing slightly better calling conventions as it avoids an
extra copy on the stack.
Fixes:
|
||
|
Thomas Hellström
|
1a3c0049b3 |
Revert "drm/pagemap: Disable device-to-device migration"
With commit |
||
|
Linus Torvalds
|
939faf71cf |
drm for 7.0-rc1
core:
- drop kgdb support
- replace system workqueue with percpu
- account for property blobs in memcg
- MAINTAINERS updates for xe + buddy
rust:
- Fix documentation for Registration constructors.
- Use pin_init::zeroed() for fops initialization.
- Annotate DRM helpers with __rust_helper.
- Improve safety documentation for gem::Object::new().
- Update AlwaysRefCounted imports.
- mm: Prevent integer overflow in page_align().
atomic:
- add drm_device pointer to drm_private_obj
- introduce gamma/degamma LUT size check
buddy:
- fix free_trees memory leak
- prevent BUG_ON
bridge:
- introduce drm_bridge_unplug/enter/exit
- add connector argument to .hpd_notify
- lots of recounting conversions
- convert rockchip inno hdmi to bridge
- lontium-lt9611uxc: switch to HDMI audio helpers
- dw-hdmi-qp: add support for HPD-less setups
- Algoltek AG6311 support
panels:
- edp: CSW MNE007QB3-1, AUO B140HAN06.4, AUO B140QAX01.H
- st75751: add SPI support
- Sitronix ST7920, Samsung LTL106HL02
- LG LH546WF1-ED01, HannStar HSD156J
- BOE NV130WUM-T08
- Innolux G150XGE-L05
- Anbernic RG-DS
dma-buf:
- improve sg_table debugging
- add tracepoints
- call clear_page instead of memset
- start to introduce cgroup memory accounting in heaps
- remove sysfs stats
dma-fence:
- add new helpers
dp:
- mst: avoid oob access with vcpi=0
hdmi:
- limit infoframes exposure to userspace
gem:
- reduce page table overhead with THP
- fix leak in drm_gem_get_unmapped_area
gpuvm:
- API sanitation for rust bindings
sched:
- introduce new helpers
panic:
- report invalid panic modes
- add kunit tests
i915/xe display:
- Expose sharpness only if num_scalers is >= 2
- Add initial Xe3P_LPD for NVL
- BMG FBC support
- Add MTL+ platforms to support dpll framework
_ fix DIMM_S DRM decoding on ICL
- Return to using AUX interrupts
- PSR/Panel replay refactoring
- use consolidation HDMI tables
- Xe3_LPD CD2X dividier changes
xe:
- vfio: add vfio_pci for intel GPU
- multi queue support
- dynamic pagemaps and multi-device SVM
- expose temp attribs in hwmon
- NO_COMPRESSION bo flag
- expose MERT OA unit
- sysfs survivability refactor
- SRIOV PF: add MERT support
- enable SR-IOV VF migration
- Enable I2C/NVM on Crescent Island
- Xe3p page reclaimation support
- introduce SRIOV scheduler groups
- add SoC remappt support in system controller
- insert compiler barriers in GuC code
- define NVL GuC firmware
- handle GT resume failure
- fix drm scheduler layering violations
- enable GSC loading and PXP for PTL
- disable GuC Power DCC strategy on PTL
- unregister drm device on probe error
i915:
- move to kernel standard fault injection
- bump recommended GuC version for DG2 and MTL
amdgpu:
- SMUIO 15.x, PSP 15.x support
- IH 6.1.1/7.1 support
- MMHUB 3.4/4.2 support
- GC 11.5.4/12.1 support
- SDMA 6.1.4/7.1/7.11.4 support
- JPEG 5.3 support
- UserQ updates
- GC 9 gfx queue reset support
- TTM memory ops parallelization
- convert legacy logging to new helpers
- DC analog fixes
amdkfd:
- GC 11.5.4/12.1 suppport
- SDMA 6.1.4/7.1 support
- per context support
- increase kfd process hash table
- Reserved SDMA rework
radeon:
- convert legacy logging to new helpers
- use devm for i2c adapters
msm:
- GPU
- Document a612/RGMU dt bindings
- UBWC 6.0 support (for A840 / Kaanapali)
- a225 support
- DPU:
- Switched to use virtual planes by default
- Fixed DSI CMD panels on DPU 3.x
- Rewrote format handling to remove intermediate representation
- Fixed watchdog on DPU 8.x+
- Fixed TE / Vsync source setting on DPU 8.x+
- Added 3D_Mux on SC7280
- Kaanapali platform support
- Fixed UBWC register programming
- Made RM reserve DSPP-enabled mixers for CRTCs with LMs.
- Gamma correction support
- DP:
- Enabled support for eDP 1.4+ link rate tables
- Fixed MDSS1 DP indices on SA8775P, making them to work
- Fixed msm_dp_ctrl_config_msa() to work with LLVM 20
- DSI:
- Documented QCS8300 as compatible with SA8775P
- Kaanapali platform support
- DSI PHY:
- switched to divider_determine_rate()
- MDP5:
- Dropped support for MSM8998, SDM660 and SDM630 (switched over
to DPU)
- MDSS:
- Kaanapali platform support
- Fixed UBWC register programming
nova-core:
- Prepare for Turing support. This includes parsing and handling
Turing-specific firmware headers and sections as well as a Turing
Falcon HAL implementation.
- Get rid of the Result<impl PinInit<T, E>> anti-pattern.
- Relocate initializer-specific code into the appropriate initializer.
- Use CStr::from_bytes_until_nul() to remove custom helpers.
- Improve handling of unexpected firmware values.
- Clean up redundant debug prints.
- Replace c_str!() with native Rust C-string literals.
- Update nova-core task list.
nova:
- Align GEM object size to system page size.
tyr:
- Use generated uAPI bindings for GpuInfo.
- Replace manual sleeps with read_poll_timeout().
- Replace c_str!() with native Rust C-string literals.
- Suppress warnings for unread fields.
- Fix incorrect register name in print statement.
nouveau:
- fix big page table support races in PTE management
- improve reclocking on tegra 186+
amdxdna:
- fix suspend race conditions
- improve handling of zero tail pointers
- fix cu_idx overwritten during command setup
- enable hardware context priority
- remove NPU2 support
- update message buffer allocation requirements
- update firmware version check
ast:
- support imported cursor buffers
- big endian fixes
etnaviv:
- add PPU flop reset support
imagination:
- add AM62P support
- introduce hw version checks
ivpu:
- implement warm boot flow
panfrost:
- add bo sync ioctl
- add GPU_PM_RT support for RZ/G3E SoC
panthor:
- add bo sync ioctl
- enable timestamp propagation
- scheduler robustness improvements
- VM termination fixes
- huge page support
rockchip:
- RK3368 HDMI Support
- get rid of atomic_check fixups
- RK3506 support
- RK3576/RK3588 improved HPD handling
rz-du:
- RZ/V2H(P) MIPI-DSI Support
v3d:
- fix DMA segment size
- convert to new logging helpers
mediatek:
- move DP training to hotplug thread
- convert logging to new helpers
- add support for HS speed DSI
- Genio 510/700/1200-EVK, Radxa NIO-12L HDMI support
atmel-hlcdc:
- switch to drmm resource
- support nomodeset
- use newer helpers
hisilicon:
- fix various DP bugs
renesas:
- fix kernel panic on reboot
exynos:
- fix vidi_connection_ioctl using wrong device
- fix vidi_connection deref user ptr
- fix concurrency regression with vidi_context
vkms:
- add configfs support for display configuration
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEEKbZHaGwW9KfbeusDHTzWXnEhr4FAmmMLAcACgkQDHTzWXnE
hr7aAw//bQ2WLhXeMqWyqrPPe51l2DmWvbdwP6TKUjzMwd9+xvs6wQcSg80Mn230
0vqSpqKq2aMB6GMmz7wdHG8JgZOvO7qDf2TZodXe5lvBiAAPjzX+UE/0bIQKuhym
Ufb7tqCIPsj6TpcD3ef/173x3BnVPA6Y7lS11KaaG5l01vUAVlTD1vfWGDQp/L6P
7g94cC+0+3eYZyKxE1+Rn7FDXdw08u+vtLchIoowcAHobgucZ8K/XtZZoqFFy3sj
ZZN580AhyZoGcgmn2KhNvU4B+3tBFFMSVZkJm7skOO0IB2AMQGdEr0uVUDzLGc7K
DrLaxYwM6HfxM4o0r0Ai0WCuoysCAJ95M2Cp58uDuNcew4lRTtIUqz32Sm2OJ8bD
Z91Rvh/kOcA0Ru11Sb/kQvy9/OJ54CqojKVaUlkFo9VhHyPCPo9hjnPvaDvCt34N
FmnhuVpZMWqcjjq5yO/192qpDJnm470eQExvkZ4YpgmWkekND0zwaT4PG4763dZJ
juPlBQ5WtUlIzlUpRxdHE7C7ht1rWRS+HdzSYPM5aHTXDvktJvcA+1b/Jyicc+x4
QZiZ/1AC0KKlLrZxpVpEcjkPdQj2CiCXHQ+0YjDfO3cHo/55EfKj4iiARzhDzokf
h7FgKwvVhc9DycSq8KPGAf09AswceGAtvB1rKk+Jh9D/GqbgGtM=
=RFJ2
-----END PGP SIGNATURE-----
Merge tag 'drm-next-2026-02-11' of https://gitlab.freedesktop.org/drm/kernel
Pull drm updates from Dave Airlie:
"Highlights:
- amdgpu support for lots of new IP blocks which means newer GPUs
- xe has a lot of SR-IOV and SVM improvements
- lots of intel display refactoring across i915/xe
- msm has more support for gen8 platforms
- Given up on kgdb/kms integration, it's too hard on modern hw
core:
- drop kgdb support
- replace system workqueue with percpu
- account for property blobs in memcg
- MAINTAINERS updates for xe + buddy
rust:
- Fix documentation for Registration constructors
- Use pin_init::zeroed() for fops initialization
- Annotate DRM helpers with __rust_helper
- Improve safety documentation for gem::Object::new()
- Update AlwaysRefCounted imports
- mm: Prevent integer overflow in page_align()
atomic:
- add drm_device pointer to drm_private_obj
- introduce gamma/degamma LUT size check
buddy:
- fix free_trees memory leak
- prevent BUG_ON
bridge:
- introduce drm_bridge_unplug/enter/exit
- add connector argument to .hpd_notify
- lots of recounting conversions
- convert rockchip inno hdmi to bridge
- lontium-lt9611uxc: switch to HDMI audio helpers
- dw-hdmi-qp: add support for HPD-less setups
- Algoltek AG6311 support
panels:
- edp: CSW MNE007QB3-1, AUO B140HAN06.4, AUO B140QAX01.H
- st75751: add SPI support
- Sitronix ST7920, Samsung LTL106HL02
- LG LH546WF1-ED01, HannStar HSD156J
- BOE NV130WUM-T08
- Innolux G150XGE-L05
- Anbernic RG-DS
dma-buf:
- improve sg_table debugging
- add tracepoints
- call clear_page instead of memset
- start to introduce cgroup memory accounting in heaps
- remove sysfs stats
dma-fence:
- add new helpers
dp:
- mst: avoid oob access with vcpi=0
hdmi:
- limit infoframes exposure to userspace
gem:
- reduce page table overhead with THP
- fix leak in drm_gem_get_unmapped_area
gpuvm:
- API sanitation for rust bindings
sched:
- introduce new helpers
panic:
- report invalid panic modes
- add kunit tests
i915/xe display:
- Expose sharpness only if num_scalers is >= 2
- Add initial Xe3P_LPD for NVL
- BMG FBC support
- Add MTL+ platforms to support dpll framework
_ fix DIMM_S DRM decoding on ICL
- Return to using AUX interrupts
- PSR/Panel replay refactoring
- use consolidation HDMI tables
- Xe3_LPD CD2X dividier changes
xe:
- vfio: add vfio_pci for intel GPU
- multi queue support
- dynamic pagemaps and multi-device SVM
- expose temp attribs in hwmon
- NO_COMPRESSION bo flag
- expose MERT OA unit
- sysfs survivability refactor
- SRIOV PF: add MERT support
- enable SR-IOV VF migration
- Enable I2C/NVM on Crescent Island
- Xe3p page reclaimation support
- introduce SRIOV scheduler groups
- add SoC remappt support in system controller
- insert compiler barriers in GuC code
- define NVL GuC firmware
- handle GT resume failure
- fix drm scheduler layering violations
- enable GSC loading and PXP for PTL
- disable GuC Power DCC strategy on PTL
- unregister drm device on probe error
i915:
- move to kernel standard fault injection
- bump recommended GuC version for DG2 and MTL
amdgpu:
- SMUIO 15.x, PSP 15.x support
- IH 6.1.1/7.1 support
- MMHUB 3.4/4.2 support
- GC 11.5.4/12.1 support
- SDMA 6.1.4/7.1/7.11.4 support
- JPEG 5.3 support
- UserQ updates
- GC 9 gfx queue reset support
- TTM memory ops parallelization
- convert legacy logging to new helpers
- DC analog fixes
amdkfd:
- GC 11.5.4/12.1 suppport
- SDMA 6.1.4/7.1 support
- per context support
- increase kfd process hash table
- Reserved SDMA rework
radeon:
- convert legacy logging to new helpers
- use devm for i2c adapters
msm:
- GPU
- Document a612/RGMU dt bindings
- UBWC 6.0 support (for A840 / Kaanapali)
- a225 support
- DPU:
- Switch to use virtual planes by default
- Fix DSI CMD panels on DPU 3.x
- Rewrite format handling to remove intermediate representation
- Fix watchdog on DPU 8.x+
- Fix TE / Vsync source setting on DPU 8.x+
- Add 3D_Mux on SC7280
- Kaanapali platform support
- Fix UBWC register programming
- Make RM reserve DSPP-enabled mixers for CRTCs with LMs
- Gamma correction support
- DP:
- Enable support for eDP 1.4+ link rate tables
- Fix MDSS1 DP indices on SA8775P, making them to work
- Fix msm_dp_ctrl_config_msa() to work with LLVM 20
- DSI:
- Document QCS8300 as compatible with SA8775P
- Kaanapali platform support
- DSI PHY:
- switch to divider_determine_rate()
- MDP5:
- Drop support for MSM8998, SDM660 and SDM630 (switch over to DPU)
- MDSS:
- Kaanapali platform support
- Fixed UBWC register programming
nova-core:
- Prepare for Turing support. This includes parsing and handling
Turing-specific firmware headers and sections as well as a Turing
Falcon HAL implementation
- Get rid of the Result<impl PinInit<T, E>> anti-pattern
- Relocate initializer-specific code into the appropriate initializer
- Use CStr::from_bytes_until_nul() to remove custom helpers
- Improve handling of unexpected firmware values
- Clean up redundant debug prints
- Replace c_str!() with native Rust C-string literals
- Update nova-core task list
nova:
- Align GEM object size to system page size
tyr:
- Use generated uAPI bindings for GpuInfo
- Replace manual sleeps with read_poll_timeout()
- Replace c_str!() with native Rust C-string literals
- Suppress warnings for unread fields
- Fix incorrect register name in print statement
nouveau:
- fix big page table support races in PTE management
- improve reclocking on tegra 186+
amdxdna:
- fix suspend race conditions
- improve handling of zero tail pointers
- fix cu_idx overwritten during command setup
- enable hardware context priority
- remove NPU2 support
- update message buffer allocation requirements
- update firmware version check
ast:
- support imported cursor buffers
- big endian fixes
etnaviv:
- add PPU flop reset support
imagination:
- add AM62P support
- introduce hw version checks
ivpu:
- implement warm boot flow
panfrost:
- add bo sync ioctl
- add GPU_PM_RT support for RZ/G3E SoC
panthor:
- add bo sync ioctl
- enable timestamp propagation
- scheduler robustness improvements
- VM termination fixes
- huge page support
rockchip:
- RK3368 HDMI Support
- get rid of atomic_check fixups
- RK3506 support
- RK3576/RK3588 improved HPD handling
rz-du:
- RZ/V2H(P) MIPI-DSI Support
v3d:
- fix DMA segment size
- convert to new logging helpers
mediatek:
- move DP training to hotplug thread
- convert logging to new helpers
- add support for HS speed DSI
- Genio 510/700/1200-EVK, Radxa NIO-12L HDMI support
atmel-hlcdc:
- switch to drmm resource
- support nomodeset
- use newer helpers
hisilicon:
- fix various DP bugs
renesas:
- fix kernel panic on reboot
exynos:
- fix vidi_connection_ioctl using wrong device
- fix vidi_connection deref user ptr
- fix concurrency regression with vidi_context
vkms:
- add configfs support for display configuration
* tag 'drm-next-2026-02-11' of https://gitlab.freedesktop.org/drm/kernel: (1610 commits)
drm/xe/pm: Disable D3Cold for BMG only on specific platforms
drm/xe: Fix kerneldoc for xe_tlb_inval_job_alloc_dep
drm/xe: Fix kerneldoc for xe_gt_tlb_inval_init_early
drm/xe: Fix kerneldoc for xe_migrate_exec_queue
drm/xe/query: Fix topology query pointer advance
drm/xe/guc: Fix kernel-doc warning in GuC scheduler ABI header
drm/xe/guc: Fix CFI violation in debugfs access.
accel/amdxdna: Move RPM resume into job run function
accel/amdxdna: Fix incorrect DPM level after suspend/resume
nouveau/vmm: start tracking if the LPT PTE is valid. (v6)
nouveau/vmm: increase size of vmm pte tracker struct to u32 (v2)
nouveau/vmm: rewrite pte tracker using a struct and bitfields.
accel/amdxdna: Fix incorrect error code returned for failed chain command
accel/amdxdna: Remove hardware context status
drm/bridge: imx8qxp-pixel-combiner: Fix bailout for imx8qxp_pc_bridge_probe()
drm/panel: ilitek-ili9882t: Remove duplicate initializers in tianma_il79900a_dsc
drm/i915/display: fix the pixel normalization handling for xe3p_lpd
drm/exynos: vidi: use ctx->lock to protect struct vidi_context member variables related to memory alloc/free
drm/exynos: vidi: fix to avoid directly dereferencing user pointer
drm/exynos: vidi: use priv->vidi_dev for ctx lookup in vidi_connection_ioctl()
...
|
||
|
Matthew Brost
|
12b2285bf3 |
mm/zone_device: reinitialize large zone device private folios
Reinitialize metadata for large zone device private folios in
zone_device_page_init prior to creating a higher-order zone device private
folio. This step is necessary when the folio's order changes dynamically
between zone_device_page_init calls to avoid building a corrupt folio. As
part of the metadata reinitialization, the dev_pagemap must be passed in
from the caller because the pgmap stored in the folio page may have been
overwritten with a compound head.
Without this fix, individual pages could have invalid pgmap fields and
flags (with PG_locked being notably problematic) due to prior different
order allocations, which can, and will, result in kernel crashes.
Link: https://lkml.kernel.org/r/20260116111325.1736137-2-francois.dugast@intel.com
Fixes:
|
||
|
Matthew Brost
|
10dd1eaa80 |
drm/pagemap: Disable device-to-device migration
Device-to-device migration is causing xe_exec_system_allocator --r
*race*no* to intermittently fail with engine resets and a kernel hang on
a page lock. This should work but is clearly buggy somewhere. Disable
device-to-device migration in the interim until the issue can be
root-caused.
The only downside of disabling device-to-device migration is that memory
will bounce through system memory during migration. However, this path
should be rare, as it only occurs when madvise attributes are changed or
atomics are used.
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Fixes:
|
||
|
Matthew Brost
|
3902846af3 |
drm/pagemap Fix error paths in drm_pagemap_migrate_to_devmem
Avoid unlocking and putting device pages unless they were successfully
locked, and do not calculate migrated_pages on error paths.
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Fixes:
|
||
|
Thomas Hellström
|
754c232384 |
drm/pagemap, drm/xe: Ensure that the devmem allocation is idle before use
In situations where no system memory is migrated to devmem, and in upcoming patches where another GPU is performing the migration to the newly allocated devmem buffer, there is nothing to ensure any ongoing clear to the devmem allocation or async eviction from the devmem allocation is complete. Address that by passing a struct dma_fence down to the copy functions, and ensure it is waited for before migration is marked complete. v3: - New patch. v4: - Update the logic used for determining when to wait for the pre_migrate_fence. - Update the logic used for determining when to warn for the pre_migrate_fence since the scheduler fences apparently can signal out-of-order. v5: - Fix a UAF (CI) - Remove references to source P2P migration (Himal) - Put the pre_migrate_fence after migration. v6: - Pipeline the pre_migrate_fence dependency (Matt Brost) Fixes: |
||
|
Thomas Hellström
|
ec265e1f1c |
drm/pagemap: Support source migration over interconnect
Support source interconnect migration by using the copy_to_ram() op of the source device private pages. Source interconnect migration is required to flush the L2 cache of the source device, which among other things is a requirement for correct global atomic operation. It also enables the source GPU to potentially decompress any compressed content which is not understood by peers, and finally for the PCIe case, it's expected that writes over PCIe will be faster than reads. The implementation can probably be improved by coalescing subregions with the same source. v5: - Update waiting for the pre_migrate_fence and comments around that, previously in another patch. (Himal). - Actually select device private pages to migrate when source_peer_migrates is true. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-24-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
75af93b3f5 |
drm/pagemap, drm/xe: Support destination migration over interconnect
Support destination migration over interconnect when migrating from device-private pages with the same dev_pagemap owner. Since we now also collect device-private pages to migrate, also abort migration if the range to migrate is already fully populated with pages from the desired pagemap. Finally return -EBUSY from drm_pagemap_populate_mm() if the migration can't be completed without first migrating all pages in the range to system. It is expected that the caller will perform that before retrying the call to drm_pagemap_populate_mm(). v3: - Fix a bug where the p2p dma-address was never used. - Postpone enabling destination interconnect migration, since xe devices require source interconnect migration to ensure the source L2 cache is flushed at migration time. - Update the drm_pagemap_migrate_to_devmem() interface to pass migration details. v4: - Define XE_INTERCONNECT_P2P unconditionally (CI) - Include a missing header (CI) v5: - Use page order increments where possible (Matt Brost). - Fix a negated value of can_migrate_same_pagemap. - Move removal of some dead code to a separate patch (Matt Brost). - Remove an unnecessary zdd get() and put() (Matt Brost). Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-23-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
5b64b23f6f |
drm/pagemap, drm/xe: Clean up the use of the device-private page owner
Use the dev_pagemap->owner field wherever possible, simplifying the code slightly. v3: New patch Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-20-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
33ac8d150a |
drm/pagemap: Remove the drm_pagemap_create() interface
With the drm_pagemap_init() interface, drm_pagemap_create() is not used anymore. v2: - Slightly more verbose commit message. (Matt Brost) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-10-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
77f14f2f2d |
drm/pagemap: Add a drm_pagemap cache and shrinker
Pagemaps are costly to set up and tear down, and they consume a lot of system memory for the struct pages. Ideally they should be created only when needed. Add a caching mechanism to allow doing just that: Create the drm_pagemaps when needed for migration. Keep them around to avoid destruction and re-creation latencies and destroy inactive/unused drm_pagemaps on memory pressure using a shrinker. Only add the helper functions. They will be hooked up to the xe driver in the upcoming patch. v2: - Add lockdep checking for drm_pagemap_put(). (Matt Brost) - Add a copyright notice. (Matt Brost) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-8-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
a26084328a |
drm/pagemap, drm/xe: Manage drm_pagemap provider lifetimes
If a device holds a reference on a foregin device's drm_pagemap, and a device unbind is executed on the foreign device, Typically that foreign device would evict its device-private pages and then continue its device-managed cleanup eventually releasing its drm device and possibly allow for module unload. However, since we're still holding a reference on a drm_pagemap, when that reference is released and the provider module is unloaded we'd execute out of undefined memory. Therefore keep a reference on the provider device and module until the last drm_pagemap reference is gone. Note that in theory, the drm_gpusvm_helper module may be unloaded as soon as the final module_put() of the provider driver module is executed, so we need to add a module_exit() function that waits for the work item executing the module_put() has completed. v2: - Better commit message (Matt Brost) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-7-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
565477dbca |
drm/pagemap: Add a refcounted drm_pagemap backpointer to struct drm_pagemap_zdd
To be able to keep track of drm_pagemap usage, add a refcounted backpointer to struct drm_pagemap_zdd. This will keep the drm_pagemap reference count from dropping to zero as long as there are drm_pagemap pages present in a CPU address space. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-6-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
a599b98607 |
drm/pagemap, drm/xe: Add refcounting to struct drm_pagemap
With the end goal of being able to free unused pagemaps and allocate them on demand, add a refcount to struct drm_pagemap, remove the xe embedded drm_pagemap, allocating and freeing it explicitly. v2: - Make the drm_pagemap pointer in drm_gpusvm_pages reference-counted. v3: - Call drm_pagemap_get() before drm_pagemap_put() in drm_gpusvm_pages (Himal Prasad Ghimiray) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> #v1 Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-5-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
16b5ad3195 |
drm/pagemap, drm/xe: Ensure that the devmem allocation is idle before use
In situations where no system memory is migrated to devmem, and in
upcoming patches where another GPU is performing the migration to
the newly allocated devmem buffer, there is nothing to ensure any
ongoing clear to the devmem allocation or async eviction from the
devmem allocation is complete.
Address that by passing a struct dma_fence down to the copy
functions, and ensure it is waited for before migration is marked
complete.
v3:
- New patch.
v4:
- Update the logic used for determining when to wait for the
pre_migrate_fence.
- Update the logic used for determining when to warn for the
pre_migrate_fence since the scheduler fences apparently
can signal out-of-order.
v5:
- Fix a UAF (CI)
- Remove references to source P2P migration (Himal)
- Put the pre_migrate_fence after migration.
v6:
- Pipeline the pre_migrate_fence dependency (Matt Brost)
Fixes:
|
||
|
Thomas Hellström
|
d1a4fc044c |
drm/pagemap: Remove some dead code
The page pointer can't be NULL. v5: - New patch. (Matt Brost) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> # For merging through drm-xe. Link: https://patch.msgid.link/20251219113320.183860-3-thomas.hellstrom@linux.intel.com |
||
|
Balbir Singh
|
3a5a065545 |
mm/zone_device: rename page_free callback to folio_free
Change page_free to folio_free to make the folio support for zone device-private more consistent. The PCI P2PDMA callback has also been updated and changed to folio_free() as a result. For drivers that do not support folios (yet), the folio is converted back into page via &folio->page and the page is used as is, in the current callback implementation. Link: https://lkml.kernel.org/r/20251001065707.920170-3-balbirs@nvidia.com Signed-off-by: Balbir Singh <balbirs@nvidia.com> Cc: David Hildenbrand <david@redhat.com> Cc: Zi Yan <ziy@nvidia.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Rakie Kim <rakie.kim@sk.com> Cc: Byungchul Park <byungchul@sk.com> Cc: Gregory Price <gourry@gourry.net> Cc: Ying Huang <ying.huang@linux.alibaba.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: "Liam R. Howlett" <Liam.Howlett@oracle.com> Cc: Nico Pache <npache@redhat.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Lyude Paul <lyude@redhat.com> Cc: Danilo Krummrich <dakr@kernel.org> Cc: David Airlie <airlied@gmail.com> Cc: Simona Vetter <simona@ffwll.ch> Cc: Ralph Campbell <rcampbell@nvidia.com> Cc: Mika Penttilä <mpenttil@redhat.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Francois Dugast <francois.dugast@intel.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Felix Kuehling <Felix.Kuehling@amd.com> Cc: Alex Deucher <alexander.deucher@amd.com> Cc: "Christian König" <christian.koenig@amd.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> |
||
|
Balbir Singh
|
d245f9b4ab |
mm/zone_device: support large zone device private folios
Patch series "mm: support device-private THP", v7.
This patch series introduces support for Transparent Huge Page (THP)
migration in zone device-private memory. The implementation enables
efficient migration of large folios between system memory and
device-private memory
Background
Current zone device-private memory implementation only supports PAGE_SIZE
granularity, leading to:
- Increased TLB pressure
- Inefficient migration between CPU and device memory
This series extends the existing zone device-private infrastructure to
support THP, leading to:
- Reduced page table overhead
- Improved memory bandwidth utilization
- Seamless fallback to base pages when needed
In my local testing (using lib/test_hmm) and a throughput test, the series
shows a 350% improvement in data transfer throughput and a 80% improvement
in latency
These patches build on the earlier posts by Ralph Campbell [1]
Two new flags are added in vma_migration to select and mark compound
pages. migrate_vma_setup(), migrate_vma_pages() and
migrate_vma_finalize() support migration of these pages when
MIGRATE_VMA_SELECT_COMPOUND is passed in as arguments.
The series also adds zone device awareness to (m)THP pages along with
fault handling of large zone device private pages. page vma walk and the
rmap code is also zone device aware. Support has also been added for
folios that might need to be split in the middle of migration (when the
src and dst do not agree on MIGRATE_PFN_COMPOUND), that occurs when src
side of the migration can migrate large pages, but the destination has not
been able to allocate large pages. The code supported and used
folio_split() when migrating THP pages, this is used when
MIGRATE_VMA_SELECT_COMPOUND is not passed as an argument to
migrate_vma_setup().
The test infrastructure lib/test_hmm.c has been enhanced to support THP
migration. A new ioctl to emulate failure of large page allocations has
been added to test the folio split code path. hmm-tests.c has new test
cases for huge page migration and to test the folio split path. A new
throughput test has been added as well.
The nouveau dmem code has been enhanced to use the new THP migration
capability.
mTHP support:
The patches hard code, HPAGE_PMD_NR in a few places, but the code has been
kept generic to support various order sizes. With additional refactoring
of the code support of different order sizes should be possible.
The future plan is to post enhancements to support mTHP with a rough
design as follows:
1. Add the notion of allowable thp orders to the HMM based test driver
2. For non PMD based THP paths in migrate_device.c, check to see if
a suitable order is found and supported by the driver
3. Iterate across orders to check the highest supported order for migration
4. Migrate and finalize
The mTHP patches can be built on top of this series, the key design
elements that need to be worked out are infrastructure and driver support
for multiple ordered pages and their migration.
HMM support for large folios was added in
|
||
|
Francois Dugast
|
ddeda61360 |
drm/pagemap: Allocate folios when possible
If the order is greater than zero, allocate a folio when populating the RAM PFNs instead of allocating individual pages one after the other. For example if 2MB folios are used instead of 4KB pages, this reduces the number of calls to the allocation API by 512. v2: - Use page order instead of extra argument (Matthew Brost) - Allocate with folio_alloc() (Matthew Brost) - Loop for mpages and free_pages based on order (Matthew Brost) v3: - Fix loops in drm_pagemap_migrate_populate_ram_pfn() (Matthew Brost) v4: - Use folio_trylock(), set local variable to NULL (Matthew Brost) Cc: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://lore.kernel.org/r/20250805140028.599361-5-francois.dugast@intel.com Signed-off-by: Francois Dugast <francois.dugast@intel.com> |
||
|
Francois Dugast
|
d755ff6063 |
drm/pagemap: DMA map folios when possible
If the page is part of a folio, DMA map the whole folio at once instead of mapping individual pages one after the other. For example if 2MB folios are used instead of 4KB pages, this reduces the number of DMA mappings by 512. The folio order (and consequently, the size) is persisted in the struct drm_pagemap_device_addr to be available at the time of unmapping. v2: - Initialize order variable (Matthew Brost) - Set proto and dir for completeness (Matthew Brost) - Do not populate drm_pagemap_addr, document it (Matthew Brost) - Add and use macro NR_PAGES(order) (Matthew Brost) Cc: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://lore.kernel.org/r/20250805140028.599361-4-francois.dugast@intel.com Signed-off-by: Francois Dugast <francois.dugast@intel.com> |
||
|
Francois Dugast
|
f35a6cdf8a |
drm/pagemap: Use struct drm_pagemap_addr in mapping and copy functions
This struct embeds more information than just the DMA address. This will help later to support folio orders greater than zero. At this point, there is no functional change as the only struct member used is addr. In Xe, adapt to the new drm_gpusvm_devmem_ops type signatures using struct drm_pagemap_addr, as well as the internal xe SVM functions implementing those operations. The use of this struct is propagated to xe_migrate as it makes indexed accesses to the next DMA address but they are no longer contiguous. v2: - Rename drm_pagemap_device_addr to drm_pagemap_addr (Matthew Brost) - Squash with patch for Xe (Matthew Brost) - Set proto and dir for completeness (Matthew Brost) - Assess DMA map protocol (Matthew Brost) Cc: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Acked-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Link: https://lore.kernel.org/r/20250805140028.599361-3-francois.dugast@intel.com Signed-off-by: Francois Dugast <francois.dugast@intel.com> |
||
|
Thomas Hellström
|
b587016878 |
drm/xe: Implement and use the drm_pagemap populate_mm op
Add runtime PM since we might call populate_mm on a foreign device. v3: - Fix a kerneldoc failure (Matt Brost) - Revert the bo type change from device to kernel (Matt Brost) v4: - Add an assert in xe_svm_alloc_vram (Matt Brost) Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250619134035.170086-4-thomas.hellstrom@linux.intel.com |
||
|
Thomas Hellström
|
2ef19be2a5 |
drm/pagemap: Add a populate_mm op
Add an operation to populate a part of a drm_mm with device private memory. Clarify how migration using it is intended to work. v3: - Kerneldoc fixes and updates (Matt Brost). v4: - More kerneldoc fixes. Rebase. Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250619134035.170086-3-thomas.hellstrom@linux.intel.com |
||
|
Matthew Brost
|
f86ad0ed62 |
drm/gpusvm, drm/pagemap: Move migration functionality to drm_pagemap
The migration functionality and track-keeping of per-pagemap VRAM mapped to the CPU mm is not per GPU_vm, but rather per pagemap. This is also reflected by the functions not needing the drm_gpusvm structures. So move to drm_pagemap. With this, drm_gpusvm shouldn't really access the page zone-device-data since its meaning is internal to drm_pagemap. Currently it's used to reject mapping ranges backed by multiple drm_pagemap allocations. For now, make the zone-device-data a void pointer. Alter the interface of drm_gpusvm_migrate_to_devmem() to ensure we don't pass a gpusvm pointer. Rename CONFIG_DRM_XE_DEVMEM_MIRROR to CONFIG_DRM_XE_PAGEMAP. Matt is listed as author of this commit since he wrote most of the code, and it makes sense to retain his git authorship. Thomas mostly moved the code around. v3: - Kerneldoc fixes (CI) - Don't update documentation about how the drm_pagemap migration should be interpreted until upcoming patches where the functionality is implemented. (Matt Brost) v4: - More kerneldoc fixes around timeslice_ms (Himal Ghimiray, Matt Brost) v6: - Fix an uninitialized pagemap pointer (CI) Co-developed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Link: https://lore.kernel.org/r/20250619134035.170086-2-thomas.hellstrom@linux.intel.com |