KFD_IOC_ALLOC_MEMORY_OF_GPU with flag
KFD_IOC_ALLOC_MEM_FLAGS_AQL_QUEUE_MEM and size=1 triggers the AQL
wraparound workaround (size >>= 1), reducing size to 0. The resulting
zero passes through PAGE_ALIGN(0) = 0 without validation, bypassing the
per-process VRAM quota check in reserve_mem_limit()
(vram_used + 0 > vram_available is always false).
The fix adds post-halving zero-size validation in the primary
allocation path (amdgpu_amdkfd_gpuvm.c). The check happens after size
halving but before reserve_mem_limit(), and uses err_alignment_size
error path to properly clean up the allocated kgd_mem structure and
mutex.
Cc: stable@vger.kernel.org
Signed-off-by: Sunday Clement <Sunday.Clement@amd.com>
Reviewed-by: Alex Deucher <Alexander.Deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The largest frame is 20 dwords, so 0xf mask is too small.
This was always wrong, but we were lucky with the VCE_CMD_END
commands inserted after fence and vm_flush.
Fixes: 8897ea8c76 ("drm/amdgpu: Implement insert_end for VCE 3")
Cc: stable@vger.kernel.org
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: David Rosca <david.rosca@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Port changes from GFX9 to GFX12.1 mostly as-is.
Minor changes to register access code.
Assisted-by: Claude:Sonnet 4.6
Signed-off-by: David Belanger <david.belanger@amd.com>
Reviewed-by: Sreekant Somasekharan <Sreekant.Somasekharan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Port changes from GFX9 to GFX12 mostly as-is.
Minor changes to register access code.
Assisted-by: Claude:Sonnet-4-6
Signed-off-by: David Belanger <david.belanger@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Sreekant Somasekharan <Sreekant.Somasekharan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Port changes from GFX9 to GFX11 mostly as-is.
Minor changes to register access code.
Assisted-by: Claude:Sonnet-4-6
Signed-off-by: David Belanger <david.belanger@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Sreekant Somasekharan <Sreekant.Somasekharan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
uvd_v7_0_sw_init() does not initialize the UVD decode ring for an
SR-IOV VF. However, amdgpu_uvd_resume() unconditionally force-completes
the decode ring when restoring its fence sequence.
Skip fence completion when the fence driver is not initialized.
Fixes: 0a33b11d26 ("drm/amdgpu: mark force completed fences with -ECANCELED")
Cc: stable@vger.kernel.org
Signed-off-by: Bob Zhou <bobzhou2@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Acked-by: Frank Min <Frank.Min@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On resume, amdgpu_userq_vm_validate_and_restore_queue() updates each queue's
wptr GPU address via amdgpu_bo_gpu_offset().
WPTR BOs are VM-mapped, but each BO has its own reservation object and is not
implicitly covered by the VM validation path here. This can leave offset reads
without proper BO locking/placement state and trigger WARN_ONs.
------------[ cut here ]------------
WARNING: amdgpu_object.c:1486 at amdgpu_bo_gpu_offset+0x75/0xa0 [amdgpu], CPU#3: kworker/3:1/116
Workqueue: events amdgpu_userq_restore_worker [amdgpu]
RIP: 0010:amdgpu_bo_gpu_offset+0x75/0xa0 [amdgpu]
Call Trace:
<TASK>
amdgpu_userq_vm_validate_and_restore_queue+0x629/0x960 [amdgpu]
amdgpu_userq_restore_worker+0xa6/0x180 [amdgpu]
process_scheduled_works+0xa6/0x460
worker_thread+0x13c/0x290
kthread+0xfb/0x140
ret_from_fork+0x1b6/0x2b0
ret_from_fork_asm+0x1a/0x30
</TASK>
---[ end trace 0000000000000000 ]---
------------[ cut here ]------------
WARNING: amdgpu_object.c:1485 at amdgpu_bo_gpu_offset+0x9a/0xa0 [amdgpu], CPU#2: kworker/2:1/127
Workqueue: events amdgpu_userq_restore_worker [amdgpu]
RIP: 0010:amdgpu_bo_gpu_offset+0x9a/0xa0 [amdgpu]
Add each queue's WPTR BO to the drm_exec ww context and validate it to its
allowed placement before the later offset update.
v2:
- Clarify that WPTR BOs are VM-mapped (fix incorrect "not part of VM" wording). (Christian)
- Describe both parts of the fix: lock BO reservations in drm_exec and
validate BO placement before offset reads.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
All remaining callers of is_cow_mapping() are invoking it in the form of
is_cow_mapping(vma->vm_flags) or an indirected version of this.
Therefore, provide a helper - vma_is_cow_mapping() to directly test the
VMA.
Additionally provide a new helper vma_flags_is_cow_mapping() which
performs the check using the new vma_flags_t type, and share this logic
between vma_is_cow_mapping() and vma_desc_is_cow_mapping().
With these changes, no callers of is_cow_mapping() remain, so remove it.
Also update the userland VMA tests to reflect the change.
No functional change intended.
[akpm@linux-foundation.org: fix kerneldoc comment typo, per Lorenzo]
Link: https://lore.kernel.org/aob1goSSPH6sTN9y@gremlin
Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-2-c21581c0c3c8@kernel.org
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Deucher <alexander.deucher@amd.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Boris Brezillon <boris.brezillon@collabora.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Chengming Zhou <chengming.zhou@linux.dev>
Cc: Chris Li <chrisl@kernel.org>
Cc: Christan König <christian.koenig@amd.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Claudio Imbrenda <imbrenda@linux.ibm.com>
Cc: Dave Airlie <airlied@gmail.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Gregory Price (Meta) <gourry@gourry.net>
Cc: Harry Yoo <harry@kernel.org>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Huang Ray <Ray.Huang@amd.com>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: James Clark <james.clark@linaro.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Jann Horn <jannh@google.com>
Cc: Janosch Frank <frankja@linux.ibm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kees Cook <kees@kernel.org>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Liviu Dudau <liviu.dudau@arm.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Marc Rutland <mark.rutland@arm.com>
Cc: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Miaohe Lin <linmiaohe@huawei.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Namhyung kim <namhyung@kernel.org>
Cc: Naoya Horiguchi <nao.horiguchi@gmail.com>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Peter Xu <peterx@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Rodrigo Vivi <rodrigo.vivi@intel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Steven Price <steven.price@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Cc: Thomas Zimemrmann <tzimmermann@suse.de>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: xu xin <xu.xin16@zte.com.cn>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
-----BEGIN PGP SIGNATURE-----
iQFSBAABCgA8FiEEq68RxlopcLEwq+PEeb4+QwBBGIYFAmqCLGoeHHRvcnZhbGRz
QGxpbnV4LWZvdW5kYXRpb24ub3JnAAoJEHm+PkMAQRiGJzYH/0SFjcgnk1Z3Km+3
2kEeGAMETajW41W7+5QQkuHk83UXDxigDRoD857/d8utK90GrZAoTMS9/6zF3tra
ht4G1yc2x7/xgVLkWii54d/sp1LEWTRDntN95fzYZwbeAXwd0AcYBlKXZYHKl4t/
4yZCgYPmYTkewaYdbyWNPiZvCwhBUl5k1E9i/drh5IJXdgXRcqoO86FY9JX+Ks9x
r0g+d6RIiSbDfwzgRpkBn0TRnqzh2OeBfgyrsgGZO2axwlKcA7SP0vwwT6c6nOUI
s8F2xXrqrUI75JbSI4YbdwOSvktwbtkz83idlRAYBOdxof3LJ6i2YaxrT8iG+KUH
l7+e18M=
=eQMh
-----END PGP SIGNATURE-----
BackMerge tag 'v7.2' into drm-next
Linux 7.2
There was a lot of conflicts this round between fixes and next,
and I'd like to get the merge resolutions that we have in drm-tip.
Signed-off-by: Dave Airlie <airlied@redhat.com>
If we end up emitting a VM fence keep pipeline sync
associated with that fence. If not, emit them as
part of the IB fence.
v2: fix need_pipe_sync handling
v3: simplify the function
Cc: David Rosca <david.rosca@amd.com>
Fixes: cb1e657cca ("drm/amdgpu: handle GDS and SPM without a VM fence")
Reviewed-by: David Rosca <david.rosca@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
If the supplied msg[2] (num_buffers) is 0x3FFFFFFF, the expression
6 + num_buffers * 4 wraps to 2 and the bounds check passes, letting
the parser loop far past the end of the message BO. Triggering it
additionally requires a ~4GiB mapping so that msg[1] survives the
earlier "header does not fit in BO" check.
Rewrite the test in division form, which is overflow-free by
construction. Also update the message to reflect that msg is invalid.
Fixes: b193019860 ("drm/amdgpu/vcn3: Prevent OOB reads when parsing dec msg")
Fixes: 0a78f2bac1 ("drm/amdgpu/vcn4: Prevent OOB reads when parsing dec msg")
Cc: stable@vger.kernel.org
Signed-off-by: David (Ming Qiang) Wu <David.Wu3@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When a queue is hung, the hang_detect_work is the
only way to recover it. However in amdgpu_userq_destroy(),
the hang_detect_work is cancelled too early,
resulting in amdgpu_userq_wait_for_last_fence()
may never return, leaving an uninterruptible dma_fence_wait()
hang there.
To fix this problem, this commit moves the cancelling of
hang_detect_work after amdgpu_userq_wait_for_last_fence(), and it has
to be before the unmap helper, because hang_detect_work resets the
queue, so it races with amdgpu_userq_unmap_helper() for MES operations
and queue state.
This commit splits amdgpu_userq_cleanup() into two parts:
1) amdgpu_userq_detach_doorbell(), which detaches the queue from
userq_doorbell_xa. This has to be called before the cancel, otherwise
the IRQ handlers (for example amdgpu_userq_process_fence_irq)
can re-schedule the hang_detect_work and the cancel is not final.
2) amdgpu_userq_fence_driver_free(), this has to be called after the
unmap helper, because it can release the seq64 slot that the GPU
writes fence values to.
Only one cancel_delayed_work_sync(&queue->hang_detect_work) is needed,
so other redundancies are removed.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When amdgpu_init_minimal_xgmi is used, SDMA engines init
is delayed so amdgpu_ttm_enable_buffer_funcs must be
called later.
Without this, the check for num_buffer_funcs_scheds will
fail and using ttm buffer funcs later will fail.
Given that amdgpu_ttm_enable_buffer_funcs is a no-op if
amdgpu_in_reset() returns true, the call has to occur
after the reset lock is dropped.
Cc: stable@vger.kernel.org
Fixes: e4029f7a94 ("drm/amdgpu: only use working sdma schedulers for ttm")
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Introduce a helper to consolidate the vga_switcheroo registration condition
used by the init and fini paths.
Keep the explicit pci_is_thunderbolt_attached() check, as dev_is_removable()
does not provide equivalent coverage for Thunderbolt-attached GPUs.
This ensures such devices remain excluded from switcheroo registration while
preserving the existing PX and Apple gmux handling.
Cc: stable@vger.kernel.org
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Like the MES scheduler ring, the KIQ ring sets no_scheduler = true and uses a
polling fence, so it is skipped by the force-completion loop in
amdgpu_device_pre_asic_reset(). Its hw fence value lives in wb (GTT) memory and
survives a MODE1 reset while fence_drv.sync_seq keeps advancing, so after a
reset the first KIQ submission can poll forever on a seq that is never written
back.
Force complete the KIQ ring fences too so their hw fence is realigned to
sync_seq.
Cc: stable@vger.kernel.org
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
rptr and wptr of a userq are 8 bytes aligned, and may
not placed on a page boundary.
This commit checks whether rptr and wptr are 8 bytes
aligned, and expectes 8 bytes when validates rptr/wptr VA.
With above changes, this commit fixes an regression
in amdgpu_userq_input_va_validate, where
end_addr is caculated by:
check_add_overflow(start_addr, expected_size - 1, &end_addr).
Wptr and rptr are very likely not to be page aligned,
when validating rptr and wptr, if they are located in the last
mapped page(or only one page is mapped)
and expected_size is PAGE_SIZE, end_addr will exceed the last
mapped page, means (end_addr >> AMDGPU_GPU_PAGE_SHIFT) > va_map->last,
and causing an -EINVAL, even it is a valid VA.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Fixes: c0122bf2cc ("drm/amdgpu: fix userq VA validation for sub-page buffers")
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The MES scheduler ring has no drm scheduler (no_scheduler = true), so it is
skipped by the force-completion loop in amdgpu_device_pre_asic_reset(). It uses
a polling fence whose hw value lives in wb (GTT) memory and survives a MODE1
reset, while fence_drv.sync_seq keeps advancing for every packet.
When the reset is triggered because MES itself stopped responding, the
timed-out packets advance sync_seq past the last hw fence value MES wrote.
After resume the first MES submission polls forever on a seq that is never
written back, failing the resume and wedging the box on a second reset:
amdgpu: MES ring buffer is full.
amdgpu: *ERROR* ring gfx_0.0.0 test failed (-110)
amdgpu: resume of IP block <gfx_v11_0> failed -110
amdgpu: GPU reset end with ret = -110
Force complete the MES scheduler ring fences together with the scheduler rings
so their hw fence is realigned to sync_seq.
v2: cover all XCCs (one scheduler ring each), not just mes.ring[0].
Cc: stable@vger.kernel.org
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
In standalone mode the ip_discovery sysfs tree is built from a verbatim
copy of the discovery binary taken before reg_base_init() collapses the
64bit base addresses in place. Decoding as 32bit there yields interleaved
zeros. Decode base_address_64[] in that case; keep reading the already
collapsed adev->discovery.bin as-is otherwise.
Fixes: 402e04f11f ("drm/amdgpu: Export ip_discovery sysfs on probe failure")
Cc: stable@vger.kernel.org
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_userq_wait_return_fence_info() calls drm_exec_init() without
DRM_EXEC_IGNORE_DUPLICATES. When the same GEM object appears more than
once across the read/write BO handle lists submitted by userspace,
drm_exec_lock_obj() returns -EALREADY the second time it locks that
object, which aborts the fence resolution pass instead of treating the
repeat as a no-op.
Add DRM_EXEC_IGNORE_DUPLICATES so duplicate objects are silently
skipped on the second lock attempt.
Signed-off-by: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_userq_wait_count_fences() calls drm_exec_init() without
DRM_EXEC_IGNORE_DUPLICATES. When the same GEM object appears more than
once across the read/write BO handle lists submitted by userspace,
drm_exec_lock_obj() returns -EALREADY the second time it locks that
object, which aborts the fence-counting pass instead of treating the
repeat as a no-op.
Add DRM_EXEC_IGNORE_DUPLICATES so duplicate objects are silently
skipped on the second lock attempt.
Signed-off-by: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_userq_signal_ioctl() calls drm_exec_init() without
DRM_EXEC_IGNORE_DUPLICATES. When the same GEM object appears more than
once across the read/write BO handle lists submitted by userspace,
drm_exec_lock_obj() returns -EALREADY the second time it locks that
object, which aborts the ioctl instead of treating the repeat as a
no-op.
Add DRM_EXEC_IGNORE_DUPLICATES so duplicate objects are silently
skipped on the second lock attempt, matching the intended semantics of
locking a set of (possibly overlapping) BOs before publishing a fence
on them.
Signed-off-by: Sunil Khatri <sunil.khatri@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_userq_buffer_vas_mapped() checks whether all VAs
of a queue are mapped before restoring it.
So that HW won't access any invalid addresses.
Currently, this function assumes all VAs are mapped if
any VA of a queue has been mapped, which is wrong.
This commit fixes this problem by examining all VAs of
a queue and reporting false if any of them is not mapped.
Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com>
Reviewed-by: Sunil Khatri <sunil.khatri@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_pci_probe() calls pm_runtime_use_autosuspend(), but
amdgpu_pci_remove() does not call the matching
pm_runtime_dont_use_autosuspend().
If the autosuspend delay is set to a negative value while autosuspend
is enabled, the runtime PM core increments usage_count to prevent
runtime suspend. Without calling pm_runtime_dont_use_autosuspend()
during teardown, this reference is not dropped and usage_count remains
unbalanced.
The documentation for pm_runtime_use_autosuspend() also notes that it
is important to undo it with pm_runtime_dont_use_autosuspend() at
driver exit time, unless runtime PM was initially enabled with
devm_pm_runtime_enable().
Add the missing pm_runtime_dont_use_autosuspend() call to the remove
path.
This issue was found by manual code inspection.
Fixes: d38ceaf99e ("drm/amdgpu: add core driver (v4)")
Cc: stable@vger.kernel.org
Signed-off-by: Guangshuo Li <lgs201920130244@gmail.com>
Reviewed-by: Mario Limonciello (AMD) <superm1@kernel.org>
Link: https://patch.msgid.link/20260808120934.2813010-1-lgs201920130244@gmail.com
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
pci_is_thunderbolt_attached() requires an upstream PCI bridge with
is_thunderbolt set from an Intel Thunderbolt VSEC. This does not cover
the affected ASM4242 USB4 PCI hierarchy:
00:02.2
\- 0f:00.0 [1b21:2421]
+- 10:01.0 [1b21:2423] -> 45:00.0 -> 46:00.0
| -> 47:00.0 -> 48:00.0 -> 49:00.0 [1002:7590]
\- 10:03.0 -> 76:00.0 [1b21:2425] USB4 Host Router
The host router is outside the GPU upstream bridge chain, leaving no
ancestor with is_thunderbolt set. PCI core propagates DEVICE_REMOVABLE
below the external-facing PCIe tunnel. Disable Runtime PM when either
pci_is_thunderbolt_attached() or dev_is_removable() is true.
Cc: stable@vger.kernel.org
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Reviewed-by: Candice Li <candice.li@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Revert fair scheduler patches and mark fair policy as experimental due
to reported regressions.
- Fix OOB read in connector/hdmi infoframe.
- Handle invalid scaling parameters and empty messages in log target.
- Skip attempting to populate unmapped pages in amdxdna.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEuXvWqAysSYEJGuVH/lWMcqZwE8MFAmp936wACgkQ/lWMcqZw
E8MzVA//aKT/DVfDA9dUljxQS70lC17W0a9sf16rpQsftuRxCUci1x2kXJG5sIGa
MaKdxmP8PaR0J+xNC/2rGuk2cyKiQxd4BaRnNYXHY4pLG8FbUmSbhWMrICgxj9XE
DUBDXzwqjfq7lxXRsKsqARhkn4rHzb9TS84YeQaUZuuwgL2AbDeSEGZkXMTxKCuJ
0sDD8kklVNJKn4ofke5za3KWJ1gIWP/y+kosmi38kG8oZ5M3Uen6JXv/mp0VhMMd
UAM+vRUFtWmkYVTsvxC9BXxdo6nqMUWRviXAeHnt6BijuysQ0ZAIuFvnf7/o+Knu
NXDABvoZFuFm6pP4t3BSEX2M2RrwczxfvT465UvYaoLwgLutGxwP4AvmWE4vLe32
5pNQLpfllCkxZeOYuxewTEfWLhykxF8wnSeVy9l6II+/OpkPnUjDAFJkDHcaTk+V
dPPmPZrOGC/LeLtD7JYbhRX6FkNQzTKNLjXSDjt3w/Sori3nhXkSAuj0zFDM7rHj
KAnXh6B4j3vOAbqUEOHmabLqoRmlgpfP30NSyvLc47hMqvHJj+YV9V+5Ay1Yg3Sh
otMusvCZY/vpwvVhlFS45XA8k81YdjnxmUqLrQceJS/KGYHt5Qzc1kP2JynjX+D7
/byp/TzTuBnO8cdXaDz58coAEPdoyfGSjH2qVYs4FgeT06ZPOYQ=
=Stgd
-----END PGP SIGNATURE-----
Merge tag 'drm-misc-fixes-2026-08-13' of https://gitlab.freedesktop.org/drm/misc/kernel into drm-fixes
drm-misc-fixes for v7.3:
- Revert fair scheduler patches and mark fair policy as experimental due
to reported regressions.
- Fix OOB read in connector/hdmi infoframe.
- Handle invalid scaling parameters and empty messages in log target.
- Skip attempting to populate unmapped pages in amdxdna.
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patch.msgid.link/a9b38792-bdd0-42da-a46a-7a048c26c0c2@linux.intel.com
If a valid signature is seen at the default offset, use the default
size/offset for discovery.
Fixes: 01bdc7e219 ("drm/amdgpu: New interface to get IP discovery binary v3")
Closes: https://gitlab.freedesktop.org/drm/amd/-/work_items/5447
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 46a0df99a0)
Cc: stable@vger.kernel.org
Same change as for h264, avoids overflow later when calculating
min dpb size.
Signed-off-by: David Rosca <david.rosca@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit a4b0720e4f)
Cc: stable@vger.kernel.org
The PCIe L1 low‑power settings for NBIF 6.3.1 were never applied due to
unresolved register mapping, which caused the relevant code to be compiled out.
As a result, the PCIe link could not enter L1/L23 power‑down states or transition to L0s.
Properly configure the link control register to enable L1 and L23 power‑down,
and permit L0s link transitions. Keep LTR disabled and let the PCI core enable it
only after verifying end‑to‑end root complex support across switches.
Fixes: 894c6d3522 ("drm/amdgpu: Add nbif v6_3_1 ip block support")
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Signed-off-by: Kenneth Feng <Kenneth.feng@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit c2417f9fd7)
Cc: stable@vger.kernel.org
After a recent change VCE now hangs when VCE_CMD_END is emitted
after a pipeline sync without VM flush.
Implement insert_end to correctly insert only one VCE_CMD_END per job.
Fixes: bc639a9ead ("drm/amdgpu: always emit the job vm fence")
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 8897ea8c76)
Cc: stable@vger.kernel.org
Use correct size for message buffer = sizeof(struct ruvd_msg).
Add ITSCALING_TABLE_BUFFER size.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 37519d007e)
Cc: stable@vger.kernel.org
This needs to use pitch instead of width. Also reject pitch
over 4096 to avoid overflow.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit b41c8cb12e)
Cc: stable@vger.kernel.org
This should use actual number of references from the decode
message, instead of maximum derived from level.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 64b525edb7)
Cc: stable@vger.kernel.org
Fixes potential overflow in DPB size calculations.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 05e1387d15)
Cc: stable@vger.kernel.org
dGPUs with an internal PCIe switch expose graphics functions below the
switch downstream port. The automatic ASPM check uses the display
endpoint and evaluates the internal link instead of the host link.
Use the switch upstream port for the check and report the selected
link.
Fixes: 0ab5d711ec ("drm/amd: Refactor `amdgpu_aspm` to be evaluated per device")
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 4e0d6f2876)
Cc: stable@vger.kernel.org
We shouldn't return early if we need to emit spm update.
Reviewed-by: David Rosca <david.rosca@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 54a118f1d7)
Cc: stable@vger.kernel.org
AMDGPU_GEM_CREATE checked domain bits against AMDGPU_GEM_DOMAIN_MASK,
but did not validate domain combinations. Userspace could combine
CPU|GTT|VRAM with DOORBELL, GDS, GWS, or OA, making
amdgpu_bo_placement_from_domain() exceed AMDGPU_BO_MAX_PLACEMENTS and
hit BUG_ON().
Allow combinations only within CPU/GTT/VRAM, and require non-CPU/GTT/
VRAM domains to be specified one at a time. Return -EINVAL for invalid
combinations in amdgpu_gem_create_ioctl().
v2: Rename helper from amdgpu_gem_domain_valid() to
amdgpu_gem_are_domains_valid() (Christian)
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit db39852d0c)
Cc: stable@vger.kernel.org
amdgpu_cs_pass1() dispatches on chunk_id once per chunk without
rejecting repeated ids. p->uf_bo is a single-slot field, so a
submission carrying two AMDGPU_CHUNK_ID_FENCE chunks runs
amdgpu_cs_p1_user_fence() twice, and the second run overwrites
p->uf_bo with a freshly referenced BO without dropping the reference
taken by the first.
amdgpu_cs_parser_fini() only unrefs the final p->uf_bo, so every FENCE
chunk but the last leaks a BO reference. The leaked BO outlives handle
close and process exit.
Reject duplicate FENCE chunks the same way commit fec5f8e8c6
("drm/amdgpu: disallow multiple BO_HANDLES chunks in one submit") did
for p->bo_list.
Fixes: d38ceaf99e ("drm/amdgpu: add core driver (v4)")
Reported-by: Yuhao Jiang <danisjiang@gmail.com>
Assisted-by: Claude:claude-opus-5
Cc: stable@vger.kernel.org
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Junrui Luo <moonafterrain@outlook.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 665b1fc2a1)
A GPU reset can race with BO teardown after the BO's GTT resource has
been marked for deletion but before its drm_mm node is removed. In this
window, amdgpu_gtt_mgr_recover() can treat the node as a live BO and try
to restore its GART mapping while its TT backing is being destroyed.
Recolor the GTT node from amdgpu_bo_delete_mem_notify() so that recovery
skips it, reusing the existing color for ranges without a BO. The range
stays allocated until the resource is freed.
This prevents reset recovery from accessing a BO whose backing storage
is no longer valid.
v2: refine commit message. (David Francis)
v3: Remove new BO color. (Christian)
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Perry Yuan <perry.yuan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Same change as for h264, avoids overflow later when calculating
min dpb size.
Signed-off-by: David Rosca <david.rosca@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
A PRT/sparse mapping has no backing BO, so its bo_va->base.bo is NULL.
amdgpu_vm_bo_base_init() deliberately keeps such a bo_va off the vm_bo
state lists, but the tail of amdgpu_vm_bo_update() unconditionally called
amdgpu_vm_bo_idle() for the !always_valid case, putting the NULL-bo PRT
bo_va onto the individual.idle list.
On a GPU reset amdgpu_vm_bo_reset_state_machine() moves individual.idle
to individual.needs_update with moved=true, and amdgpu_vm_handle_moved()
then dereferences bo_va->base.bo to read its reservation object,
crashing on the NULL bo (e.g. the userq eviction restore worker running
during a reset while a user queue is torn down):
BUG: kernel NULL pointer dereference, address: 0000000000000158
RIP: 0010:amdgpu_vm_handle_moved+0x17a/0x200 [amdgpu]
Call Trace:
amdgpu_userq_vm_validate_and_restore_queue+0x2ce/0x920 [amdgpu]
amdgpu_userq_restore_worker+0xce/0x210 [amdgpu]
Skip amdgpu_vm_bo_idle() when bo is NULL so a PRT mapping never lands on
a state list in the first place, and refresh the PRT page tables
explicitly in the userq restore path (as the CS path already does) so
sparse mappings survive a VRAM-lost reset. Because the PRT bo_va is off
the state lists, its PTE update fence lands in prt_va->last_pt_update
rather than vm->last_update, so wait on it explicitly before restarting
the queues (mirroring how the CS path syncs that fence).
v2:
- keep the PRT bo_va off the vm_bo state lists instead of NULL-guarding
bo inside amdgpu_vm_handle_moved(); a PRT mapping should never be on
the moved list in the first place (Christian)
v3:
- the PRT PTEs are updated separately, so their fence is in
prt_va->last_pt_update, not vm->last_update; wait on it in the userq
restore path before restarting queues, otherwise the queues could
restart before the sparse PTEs are written (Christian)
Suggested-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The PCIe L1 low‑power settings for NBIF 6.3.1 were never applied due to
unresolved register mapping, which caused the relevant code to be compiled out.
As a result, the PCIe link could not enter L1/L23 power‑down states or transition to L0s.
Properly configure the link control register to enable L1 and L23 power‑down,
and permit L0s link transitions. Keep LTR disabled and let the PCI core enable it
only after verifying end‑to‑end root complex support across switches.
Fixes: 894c6d3522 ("drm/amdgpu: Add nbif v6_3_1 ip block support")
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Signed-off-by: Kenneth Feng <Kenneth.feng@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
After a recent change VCE now hangs when VCE_CMD_END is emitted
after a pipeline sync without VM flush.
Implement insert_end to correctly insert only one VCE_CMD_END per job.
Fixes: bc639a9ead ("drm/amdgpu: always emit the job vm fence")
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Use correct size for message buffer = sizeof(struct ruvd_msg).
Add ITSCALING_TABLE_BUFFER size.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This needs to use pitch instead of width. Also reject pitch
over 4096 to avoid overflow.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This should use actual number of references from the decode
message, instead of maximum derived from level.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Fixes potential overflow in DPB size calculations.
Signed-off-by: David Rosca <david.rosca@amd.com>
Acked-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
dGPUs with an internal PCIe switch expose graphics functions below the
switch downstream port. The automatic ASPM check uses the display
endpoint and evaluates the internal link instead of the host link.
Use the switch upstream port for the check and report the selected
link.
Fixes: 0ab5d711ec ("drm/amd: Refactor `amdgpu_aspm` to be evaluated per device")
Signed-off-by: Yang Wang <kevinyang.wang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
If we end up emitting a VM fence keep GDS and SPM
associated with that fence. If not, emit them as
part of the IB fence.
Reviewed-by: David Rosca <david.rosca@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
We shouldn't return early if we need to emit spm update.
Reviewed-by: David Rosca <david.rosca@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The MES RS64 local memory process/gang context index may be updated,
saved, or restored improperly during queue eviction and restore cycles.
Enabling RS64 memory for MES process/gang context loading introduces
two known issues:
1. Performance regression on userq due to improper context index
handling during RS64 memory access.
2. KFDIPCTest.BasicTest failure on Navi31/32/33: the IPC buffer
GPU VA mapping is not correctly reflected in the restored RS64
process context, causing GPU page faults on the IPC shared
buffer address, which eventually leads to MES firmware becoming
unresponsive and requiring a full GPU reset to recover.
Temporarily disable MES RS64 process/gang context loading until both
issues are resolved.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Access to hw is already suspended so there are no reasons to
disable ttm buffer funcs.
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
AMDGPU_GEM_CREATE checked domain bits against AMDGPU_GEM_DOMAIN_MASK,
but did not validate domain combinations. Userspace could combine
CPU|GTT|VRAM with DOORBELL, GDS, GWS, or OA, making
amdgpu_bo_placement_from_domain() exceed AMDGPU_BO_MAX_PLACEMENTS and
hit BUG_ON().
Allow combinations only within CPU/GTT/VRAM, and require non-CPU/GTT/
VRAM domains to be specified one at a time. Return -EINVAL for invalid
combinations in amdgpu_gem_create_ioctl().
v2: Rename helper from amdgpu_gem_domain_valid() to
amdgpu_gem_are_domains_valid() (Christian)
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_cs_pass1() dispatches on chunk_id once per chunk without
rejecting repeated ids. p->uf_bo is a single-slot field, so a
submission carrying two AMDGPU_CHUNK_ID_FENCE chunks runs
amdgpu_cs_p1_user_fence() twice, and the second run overwrites
p->uf_bo with a freshly referenced BO without dropping the reference
taken by the first.
amdgpu_cs_parser_fini() only unrefs the final p->uf_bo, so every FENCE
chunk but the last leaks a BO reference. The leaked BO outlives handle
close and process exit.
Reject duplicate FENCE chunks the same way commit fec5f8e8c6
("drm/amdgpu: disallow multiple BO_HANDLES chunks in one submit") did
for p->bo_list.
Fixes: d38ceaf99e ("drm/amdgpu: add core driver (v4)")
Reported-by: Yuhao Jiang <danisjiang@gmail.com>
Assisted-by: Claude:claude-opus-5
Cc: stable@vger.kernel.org
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Junrui Luo <moonafterrain@outlook.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
UAPI Changes:
- Remove the default udmabuf size limit of 64MB.
Cross-subsystem Changes:
- Add dmemcg support for eviction, and hook it up for amdgpu and xe.
Core Changes:
- Changes to TTM to be more aggressive when allocating below protection limit!
- Improve dt binding documentation for renesas.
- Add helper to convert physical address back to buddy block,
add that to and improve its kunit test.
Driver Changes:
- Assorted small fixes to ti-sn65dsi86, panthor, imagination, omapdrm,
bridge/synopsys, panel-edp, ssd130x, panel/tdo-tl070wsh30.
- Add Sharp LQ120P1JX51 panel.
- Add dmemcg support to nouveau.
- Various updates and improvements to sun4i, among which YUV and 4k support.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEuXvWqAysSYEJGuVH/lWMcqZwE8MFAmp1AMkACgkQ/lWMcqZw
E8PASxAApPBUp2ArpFt9+gE33YWRb7sqn42AqCbQfbNgL7IZ4u0TkpnY/4SYyhkf
/6ksiS9UReSdtm+1sVEJQuwN7p8ujvfiugvU3uD6EdUxyNeE4u1CHCUex97HeNxn
JXdGJPoY6Mci6gC2fYU4xknGOH/sMlY+sDZ/cXSylVKt0/qJQ/HpiALnrwW+G2nm
p+wj57fp+gB4T9a0ojJ9WldFdR4e2/ilgj5f+ywGza2XZdegi/f/t+H+EVN+MRfx
OyhTO2LepvlYZo2ZEeFlX49dIEquLwEpva2apQE5YsYn/dB2GYtTnI3MGw/eqMFV
e1U4oK88cQQEKtQsBKYYUwuU3rNrFFJUAa0mlj168KHid+7kQHBCj8oSux8mSU7C
aoK8oK/puflJl+vPP/4FMmGuqEl7CvLkiB5xAyE8BJSM4F1fKfm+72SIRFbAnS5X
l5OWsjvXGmZFU4dwiNElh2OGi3COOyp4nc0BvJZh64p5+4Drj2eslCo9uZrOEKqZ
Pnuu/j9NnajvRNl6cwWa5DQ2nNn0jdwQjGYSvaiYwy5NFZLSAK4cKakm8ZPsjJS3
K1sHlAhKKcJzRdBB/iZTNFBwOq3l0+rFCWfMqeb4SGev8nsQaX4iFVOFU77MabAG
0W7+hGIr+76C6MX1rJJR9+GvSnNYYNti+4HdGmio3w+rlC19qHA=
=pXJR
-----END PGP SIGNATURE-----
Merge tag 'drm-misc-next-2026-08-06' of https://gitlab.freedesktop.org/drm/misc/kernel into drm-next
drm-misc-next for v7.3:
UAPI Changes:
- Remove the default udmabuf size limit of 64MB.
Cross-subsystem Changes:
- Add dmemcg support for eviction, and hook it up for amdgpu and xe.
Core Changes:
- Changes to TTM to be more aggressive when allocating below protection limit!
- Improve dt binding documentation for renesas.
- Add helper to convert physical address back to buddy block,
add that to and improve its kunit test.
Driver Changes:
- Assorted small fixes to ti-sn65dsi86, panthor, imagination, omapdrm,
bridge/synopsys, panel-edp, ssd130x, panel/tdo-tl070wsh30.
- Add Sharp LQ120P1JX51 panel.
- Add dmemcg support to nouveau.
- Various updates and improvements to sun4i, among which YUV and 4k support.
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patch.msgid.link/917d462a-8976-4a15-bec4-4513ec51c5c0@linux.intel.com
Register the VRAM manager with the dmem cgroup reclaim infrastructure
so that lowering dmem.max below current VRAM usage triggers TTM
eviction rather than failing with -EBUSY.
Guard place->flags in amdgpu_ttm_bo_eviction_valuable() against NULL,
as the TTM reclaim path passes a NULL place in cgroup drain mode.
Use drmm_cgroup_register_region() so that the region is automatically
unregistered at DRM device release, after drm_dev_unplug() has already
made drm_dev_enter() return false. The drm_dev_enter/exit guard in the
reclaim callback ensures no reclaim work touches the TTM manager after
driver unbind, closing the window between vram_mgr_fini() (called from
drm_driver.release) and the drmm cleanup that unregisters the region.
v3:
- Rebased on fix for uninitialized list and buddy allocator on the
drmm_cgroup_register_region() error path.
v5:
- Rebased on the introduction of struct dmem_cgroup_init.
- Clear the reclaim callback in amdgpu_vram_mgr_fini() to prevent
use-after-free if cgroup reclaim is triggered after driver unbind
while userspace holds an open DRM file descriptor. (Sashiko-bot)
- Switch from drmm_cgroup_register_region() to the raw
dmem_cgroup_register_region() and store the region in
amdgpu_vram_mgr.cg_region. Call dmem_cgroup_unregister_region()
in amdgpu_vram_mgr_fini() after ttm_resource_manager_evict_all()
to drain in-flight reclaim callbacks, and clear man->cg afterwards.
This is required because amdgpu's vram manager fini is called
explicitly during driver unbind, which may precede the DRM device
release and thus precede any drmm-based cleanup. (Sashiko-bot)
v6:
- Fix mgr->cg_region never being assigned, so
dmem_cgroup_unregister_region() in fini silently no-ops on NULL
and leaks the region. (Sashiko-bot)
- Reorder fini to call set_used(false) and evict_all() before
dmem_cgroup_unregister_region(), so ttm_resource_free() can
uncharge via man->cg during eviction; clear man->cg after
unregister. (Sashiko-bot)
v7:
- Move dmem_cgroup_unregister_region() before the early return on
evict_all() failure; not doing so leaves a dangling reclaim callback
pointing to the partially-torn-down VRAM manager, causing a
use-after-free when the cgroup later triggers reclaim. (Sashiko-bot)
- Switch back to drmm_cgroup_register_region() with a drm_dev_enter/
exit guard in the reclaim callback (matching xe), rather than manual
register/unregister. drm_dev_unplug() fires before vram_mgr_fini(),
so drm_dev_enter() returning false prevents any reclaim from touching
the manager during teardown. This also fixes the "vram" name
collision on multi-GPU systems, since drmm_cgroup_register_region()
automatically prefixes with "drm/<pci-addr>/". (Sashiko-bot)
v8:
- Move the dmem cgroup region teardown back into
amdgpu_vram_mgr_fini(): register the region with
dmem_cgroup_register_region() (keeping the "drm/<unique>/vram"
prefix), store it in amdgpu_vram_mgr.cg_region, and unregister it
explicitly after ttm_resource_manager_evict_all(). The v7
drmm-only approach left a use-after-free on the probe-error
teardown path: drm_dev_unplug() is not called there, so the
drm_dev_enter() guard stays open while drm_driver.release destroys
the manager before the drmm action unregisters the region. The
explicit unregister drains in-flight reclaim on every teardown
path; the drm_dev_enter() guard is kept as defense against reclaim
after unplug. (Sashiko-bot)
v9:
- Don't leak a gpu buddy manager on cgroup init failure. (Sashiko-bot)
- Acquire a runtime PM reference (pm_runtime_get_sync/put_autosuspend)
around the TTM reclaim call in amdgpu_vram_mgr_dmem_reclaim().
(Sashiko-bot)
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-7-thomas.hellstrom@linux.intel.com
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
Replace the bare u64 size argument to dmem_cgroup_register_region() and
drmm_cgroup_register_region() with a const struct dmem_cgroup_init *
pointer. The struct currently carries only the size field, but using a
struct makes the API extensible: future callers can supply additional
initialization parameters without adding more positional arguments.
Update all in-tree callers (amdgpu, xe) to use a compound-literal
initializer.
v5:
- Commit introduced.
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-3-thomas.hellstrom@linux.intel.com
Acked-by: Dave Airlie <airlied@redhat.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
drmm_cgroup_register_region() is called before INIT_LIST_HEAD() and
gpu_buddy_init() in amdgpu_vram_mgr_init(). If it fails, the function
returns early and bypasses those initializations.
Since adev->mman.initialized is set to true before amdgpu_vram_mgr_init()
is called, a failure triggers amdgpu_ttm_fini(), which calls
amdgpu_vram_mgr_fini(), which then:
- Calls list_for_each_entry_safe() on reservations_pending and
reserved_pages, whose list_head::next pointers are zero-initialized
(NULL). The loop does not recognize them as empty and dereferences NULL.
- Calls gpu_buddy_fini(), which iterates free_trees[] unconditionally
via for_each_free_tree(). Since mm->free_trees is NULL
(never allocated), this dereferences NULL.
Both result in a kernel panic on the module load error path.
Fix by moving drmm_cgroup_register_region() to after the list and buddy
allocator are fully initialized, so the teardown path is safe to run.
Reported-by: Sashiko-bot <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260428073116.15687-1-thomas.hellstrom@linux.intel.com?part=4
Fixes: 2b624a2c18 ("drm/ttm: Handle cgroup based eviction in TTM")
Cc: Friedrich Vock <friedrich.vock@gmx.de>
Cc: Maarten Lankhorst <dev@lankhorst.se>
Cc: Tejun Heo <tj@kernel.org>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: amd-gfx@lists.freedesktop.org
Cc: dri-devel@lists.freedesktop.org
Cc: stable@vger.kernel.org # v6.14+
Assisted-by: GitHub_Copilot:claude-sonnet-4.6
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com>
Reviewed-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Reviewed-By: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Tested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
Link: https://patch.msgid.link/20260725100036.2372-2-thomas.hellstrom@linux.intel.com
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Maarten Lankhorst <dev@lankhorst.se>
amdgpu_pci_remove() calls drm_dev_unplug() before invoking the fini
routines. After drm_dev_unplug() the drm_dev_enter() guard in
amdgpu_ttm_fini() always returns false, so iounmap() for
aper_base_kaddr is silently skipped. On connected_to_cpu hardware
ioremap_cache() maps the aperture as WB; when iounmap() is skipped the
stale WB PAT entry persists. On reload IP discovery's
memremap(MEMREMAP_WC) on the same aperture range hits a WB/WC conflict,
producing an ioremap error and failing re-probe.
Remove the drm_dev_enter() guard and call iounmap() unconditionally.
The aperture mapping is plain MMIO and does not require device-presence
protection. Surprise-removal cleanup of aper_base_kaddr is already
handled unconditionally by amdgpu_device_unmap_mmio().
Fixes: 62d5f9f711 ("drm/amdgpu: Unmap MMIO mappings when device is not unplugged")
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit fb3f68af9f)
Cc: stable@vger.kernel.org
devres teardown is LIFO. The aperture devres node was registered after
the DRM device node, so devres_release_all() unmaps the aperture before
the DRM device release callback fires amdgpu_device_fini_sw(). IP
sw_fini callbacks (e.g. vcn_v4_0_sw_fini) write to fw_shared through a
pointer derived from aper_base_kaddr, causing a kernel page fault on
probe failure / rollback:
BUG: unable to handle page fault ... PMD 0
RIP: vcn_v4_0_sw_fini+0x7b/0x170 [amdgpu]
Call Trace:
amdgpu_device_fini_sw
amdgpu_driver_release_kms
devm_drm_dev_init_release
devres_release_all
This reverts commit d871e99879.
Fixes: d871e99879 ("drm/amdgpu: fix aperture mapping leak")
Reported-by: Yuansheng Mao <yuansheng.mao@amd.com>
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 336e0cd576)
Cc: stable@vger.kernel.org
Check for mmhub0 rather than mmhub1. Looks like a copy
paste typo.
Fixes: d0c989a0aa ("drm/amd/amdgpu : Use the MES INV_TLBS API for tlb invalidation on gfx12_1")
Cc: Shaoyun Liu <shaoyun.liu@amd.com>
Reviewed-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 0e8faef0aa)
Cc: stable@vger.kernel.org
Allocate each ring buffer separately. A single allocation summing all
ring sizes can exceed the page allocator's MAX_ORDER limit and fail;
per-ring buffers stay small enough to satisfy. The existing allocation
style doesn't capture any ring data if the huge allocation fails.
Splitting into multiple allocations helps to capture as much data as
possible for the core dump.
A failed ring is left with a NULL buffer and skipped when formatting.
Fixes: eea85914d1 ("drm/amdgpu: save ring content before resetting the device")
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Assisted-by: Claude Code
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 3e8e92b789)
Cc: stable@vger.kernel.org
The number of rings with outstanding fences can be large, requiring a
bigger allocation. Such allocations don't need to be physically
contiguous, so use kvzalloc/kvcalloc which fall back to vmalloc when
contiguous memory isn't available. This also matches the existing
kvfree used to free these allocations.
Also guard the allocation with ring_count to avoid passing 0 size to
allocation routines.
Fixes: eea85914d1 ("drm/amdgpu: save ring content before resetting the device")
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 74d48bd6b7)
Cc: stable@vger.kernel.org
On GFX rings, amdgpu_cs_p2_ib() passed user-supplied ib_bytes through
to ib->length_dw without a limit, while ring_emit_ib() encodes length
into packet fields. Oversized values can corrupt adjacent control bits
and destabilize command submission.
Add a per-ring IB packet size limit helper and reject command
submissions exceeding the corresponding dword limit before IB
allocation. Use the documented 20-bit limit for GFX/compute/SDMA/VPE,
and apply the MM fallback limit for other ring types.
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 7f48fa2cf6)
Cc: stable@vger.kernel.org
Creating a user queue can race with a GPU reset. While recovery holds
reset_domain->sem for write, MES is unresponsive, so the ADD_QUEUE from
amdgpu_userq_map_helper() times out (-110) and an otherwise valid queue
create fails:
amdgpu: MES(0) failed to respond to msg=ADD_QUEUE
[drm:mes_userq_map [amdgpu]] *ERROR* Failed to map queue in HW, err (-110)
amdgpu: [drm] *ERROR* ... Failed to map Queue
amdgpu: [drm] *ERROR* ... Failed to create usermode queue
Take reset_domain->sem for read around the map so it runs only once MES
is back up. This mirrors amdgpu_userq_cleanup() and honors the
userq_mutex -> reset_domain->sem order; the reset path never takes
userq_mutex, so there is no deadlock.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit a8e151fe62)
Cc: stable@vger.kernel.org
Move fs_reclaim_acquire() to before all lock acquisitions to eliminate
false positive circular locking dependency warning.
This is a 7.2-cycle regression fix suitable for stable backport.
v3: Address Mikhail Gavrilov technical review:
- Clarify that fs_reclaim_acquire/release pair only REGISTERS the
fs_reclaim lock class, does NOT create a static edge when called
with no locks held
- Explain that the actual fs_reclaim -> notifier_lock edge is
established at runtime during memory reclaim -> MMU notifier path
- Add Cc: Arunpravin PaneerSelvam
v2: Address Mikhail Gavrilov review feedback:
- Fix author name: Michael -> Mikhail Gavrilov in all trailers
- Add Fixes: tag to link regression to original commit
- Add Tested-by: Mikhail Gavrilov (tested on RX 7900 XTX)
Fixes: 1d0f5838b1 ("drm/amdgpu: Add lockdep annotations for lock ordering validation")
Reported-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Analyzed-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Test-case-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Suggested-by: Christian König <christian.koenig@amd.com>
Tested-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Arunpravin PaneerSelvam <Arunpravin.PaneerSelvam@amd.com>
Signed-off-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Acked-by: Arunpravin Paneer Selvam <Arunpravin.PaneerSelvam@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 70a1e9849e)
Cc: stable@vger.kernel.org
Needed to properly lock the interface before using it.
Cc: Sonny Jiang <sonny.jiang@amd.com>
Reviewed-by: Sonny Jiang <sonny.jiang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 8e37aa0bd5)
Cc: stable@vger.kernel.org
TA_CNTL2.TRUNCATE_COORD_MODE selects whether texture coordinate
truncation is D3D9/GL/Vulkan conformant. gfx11 reads it and reports it to
userspace via AMDGPU_IDS_FLAGS_CONFORMANT_TRUNC_COORD, but gfx12 never
read it, so the flag was always reported as 0 and userspace fell back to
the non-conformant path.
Read it in gfx_v12_0_constants_init() like gfx11 does.
Fixes: 52cb80c12e ("drm/amdgpu: Add gfx v12_0 ip block support (v6)")
Signed-off-by: Qiang Yu <Qiang.Yu@amd.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 4261cbc7b0)
Cc: stable@vger.kernel.org
Like jpeg_v5_0_0, in DPG mode the ring reset path only clears the
JPEG_PG_MODE bit and never resets a hung JRBC, so the post-reset ring test
times out and the driver falls back to a full MODE1 reset.
Temporarily force the static power-gating path during the reset so the
stop/start sequence power-cycles the JPEG block (JMI soft reset + power
off/on), matching the jpeg_v4_0 reset.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit e93659cab1)
Cc: stable@vger.kernel.org
Like jpeg_v5_0_0, in DPG mode the ring reset path only clears the
JPEG_PG_MODE bit and never resets a hung JRBC, so the post-reset ring test
times out and the driver falls back to a full MODE1 reset.
Temporarily force the static power-gating path during the reset so the
stop/start sequence power-cycles the JPEG block (JMI soft reset + power
off/on), matching the jpeg_v4_0 reset.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 75a308eef4)
Cc: stable@vger.kernel.org
In DPG mode jpeg_v5_0_0_ring_reset() takes the DPG stop path, which only
clears the JPEG_PG_MODE bit and never resets the JRBC. A hung ring is not
recovered: the post-reset ring test times out and the driver falls back to
a full MODE1 reset.
Temporarily force the static power-gating path during the reset so the
stop/start sequence power-cycles the JPEG block (JMI soft reset + power
off/on), matching the jpeg_v4_0 reset which has no DPG path.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 79b3612827)
Cc: stable@vger.kernel.org
amdgpu_pci_remove() calls drm_dev_unplug() before invoking the fini
routines. After drm_dev_unplug() the drm_dev_enter() guard in
amdgpu_ttm_fini() always returns false, so iounmap() for
aper_base_kaddr is silently skipped. On connected_to_cpu hardware
ioremap_cache() maps the aperture as WB; when iounmap() is skipped the
stale WB PAT entry persists. On reload IP discovery's
memremap(MEMREMAP_WC) on the same aperture range hits a WB/WC conflict,
producing an ioremap error and failing re-probe.
Remove the drm_dev_enter() guard and call iounmap() unconditionally.
The aperture mapping is plain MMIO and does not require device-presence
protection. Surprise-removal cleanup of aper_base_kaddr is already
handled unconditionally by amdgpu_device_unmap_mmio().
Fixes: 62d5f9f711 ("drm/amdgpu: Unmap MMIO mappings when device is not unplugged")
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
devres teardown is LIFO. The aperture devres node was registered after
the DRM device node, so devres_release_all() unmaps the aperture before
the DRM device release callback fires amdgpu_device_fini_sw(). IP
sw_fini callbacks (e.g. vcn_v4_0_sw_fini) write to fw_shared through a
pointer derived from aper_base_kaddr, causing a kernel page fault on
probe failure / rollback:
BUG: unable to handle page fault ... PMD 0
RIP: vcn_v4_0_sw_fini+0x7b/0x170 [amdgpu]
Call Trace:
amdgpu_device_fini_sw
amdgpu_driver_release_kms
devm_drm_dev_init_release
devres_release_all
This reverts commit d871e99879.
Fixes: d871e99879 ("drm/amdgpu: fix aperture mapping leak")
Reported-by: Yuansheng Mao <yuansheng.mao@amd.com>
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Enable IP block soft reset as a GPU recovery method for GFX6
graphics and compute rings.
This improves current user experience on all GFX6 chips.
The current GPU recovery method is a legacy ASIC reset which
always clears the contents of VRAM, which means that a buggy
(hanging) app can crash the whole graphical session, which
is less than ideal. Also on some GPUs the ASIC reset causes
the GPU to fall off the PCIe bus so it's not desireable.
Using GFX IP block soft reset means that we can now
move on from GFX hangs on GFX6 dGPUs without crashing the
whole system.
Tested with the "hard_reset_cp_wait" test case from the
Hang Test Suite created by Natalie Vock and Konstantin Seurer.
This Vulkan testcase waits for an event that never occurs,
effectively a WAIT_REG_MEM packet that intentionally hangs.
IP block soft reset can resolve that hang and allow
the rest of the system to move on and keep functioning
without needing a full ASIC reset.
Tested on the following chips:
Tahiti (FirePro W9000, Radeon HD 7870 XT)
Cape Verde (Radeon R7 450)
Pitcairn (Radeon R9 270X)
Oland (Radeon 430)
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Update the register definition for GRBM_SOFT_RESET
to match what was in the old radeon driver and use
these bits in the soft reset implementation.
For the soft reset, use basically the same
implementation as GFX7-8, the main difference being
the GRBM_SOFT_RESET bit fields and the fact that
GFX6 doesn't have MQD/HQD. Reset every block using
the GRBM, then proceed to reset the GRBM and SEM blocks
using the SRBM.
The soft reset also calls the clock and powergating
functions of the IP block. This is necessary for
correct operation, otherwise the GPU might fall
off the PCIe bus.
Add a gfx_v6_0_late_init() function for consistency
with other GPU generations. This function will
later serve the same purpose as it does on GFX7+
when we get around to enable more IRQs on GFX6.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
COND_EXEC tells the CP to discard the dwords following it
when its condition is zero (false).
This is useful for GPU recovery because it can help reduce
collateral damage during GFX IP block soft reset, meaning
that it reduces the likelyhood that we fail some jobs which
are not guilty of the hang as the IP block soft reset
mechanism clears the condition before doing the reset.
Note that this packet is only 4 DW on GFX6 (as opposed
to GFX7 and newer where it's 5 DW).
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Adjust gfx_v6_0_setup_tcc() to keep it working after
a GFX IP block soft reset. On a soft reset, the
TCP_CHAN_STEER_LO/HI registers are not cleared so
the function needs a slight adjustment to how the
number of active TCCs are calculated.
Additionally, let's expose the disabled TCC mask
in the tcc_disabled_mask field, like on other GPUs.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Emit a compute CLEAR_STATE packet on the compute rings after
the GFX ring already finished executing ME_INITIALIZE and
before the ring test, so that gfx_v6_0_cp_gfx_resume() can
wait until the CLEAR_STATE is complete.
For reference, see si_cp_start() in the old radeon driver.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Clear the WPTR and RPTR at ring initialization.
Additionally clear the ring contents too.
This is necessary so that the IP block soft reset can
bring the rings back to a clean state.
After a reset, the ring contents could contain packets
emitted before the reset, and thus need to be cleared
to prevent the command processor from executing packets
left over in the ring from before the reset.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
In GFX6 GPUs, compute takes the same CP path as graphics.
CP ME command parser executes packets for each ring buffer:
RB0 supports graphics, RB1 and RB2 are compute only.
Initialize all three rings before calling gfx_v6_0_cp_gfx_start()
to make sure they are all in a sane state before execution starts.
Previously, the two compute-only rings were initialized after
the ME had been already started, which could cause the ME to
start executing the ring contents before the rings could be
properly initialized. This happens to work when the HW is first
initialized, but not during an IP block reset where we want
to reinitialize the compute rings before starting the ME
to prevent it from executing garbage from these rings.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On GFX6, the compute rings use the same CP path as
the graphics ring. The only difference is that they
don't support draw commands. (As opposed to GFX7 and
newer which have a separate command parser that is
called MEC for compute queues.)
This means that we have to take into consideration
that the PFP also exists on compute queues on GFX6:
Use PFP for register writes on both graphics and
compute queues.
In the pipeline sync, use the PFP to wait for the
previous fence (and not the ME) to prevent the PFP
from starting to execute the next submission while
the ME is still in the previous submission.
After a VM flush, emit PFP_SYNC_ME on compute
queues as well.
Fixes: 2cd46ad223 ("drm/amdgpu: add graphic pipeline implementation for si v8")
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Implement the emit_switch_buffer() function instead of emitting
them duing emit_ib, emit_pipeline_sync and emit_vm_flush.
Note that it isn't necessary to emit these in both
emit_pipeline_sync() and emit_vm_flush() because
amdgpu_vm_flush() already calls these when calling
either of those functions.
Fixes: 2cd46ad223 ("drm/amdgpu: add graphic pipeline implementation for si v8")
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Enable the MES process/gang context load from MES local memory,
this will reduce the MES firmware and driver handshake latency.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The RAS manager uses the rascore debug mode setting to control whether
MCA data is read directly or collected through PMFW MCA polling and
ClearMcaOnRead. Track that debug mode state in ras_mgr so
amdgpu_ras_get_error_query_mode() can classify queries as direct or
firmware-backed.
Limit the PMFW polling wait in recovery to firmware-backed MODE1 fatal
recovery so other reset paths, such as MODE2, do not pay the delay
unnecessarily.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Unify amdgpu_mes_alloc/free_proc|gang_ctx_index to provide
centralized RS64mem bitmap management for both KGD and KFD.
- Retrieve the bitmap bit for userq contex index based on a
per process granularity.
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On gfx12, when more queues are ready than there are HQD slots
(oversubscription), MES keeps queues unmapped and maps a queue to an HQD
on demand when its doorbell rings; a queue is only refetched once its
per-queue work_done is cleared.
Historically oversubscription was handled via the aggregated doorbell,
but that is a shared level-wide signal: MES cannot tell which queue rang
and never clears a queue's work_done, so it relies on the coarse
oversubscription timer to rescan. gfx12 therefore drives GFX/compute
through the per-queue unmapped-doorbell path instead (the aggregated
doorbell now only serves SDMA), which can pinpoint the ringing queue and
lets MES stay idle otherwise.
But with unmapped-doorbell mode 1 (basic) a ring only sets a coarse
level-wide ready flag and does not read the per-queue bitmap or clear the
ringing queue's work_done. A queue still marked done from a prior dispatch
is skipped and its new work is never fetched: the completion never
signals, while queue creation succeeds and no dmesg error is reported.
Switch to mode 2 (basic+): MES reads the per-queue CP_UNMAPPED_QUEUE
bitmap, identifies which queue rang, and clears its work_done so it is
rescheduled promptly. This affects every queue MES schedules on gfx12
(KFD and DRM user queues), which share the one MES instance and this
global setting.
Tested on gfx1201: concurrent multi-queue dispatch that intermittently
stalled under mode 1 now passes on both the KFD and DRM userq paths.
v2: update commit message: describe the oversubscription case, why the
aggregated doorbell is not used for GFX/compute on gfx12, and why
basic+ (not basic) is required.
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Reviewed-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Set bits on dword 2 like GFX7-8 except load_global_uconfig
which doesn't exist on GFX6.
Emit VS_PARTIAL_FLUSH before VGT_FLUSH like GFX7-8.
For reference see old PAL which explains the bit fields in
this register and that load_global_uconfig doesn't exist on GFX6
and also see gfx_v7_ring_emit_cntxcntl() for the GFX7 code
which this commit follows.
Fixes: 2cd46ad223 ("drm/amdgpu: add graphic pipeline implementation for si v8")
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Check for mmhub0 rather than mmhub1. Looks like a copy
paste typo.
Fixes: d0c989a0aa ("drm/amd/amdgpu : Use the MES INV_TLBS API for tlb invalidation on gfx12_1")
Cc: Shaoyun Liu <shaoyun.liu@amd.com>
Reviewed-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
- Add hooks in various entry points to perform hw/sw init for DCN6 asic
- Add dependent changes needed to enable DCN6 asic
- Update the Makefiles so that DCN6 related newly added sources are compiled
Signed-off-by: Aurabindo Pillai <aurabindo.pillai@amd.com>
Signed-off-by: Roman Li <Roman.Li@amd.com>
Reviewed-by: Ivan Lipski <ivan.lipski@amd.com>
Tested-by: Dan Wheeler <daniel.wheeler@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Allocate each ring buffer separately. A single allocation summing all
ring sizes can exceed the page allocator's MAX_ORDER limit and fail;
per-ring buffers stay small enough to satisfy. The existing allocation
style doesn't capture any ring data if the huge allocation fails.
Splitting into multiple allocations helps to capture as much data as
possible for the core dump.
A failed ring is left with a NULL buffer and skipped when formatting.
Fixes: eea85914d1 ("drm/amdgpu: save ring content before resetting the device")
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Assisted-by: Claude Code
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The number of rings with outstanding fences can be large, requiring a
bigger allocation. Such allocations don't need to be physically
contiguous, so use kvzalloc/kvcalloc which fall back to vmalloc when
contiguous memory isn't available. This also matches the existing
kvfree used to free these allocations.
Also guard the allocation with ring_count to avoid passing 0 size to
allocation routines.
Fixes: eea85914d1 ("drm/amdgpu: save ring content before resetting the device")
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
On GFX rings, amdgpu_cs_p2_ib() passed user-supplied ib_bytes through
to ib->length_dw without a limit, while ring_emit_ib() encodes length
into packet fields. Oversized values can corrupt adjacent control bits
and destabilize command submission.
Add a per-ring IB packet size limit helper and reject command
submissions exceeding the corresponding dword limit before IB
allocation. Use the documented 20-bit limit for GFX/compute/SDMA/VPE,
and apply the MM fallback limit for other ring types.
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Creating a user queue can race with a GPU reset. While recovery holds
reset_domain->sem for write, MES is unresponsive, so the ADD_QUEUE from
amdgpu_userq_map_helper() times out (-110) and an otherwise valid queue
create fails:
amdgpu: MES(0) failed to respond to msg=ADD_QUEUE
[drm:mes_userq_map [amdgpu]] *ERROR* Failed to map queue in HW, err (-110)
amdgpu: [drm] *ERROR* ... Failed to map Queue
amdgpu: [drm] *ERROR* ... Failed to create usermode queue
Take reset_domain->sem for read around the map so it runs only once MES
is back up. This mirrors amdgpu_userq_cleanup() and honors the
userq_mutex -> reset_domain->sem order; the reset path never takes
userq_mutex, so there is no deadlock.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
mes_userq_reset() unmaps the queue via the low-level mes_userq_unmap()
(REMOVE_QUEUE) but does not update queue->state, so the queue still looks
MAPPED. The destroy path then issues a second, redundant REMOVE_QUEUE for
the already-removed queue; for gfx that unmap waits on an EOP that never
arrives, times out (-110) and escalates to a full GPU reset.
Mark the queue UNMAPPED on a successful reset-path unmap so destroy skips
the redundant REMOVE_QUEUE.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
