mirror of
https://github.com/torvalds/linux.git
synced 2026-09-26 01:52:03 +02:00
drm fixes for 7.3-rc5
client: - fix restore of partially initialized client i915: - Fix incorrect RCU teardown order leading to endless loop - Fix DP MST TU and FEC handling for disconnected streams - Fix selective fetch disable, again - Fix export namespace for kunit helpers - Workaround eDP flicker on a specific laptop model xe: - CRI throttle reasons report - TLB invalidation at wedge - SVM eviction and VM close - Display corruption on LNL on Xen PV - W/a fix and addition amdgpu: - Display ref count fix - Userq fixes - VCN 4, 5 reset fixes - Fixes for various error paths - Stack frame size fixes for various combinations of compilers and configs amdkfd: - Possible UAF fix nouveau: - runtime suspend/resume fixes for newer firmware - rcu free the scheduler - fix VRAM pinning - fix double free - fix reference leaks - fix runtime PM leak - fix cursor list usage problems - fix HDMI config rejection without SCDC virtio: - fix a bunch of object/memory leaks in failure paths - add pixel blend mode property to cursor plane - revert prime buffers import - sync shmem backing on guest transfers imagination: - propogate map failures properly - fix page count in map interface - clamp freelist reconstruction requests ivpu: - use separate flag for job timeout bridge: - samsung-dsim: fix TE GPIO lifetime for host attach -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEEKbZHaGwW9KfbeusDHTzWXnEhr4FAmq29FYACgkQDHTzWXnE hr4pnw/+JhcAdts2saCNt/j7EtD6DqXJvyfo4BVyjYCSVP3YEZuZCrpBJh02UUUo sU8MgjmzwXHRxflTk8DLQMMKhN7cvFmzdREro5x7gJkCi2m/x3eanafQJS0iyvqN rl+K+rVeEmrOELN7auNFSl3kBtvR9yvbPZtBKNNe1IhCSq/iWKm5I4HLwP85diJZ OHOPuXXh+QFhtXsbwH3p1NsbWb/Kiovbevqitd/X/9CFZRMiiwuy5QFX8xOAW4CR 0id0njQPWVbLupuqZRGSDMAsqiVq/ClKIb8X0uMzU92y6l8oMsamPXxbs7Z6pfsT d8k9+zMP0LXXkwdS/sTI/9axJ5LZmd4mqEL+iAkoKkO2EGnDerzyklOCWtk38qvc qpjhQDW77AGLoQbISKZzI2Ce9B3RotdDghyd0Bt++GJegKQAAvevEQKNzWVvlMd3 aQcpI/EKf1PXJnX44LtDqNmoEmmEy4fKOICu52GzpPYOGaSQL25tuA7/KGNX35D8 hY7KiJRYaPcU0hAwAPLvQDHUvFBhDJD4BHGO01YOZ0fZzBARwE57Hwp3hxkHez64 CbcVZ5bzTREmZ3KHKI79OMhKUfCbq+E6YoJ1jxSDs902LWTrULx/iMCA+7NTZL1U f/9nIe2fkA0/D+++CraKd+HYkfdxvZZwuRuZtp5jWqRLxB4bdes= =oRa2 -----END PGP SIGNATURE----- Merge tag 'drm-fixes-2026-09-26' of https://gitlab.freedesktop.org/drm/kernel Pull drm fixes from Dave Airlie: "While most of this is AI inspired fixes for error handling paths, leaks and use after frees, there are some normal things. nouveau has probably the biggest changes with some fixes to stabilise runtime suspend/resume on 570 firmware which regressed after we moved from 535, there are some fixes to stackframe issues seen with amdgpu, and otherwise the usual bunch of i915/xe/amdgpu fixes, and some virtio-gpu fixes. Hopefully it will start to quiten down a bit from here. client: - fix restore of partially initialized client i915: - Fix incorrect RCU teardown order leading to endless loop - Fix DP MST TU and FEC handling for disconnected streams - Fix selective fetch disable, again - Fix export namespace for kunit helpers - Workaround eDP flicker on a specific laptop model xe: - CRI throttle reasons report - TLB invalidation at wedge - SVM eviction and VM close - Display corruption on LNL on Xen PV - W/a fix and addition amdgpu: - Display ref count fix - Userq fixes - VCN 4, 5 reset fixes - Fixes for various error paths - Stack frame size fixes for various combinations of compilers and configs amdkfd: - Possible UAF fix nouveau: - runtime suspend/resume fixes for newer firmware - rcu free the scheduler - fix VRAM pinning - fix double free - fix reference leaks - fix runtime PM leak - fix cursor list usage problems - fix HDMI config rejection without SCDC virtio: - fix a bunch of object/memory leaks in failure paths - add pixel blend mode property to cursor plane - revert prime buffers import - sync shmem backing on guest transfers imagination: - propogate map failures properly - fix page count in map interface - clamp freelist reconstruction requests ivpu: - use separate flag for job timeout bridge: - samsung-dsim: fix TE GPIO lifetime for host attach" * tag 'drm-fixes-2026-09-26' of https://gitlab.freedesktop.org/drm/kernel: (60 commits) drm/amd/display: Bump frame warning limit for all builds of dml drm/imagination: clamp freelist reconstruction requests drm/imagination: Fix page count for page table for map() interface drm/imagination: Propagate map failures correctly from pvr_mmu_map_sgl() drm/amd/display: Bump frame warning limit for clang builds of dml drm/amd/display: Relax DML frame limit with UBSAN drm/amdgpu: Fix runtime PM leak in amdgpu_debugfs_test_ib_show() drm/amdgpu: Fix last_update fence leak in amdgpu_vm_init() drm/amdgpu: Fix acpi device leak in amdgpu_acpi_enumerate_xcc() drm/amdgpu: Fix vmid_wait fence leak in amdgpu_ring_init() drm/amdkfd: fix use-after-free and multi-container gap in kfd_dev_mapping drm/amdgpu/vcn4.0.3: fix video_timeout unit mismatch in jpeg reset wait drm/amdgpu/vcn5.0.1: fix video_timeout unit mismatch in jpeg reset wait drm/amdgpu/userq: fix double jiffies conversion in hang detect timeout drm/amdgpu: move userq fence wait out of signalling section drm/amd/display: Fix dc stream excess put in dm_update_crtc_state() drm/xe: Add wa_14025941587 to xe2, xe3 and xe3p platforms drm/xe: harden adjust_idledly() against divide-by-zero and overflow drm/xe: Limit sg segment size to PAGE_SIZE on Xen PV drm/i915: fix incorrect RCU teardown order ...
This commit is contained in:
Linus Torvalds
commit
6812ce4e43
|
|
@ -515,6 +515,7 @@ void ivpu_prepare_for_reset(struct ivpu_device *vdev)
|
|||
{
|
||||
ivpu_hw_irq_disable(vdev);
|
||||
disable_irq(vdev->irq);
|
||||
atomic_set(&vdev->job_timeout_detected, 0);
|
||||
flush_work(&vdev->irq_dct_work);
|
||||
flush_work(&vdev->context_abort_work);
|
||||
flush_work(&vdev->job_destroy_work);
|
||||
|
|
@ -710,7 +711,7 @@ static int ivpu_dev_init(struct ivpu_device *vdev)
|
|||
vdev->context_xa_limit.max = IVPU_USER_CONTEXT_MAX_SSID;
|
||||
atomic64_set(&vdev->unique_id_counter, 0);
|
||||
atomic_set(&vdev->job_timeout_counter, 0);
|
||||
atomic_set(&vdev->faults_detected, 0);
|
||||
atomic_set(&vdev->job_timeout_detected, 0);
|
||||
xa_init_flags(&vdev->context_xa, XA_FLAGS_ALLOC | XA_FLAGS_LOCK_IRQ);
|
||||
xa_init_flags(&vdev->submitted_jobs_xa, XA_FLAGS_ALLOC1);
|
||||
xa_init_flags(&vdev->db_xa, XA_FLAGS_ALLOC1);
|
||||
|
|
|
|||
|
|
@ -171,7 +171,7 @@ struct ivpu_device {
|
|||
struct xarray submitted_jobs_xa;
|
||||
struct ivpu_ipc_consumer job_done_consumer;
|
||||
atomic_t job_timeout_counter;
|
||||
atomic_t faults_detected;
|
||||
atomic_t job_timeout_detected;
|
||||
|
||||
atomic64_t unique_id_counter;
|
||||
|
||||
|
|
|
|||
|
|
@ -621,7 +621,6 @@ bool ivpu_job_handle_engine_error(struct ivpu_device *vdev, u32 job_id, u32 job_
|
|||
* status and ensure both are handled in the same way
|
||||
*/
|
||||
job->file_priv->has_mmu_faults = true;
|
||||
atomic_set(&vdev->faults_detected, 1);
|
||||
queue_work(system_percpu_wq, &vdev->context_abort_work);
|
||||
return true;
|
||||
}
|
||||
|
|
@ -1175,10 +1174,10 @@ static int reset_engine_and_mark_faulty_contexts(struct ivpu_device *vdev)
|
|||
return ret;
|
||||
|
||||
/*
|
||||
* If faults are detected, ignore guilty contexts from engine reset as NPU may not be stuck
|
||||
* and could return currently running good context and faulty contexts are already marked
|
||||
* If job timeout is detected, read guilty context from engine reset, for other reasons
|
||||
* faulty context is already known
|
||||
*/
|
||||
if (atomic_cmpxchg(&vdev->faults_detected, 1, 0) == 1)
|
||||
if (atomic_cmpxchg(&vdev->job_timeout_detected, 1, 0) == 0)
|
||||
return 0;
|
||||
|
||||
num_impacted_contexts = resp.payload.engine_reset_done.num_impacted_contexts;
|
||||
|
|
|
|||
|
|
@ -964,7 +964,6 @@ void ivpu_mmu_irq_evtq_handler(struct ivpu_device *vdev)
|
|||
file_priv = xa_load(&vdev->context_xa, ssid);
|
||||
if (file_priv) {
|
||||
if (!READ_ONCE(file_priv->has_mmu_faults)) {
|
||||
atomic_set(&vdev->faults_detected, 1);
|
||||
ivpu_mmu_dump_event(vdev, event);
|
||||
WRITE_ONCE(file_priv->has_mmu_faults, true);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -229,6 +229,7 @@ static void ivpu_job_timeout_work(struct work_struct *work)
|
|||
|
||||
ivpu_jsm_state_dump(vdev);
|
||||
ivpu_dev_coredump(vdev);
|
||||
atomic_set(&vdev->job_timeout_detected, 1);
|
||||
queue_work(system_percpu_wq, &vdev->context_abort_work);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1167,8 +1167,10 @@ int amdgpu_acpi_enumerate_xcc(void)
|
|||
}
|
||||
|
||||
xcc_info = kzalloc_obj(struct amdgpu_acpi_xcc_info);
|
||||
if (!xcc_info)
|
||||
if (!xcc_info) {
|
||||
acpi_dev_put(acpi_dev);
|
||||
return -ENOMEM;
|
||||
}
|
||||
|
||||
INIT_LIST_HEAD(&xcc_info->list);
|
||||
xcc_info->handle = acpi_device_handle(acpi_dev);
|
||||
|
|
|
|||
|
|
@ -1780,8 +1780,10 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)
|
|||
|
||||
/* Avoid accidently unparking the sched thread during GPU reset */
|
||||
r = down_write_killable(&adev->reset_domain->sem);
|
||||
if (r)
|
||||
if (r) {
|
||||
pm_runtime_put_autosuspend(dev->dev);
|
||||
return r;
|
||||
}
|
||||
|
||||
/* hold on the scheduler */
|
||||
for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
|
||||
|
|
|
|||
|
|
@ -68,6 +68,9 @@ amdgpu_eviction_fence_suspend_worker(struct work_struct *work)
|
|||
|
||||
mutex_lock(&uq_mgr->userq_mutex);
|
||||
|
||||
/* Fence waits are not allowed in a fence signalling critical section. */
|
||||
amdgpu_userq_wait_for_signal(uq_mgr);
|
||||
|
||||
/*
|
||||
* This is intentionally after taking the userq_mutex since we do
|
||||
* allocate memory while holding this lock, but only after ensuring that
|
||||
|
|
|
|||
|
|
@ -254,7 +254,6 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
|
|||
ring->adev = adev;
|
||||
ring->num_hw_submission = sched_hw_submission;
|
||||
ring->sched_score = sched_score;
|
||||
ring->vmid_wait = dma_fence_get_stub();
|
||||
|
||||
ring->idx = adev->num_rings++;
|
||||
adev->rings[ring->idx] = ring;
|
||||
|
|
@ -374,6 +373,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
|
|||
|
||||
ring->max_dw = max_dw;
|
||||
ring->hw_prio = hw_prio;
|
||||
ring->vmid_wait = dma_fence_get_stub();
|
||||
|
||||
if (!ring->no_scheduler && ring->funcs->type < AMDGPU_HW_IP_NUM) {
|
||||
hw_ip = ring->funcs->type;
|
||||
|
|
|
|||
|
|
@ -184,27 +184,27 @@ static void amdgpu_userq_hang_detect_work(struct work_struct *work)
|
|||
void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue)
|
||||
{
|
||||
struct amdgpu_device *adev;
|
||||
unsigned long timeout_ms;
|
||||
unsigned long timeout_jiffies;
|
||||
|
||||
adev = queue->userq_mgr->adev;
|
||||
/* Determine timeout based on queue type */
|
||||
switch (queue->queue_type) {
|
||||
case AMDGPU_RING_TYPE_GFX:
|
||||
timeout_ms = adev->gfx_timeout;
|
||||
timeout_jiffies = adev->gfx_timeout;
|
||||
break;
|
||||
case AMDGPU_RING_TYPE_COMPUTE:
|
||||
timeout_ms = adev->compute_timeout;
|
||||
timeout_jiffies = adev->compute_timeout;
|
||||
break;
|
||||
case AMDGPU_RING_TYPE_SDMA:
|
||||
timeout_ms = adev->sdma_timeout;
|
||||
timeout_jiffies = adev->sdma_timeout;
|
||||
break;
|
||||
default:
|
||||
timeout_ms = adev->gfx_timeout;
|
||||
timeout_jiffies = adev->gfx_timeout;
|
||||
break;
|
||||
}
|
||||
|
||||
queue_delayed_work(adev->reset_domain->wq, &queue->hang_detect_work,
|
||||
msecs_to_jiffies(timeout_ms));
|
||||
timeout_jiffies);
|
||||
}
|
||||
|
||||
void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
|
||||
|
|
@ -1272,7 +1272,7 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr)
|
|||
return ret;
|
||||
}
|
||||
|
||||
static void
|
||||
void
|
||||
amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr)
|
||||
{
|
||||
struct amdgpu_usermode_queue *queue;
|
||||
|
|
@ -1291,8 +1291,6 @@ amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr)
|
|||
void
|
||||
amdgpu_userq_evict(struct amdgpu_userq_mgr *uq_mgr)
|
||||
{
|
||||
/* Wait for any pending userqueue fence work to finish */
|
||||
amdgpu_userq_wait_for_signal(uq_mgr);
|
||||
amdgpu_userq_evict_all(uq_mgr);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -162,6 +162,7 @@ void amdgpu_userq_mgr_cancel_reset_work(struct amdgpu_device *adev);
|
|||
void amdgpu_userq_mgr_cancel_resume(struct amdgpu_userq_mgr *userq_mgr);
|
||||
void amdgpu_userq_mgr_fini(struct amdgpu_userq_mgr *userq_mgr);
|
||||
|
||||
void amdgpu_userq_wait_for_signal(struct amdgpu_userq_mgr *uq_mgr);
|
||||
void amdgpu_userq_evict(struct amdgpu_userq_mgr *uq_mgr);
|
||||
|
||||
void amdgpu_userq_ensure_ev_fence(struct amdgpu_userq_mgr *userq_mgr,
|
||||
|
|
|
|||
|
|
@ -2678,6 +2678,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm,
|
|||
amdgpu_bo_unref(&root_bo);
|
||||
|
||||
error_free_delayed:
|
||||
dma_fence_put(vm->last_update);
|
||||
dma_fence_put(vm->last_tlb_flush);
|
||||
dma_fence_put(vm->last_unlocked);
|
||||
ttm_lru_bulk_move_fini(&adev->mman.bdev, &vm->lru_bulk_move);
|
||||
|
|
|
|||
|
|
@ -1689,7 +1689,8 @@ static int vcn_v4_0_3_reset_jpeg_pre_helper(struct amdgpu_device *adev, int inst
|
|||
/* if Jobs are still pending after timeout,
|
||||
* We'll handle them in the bottom helper
|
||||
*/
|
||||
amdgpu_fence_wait_polling(ring, wait_seq, adev->video_timeout);
|
||||
amdgpu_fence_wait_polling(ring, wait_seq,
|
||||
jiffies_to_usecs(adev->video_timeout));
|
||||
}
|
||||
|
||||
return 0;
|
||||
|
|
|
|||
|
|
@ -1335,7 +1335,8 @@ static int vcn_v5_0_1_reset_jpeg_pre_helper(struct amdgpu_device *adev, int inst
|
|||
/* if Jobs are still pending after timeout,
|
||||
* We'll handle them in the bottom helper
|
||||
*/
|
||||
amdgpu_fence_wait_polling(ring, wait_seq, adev->video_timeout);
|
||||
amdgpu_fence_wait_polling(ring, wait_seq,
|
||||
jiffies_to_usecs(adev->video_timeout));
|
||||
}
|
||||
|
||||
return 0;
|
||||
|
|
|
|||
|
|
@ -35,6 +35,7 @@
|
|||
#include <linux/time.h>
|
||||
#include <linux/mm.h>
|
||||
#include <linux/mman.h>
|
||||
#include <linux/pseudo_fs.h>
|
||||
#include <linux/ptrace.h>
|
||||
#include <linux/dma-buf.h>
|
||||
#include <linux/processor.h>
|
||||
|
|
@ -70,18 +71,54 @@ static const struct class kfd_class = {
|
|||
};
|
||||
|
||||
/*
|
||||
* Cache the address space of the chardev on first open so that the reset
|
||||
* path can drop all userspace mappings of doorbell and MMIO ranges via
|
||||
* unmap_mapping_range().
|
||||
* Private pseudo-filesystem for KFD, Provides a stable, module-owned
|
||||
* inode whose address_space is the unmap target for all /dev/kfd
|
||||
* openers during GPU reset.
|
||||
*/
|
||||
static struct address_space *kfd_dev_mapping;
|
||||
static struct vfsmount *kfd_fs_mnt;
|
||||
static int kfd_fs_cnt;
|
||||
|
||||
static int kfd_fs_init_fs_context(struct fs_context *fc)
|
||||
{
|
||||
return init_pseudo(fc, 0x4b464400 /* "KFD" */) ? 0 : -ENOMEM;
|
||||
}
|
||||
|
||||
static struct file_system_type kfd_fs_type = {
|
||||
.name = "kfd",
|
||||
.init_fs_context = kfd_fs_init_fs_context,
|
||||
.kill_sb = kill_anon_super,
|
||||
};
|
||||
|
||||
static struct inode *kfd_fs_inode_new(void)
|
||||
{
|
||||
struct inode *inode;
|
||||
int r;
|
||||
|
||||
r = simple_pin_fs(&kfd_fs_type, &kfd_fs_mnt, &kfd_fs_cnt);
|
||||
if (r < 0)
|
||||
return ERR_PTR(r);
|
||||
|
||||
inode = alloc_anon_inode(kfd_fs_mnt->mnt_sb);
|
||||
if (IS_ERR(inode))
|
||||
simple_release_fs(&kfd_fs_mnt, &kfd_fs_cnt);
|
||||
|
||||
return inode;
|
||||
}
|
||||
|
||||
static void kfd_fs_inode_free(struct inode *inode)
|
||||
{
|
||||
if (inode) {
|
||||
iput(inode);
|
||||
simple_release_fs(&kfd_fs_mnt, &kfd_fs_cnt);
|
||||
}
|
||||
}
|
||||
|
||||
static struct inode *kfd_anon_inode;
|
||||
|
||||
void kfd_dev_unmap_mapping_range(loff_t const holebegin, loff_t const holelen)
|
||||
{
|
||||
struct address_space *mapping = READ_ONCE(kfd_dev_mapping);
|
||||
|
||||
if (mapping)
|
||||
unmap_mapping_range(mapping, holebegin, holelen, 1);
|
||||
if (kfd_anon_inode)
|
||||
unmap_mapping_range(kfd_anon_inode->i_mapping, holebegin, holelen, 1);
|
||||
}
|
||||
|
||||
static inline struct kfd_process_device *kfd_lock_pdd_by_id(struct kfd_process *p, __u32 gpu_id)
|
||||
|
|
@ -107,6 +144,13 @@ int kfd_chardev_init(void)
|
|||
{
|
||||
int err = 0;
|
||||
|
||||
kfd_anon_inode = kfd_fs_inode_new();
|
||||
if (IS_ERR(kfd_anon_inode)) {
|
||||
err = PTR_ERR(kfd_anon_inode);
|
||||
kfd_anon_inode = NULL;
|
||||
return err;
|
||||
}
|
||||
|
||||
kfd_char_dev_major = register_chrdev(0, kfd_dev_name, &kfd_fops);
|
||||
err = kfd_char_dev_major;
|
||||
if (err < 0)
|
||||
|
|
@ -130,6 +174,8 @@ int kfd_chardev_init(void)
|
|||
err_class_create:
|
||||
unregister_chrdev(kfd_char_dev_major, kfd_dev_name);
|
||||
err_register_chrdev:
|
||||
kfd_fs_inode_free(kfd_anon_inode);
|
||||
kfd_anon_inode = NULL;
|
||||
return err;
|
||||
}
|
||||
|
||||
|
|
@ -138,6 +184,8 @@ void kfd_chardev_exit(void)
|
|||
device_destroy(&kfd_class, MKDEV(kfd_char_dev_major, 0));
|
||||
class_unregister(&kfd_class);
|
||||
unregister_chrdev(kfd_char_dev_major, kfd_dev_name);
|
||||
kfd_fs_inode_free(kfd_anon_inode);
|
||||
kfd_anon_inode = NULL;
|
||||
kfd_device = NULL;
|
||||
}
|
||||
|
||||
|
|
@ -150,12 +198,7 @@ static int kfd_open(struct inode *inode, struct file *filep)
|
|||
if (iminor(inode) != 0)
|
||||
return -ENODEV;
|
||||
|
||||
/*
|
||||
* /dev/kfd is a single chardev so all opens share one inode. Cache
|
||||
* its address_space on the first open for use by the reset path.
|
||||
*/
|
||||
if (!READ_ONCE(kfd_dev_mapping))
|
||||
cmpxchg(&kfd_dev_mapping, NULL, inode->i_mapping);
|
||||
filep->f_mapping = kfd_anon_inode->i_mapping;
|
||||
|
||||
is_32bit_user_mode = in_compat_syscall();
|
||||
|
||||
|
|
|
|||
|
|
@ -5583,8 +5583,10 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm,
|
|||
|
||||
skip_modeset:
|
||||
/* Release extra reference */
|
||||
if (new_stream)
|
||||
if (new_stream) {
|
||||
dc_stream_release(new_stream);
|
||||
new_stream = NULL;
|
||||
}
|
||||
new_stream = NULL;
|
||||
|
||||
/*
|
||||
|
|
|
|||
|
|
@ -29,14 +29,14 @@ dml_ccflags := $(CC_FLAGS_FPU)
|
|||
dml_rcflags := $(CC_FLAGS_NO_FPU)
|
||||
|
||||
ifneq ($(CONFIG_FRAME_WARN),0)
|
||||
ifeq ($(filter y,$(CONFIG_KASAN)$(CONFIG_KCSAN)),y)
|
||||
ifeq ($(filter y,$(CONFIG_KASAN)$(CONFIG_KCSAN)$(CONFIG_UBSAN)),y)
|
||||
ifeq ($(CONFIG_CC_IS_CLANG)$(CONFIG_COMPILE_TEST),yy)
|
||||
frame_warn_limit := 4096
|
||||
else
|
||||
frame_warn_limit := 3072
|
||||
endif
|
||||
else
|
||||
frame_warn_limit := 2048
|
||||
frame_warn_limit := 3072
|
||||
endif
|
||||
|
||||
ifeq ($(call test-lt, $(CONFIG_FRAME_WARN), $(frame_warn_limit)),y)
|
||||
|
|
|
|||
|
|
@ -28,14 +28,14 @@ dml2_ccflags := $(CC_FLAGS_FPU)
|
|||
dml2_rcflags := $(CC_FLAGS_NO_FPU)
|
||||
|
||||
ifneq ($(CONFIG_FRAME_WARN),0)
|
||||
ifeq ($(filter y,$(CONFIG_KASAN)$(CONFIG_KCSAN)),y)
|
||||
ifeq ($(filter y,$(CONFIG_KASAN)$(CONFIG_KCSAN)$(CONFIG_UBSAN)),y)
|
||||
ifeq ($(CONFIG_CC_IS_CLANG)$(CONFIG_COMPILE_TEST),yy)
|
||||
frame_warn_limit := 4096
|
||||
else
|
||||
frame_warn_limit := 3072
|
||||
endif
|
||||
else
|
||||
frame_warn_limit := 2056
|
||||
frame_warn_limit := 3072
|
||||
endif
|
||||
|
||||
ifeq ($(call test-lt, $(CONFIG_FRAME_WARN), $(frame_warn_limit)),y)
|
||||
|
|
|
|||
|
|
@ -1862,7 +1862,7 @@ static int samsung_dsim_register_te_irq(struct samsung_dsim *dsi, struct device
|
|||
int te_gpio_irq;
|
||||
int ret;
|
||||
|
||||
dsi->te_gpio = devm_gpiod_get_optional(dev, "te", GPIOD_IN);
|
||||
dsi->te_gpio = gpiod_get_optional(dev, "te", GPIOD_IN);
|
||||
if (!dsi->te_gpio)
|
||||
return 0;
|
||||
else if (IS_ERR(dsi->te_gpio))
|
||||
|
|
|
|||
|
|
@ -42,6 +42,14 @@ static int drm_fbdev_client_restore(struct drm_client_dev *client, bool force)
|
|||
{
|
||||
struct drm_fb_helper *fb_helper = drm_fb_helper_from_client(client);
|
||||
|
||||
/*
|
||||
* The client is registered before the initial fbdev probe.
|
||||
* If probing failed, the client remains registered but there
|
||||
* is no valid fbdev framebuffer to restore.
|
||||
*/
|
||||
if (!fb_helper->info || !fb_helper->fb)
|
||||
return 0;
|
||||
|
||||
drm_fb_helper_restore_fbdev_mode_unlocked(fb_helper, force);
|
||||
|
||||
return 0;
|
||||
|
|
|
|||
|
|
@ -536,7 +536,8 @@ static void i9xx_cursor_disable_sel_fetch_arm(struct intel_dsb *dsb,
|
|||
struct intel_display *display = to_intel_display(plane);
|
||||
enum pipe pipe = plane->pipe;
|
||||
|
||||
if (!crtc_state->enable_psr2_sel_fetch)
|
||||
if (!crtc_state->enable_psr2_sel_fetch &&
|
||||
!crtc_state->clear_psr2_sel_fetch)
|
||||
return;
|
||||
|
||||
intel_de_write_dsb(display, dsb, SEL_FETCH_CUR_CTL(pipe), 0);
|
||||
|
|
@ -569,8 +570,10 @@ static void i9xx_cursor_update_sel_fetch_arm(struct intel_dsb *dsb,
|
|||
struct intel_display *display = to_intel_display(plane);
|
||||
enum pipe pipe = plane->pipe;
|
||||
|
||||
if (!crtc_state->enable_psr2_sel_fetch)
|
||||
if (!crtc_state->enable_psr2_sel_fetch) {
|
||||
i9xx_cursor_disable_sel_fetch_arm(dsb, plane, crtc_state);
|
||||
return;
|
||||
}
|
||||
|
||||
if (drm_rect_height(&plane_state->psr2_sel_fetch_area) > 0) {
|
||||
if (crtc_state->enable_psr2_su_region_et) {
|
||||
|
|
|
|||
|
|
@ -1187,6 +1187,8 @@ struct intel_crtc_state {
|
|||
bool has_sel_update;
|
||||
bool enable_psr2_sel_fetch;
|
||||
bool enable_psr2_su_region_et;
|
||||
/* Drop the stale selective fetch enable bits as selective fetch is turned off */
|
||||
bool clear_psr2_sel_fetch;
|
||||
bool req_psr2_sdp_prior_scanline;
|
||||
bool has_panel_replay;
|
||||
bool link_off_after_as_sdp_when_pr_active;
|
||||
|
|
|
|||
|
|
@ -3,6 +3,8 @@
|
|||
* Copyright © 2026 Intel Corporation
|
||||
*/
|
||||
|
||||
#include <kunit/visibility.h>
|
||||
|
||||
#include <linux/bitops.h>
|
||||
#include <linux/debugfs.h>
|
||||
#include <linux/log2.h>
|
||||
|
|
@ -1302,14 +1304,14 @@ void intel_dp_link_caps_cleanup(struct intel_dp_link_caps *link_caps)
|
|||
const struct intel_dp_link_caps_test_ops i915_display_dp_link_caps_test_ops = {
|
||||
INTEL_DP_LINK_CAPS_TEST_OPS_INIT
|
||||
};
|
||||
EXPORT_SYMBOL(i915_display_dp_link_caps_test_ops);
|
||||
EXPORT_SYMBOL_IF_KUNIT(i915_display_dp_link_caps_test_ops);
|
||||
|
||||
#else
|
||||
|
||||
const struct intel_dp_link_caps_test_ops intel_display_dp_link_caps_test_ops = {
|
||||
INTEL_DP_LINK_CAPS_TEST_OPS_INIT
|
||||
};
|
||||
EXPORT_SYMBOL(intel_display_dp_link_caps_test_ops);
|
||||
EXPORT_SYMBOL_IF_KUNIT(intel_display_dp_link_caps_test_ops);
|
||||
|
||||
#endif /* I915 */
|
||||
|
||||
|
|
|
|||
|
|
@ -2825,14 +2825,14 @@ void intel_dp_link_training_cleanup(struct intel_dp_link_training *link_training
|
|||
const struct intel_dp_link_training_test_ops i915_display_dp_link_training_test_ops = {
|
||||
INTEL_DP_LINK_TRAINING_TEST_OPS_INIT
|
||||
};
|
||||
EXPORT_SYMBOL(i915_display_dp_link_training_test_ops);
|
||||
EXPORT_SYMBOL_IF_KUNIT(i915_display_dp_link_training_test_ops);
|
||||
|
||||
#else
|
||||
|
||||
const struct intel_dp_link_training_test_ops intel_display_dp_link_training_test_ops = {
|
||||
INTEL_DP_LINK_TRAINING_TEST_OPS_INIT
|
||||
};
|
||||
EXPORT_SYMBOL(intel_display_dp_link_training_test_ops);
|
||||
EXPORT_SYMBOL_IF_KUNIT(intel_display_dp_link_training_test_ops);
|
||||
|
||||
#endif /* I915 */
|
||||
|
||||
|
|
|
|||
|
|
@ -852,7 +852,8 @@ static u8 get_pipes_downstream_of_mst_port(struct intel_atomic_state *state,
|
|||
if (&connector->mst.dp->mst.mgr != mst_mgr)
|
||||
continue;
|
||||
|
||||
if (connector->mst.port != parent_port &&
|
||||
if (parent_port &&
|
||||
connector->mst.port != parent_port &&
|
||||
!drm_dp_mst_port_downstream_of_parent(mst_mgr,
|
||||
connector->mst.port,
|
||||
parent_port))
|
||||
|
|
@ -2167,6 +2168,27 @@ bool intel_dp_mst_crtc_needs_modeset(struct intel_atomic_state *state,
|
|||
return false;
|
||||
}
|
||||
|
||||
bool intel_dp_mst_stream_disconnected(struct intel_atomic_state *state,
|
||||
const struct intel_crtc *crtc)
|
||||
{
|
||||
struct intel_connector *connector;
|
||||
|
||||
connector = get_connector_in_state_for_crtc(state, crtc);
|
||||
if (!connector)
|
||||
return false;
|
||||
|
||||
if (!connector->mst.dp)
|
||||
return false;
|
||||
|
||||
if (!connector->mst.dp->mst.mgr.mst_state)
|
||||
return true;
|
||||
|
||||
if (drm_connector_is_unregistered(&connector->base))
|
||||
return true;
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* intel_dp_mst_prepare_probe - Prepare an MST link for topology probing
|
||||
* @intel_dp: DP port object
|
||||
|
|
|
|||
|
|
@ -28,6 +28,8 @@ int intel_dp_mst_atomic_check_link(struct intel_atomic_state *state,
|
|||
struct intel_link_bw_limits *limits);
|
||||
bool intel_dp_mst_crtc_needs_modeset(struct intel_atomic_state *state,
|
||||
struct intel_crtc *crtc);
|
||||
bool intel_dp_mst_stream_disconnected(struct intel_atomic_state *state,
|
||||
const struct intel_crtc *crtc);
|
||||
void intel_dp_mst_prepare_probe(struct intel_dp *intel_dp);
|
||||
bool intel_dp_mst_verify_dpcd_state(struct intel_dp *intel_dp);
|
||||
|
||||
|
|
|
|||
|
|
@ -64,7 +64,8 @@ void intel_link_bw_init_limits(struct intel_atomic_state *state,
|
|||
intel_atomic_get_new_crtc_state(state, crtc);
|
||||
int forced_bpp_x16 = get_forced_link_bpp_x16(state, crtc);
|
||||
|
||||
if (state->base.duplicated && crtc_state) {
|
||||
if ((state->base.duplicated && crtc_state) ||
|
||||
intel_dp_mst_stream_disconnected(state, crtc)) {
|
||||
limits->max_bpp_x16[pipe] = crtc_state->max_link_bpp_x16;
|
||||
if (intel_dsc_enabled_on_link(crtc_state))
|
||||
limits->link_dsc_pipes |= BIT(pipe);
|
||||
|
|
|
|||
|
|
@ -2883,6 +2883,8 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
|
|||
struct intel_crtc *crtc)
|
||||
{
|
||||
struct intel_display *display = to_intel_display(state);
|
||||
const struct intel_crtc_state *old_crtc_state =
|
||||
intel_atomic_get_old_crtc_state(state, crtc);
|
||||
struct intel_crtc_state *crtc_state = intel_atomic_get_new_crtc_state(state, crtc);
|
||||
struct intel_plane_state *new_plane_state, *old_plane_state;
|
||||
struct intel_plane *plane;
|
||||
|
|
@ -2895,6 +2897,19 @@ int intel_psr2_sel_fetch_update(struct intel_atomic_state *state,
|
|||
bool full_update = false, su_area_changed;
|
||||
int i, ret;
|
||||
|
||||
/*
|
||||
* Selective fetch is not always usable, for instance it is dropped
|
||||
* while pipe CRC is active. The planes keep their selective fetch
|
||||
* enable bit set in hardware over that, and a plane disabled while
|
||||
* selective fetch is off never gets the bit cleared. Once selective
|
||||
* fetch comes back the hardware would resume fetching for a plane that
|
||||
* is no longer enabled and keep its DDB range reserved, so have the
|
||||
* plane update drop the bit for every plane of the pipe as selective
|
||||
* fetch is turned off.
|
||||
*/
|
||||
crtc_state->clear_psr2_sel_fetch = old_crtc_state->enable_psr2_sel_fetch &&
|
||||
!crtc_state->enable_psr2_sel_fetch;
|
||||
|
||||
if (!crtc_state->enable_psr2_sel_fetch)
|
||||
return 0;
|
||||
|
||||
|
|
|
|||
|
|
@ -257,6 +257,9 @@ static struct intel_quirk intel_quirks[] = {
|
|||
/* Dell XPS 13 7390 2-in-1 */
|
||||
{ 0x8a52, 0x1028, 0x08b0, quirk_edp_limit_rate_hbr2 },
|
||||
|
||||
/* HP Pavilion Plus Laptop 14-ew1xxx */
|
||||
{ 0x7d55, 0x103c, 0x8c31, quirk_edp_limit_rate_hbr2 },
|
||||
|
||||
/* Xiaomi Book Pro 14 2026 */
|
||||
{ 0xb081, 0x1d72, 0x2424, quirk_disable_psr2 },
|
||||
};
|
||||
|
|
|
|||
|
|
@ -885,7 +885,8 @@ static void icl_plane_disable_sel_fetch_arm(struct intel_dsb *dsb,
|
|||
struct intel_display *display = to_intel_display(plane);
|
||||
enum pipe pipe = plane->pipe;
|
||||
|
||||
if (!crtc_state->enable_psr2_sel_fetch)
|
||||
if (!crtc_state->enable_psr2_sel_fetch &&
|
||||
!crtc_state->clear_psr2_sel_fetch)
|
||||
return;
|
||||
|
||||
intel_de_write_dsb(display, dsb, SEL_FETCH_PLANE_CTL(pipe, plane->id), 0);
|
||||
|
|
@ -1634,10 +1635,8 @@ static void icl_plane_update_sel_fetch_arm(struct intel_dsb *dsb,
|
|||
struct intel_display *display = to_intel_display(plane);
|
||||
enum pipe pipe = plane->pipe;
|
||||
|
||||
if (!crtc_state->enable_psr2_sel_fetch)
|
||||
return;
|
||||
|
||||
if (drm_rect_height(&plane_state->psr2_sel_fetch_area) > 0)
|
||||
if (crtc_state->enable_psr2_sel_fetch &&
|
||||
drm_rect_height(&plane_state->psr2_sel_fetch_area) > 0)
|
||||
intel_de_write_dsb(display, dsb, SEL_FETCH_PLANE_CTL(pipe, plane->id),
|
||||
SEL_FETCH_PLANE_CTL_ENABLE);
|
||||
else
|
||||
|
|
|
|||
|
|
@ -89,6 +89,7 @@ struct drm_i915_gem_object *i915_gem_object_alloc(void)
|
|||
|
||||
void i915_gem_object_free(struct drm_i915_gem_object *obj)
|
||||
{
|
||||
dma_resv_fini(&obj->base._resv);
|
||||
return kmem_cache_free(slab_objects, obj);
|
||||
}
|
||||
|
||||
|
|
@ -144,7 +145,6 @@ void __i915_gem_object_fini(struct drm_i915_gem_object *obj)
|
|||
{
|
||||
mutex_destroy(&obj->mm.get_page.lock);
|
||||
mutex_destroy(&obj->mm.get_dma_page.lock);
|
||||
dma_resv_fini(&obj->base._resv);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@
|
|||
#include "pvr_vm.h"
|
||||
|
||||
#include <drm/drm_gem.h>
|
||||
#include <drm/drm_print.h>
|
||||
#include <linux/slab.h>
|
||||
#include <linux/xarray.h>
|
||||
#include <uapi/drm/pvr_drm.h>
|
||||
|
|
@ -612,13 +613,21 @@ pvr_free_list_process_reconstruct_req(struct pvr_device *pvr_dev,
|
|||
};
|
||||
struct rogue_fwif_freelists_reconstruction_data *resp =
|
||||
&resp_cmd.cmd_data.free_lists_reconstruction_data;
|
||||
u32 count = min_t(u32, req->freelist_count,
|
||||
ARRAY_SIZE(req->freelist_ids));
|
||||
|
||||
for (u32 i = 0; i < req->freelist_count; i++)
|
||||
if (count != req->freelist_count) {
|
||||
drm_warn_once(from_pvr_device(pvr_dev),
|
||||
"Requested reconstruction of %u freelists, limiting to %u\n",
|
||||
req->freelist_count, count);
|
||||
}
|
||||
|
||||
for (u32 i = 0; i < count; i++)
|
||||
pvr_free_list_reconstruct(pvr_dev, req->freelist_ids[i]);
|
||||
|
||||
resp->freelist_count = req->freelist_count;
|
||||
resp->freelist_count = count;
|
||||
memcpy(resp->freelist_ids, req->freelist_ids,
|
||||
req->freelist_count * sizeof(resp->freelist_ids[0]));
|
||||
count * sizeof(resp->freelist_ids[0]));
|
||||
|
||||
WARN_ON(pvr_kccb_send_cmd(pvr_dev, &resp_cmd, NULL));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@
|
|||
#include "pvr_rogue_mmu_defs.h"
|
||||
|
||||
#include <drm/drm_drv.h>
|
||||
#include <drm/drm_print.h>
|
||||
#include <linux/atomic.h>
|
||||
#include <linux/bitops.h>
|
||||
#include <linux/dma-mapping.h>
|
||||
|
|
@ -2335,6 +2336,7 @@ void pvr_mmu_op_context_destroy(struct pvr_mmu_op_context *op_ctx)
|
|||
* pvr_mmu_op_context_create() - Create an MMU op context.
|
||||
* @ctx: MMU context associated with owning VM context.
|
||||
* @sgt: Scatter gather table containing pages pinned for use by this context.
|
||||
* @device_addr: Virtual device address at the start of the requested mapping.
|
||||
* @sgt_offset: Start offset of the requested device-virtual memory mapping.
|
||||
* @size: Size in bytes of the requested device-virtual memory mapping. For an
|
||||
* unmapping, this should be zero so that no page tables are allocated.
|
||||
|
|
@ -2346,8 +2348,9 @@ void pvr_mmu_op_context_destroy(struct pvr_mmu_op_context *op_ctx)
|
|||
*/
|
||||
struct pvr_mmu_op_context *
|
||||
pvr_mmu_op_context_create(struct pvr_mmu_context *ctx, struct sg_table *sgt,
|
||||
u64 sgt_offset, u64 size)
|
||||
u64 device_addr, u64 sgt_offset, u64 size)
|
||||
{
|
||||
u64 start_addr = device_addr + sgt_offset;
|
||||
int err;
|
||||
|
||||
struct pvr_mmu_op_context *op_ctx = kzalloc_obj(*op_ctx);
|
||||
|
|
@ -2363,16 +2366,16 @@ pvr_mmu_op_context_create(struct pvr_mmu_context *ctx, struct sg_table *sgt,
|
|||
if (size) {
|
||||
/*
|
||||
* The number of page table objects we need to prealloc is
|
||||
* indicated by the mapping size, start offset and the sizes
|
||||
* indicated by the mapping size, start address and the sizes
|
||||
* of the areas mapped per PT or PD. The range calculation is
|
||||
* identical to that for the index into a table for a device
|
||||
* address, so we reuse those functions here.
|
||||
*/
|
||||
const u32 l1_start_idx = pvr_page_table_l2_idx(sgt_offset);
|
||||
const u32 l1_end_idx = pvr_page_table_l2_idx(sgt_offset + size);
|
||||
const u32 l1_start_idx = pvr_page_table_l2_idx(start_addr);
|
||||
const u32 l1_end_idx = pvr_page_table_l2_idx(start_addr + size);
|
||||
const u32 l1_count = l1_end_idx - l1_start_idx + 1;
|
||||
const u32 l0_start_idx = pvr_page_table_l1_idx(sgt_offset);
|
||||
const u32 l0_end_idx = pvr_page_table_l1_idx(sgt_offset + size);
|
||||
const u32 l0_start_idx = pvr_page_table_l1_idx(start_addr);
|
||||
const u32 l0_end_idx = pvr_page_table_l1_idx(start_addr + size);
|
||||
const u32 l0_count = l0_end_idx - l0_start_idx + 1;
|
||||
|
||||
/*
|
||||
|
|
@ -2553,7 +2556,9 @@ pvr_mmu_map_sgl(struct pvr_mmu_op_context *op_ctx, struct scatterlist *sgl,
|
|||
|
||||
err_destroy_pages:
|
||||
memcpy(&op_ctx->curr_page, &ptr_copy, sizeof(op_ctx->curr_page));
|
||||
err = pvr_mmu_op_context_unmap_curr_page(op_ctx, page);
|
||||
if (pvr_mmu_op_context_unmap_curr_page(op_ctx, page))
|
||||
drm_err(from_pvr_device(op_ctx->mmu_ctx->pvr_dev),
|
||||
"%s : Failure in unmapping pages\n", __func__);
|
||||
|
||||
return err;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -99,7 +99,7 @@ dma_addr_t pvr_mmu_get_root_table_dma_addr(struct pvr_mmu_context *ctx);
|
|||
void pvr_mmu_op_context_destroy(struct pvr_mmu_op_context *op_ctx);
|
||||
struct pvr_mmu_op_context *
|
||||
pvr_mmu_op_context_create(struct pvr_mmu_context *ctx,
|
||||
struct sg_table *sgt, u64 sgt_offset, u64 size);
|
||||
struct sg_table *sgt, u64 device_addr, u64 sgt_offset, u64 size);
|
||||
|
||||
int pvr_mmu_map(struct pvr_mmu_op_context *op_ctx, u64 size, u64 flags,
|
||||
u64 device_addr);
|
||||
|
|
|
|||
|
|
@ -276,7 +276,7 @@ pvr_vm_bind_op_map_init(struct pvr_vm_bind_op *bind_op,
|
|||
goto err_bind_op_fini;
|
||||
|
||||
bind_op->mmu_op_ctx =
|
||||
pvr_mmu_op_context_create(vm_ctx->mmu_ctx, sgt, offset, size);
|
||||
pvr_mmu_op_context_create(vm_ctx->mmu_ctx, sgt, device_addr, offset, size);
|
||||
err = PTR_ERR_OR_ZERO(bind_op->mmu_op_ctx);
|
||||
if (err) {
|
||||
bind_op->mmu_op_ctx = NULL;
|
||||
|
|
@ -318,7 +318,7 @@ pvr_vm_bind_op_unmap_init(struct pvr_vm_bind_op *bind_op,
|
|||
}
|
||||
|
||||
bind_op->mmu_op_ctx =
|
||||
pvr_mmu_op_context_create(vm_ctx->mmu_ctx, NULL, 0, 0);
|
||||
pvr_mmu_op_context_create(vm_ctx->mmu_ctx, NULL, device_addr, 0, 0);
|
||||
err = PTR_ERR_OR_ZERO(bind_op->mmu_op_ctx);
|
||||
if (err) {
|
||||
bind_op->mmu_op_ctx = NULL;
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@
|
|||
|
||||
#define NV_DEVICE_V0_INFO 0x00
|
||||
#define NV_DEVICE_V0_TIME 0x01
|
||||
#define NV_DEVICE_V0_GCX_READY 0x02
|
||||
|
||||
struct nv_device_info_v0 {
|
||||
__u8 version;
|
||||
|
|
@ -55,6 +56,15 @@ struct nv_device_time_v0 {
|
|||
__u64 time;
|
||||
};
|
||||
|
||||
#define NV_DEVICE_GC6_READY BIT(0)
|
||||
#define NV_DEVICE_GCOFF_READY BIT(1)
|
||||
|
||||
struct nv_device_gcx_ready_v0 {
|
||||
__u8 version;
|
||||
__u8 pad01[6];
|
||||
__u8 ready;
|
||||
};
|
||||
|
||||
#define NV_DEVICE_INFO_UNIT (0xffffffffULL << 32)
|
||||
#define NV_DEVICE_INFO(n) ((n) | (0x00000000ULL << 32))
|
||||
#define NV_DEVICE_HOST(n) ((n) | (0x00000001ULL << 32))
|
||||
|
|
|
|||
|
|
@ -22,4 +22,5 @@ int nvif_device_ctor(struct nvif_client *, const char *name, struct nvif_device
|
|||
void nvif_device_dtor(struct nvif_device *);
|
||||
int nvif_device_map(struct nvif_device *);
|
||||
u64 nvif_device_time(struct nvif_device *);
|
||||
int nvif_device_gcx_ready(struct nvif_device *);
|
||||
#endif
|
||||
|
|
|
|||
|
|
@ -156,6 +156,10 @@ struct nvkm_gsp {
|
|||
struct sg_table fbsr;
|
||||
} sr;
|
||||
|
||||
struct {
|
||||
bool use_raw_mode_comptagline_alloc;
|
||||
} memsys;
|
||||
|
||||
struct {
|
||||
struct nvkm_gsp_mem mem;
|
||||
|
||||
|
|
@ -495,6 +499,8 @@ nvkm_gsp_event_dtor(struct nvkm_gsp_event *event)
|
|||
int nvkm_gsp_intr_stall(struct nvkm_gsp *, enum nvkm_subdev_type, int);
|
||||
int nvkm_gsp_intr_nonstall(struct nvkm_gsp *, enum nvkm_subdev_type, int);
|
||||
|
||||
int nvkm_gsp_gcx_ready(struct nvkm_gsp *gsp);
|
||||
|
||||
int gv100_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_gsp **);
|
||||
int tu102_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_gsp **);
|
||||
int tu116_gsp_new(struct nvkm_device *, enum nvkm_subdev_type, int, struct nvkm_gsp **);
|
||||
|
|
|
|||
|
|
@ -578,8 +578,9 @@ int nouveau_bo_pin_locked(struct nouveau_bo *nvbo, uint32_t domain, bool contig)
|
|||
"0x%08x vs 0x%08x\n", bo,
|
||||
bo->resource->mem_type, domain);
|
||||
ret = -EBUSY;
|
||||
} else {
|
||||
ttm_bo_pin(&nvbo->bo);
|
||||
}
|
||||
ttm_bo_pin(&nvbo->bo);
|
||||
goto out;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -600,8 +600,11 @@ nouveau_connector_detect(struct drm_connector *connector, bool force)
|
|||
new_edid = drm_get_edid(connector, nv_encoder->i2c);
|
||||
} else {
|
||||
ret = nvif_outp_edid_get(&nv_encoder->outp, (u8 **)&new_edid);
|
||||
if (ret < 0)
|
||||
if (ret < 0) {
|
||||
pm_runtime_mark_last_busy(dev->dev);
|
||||
pm_runtime_put_autosuspend(dev->dev);
|
||||
return connector_status_disconnected;
|
||||
}
|
||||
}
|
||||
|
||||
nouveau_connector_set_edid(nv_connector, new_edid);
|
||||
|
|
|
|||
|
|
@ -339,8 +339,8 @@ nouveau_dmem_chunk_alloc(struct nouveau_drm *drm, struct page **ppage,
|
|||
chunk->pagemap.ops = &nouveau_dmem_pagemap_ops;
|
||||
chunk->pagemap.owner = drm->dev;
|
||||
|
||||
ret = nouveau_bo_new_pin(&drm->client, NOUVEAU_GEM_DOMAIN_VRAM, DMEM_CHUNK_SIZE,
|
||||
&chunk->bo);
|
||||
ret = nouveau_bo_new_pin(&drm->client, NOUVEAU_GEM_DOMAIN_VRAM,
|
||||
DMEM_CHUNK_SIZE * NR_CHUNKS, &chunk->bo);
|
||||
if (ret)
|
||||
goto out_release;
|
||||
|
||||
|
|
|
|||
|
|
@ -585,6 +585,7 @@ nouveau_drm_device_fini(struct nouveau_drm *drm)
|
|||
if (nouveau_pmops_runtime()) {
|
||||
pm_runtime_get_sync(dev->dev);
|
||||
pm_runtime_forbid(dev->dev);
|
||||
pm_runtime_dont_use_autosuspend(dev->dev);
|
||||
}
|
||||
|
||||
nouveau_led_fini(dev);
|
||||
|
|
@ -1148,6 +1149,7 @@ nouveau_pmops_runtime_suspend(struct device *dev)
|
|||
{
|
||||
struct pci_dev *pdev = to_pci_dev(dev);
|
||||
struct nouveau_drm *drm = pci_get_drvdata(pdev);
|
||||
struct nvif_device *nvif = &drm->client.device;
|
||||
int ret;
|
||||
|
||||
if (!nouveau_pmops_runtime()) {
|
||||
|
|
@ -1155,6 +1157,18 @@ nouveau_pmops_runtime_suspend(struct device *dev)
|
|||
return -EBUSY;
|
||||
}
|
||||
|
||||
// Check if the GPU itself is ready for runtime suspend, otherwise mark as busy and check
|
||||
// again in a bit.
|
||||
ret = nvif_device_gcx_ready(nvif);
|
||||
if (ret < 0) {
|
||||
NV_ERROR(drm, "Failed to query GCX readiness (returned %d)\n", ret);
|
||||
return -EBUSY;
|
||||
} else if (!(ret & NV_DEVICE_GCOFF_READY)) {
|
||||
NV_DEBUG(drm, "GPU isn't ready for suspend yet, delaying...\n");
|
||||
pm_runtime_mark_last_busy(dev);
|
||||
return -EBUSY;
|
||||
}
|
||||
|
||||
nouveau_switcheroo_optimus_dsm();
|
||||
ret = nouveau_do_suspend(drm, true);
|
||||
pci_save_state(pdev);
|
||||
|
|
@ -1250,10 +1264,8 @@ nouveau_drm_open(struct drm_device *dev, struct drm_file *fpriv)
|
|||
mutex_unlock(&drm->clients_lock);
|
||||
|
||||
done:
|
||||
if (ret && cli) {
|
||||
nouveau_cli_fini(cli);
|
||||
if (ret && cli)
|
||||
kfree(cli);
|
||||
}
|
||||
|
||||
pm_runtime_mark_last_busy(dev->dev);
|
||||
pm_runtime_put_autosuspend(dev->dev);
|
||||
|
|
|
|||
|
|
@ -522,6 +522,7 @@ validate_init(struct nouveau_channel *chan, struct drm_file *file_priv,
|
|||
if (unlikely(ret)) {
|
||||
if (ret != -ERESTARTSYS)
|
||||
NV_PRINTK(err, cli, "fail reserve\n");
|
||||
drm_gem_object_put(gem);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
|
@ -531,6 +532,7 @@ validate_init(struct nouveau_channel *chan, struct drm_file *file_priv,
|
|||
struct nouveau_vma *vma = nouveau_vma_find(nvbo, vmm);
|
||||
if (!vma) {
|
||||
NV_PRINTK(err, cli, "vma not found!\n");
|
||||
drm_gem_object_put(gem);
|
||||
ret = -EINVAL;
|
||||
break;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -517,7 +517,7 @@ nouveau_sched_destroy(struct nouveau_sched **psched)
|
|||
struct nouveau_sched *sched = *psched;
|
||||
|
||||
nouveau_sched_fini(sched);
|
||||
kfree(sched);
|
||||
kfree_rcu(sched, rcu);
|
||||
|
||||
*psched = NULL;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -98,6 +98,7 @@ void nouveau_job_free(struct nouveau_job *job);
|
|||
|
||||
struct nouveau_sched {
|
||||
struct drm_gpu_scheduler base;
|
||||
struct rcu_head rcu;
|
||||
struct drm_sched_entity entity;
|
||||
struct workqueue_struct *wq;
|
||||
struct mutex mutex;
|
||||
|
|
|
|||
|
|
@ -846,6 +846,9 @@ op_map(struct nouveau_uvma *uvma)
|
|||
{
|
||||
struct nouveau_bo *nvbo = nouveau_gem_object(uvma->va.gem.obj);
|
||||
|
||||
if (drm_gpuva_invalidated(&uvma->va))
|
||||
return;
|
||||
|
||||
nouveau_uvma_map(uvma, nouveau_mem(nvbo->bo.resource));
|
||||
}
|
||||
|
||||
|
|
@ -1232,6 +1235,7 @@ bind_lock_validate(struct nouveau_job *job, struct drm_exec *exec,
|
|||
|
||||
drm_gpuva_for_each_op(va_op, op->ops) {
|
||||
struct drm_gem_object *obj = op_gem_obj(va_op);
|
||||
struct nouveau_bo *nvbo;
|
||||
|
||||
if (unlikely(!obj))
|
||||
continue;
|
||||
|
|
@ -1246,8 +1250,13 @@ bind_lock_validate(struct nouveau_job *job, struct drm_exec *exec,
|
|||
if (va_op->op == DRM_GPUVA_OP_UNMAP)
|
||||
continue;
|
||||
|
||||
ret = nouveau_bo_validate(nouveau_gem_object(obj),
|
||||
true, false);
|
||||
nvbo = nouveau_gem_object(obj);
|
||||
if (!(nvbo->valid_domains &
|
||||
(NOUVEAU_GEM_DOMAIN_VRAM | NOUVEAU_GEM_DOMAIN_GART)))
|
||||
return -EINVAL;
|
||||
|
||||
nouveau_bo_placement_set(nvbo, nvbo->valid_domains, 0);
|
||||
ret = nouveau_bo_validate(nvbo, true, false);
|
||||
if (ret)
|
||||
return ret;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -38,6 +38,19 @@ nvif_device_time(struct nvif_device *device)
|
|||
return device->user.func->time(&device->user);
|
||||
}
|
||||
|
||||
int
|
||||
nvif_device_gcx_ready(struct nvif_device *device)
|
||||
{
|
||||
struct nv_device_gcx_ready_v0 args = {};
|
||||
int ret;
|
||||
|
||||
ret = nvif_object_mthd(&device->object, NV_DEVICE_V0_GCX_READY, &args, sizeof(args));
|
||||
if (ret)
|
||||
return ret;
|
||||
|
||||
return args.ready;
|
||||
}
|
||||
|
||||
int
|
||||
nvif_device_map(struct nvif_device *device)
|
||||
{
|
||||
|
|
|
|||
|
|
@ -192,6 +192,7 @@ void
|
|||
nvif_vmm_dtor(struct nvif_vmm *vmm)
|
||||
{
|
||||
kfree(vmm->page);
|
||||
vmm->page = NULL;
|
||||
nvif_object_dtor(&vmm->object);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -74,6 +74,7 @@ nvkm_control_mthd_pstate_attr(struct nvkm_control *ctrl, void *data, u32 size)
|
|||
const struct nvkm_domain *domain;
|
||||
struct nvkm_pstate *pstate;
|
||||
struct nvkm_cstate *cstate;
|
||||
bool found = false;
|
||||
int i = 0, j = -1;
|
||||
u32 lo, hi;
|
||||
int ret = -ENOSYS;
|
||||
|
|
@ -104,10 +105,15 @@ nvkm_control_mthd_pstate_attr(struct nvkm_control *ctrl, void *data, u32 size)
|
|||
|
||||
if (args->v0.state != NVIF_CONTROL_PSTATE_ATTR_V0_STATE_CURRENT) {
|
||||
list_for_each_entry(pstate, &clk->states, head) {
|
||||
if (i++ == args->v0.state)
|
||||
if (i++ == args->v0.state) {
|
||||
found = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!found)
|
||||
return -EINVAL;
|
||||
|
||||
lo = pstate->base.domain[domain->name];
|
||||
hi = lo;
|
||||
list_for_each_entry(cstate, &pstate->list, head) {
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@
|
|||
|
||||
#include <core/client.h>
|
||||
#include <subdev/fb.h>
|
||||
#include <subdev/gsp.h>
|
||||
#include <subdev/instmem.h>
|
||||
#include <subdev/timer.h>
|
||||
|
||||
|
|
@ -189,6 +190,38 @@ nvkm_udevice_time(struct nvkm_udevice *udev, void *data, u32 size)
|
|||
return ret;
|
||||
}
|
||||
|
||||
static int
|
||||
nvkm_udevice_gcx_ready(struct nvkm_udevice *udev, void *data, u32 size)
|
||||
{
|
||||
struct nvkm_object *object = &udev->object;
|
||||
struct nvkm_device *device = udev->device;
|
||||
struct nvkm_gsp *gsp = device->gsp;
|
||||
union {
|
||||
struct nv_device_gcx_ready_v0 v0;
|
||||
} *args = data;
|
||||
int ret = -ENOSYS;
|
||||
|
||||
if (!gsp) {
|
||||
args->v0.ready = NV_DEVICE_GC6_READY | NV_DEVICE_GCOFF_READY;
|
||||
return 0;
|
||||
}
|
||||
|
||||
nvif_ioctl(object, "device gcx ready size %d\n", size);
|
||||
ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, false);
|
||||
if (!ret) {
|
||||
nvif_ioctl(object, "device gcx ready vers %d\n", args->v0.version);
|
||||
|
||||
ret = nvkm_gsp_gcx_ready(gsp);
|
||||
if (ret < 0)
|
||||
return ret;
|
||||
|
||||
args->v0.ready = ret;
|
||||
ret = 0;
|
||||
}
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
static int
|
||||
nvkm_udevice_mthd(struct nvkm_object *object, u32 mthd, void *data, u32 size)
|
||||
{
|
||||
|
|
@ -199,6 +232,8 @@ nvkm_udevice_mthd(struct nvkm_object *object, u32 mthd, void *data, u32 size)
|
|||
return nvkm_udevice_info(udev, data, size);
|
||||
case NV_DEVICE_V0_TIME:
|
||||
return nvkm_udevice_time(udev, data, size);
|
||||
case NV_DEVICE_V0_GCX_READY:
|
||||
return nvkm_udevice_gcx_ready(udev, data, size);
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -253,8 +253,7 @@ nvkm_uoutp_mthd_hdmi(struct nvkm_outp *outp, void *argv, u32 argc)
|
|||
|
||||
if (!ior->func->hdmi ||
|
||||
args->v0.max_ac_packet > 0x1f ||
|
||||
args->v0.rekey > 0x7f ||
|
||||
(args->v0.scdc && !ior->func->hdmi->scdc))
|
||||
args->v0.rekey > 0x7f)
|
||||
return -EINVAL;
|
||||
|
||||
if (!args->v0.enable) {
|
||||
|
|
|
|||
|
|
@ -199,16 +199,18 @@ nvkm_cstate_prog(struct nvkm_clk *clk, struct nvkm_pstate *pstate, int cstatei)
|
|||
}
|
||||
|
||||
if (volt) {
|
||||
ret = nvkm_volt_set_id(volt, cstate->voltage,
|
||||
pstate->base.voltage, clk->temp, -1);
|
||||
if (ret && ret != -ENODEV)
|
||||
nvkm_error(subdev, "failed to lower voltage: %d\n", ret);
|
||||
int err = nvkm_volt_set_id(volt, cstate->voltage,
|
||||
pstate->base.voltage, clk->temp, -1);
|
||||
|
||||
if (err && err != -ENODEV)
|
||||
nvkm_error(subdev, "failed to lower voltage: %d\n", err);
|
||||
}
|
||||
|
||||
if (therm) {
|
||||
ret = nvkm_therm_cstate(therm, pstate->fanspeed, -1);
|
||||
if (ret && ret != -ENODEV)
|
||||
nvkm_error(subdev, "failed to lower fan speed: %d\n", ret);
|
||||
int err = nvkm_therm_cstate(therm, pstate->fanspeed, -1);
|
||||
|
||||
if (err && err != -ENODEV)
|
||||
nvkm_error(subdev, "failed to lower fan speed: %d\n", err);
|
||||
}
|
||||
|
||||
return ret;
|
||||
|
|
@ -270,13 +272,19 @@ nvkm_pstate_prog(struct nvkm_clk *clk, int pstatei)
|
|||
struct nvkm_fb *fb = subdev->device->fb;
|
||||
struct nvkm_pci *pci = subdev->device->pci;
|
||||
struct nvkm_pstate *pstate;
|
||||
bool found = false;
|
||||
int ret, idx = 0;
|
||||
|
||||
list_for_each_entry(pstate, &clk->states, head) {
|
||||
if (idx++ == pstatei)
|
||||
if (idx++ == pstatei) {
|
||||
found = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!found)
|
||||
return -EINVAL;
|
||||
|
||||
nvkm_debug(subdev, "setting performance state %d\n", pstatei);
|
||||
clk->pstate = pstatei;
|
||||
|
||||
|
|
@ -473,6 +481,7 @@ static int
|
|||
nvkm_clk_ustate_update(struct nvkm_clk *clk, int req)
|
||||
{
|
||||
struct nvkm_pstate *pstate;
|
||||
bool found = false;
|
||||
int i = 0;
|
||||
|
||||
if (!clk->allow_reclock)
|
||||
|
|
@ -480,12 +489,14 @@ nvkm_clk_ustate_update(struct nvkm_clk *clk, int req)
|
|||
|
||||
if (req != -1 && req != -2) {
|
||||
list_for_each_entry(pstate, &clk->states, head) {
|
||||
if (pstate->pstate == req)
|
||||
if (pstate->pstate == req) {
|
||||
found = true;
|
||||
break;
|
||||
}
|
||||
i++;
|
||||
}
|
||||
|
||||
if (pstate->pstate != req)
|
||||
if (!found)
|
||||
return -EINVAL;
|
||||
req = i;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -51,6 +51,8 @@ nv1a_ram_new(struct nvkm_fb *fb, struct nvkm_ram **pram)
|
|||
mib = ((mem >> 4) & 127) + 1;
|
||||
}
|
||||
|
||||
pci_dev_put(bridge);
|
||||
|
||||
return nvkm_ram_new_(&nv04_ram_func, fb, NVKM_RAM_TYPE_STOLEN,
|
||||
mib * 1024 * 1024, pram);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@
|
|||
* OTHER DEALINGS IN THE SOFTWARE.
|
||||
*/
|
||||
#include "priv.h"
|
||||
#include <nvif/cl0080.h>
|
||||
|
||||
int
|
||||
nvkm_gsp_intr_nonstall(struct nvkm_gsp *gsp, enum nvkm_subdev_type type, int inst)
|
||||
|
|
@ -47,6 +48,15 @@ nvkm_gsp_intr_stall(struct nvkm_gsp *gsp, enum nvkm_subdev_type type, int inst)
|
|||
return -ENOENT;
|
||||
}
|
||||
|
||||
int
|
||||
nvkm_gsp_gcx_ready(struct nvkm_gsp *gsp)
|
||||
{
|
||||
if (!gsp->rm->api->gsp->gcx_ready)
|
||||
return NV_DEVICE_GC6_READY | NV_DEVICE_GCOFF_READY;
|
||||
|
||||
return gsp->rm->api->gsp->gcx_ready(gsp);
|
||||
}
|
||||
|
||||
static int
|
||||
nvkm_gsp_fini(struct nvkm_subdev *subdev, enum nvkm_suspend_state suspend)
|
||||
{
|
||||
|
|
|
|||
|
|
@ -93,6 +93,7 @@ void r535_gsp_dtor(struct nvkm_gsp *);
|
|||
int r535_gsp_oneinit(struct nvkm_gsp *);
|
||||
int r535_gsp_init(struct nvkm_gsp *);
|
||||
int r535_gsp_fini(struct nvkm_gsp *, enum nvkm_suspend_state suspend);
|
||||
int r535_gsp_gcx_ready(struct nvkm_gsp *gsp);
|
||||
|
||||
int nvkm_gsp_new_(const struct nvkm_gsp_fwif *, struct nvkm_device *, enum nvkm_subdev_type, int,
|
||||
struct nvkm_gsp **);
|
||||
|
|
|
|||
|
|
@ -1782,6 +1782,23 @@ r535_gsp_fini(struct nvkm_gsp *gsp, enum nvkm_suspend_state suspend)
|
|||
return 0;
|
||||
}
|
||||
|
||||
int
|
||||
r535_gsp_get_static_memsys_info(struct nvkm_gsp *gsp)
|
||||
{
|
||||
NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS *ctrl;
|
||||
|
||||
ctrl = nvkm_gsp_rm_ctrl_rd(&gsp->internal.device.subdevice,
|
||||
NV2080_CTRL_CMD_INTERNAL_MEMSYS_GET_STATIC_CONFIG,
|
||||
sizeof(*ctrl));
|
||||
if (IS_ERR(ctrl))
|
||||
return PTR_ERR(ctrl);
|
||||
|
||||
gsp->memsys.use_raw_mode_comptagline_alloc = ctrl->bUseRawModeComptaglineAllocation;
|
||||
|
||||
nvkm_gsp_rm_ctrl_done(&gsp->internal.device.subdevice, ctrl);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int
|
||||
r535_gsp_init(struct nvkm_gsp *gsp)
|
||||
{
|
||||
|
|
|
|||
|
|
@ -782,6 +782,51 @@ typedef struct NV2080_CTRL_INTERNAL_INTR_GET_KERNEL_TABLE_PARAMS {
|
|||
|
||||
#define GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE ((48 << 10) * 2048) // Support 2048 channels
|
||||
|
||||
typedef struct NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS {
|
||||
/*! Determines if RM should use 1 to 1 Comptagline allocation policy */
|
||||
NvBool bOneToOneComptagLineAllocation;
|
||||
|
||||
/*! Determines if RM should use 1 to 4 Comptagline allocation policy */
|
||||
NvBool bUseOneToFourComptagLineAllocation;
|
||||
|
||||
/*! Determines if RM should use raw Comptagline allocation policy */
|
||||
NvBool bUseRawModeComptaglineAllocation;
|
||||
|
||||
/*! Has COMPBIT_BACKING_SIZE been overridden to zero (i.e. disabled)? */
|
||||
NvBool bDisableCompbitBacking;
|
||||
|
||||
/*! Determine if we need to disable post L2 compression */
|
||||
NvBool bDisablePostL2Compression;
|
||||
|
||||
/*! Is ECC DRAM feature supported? */
|
||||
NvBool bEnabledEccFBPA;
|
||||
|
||||
NvBool bL2PreFill;
|
||||
|
||||
/*! L2 cache size */
|
||||
NV_DECLARE_ALIGNED(NvU64 l2CacheSize, 8);
|
||||
|
||||
/*! Indicate whether fpba is present or not */
|
||||
NvBool bFbpaPresent;
|
||||
|
||||
/*! Size covered by one comptag */
|
||||
NvU32 comprPageSize;
|
||||
|
||||
/*! log32(comprPageSize) */
|
||||
NvU32 comprPageShift;
|
||||
|
||||
/*! RAM type */
|
||||
NvU32 ramType;
|
||||
|
||||
/*! LTC count */
|
||||
NvU32 ltcCount;
|
||||
|
||||
/*! LTS per LTC count */
|
||||
NvU32 ltsPerLtcCount;
|
||||
} NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS;
|
||||
|
||||
#define NV2080_CTRL_CMD_INTERNAL_MEMSYS_GET_STATIC_CONFIG (0x20800a1c) /* finn: Evaluated from "(FINN_NV20_SUBDEVICE_0_INTERNAL_INTERFACE_ID << 8) | NV2080_CTRL_INTERNAL_MEMSYS_GET_STATIC_CONFIG_PARAMS_MESSAGE_ID" */
|
||||
|
||||
typedef union rpc_message_rpc_union_field_v03_00
|
||||
{
|
||||
NvU32 spare;
|
||||
|
|
|
|||
|
|
@ -26,6 +26,52 @@ r570_fbsr_suspend_channels(struct nvkm_gsp *gsp, bool suspend)
|
|||
return nvkm_gsp_rm_ctrl_wr(&gsp->internal.device.subdevice, ctrl);
|
||||
}
|
||||
|
||||
static int
|
||||
r570_fb_get_compbit_store_size(struct nvkm_gsp *gsp, u64 *size)
|
||||
{
|
||||
NV0080_CTRL_FB_GET_COMPBIT_STORE_INFO_PARAMS *ctrl;
|
||||
|
||||
ctrl = nvkm_gsp_rm_ctrl_rd(&gsp->internal.device.object,
|
||||
NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO,
|
||||
sizeof(*ctrl));
|
||||
if (IS_ERR(ctrl))
|
||||
return PTR_ERR(ctrl);
|
||||
|
||||
*size = ctrl->Size;
|
||||
|
||||
nvkm_gsp_rm_ctrl_done(&gsp->internal.device.object, ctrl);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int
|
||||
r570_memsys_enable_raw_comp_mode(struct nvkm_gsp *gsp, bool enable)
|
||||
{
|
||||
NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS *ctrl;
|
||||
int ret;
|
||||
|
||||
ctrl = nvkm_gsp_rm_ctrl_get(&gsp->internal.device.subdevice,
|
||||
NV2080_CTRL_CMD_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE,
|
||||
sizeof(*ctrl));
|
||||
if (IS_ERR(ctrl))
|
||||
return PTR_ERR(ctrl);
|
||||
|
||||
ctrl->bRawMode = enable;
|
||||
|
||||
ret = nvkm_gsp_rm_ctrl_wr(&gsp->internal.device.subdevice, ctrl);
|
||||
if (!ret)
|
||||
nvkm_debug(&gsp->subdev, "memsys: Raw compression mode %s\n",
|
||||
str_enabled_disabled(enable));
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
static bool
|
||||
r570_need_raw_comp_war(struct nvkm_gsp *gsp, struct nvkm_device *device)
|
||||
{
|
||||
return (device->card_type == GA100 || device->card_type == AD100) &&
|
||||
gsp->memsys.use_raw_mode_comptagline_alloc;
|
||||
}
|
||||
|
||||
static void
|
||||
r570_fbsr_resume(struct nvkm_gsp *gsp)
|
||||
{
|
||||
|
|
@ -33,6 +79,7 @@ r570_fbsr_resume(struct nvkm_gsp *gsp)
|
|||
struct nvkm_instmem *imem = device->imem;
|
||||
struct nvkm_instobj *iobj;
|
||||
struct nvkm_vmm *vmm;
|
||||
int ret;
|
||||
|
||||
/* Restore BAR2 page tables via BAR0 window, and re-enable BAR2. */
|
||||
list_for_each_entry(iobj, &imem->boot, head) {
|
||||
|
|
@ -54,6 +101,13 @@ r570_fbsr_resume(struct nvkm_gsp *gsp)
|
|||
vmm = nvkm_bar_bar1_vmm(device);
|
||||
vmm->func->flush(vmm, 0);
|
||||
|
||||
/* Re-enable raw mode if it was previously disabled */
|
||||
if (r570_need_raw_comp_war(gsp, device)) {
|
||||
ret = r570_memsys_enable_raw_comp_mode(gsp, true);
|
||||
if (ret)
|
||||
nvkm_error(&gsp->subdev, "Failed to re-enable raw comp mode\n");
|
||||
}
|
||||
|
||||
/* Resume channel scheduling. */
|
||||
r570_fbsr_suspend_channels(device->gsp, false);
|
||||
|
||||
|
|
@ -81,7 +135,7 @@ r570_fbsr_init(struct nvkm_gsp *gsp, struct sg_table *sgt, u64 size)
|
|||
ctrl->hClient = gsp->internal.client.object.handle;
|
||||
ctrl->hSysMem = memlist.handle;
|
||||
ctrl->sysmemAddrOfSuspendResumeData = gsp->sr.meta.addr;
|
||||
ctrl->bEnteringGcoffState = 0;
|
||||
ctrl->bEnteringGcoffState = 1;
|
||||
|
||||
ret = nvkm_gsp_rm_ctrl_wr(&gsp->internal.device.subdevice, ctrl);
|
||||
if (ret)
|
||||
|
|
@ -98,12 +152,29 @@ r570_fbsr_suspend(struct nvkm_gsp *gsp)
|
|||
struct nvkm_device *device = subdev->device;
|
||||
struct nvkm_instmem *imem = device->imem;
|
||||
struct nvkm_instobj *iobj;
|
||||
u64 size;
|
||||
u64 size, compbit_store_size;
|
||||
int ret;
|
||||
|
||||
/* Stop channel scheduling. */
|
||||
r570_fbsr_suspend_channels(gsp, true);
|
||||
|
||||
/* Temporarily disable raw mode to prevent FBSR restore operations from corrupting
|
||||
* compressed surfaces. Required for ampere and ada.
|
||||
*
|
||||
* Nvidia bug #3172217
|
||||
*/
|
||||
if (r570_need_raw_comp_war(gsp, device)) {
|
||||
ret = r570_memsys_enable_raw_comp_mode(gsp, false);
|
||||
if (ret)
|
||||
return ret;
|
||||
}
|
||||
|
||||
ret = r570_fb_get_compbit_store_size(gsp, &compbit_store_size);
|
||||
if (ret < 0)
|
||||
return ret;
|
||||
nvkm_debug(&gsp->subdev, "fbsr: Compbit backing store size: 0x%llx bytes\n",
|
||||
compbit_store_size);
|
||||
|
||||
/* Save BAR2 allocations to system memory. */
|
||||
list_for_each_entry(iobj, &imem->list, head) {
|
||||
if (iobj->preserve) {
|
||||
|
|
@ -126,6 +197,8 @@ r570_fbsr_suspend(struct nvkm_gsp *gsp)
|
|||
size = gsp->fb.heap.size;
|
||||
size += gsp->fb.rsvd_size;
|
||||
size += gsp->fb.bios.vga_workspace.size;
|
||||
size += compbit_store_size;
|
||||
|
||||
nvkm_debug(subdev, "fbsr: size: 0x%llx bytes\n", size);
|
||||
|
||||
ret = nvkm_gsp_sg(device, size, &gsp->sr.fbsr);
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@
|
|||
#include "nvrm/gsp.h"
|
||||
#include "nvrm/rpcfn.h"
|
||||
#include "nvrm/msgfn.h"
|
||||
#include "nvif/cl0080.h"
|
||||
|
||||
#include <core/pci.h>
|
||||
#include <subdev/pci/priv.h>
|
||||
|
|
@ -137,6 +138,14 @@ r570_gsp_get_static_info(struct nvkm_gsp *gsp)
|
|||
}
|
||||
}
|
||||
|
||||
ret = r535_gsp_get_static_memsys_info(gsp);
|
||||
if (ret) {
|
||||
nvkm_error(&gsp->subdev, "Retrieving static memsys info failed\n");
|
||||
return ret;
|
||||
}
|
||||
nvkm_debug(&gsp->subdev, "memsys: Use raw mode for comptag allocations? %s\n",
|
||||
str_yes_no(gsp->memsys.use_raw_mode_comptagline_alloc));
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -215,6 +224,32 @@ r570_gsp_set_rmargs(struct nvkm_gsp *gsp, bool resume)
|
|||
args->bDmemStack = 1;
|
||||
}
|
||||
|
||||
int
|
||||
r570_gsp_gcx_ready(struct nvkm_gsp *gsp)
|
||||
{
|
||||
NV2080_CTRL_INTERNAL_GCX_ENTRY_PREREQUISITE_PARAMS *ctrl;
|
||||
int ret = 0;
|
||||
|
||||
ctrl = nvkm_gsp_rm_ctrl_rd(&gsp->internal.device.subdevice,
|
||||
NV2080_CTRL_CMD_INTERNAL_GCX_ENTRY_PREREQUISITE,
|
||||
sizeof(*ctrl));
|
||||
if (IS_ERR(ctrl))
|
||||
return PTR_ERR(ctrl);
|
||||
|
||||
if (ctrl->bIsGC6Satisfied)
|
||||
ret |= NV_DEVICE_GC6_READY;
|
||||
if (ctrl->bIsGCOFFSatisfied)
|
||||
ret |= NV_DEVICE_GCOFF_READY;
|
||||
|
||||
nvkm_debug(&gsp->subdev,
|
||||
"GCX ready status: GC6=%s GCOFF=%s\n",
|
||||
str_yes_no(ctrl->bIsGC6Satisfied), str_yes_no(ctrl->bIsGCOFFSatisfied));
|
||||
|
||||
nvkm_gsp_rm_ctrl_done(&gsp->internal.device.subdevice, ctrl);
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
||||
const struct nvkm_rm_api_gsp
|
||||
r570_gsp = {
|
||||
.set_rmargs = r570_gsp_set_rmargs,
|
||||
|
|
@ -223,4 +258,5 @@ r570_gsp = {
|
|||
.xlat_mc_engine_idx = r570_gsp_xlat_mc_engine_idx,
|
||||
.drop_post_nocat_record = r570_gsp_drop_post_nocat_record,
|
||||
.sr_data_size = r570_gsp_sr_data_size,
|
||||
.gcx_ready = r570_gsp_gcx_ready,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -16,4 +16,33 @@ typedef struct NV2080_CTRL_INTERNAL_FBSR_INIT_PARAMS {
|
|||
NV_DECLARE_ALIGNED(NvU64 sysmemAddrOfSuspendResumeData, 8);
|
||||
} NV2080_CTRL_INTERNAL_FBSR_INIT_PARAMS;
|
||||
|
||||
#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO (0x801306) /* finn: Evaluated from "(FINN_NV01_DEVICE_0_FB_INTERFACE_ID << 8) | NV0080_CTRL_FB_GET_COMPBIT_STORE_INFO_PARAMS_MESSAGE_ID" */
|
||||
|
||||
typedef struct NV0080_CTRL_FB_GET_COMPBIT_STORE_INFO_PARAMS {
|
||||
NV_DECLARE_ALIGNED(NvU64 Size, 8);
|
||||
NV_DECLARE_ALIGNED(NvU64 Address, 8);
|
||||
NvU32 AddressSpace;
|
||||
NvU32 MaxCompbitLine;
|
||||
NvU32 comptagsPerCacheLine;
|
||||
NvU32 cacheLineSize;
|
||||
NvU32 cacheLineSizePerSlice;
|
||||
NvU32 cacheLineFetchAlignment;
|
||||
NV_DECLARE_ALIGNED(NvU64 backingStoreBase, 8);
|
||||
NvU32 gobsPerComptagPerSlice;
|
||||
NvU32 backingStoreCbcBase;
|
||||
NvU32 comptaglineAllocationPolicy;
|
||||
NV_DECLARE_ALIGNED(NvU64 privRegionStartOffset, 8);
|
||||
NvU32 cbcCoveragePerSlice;
|
||||
} NV0080_CTRL_FB_GET_COMPBIT_STORE_INFO_PARAMS;
|
||||
|
||||
#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO_ADDRESS_SPACE_UNKNOWN 0 // ADDR_UNKNOWN
|
||||
#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO_ADDRESS_SPACE_SYSMEM 1 // ADDR_SYSMEM
|
||||
#define NV0080_CTRL_CMD_FB_GET_COMPBIT_STORE_INFO_ADDRESS_SPACE_FBMEM 2 // ADDR_FBMEM
|
||||
|
||||
#define NV2080_CTRL_CMD_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE (0x20800a6f) /* finn: Evaluated from "(FINN_NV20_SUBDEVICE_0_INTERNAL_INTERFACE_ID << 8) | NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS_MESSAGE_ID" */
|
||||
|
||||
typedef struct NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS {
|
||||
NvBool bRawMode;
|
||||
} NV2080_CTRL_INTERNAL_MEMSYS_PROGRAM_RAW_COMPRESSION_MODE_PARAMS;
|
||||
|
||||
#endif
|
||||
|
|
|
|||
|
|
@ -639,4 +639,11 @@ typedef struct GSP_FMC_BOOT_PARAMS
|
|||
} GSP_FMC_BOOT_PARAMS;
|
||||
|
||||
#define GSP_FW_HEAP_PARAM_BASE_RM_SIZE_GH100 (14 << 20) // Hopper+
|
||||
|
||||
#define NV2080_CTRL_CMD_INTERNAL_GCX_ENTRY_PREREQUISITE (0x2080a7d7)
|
||||
|
||||
typedef struct NV2080_CTRL_INTERNAL_GCX_ENTRY_PREREQUISITE_PARAMS {
|
||||
NvBool bIsGC6Satisfied;
|
||||
NvBool bIsGCOFFSatisfied;
|
||||
} NV2080_CTRL_INTERNAL_GCX_ENTRY_PREREQUISITE_PARAMS;
|
||||
#endif
|
||||
|
|
|
|||
|
|
@ -40,6 +40,7 @@ struct nvkm_rm_api {
|
|||
void (*drop_send_user_shared_data)(struct nvkm_gsp *);
|
||||
void (*drop_post_nocat_record)(struct nvkm_gsp *);
|
||||
u32 (*sr_data_size)(struct nvkm_gsp *);
|
||||
int (*gcx_ready)(struct nvkm_gsp *gsp);
|
||||
} *gsp;
|
||||
|
||||
const struct nvkm_rm_api_rpc {
|
||||
|
|
@ -174,6 +175,8 @@ int r535_gr_chan_new(struct nvkm_gr *, struct nvkm_chan *, const struct nvkm_ocl
|
|||
int r535_gr_promote_ctx(struct r535_gr *, bool golden, struct nvkm_vmm *,
|
||||
struct nvkm_memory **pctxbuf_mem, struct nvkm_vma **pctxbuf_vma,
|
||||
struct nvkm_gsp_object *chan);
|
||||
int r570_gsp_gcx_ready(struct nvkm_gsp *gsp);
|
||||
int r535_gsp_get_static_memsys_info(struct nvkm_gsp *gsp);
|
||||
extern const struct nvkm_rm_api_engine r535_nvdec;
|
||||
extern const struct nvkm_rm_api_engine r535_nvenc;
|
||||
extern const struct nvkm_rm_api_engine r535_nvjpg;
|
||||
|
|
|
|||
|
|
@ -114,6 +114,8 @@ struct virtio_gpu_object {
|
|||
bool dumb;
|
||||
bool created;
|
||||
bool attached;
|
||||
/* a guest-bound transfer is queued and its mapping not yet synced */
|
||||
bool from_host_pending;
|
||||
bool host3d_blob, guest_blob;
|
||||
uint32_t blob_mem, blob_flags;
|
||||
|
||||
|
|
@ -196,6 +198,9 @@ struct virtio_gpu_vbuffer {
|
|||
struct list_head list;
|
||||
|
||||
uint32_t seqno;
|
||||
|
||||
/* guest-bound transfer whose shmem backing needs a CPU sync */
|
||||
bool sync_for_cpu;
|
||||
};
|
||||
|
||||
struct virtio_gpu_output {
|
||||
|
|
|
|||
|
|
@ -45,7 +45,7 @@ static int virtio_gpu_gem_create(struct drm_file *file,
|
|||
|
||||
ret = drm_gem_handle_create(file, &obj->base.base, &handle);
|
||||
if (ret) {
|
||||
drm_gem_object_release(&obj->base.base);
|
||||
drm_gem_object_put(&obj->base.base);
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -185,7 +185,7 @@ static int virtio_gpu_resource_create_ioctl(struct drm_device *dev, void *data,
|
|||
|
||||
ret = drm_gem_handle_create(file, obj, &handle);
|
||||
if (ret) {
|
||||
drm_gem_object_release(obj);
|
||||
drm_gem_object_put(obj);
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
|
@ -261,6 +261,27 @@ static int virtio_gpu_transfer_from_host_ioctl(struct drm_device *dev,
|
|||
if (ret != 0)
|
||||
goto err_put_free;
|
||||
|
||||
if (virtio_gpu_is_shmem(bo) && virtio_gpu_use_dma_api(vgdev->vdev)) {
|
||||
/*
|
||||
* The sync on completion restores the whole mapping, so an
|
||||
* earlier transfer has to be done before this one snapshots it.
|
||||
* Otherwise the snapshot predates anything the CPU wrote once
|
||||
* that transfer's fence signalled, and the later sync would
|
||||
* discard it. Nothing can add a fence behind our back here,
|
||||
* since doing so takes the reservation we already hold.
|
||||
* This writes the pages, so it waits as a writer does. READ
|
||||
* usage covers existing readers.
|
||||
*/
|
||||
long wait = dma_resv_wait_timeout(objs->objs[0]->resv,
|
||||
DMA_RESV_USAGE_READ, true,
|
||||
MAX_SCHEDULE_TIMEOUT);
|
||||
|
||||
if (wait < 0) {
|
||||
ret = wait;
|
||||
goto err_unlock;
|
||||
}
|
||||
}
|
||||
|
||||
fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context, 0);
|
||||
if (!fence) {
|
||||
ret = -ENOMEM;
|
||||
|
|
@ -320,6 +341,28 @@ static int virtio_gpu_transfer_to_host_ioctl(struct drm_device *dev, void *data,
|
|||
if (ret != 0)
|
||||
goto err_put_free;
|
||||
|
||||
/*
|
||||
* A transfer the other way may have queued without yet syncing
|
||||
* its mapping. Pushing the guest pages into it now would
|
||||
* discard what the device wrote there, so wait for that sync:
|
||||
* it runs before the fence it belongs to is signalled. The
|
||||
* flag is only set under this reservation, so it cannot appear
|
||||
* behind our back, and the acquire pairs with the release in
|
||||
* that sync, so finding it clear means the pages it wrote are
|
||||
* visible here too.
|
||||
*/
|
||||
if (smp_load_acquire(&bo->from_host_pending)) {
|
||||
long wait = dma_resv_wait_timeout(objs->objs[0]->resv,
|
||||
DMA_RESV_USAGE_WRITE,
|
||||
true,
|
||||
MAX_SCHEDULE_TIMEOUT);
|
||||
|
||||
if (wait < 0) {
|
||||
ret = wait;
|
||||
goto err_unlock;
|
||||
}
|
||||
}
|
||||
|
||||
ret = -ENOMEM;
|
||||
fence = virtio_gpu_fence_alloc(vgdev, vgdev->fence_drv.context,
|
||||
0);
|
||||
|
|
@ -557,14 +600,14 @@ static int virtio_gpu_resource_create_blob_ioctl(struct drm_device *dev,
|
|||
if (params.blob_flags & VIRTGPU_BLOB_FLAG_USE_CROSS_DEVICE) {
|
||||
ret = virtio_gpu_resource_assign_uuid(vgdev, bo);
|
||||
if (ret) {
|
||||
drm_gem_object_release(obj);
|
||||
drm_gem_object_put(obj);
|
||||
return ret;
|
||||
}
|
||||
}
|
||||
|
||||
ret = drm_gem_handle_create(file, obj, &handle);
|
||||
if (ret) {
|
||||
drm_gem_object_release(obj);
|
||||
drm_gem_object_put(obj);
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -589,6 +589,7 @@ struct drm_plane *virtio_gpu_plane_init(struct virtio_gpu_device *vgdev,
|
|||
struct drm_plane *plane;
|
||||
const uint32_t *formats;
|
||||
int nformats;
|
||||
int ret;
|
||||
|
||||
if (type == DRM_PLANE_TYPE_CURSOR) {
|
||||
formats = virtio_gpu_cursor_formats;
|
||||
|
|
@ -614,5 +615,17 @@ struct drm_plane *virtio_gpu_plane_init(struct virtio_gpu_device *vgdev,
|
|||
drm_plane_create_blend_mode_property(plane,
|
||||
BIT(DRM_MODE_BLEND_PREMULTI));
|
||||
|
||||
if (type == DRM_PLANE_TYPE_CURSOR) {
|
||||
/*
|
||||
* The cursor plane exposes a format with an alpha channel,
|
||||
* which requires a blend mode property. The host blends
|
||||
* premultiplied alpha, matching the property's default.
|
||||
*/
|
||||
ret = drm_plane_create_blend_mode_property(plane,
|
||||
BIT(DRM_MODE_BLEND_PREMULTI));
|
||||
if (ret)
|
||||
return ERR_PTR(ret);
|
||||
}
|
||||
|
||||
return plane;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -349,7 +349,7 @@ struct drm_gem_object *virtgpu_gem_prime_import(struct drm_device *dev,
|
|||
}
|
||||
}
|
||||
|
||||
if (!vgdev->has_resource_blob)
|
||||
if (!vgdev->has_resource_blob || vgdev->has_virgl_3d)
|
||||
return drm_gem_prime_import(dev, buf);
|
||||
|
||||
bo = kzalloc_obj(*bo);
|
||||
|
|
|
|||
|
|
@ -389,10 +389,13 @@ static int virtio_gpu_init_submit(struct virtio_gpu_submit *submit,
|
|||
if ((exbuf->flags & VIRTGPU_EXECBUF_FENCE_FD_OUT) ||
|
||||
exbuf->num_out_syncobjs ||
|
||||
exbuf->num_bo_handles ||
|
||||
drm_fence_event)
|
||||
drm_fence_event) {
|
||||
out_fence = virtio_gpu_fence_alloc(vgdev, fence_ctx, ring_idx);
|
||||
else
|
||||
if (!out_fence)
|
||||
return -ENOMEM;
|
||||
} else {
|
||||
out_fence = NULL;
|
||||
}
|
||||
|
||||
if (drm_fence_event) {
|
||||
err = virtio_gpu_fence_event_create(dev, file, out_fence, ring_idx);
|
||||
|
|
@ -538,6 +541,10 @@ int virtio_gpu_execbuffer_ioctl(struct drm_device *dev, void *data,
|
|||
virtio_gpu_process_post_deps(&submit);
|
||||
virtio_gpu_complete_submit(&submit);
|
||||
cleanup:
|
||||
if (ret && submit.out_fence && submit.out_fence->e) {
|
||||
drm_event_cancel_free(dev, &submit.out_fence->e->base);
|
||||
submit.out_fence->e = NULL;
|
||||
}
|
||||
virtio_gpu_cleanup_submit(&submit);
|
||||
|
||||
return ret;
|
||||
|
|
|
|||
|
|
@ -256,6 +256,33 @@ void virtio_gpu_dequeue_ctrl_func(struct work_struct *work)
|
|||
} while (!virtqueue_enable_cb(vgdev->ctrlq.vq));
|
||||
spin_unlock(&vgdev->ctrlq.qlock);
|
||||
|
||||
/*
|
||||
* Sync guest-bound transfers before signalling anything, so that a
|
||||
* waiter cannot read the backing pages while what the device wrote is
|
||||
* still in a bounce buffer. This cannot be folded into the loop below:
|
||||
* virtio_gpu_fence_event_process() also signals every earlier fence in
|
||||
* the same context, so any entry there may signal this entry's fence.
|
||||
*/
|
||||
list_for_each_entry(entry, &reclaim_list, list) {
|
||||
if (entry->sync_for_cpu) {
|
||||
struct virtio_gpu_object *bo =
|
||||
gem_to_virtio_gpu_obj(entry->objs->objs[0]);
|
||||
|
||||
dma_sync_sgtable_for_cpu(vgdev->vdev->dev.parent,
|
||||
bo->base.sgt, DMA_FROM_DEVICE);
|
||||
/*
|
||||
* Release, so a transfer the other way that skips its
|
||||
* wait on the strength of this cannot go on to read
|
||||
* the backing pages before the sync above is visible.
|
||||
* Nothing orders the two otherwise: where the mapping
|
||||
* bounces on a coherent device the sync is a plain
|
||||
* copy, and dma_direct_sync_sg_for_cpu() emits its
|
||||
* barrier only for the non-coherent case.
|
||||
*/
|
||||
smp_store_release(&bo->from_host_pending, false);
|
||||
}
|
||||
}
|
||||
|
||||
list_for_each_entry(entry, &reclaim_list, list) {
|
||||
resp = (struct virtio_gpu_ctrl_hdr *)entry->resp_buf;
|
||||
|
||||
|
|
@ -1278,12 +1305,31 @@ void virtio_gpu_cmd_transfer_from_host_3d(struct virtio_gpu_device *vgdev,
|
|||
struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(objs->objs[0]);
|
||||
struct virtio_gpu_transfer_host_3d *cmd_p;
|
||||
struct virtio_gpu_vbuffer *vbuf;
|
||||
bool use_dma_api = virtio_gpu_use_dma_api(vgdev->vdev);
|
||||
|
||||
cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
|
||||
memset(cmd_p, 0, sizeof(*cmd_p));
|
||||
|
||||
vbuf->objs = objs;
|
||||
|
||||
if (virtio_gpu_is_shmem(bo) && use_dma_api) {
|
||||
/*
|
||||
* The device writes only the requested box, so prime the
|
||||
* mapping with the current contents: otherwise the sync on
|
||||
* completion would hand back whatever a bounce buffer held for
|
||||
* the regions the device does not touch.
|
||||
*/
|
||||
dma_sync_sgtable_for_device(vgdev->vdev->dev.parent,
|
||||
bo->base.sgt, DMA_TO_DEVICE);
|
||||
vbuf->sync_for_cpu = true;
|
||||
/*
|
||||
* Set under the reservation the caller holds, so a transfer
|
||||
* the other way cannot miss it and push the guest pages into
|
||||
* the mapping while the device still owns it.
|
||||
*/
|
||||
WRITE_ONCE(bo->from_host_pending, true);
|
||||
}
|
||||
|
||||
cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_TRANSFER_FROM_HOST_3D);
|
||||
cmd_p->hdr.ctx_id = cpu_to_le32(ctx_id);
|
||||
cmd_p->resource_id = cpu_to_le32(bo->hw_res_handle);
|
||||
|
|
|
|||
|
|
@ -215,16 +215,12 @@ int virtio_gpu_vram_create(struct virtio_gpu_device *vgdev,
|
|||
|
||||
/* Create fake offset */
|
||||
ret = drm_gem_create_mmap_offset(obj);
|
||||
if (ret) {
|
||||
kfree(vram);
|
||||
return ret;
|
||||
}
|
||||
if (ret)
|
||||
goto err_release_obj;
|
||||
|
||||
ret = virtio_gpu_resource_id_get(vgdev, &vram->base.hw_res_handle);
|
||||
if (ret) {
|
||||
kfree(vram);
|
||||
return ret;
|
||||
}
|
||||
if (ret)
|
||||
goto err_release_obj;
|
||||
|
||||
virtio_gpu_cmd_resource_create_blob(vgdev, &vram->base, params, NULL,
|
||||
0);
|
||||
|
|
@ -240,6 +236,11 @@ int virtio_gpu_vram_create(struct virtio_gpu_device *vgdev,
|
|||
|
||||
*bo_ptr = &vram->base;
|
||||
return 0;
|
||||
|
||||
err_release_obj:
|
||||
drm_gem_object_release(obj);
|
||||
kfree(vram);
|
||||
return ret;
|
||||
}
|
||||
|
||||
void virtio_gpu_vram_map_deferred(struct virtio_gpu_object_vram *vram)
|
||||
|
|
|
|||
|
|
@ -651,6 +651,7 @@
|
|||
#define MEM_THERMAL_MASK REG_BIT(2)
|
||||
#define VR_THERMAL_MASK REG_BIT(3)
|
||||
#define ICCMAX_MASK REG_BIT(4)
|
||||
#define PWRBRK_MASK REG_BIT(5)
|
||||
#define SOC_AVG_THERMAL_MASK REG_BIT(6)
|
||||
#define FASTVMODE_MASK REG_BIT(7)
|
||||
#define PSYS_PL1_MASK REG_BIT(12)
|
||||
|
|
|
|||
|
|
@ -1037,6 +1037,13 @@ static int xe_bo_move(struct ttm_buffer_object *ttm_bo, bool evict,
|
|||
} else {
|
||||
drm_dbg(&xe->drm, "Evict system allocator BO failed=%pe\n",
|
||||
ERR_PTR(ret));
|
||||
/*
|
||||
* The semantic we want upon SVM eviction failure
|
||||
* because of racing access is keep walking for
|
||||
* eviction, which is -ENOSPC.
|
||||
*/
|
||||
if (ret == -EBUSY)
|
||||
ret = -ENOSPC;
|
||||
}
|
||||
|
||||
goto out;
|
||||
|
|
|
|||
|
|
@ -9,6 +9,8 @@
|
|||
#include <drm/drm_prime.h>
|
||||
#include <drm/ttm/ttm_tt.h>
|
||||
|
||||
#include <xen/xen.h>
|
||||
|
||||
#include "xe_bo_types.h"
|
||||
#include "xe_ggtt.h"
|
||||
#include "xe_macros.h"
|
||||
|
|
@ -575,6 +577,23 @@ static inline unsigned int xe_sg_segment_size(struct device *dev)
|
|||
struct scatterlist __maybe_unused sg;
|
||||
size_t max = BIT_ULL(sizeof(sg.length) * 8) - 1;
|
||||
|
||||
/*
|
||||
* For Xen PV guests pages aren't contiguous in DMA (machine) address
|
||||
* space. The DMA API takes care of that both in dma_alloc_* (by
|
||||
* calling into the hypervisor to make the pages contiguous) and in
|
||||
* dma_map_* (by bounce buffering). But xe (like i915, see commit
|
||||
* 78a07fe777c4) ignores the coherency aspects of the DMA API and thus
|
||||
* can't cope with bounce buffering actually happening, so add a hack
|
||||
* here to force small allocations and mappings when running in PV
|
||||
* mode on Xen.
|
||||
*
|
||||
* Note this will still break if bounce buffering is required for other
|
||||
* reasons, like confidential computing hypervisors or PCIe root ports
|
||||
* with addressing limitations.
|
||||
*/
|
||||
if (xen_pv_domain())
|
||||
return PAGE_SIZE;
|
||||
|
||||
max = min_t(size_t, max, dma_max_mapping_size(dev));
|
||||
|
||||
/*
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@
|
|||
* - ``reason_mem_thermal``: Memory thermal
|
||||
* - ``reason_vr_thermal``: VR thermal
|
||||
* - ``reason_iccmax``: ICCMAX
|
||||
* - ``reason_ratl``: RATL thermal algorithm
|
||||
* - ``reason_pwrbrk``: Power brake
|
||||
* - ``reason_soc_avg_thermal``: SoC average temp
|
||||
* - ``reason_fastvmode``: VR is hitting FastVMode
|
||||
* - ``reason_psys_pl1``: PSYS PL1
|
||||
|
|
@ -200,6 +200,7 @@ static THROTTLE_ATTR_RO(reason_psys_pl1, PSYS_PL1_MASK);
|
|||
static THROTTLE_ATTR_RO(reason_psys_pl2, PSYS_PL2_MASK);
|
||||
static THROTTLE_ATTR_RO(reason_p0_freq, P0_FREQ_MASK);
|
||||
static THROTTLE_ATTR_RO(reason_psys_crit, PSYS_CRIT_MASK);
|
||||
static THROTTLE_ATTR_RO(reason_pwrbrk, PWRBRK_MASK);
|
||||
|
||||
static struct attribute *cri_throttle_attrs[] = {
|
||||
/* Common */
|
||||
|
|
@ -209,12 +210,12 @@ static struct attribute *cri_throttle_attrs[] = {
|
|||
&attr_reason_pl2.attr.attr,
|
||||
&attr_reason_pl4.attr.attr,
|
||||
&attr_reason_prochot.attr.attr,
|
||||
&attr_reason_ratl.attr.attr,
|
||||
/* CRI */
|
||||
&attr_reason_vr_thermal.attr.attr,
|
||||
&attr_reason_soc_thermal.attr.attr,
|
||||
&attr_reason_mem_thermal.attr.attr,
|
||||
&attr_reason_iccmax.attr.attr,
|
||||
&attr_reason_pwrbrk.attr.attr,
|
||||
&attr_reason_soc_avg_thermal.attr.attr,
|
||||
&attr_reason_fastvmode.attr.attr,
|
||||
&attr_reason_psys_pl1.attr.attr,
|
||||
|
|
|
|||
|
|
@ -864,7 +864,7 @@ static unsigned int guc_mmio_regset_write(struct xe_guc_ads *ads,
|
|||
}
|
||||
}
|
||||
|
||||
if (XE_GT_WA(hwe->gt, 16023105232))
|
||||
if (XE_GT_WA(hwe->gt, 16023105232) || XE_GT_WA(hwe->gt, 14025941587))
|
||||
guc_mmio_regset_write_one(ads, regset_map,
|
||||
RING_IDLEDLY(hwe->mmio_base),
|
||||
count++);
|
||||
|
|
|
|||
|
|
@ -585,28 +585,102 @@ static void hw_engine_init_early(struct xe_gt *gt, struct xe_hw_engine *hwe,
|
|||
xe_reg_whitelist_process_engine(hwe);
|
||||
}
|
||||
|
||||
static u32 idledly_floor_ticks(u32 idledly_ns, u32 idledly_units_ps)
|
||||
{
|
||||
return DIV_ROUND_DOWN_ULL((u64)idledly_ns * 1000, idledly_units_ps);
|
||||
}
|
||||
|
||||
static void adjust_idledly(struct xe_hw_engine *hwe)
|
||||
{
|
||||
struct xe_gt *gt = hwe->gt;
|
||||
u32 idledly, maxcnt;
|
||||
u32 idledly, idledly_hw, idledly_reg_val, maxcnt;
|
||||
u32 idledly_units_ps = 8 * gt->info.timestamp_base;
|
||||
u32 maxcnt_units_ns = 640;
|
||||
bool inhibit_switch = 0;
|
||||
bool inhibit_switch = false;
|
||||
bool wa_applied = false;
|
||||
bool clamped_below_maxcnt = false;
|
||||
|
||||
if (!IS_SRIOV_VF(gt_to_xe(hwe->gt)) && XE_GT_WA(gt, 16023105232)) {
|
||||
idledly = xe_mmio_read32(>->mmio, RING_IDLEDLY(hwe->mmio_base));
|
||||
if ((!IS_SRIOV_VF(gt_to_xe(gt)) && XE_GT_WA(gt, 16023105232)) ||
|
||||
XE_GT_WA(gt, 14025941587)) {
|
||||
u32 mincnt_idledly_ns = 5000;
|
||||
|
||||
/* xe_gt_clock_init() warns and zeroes timestamp_base on unknown crystal clock. */
|
||||
if (!idledly_units_ps)
|
||||
return;
|
||||
|
||||
idledly_reg_val = xe_mmio_read32(>->mmio, RING_IDLEDLY(hwe->mmio_base));
|
||||
maxcnt = xe_mmio_read32(>->mmio, RING_PWRCTX_MAXCNT(hwe->mmio_base));
|
||||
|
||||
inhibit_switch = idledly & INHIBIT_SWITCH_UNTIL_PREEMPTED;
|
||||
idledly = REG_FIELD_GET(IDLE_DELAY, idledly);
|
||||
idledly = DIV_ROUND_CLOSEST(idledly * idledly_units_ps, 1000);
|
||||
inhibit_switch = idledly_reg_val & INHIBIT_SWITCH_UNTIL_PREEMPTED;
|
||||
idledly = REG_FIELD_GET(IDLE_DELAY, idledly_reg_val);
|
||||
idledly = DIV_ROUND_CLOSEST_ULL((u64)idledly * idledly_units_ps, 1000);
|
||||
idledly_hw = idledly;
|
||||
maxcnt = REG_FIELD_GET(IDLE_WAIT_TIME, maxcnt);
|
||||
maxcnt *= maxcnt_units_ns;
|
||||
|
||||
if (xe_gt_WARN_ON(gt, idledly >= maxcnt || inhibit_switch)) {
|
||||
idledly = DIV_ROUND_CLOSEST(((maxcnt - 1) * 1000),
|
||||
idledly_units_ps);
|
||||
xe_mmio_write32(>->mmio, RING_IDLEDLY(hwe->mmio_base), idledly);
|
||||
/*
|
||||
* Wa_14025941587 is applied before Wa_16023105232, which takes
|
||||
* priority if the two ever conflict (not expected in practice).
|
||||
*/
|
||||
if (XE_GT_WA(gt, 14025941587) &&
|
||||
idledly < mincnt_idledly_ns) {
|
||||
idledly = mincnt_idledly_ns;
|
||||
wa_applied = true;
|
||||
}
|
||||
|
||||
if (XE_GT_WA(gt, 16023105232)) {
|
||||
/* Clear the inhibit switch without disturbing a valid delay. */
|
||||
if (inhibit_switch) {
|
||||
idledly_reg_val &= ~INHIBIT_SWITCH_UNTIL_PREEMPTED;
|
||||
wa_applied = true;
|
||||
}
|
||||
|
||||
/* Warn only on the value read from hardware. */
|
||||
xe_gt_WARN_ON(gt, idledly_hw >= maxcnt);
|
||||
|
||||
if (idledly >= maxcnt) {
|
||||
/* maxcnt may be 0 if IDLE_WAIT_TIME is unprogrammed. */
|
||||
idledly = maxcnt ? maxcnt - 1 : 0;
|
||||
clamped_below_maxcnt = true;
|
||||
wa_applied = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (wa_applied) {
|
||||
u32 idledly_ticks;
|
||||
|
||||
/*
|
||||
* Wa_16023105232 requires idledly < maxcnt, so floor
|
||||
* that clamp; otherwise round up to guarantee the
|
||||
* Wa_14025941587 minimum survives tick quantization.
|
||||
*/
|
||||
if (clamped_below_maxcnt)
|
||||
idledly_ticks = idledly_floor_ticks(idledly, idledly_units_ps);
|
||||
else
|
||||
idledly_ticks = DIV_ROUND_UP_ULL((u64)idledly * 1000,
|
||||
idledly_units_ps);
|
||||
|
||||
/*
|
||||
* Tick quantization can still push the rounded-up value
|
||||
* to/above maxcnt; re-floor here so Wa_16023105232 keeps
|
||||
* priority even in that case.
|
||||
*/
|
||||
if (!clamped_below_maxcnt && XE_GT_WA(gt, 16023105232) &&
|
||||
(u64)idledly_ticks * idledly_units_ps >= (u64)maxcnt * 1000) {
|
||||
xe_gt_dbg(gt, "idledly %s: %u ticks would exceed maxcnt=%u, so flooring\n",
|
||||
hwe->name, idledly_ticks, maxcnt);
|
||||
idledly = maxcnt ? maxcnt - 1 : 0;
|
||||
idledly_ticks = idledly_floor_ticks(idledly, idledly_units_ps);
|
||||
}
|
||||
|
||||
idledly_reg_val &= ~IDLE_DELAY;
|
||||
idledly_reg_val |= REG_FIELD_PREP(IDLE_DELAY, idledly_ticks);
|
||||
xe_gt_dbg(gt, "idledly %s: set %u max=%u inh=%u ts=%u\n",
|
||||
hwe->name, idledly, maxcnt,
|
||||
!!inhibit_switch, gt->info.timestamp_base);
|
||||
xe_mmio_write32(>->mmio,
|
||||
RING_IDLEDLY(hwe->mmio_base),
|
||||
idledly_reg_val);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -280,7 +280,8 @@ static void xe_tlb_inval_fence_prep(struct xe_tlb_inval_fence *fence)
|
|||
xe_tlb_inval_fence_signal_unlocked((__fence)); \
|
||||
mutex_unlock(&(__tlb_inval)->seqno_lock); \
|
||||
\
|
||||
__ret == -ECANCELED ? 0 : __ret; \
|
||||
/* Undelivered: fence already signalled, report done */ \
|
||||
(__ret == -ECANCELED || __ret == -ENOTRECOVERABLE) ? 0 : __ret; \
|
||||
})
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -1947,21 +1947,13 @@ void xe_vm_close_and_put(struct xe_vm *vm)
|
|||
vma->gpuva.flags |= XE_VMA_DESTROYED;
|
||||
}
|
||||
|
||||
/*
|
||||
* All vm operations will add shared fences to resv.
|
||||
* The only exception is eviction for a shared object,
|
||||
* but even so, the unbind when evicted would still
|
||||
* install a fence to resv. Hence it's safe to
|
||||
* destroy the pagetables immediately.
|
||||
*/
|
||||
xe_vm_free_scratch(vm);
|
||||
xe_vm_pt_destroy(vm);
|
||||
xe_vm_unlock(vm);
|
||||
|
||||
/*
|
||||
* VM is now dead, cannot re-add nodes to vm->vmas if it's NULL
|
||||
* Since we hold a refcount to the bo, we can remove and free
|
||||
* the members safely without locking.
|
||||
* Unlink and destroy all contested external-BO VMAs before destroying
|
||||
* the page tables. Otherwise, concurrent eviction holding only bo->resv
|
||||
* can walk the BO's VMAs and attempt to invalidate/zap page tables that
|
||||
* have already been freed.
|
||||
*/
|
||||
list_for_each_entry_safe(vma, next_vma, &contested,
|
||||
combined_links.destroy) {
|
||||
|
|
@ -1969,6 +1961,11 @@ void xe_vm_close_and_put(struct xe_vm *vm)
|
|||
xe_vma_destroy_unlocked(vma);
|
||||
}
|
||||
|
||||
xe_vm_lock(vm, false);
|
||||
xe_vm_free_scratch(vm);
|
||||
xe_vm_pt_destroy(vm);
|
||||
xe_vm_unlock(vm);
|
||||
|
||||
xe_svm_fini(vm);
|
||||
|
||||
up_write(&vm->lock);
|
||||
|
|
|
|||
|
|
@ -72,3 +72,5 @@
|
|||
16029897822 MEDIA_VERSION(3500)
|
||||
GRAPHICS_VERSION(3510)
|
||||
14027054324 GRAPHICS_VERSION(3511)
|
||||
14025941587 GRAPHICS_VERSION_RANGE(2001, 3511), FUNC(xe_rtp_match_not_sriov_vf)
|
||||
MEDIA_VERSION_RANGE(1301, 3503), FUNC(xe_rtp_match_not_sriov_vf)
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user