mirror of
https://github.com/torvalds/linux.git
synced 2026-11-12 20:54:48 +01:00
c4126f1db3
A device-private folio migrated to system memory by a CPU fault can
remain reachable through the raw-PFN eviction path until migration
finalization drops the source reference.
If eviction selects the same device-private folio during this window,
it can attempt to migrate the folio again. The second migration can leave
an uncharged folio on an LRU list, causing folio_lruvec_lock_irqsave() to
retry indefinitely and resulting in a soft lockup and RCU stall.
Mark successfully migrated device-private folios using a low bit of
their zone_device_data before migration finalization. Make both CPU-fault
and raw-PFN migration paths skip device-private folios carrying this
flag.
Mask the flag when retrieving the drm_pagemap_zdd pointer and preserve
it when a device-private folio is split. Keeping the state on the physical
folio also avoids depending on a virtual address that may change before a
fault occurs.
v2:
- Replace the retired-PFN XArray with an embedded bitmap. (Matthew Brost)
- Mark every base page covered by a migrated folio so retirement remains
valid if the folio is later split.
v3:
- Store the migrated state in a low bit of zone_device_data instead of
adding virtual-range and bitmap tracking to the ZDD. (Matthew Brost)
- Mask the flag when retrieving the ZDD and preserve it when splitting
a folio.
- Drop the pre-existing fixes already covered by Matthew Brost's series:
https://patchwork.freedesktop.org/series/171651/
v4:
- Advance by the folio size only for migration entries marked with
MIGRATE_PFN_COMPOUND. (Sashiko)
v5:
- Simplify ZDD flag updates and folio iteration. (Matthew Brost)
- Skip retired device-private folios in the CPU-fault path. (Matthew Brost)
- Preserve flag bits while taking a new ZDD reference for split folios.
v6:
- Restore MIGRATE_PFN_COMPOUND-aware stepping so non-compound migration
entries are processed one at a time. (Sashiko)
- Drop the pre-existing fixes already covered by Matthew Brost's series:
https://patchwork.freedesktop.org/series/171651/
The lockup was observed as:
[10109.860465] watchdog: BUG: soft lockup - CPU#9 stuck for 26s! [kworker/u65:5:6557]
[10109.860524] Tainted: [S]=CPU_OUT_OF_SPEC, [O]=OOT_MODULE
[10109.860524] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
[10109.860525] Workqueue: xe_page_fault_work_queue xe_pagefault_queue_work [xe]
[10109.860644] RIP: 0010:_raw_spin_unlock_irqrestore+0x57/0x80
[10109.860655] Call Trace:
[10109.860655] <TASK>
[10109.860657] folio_lruvec_lock_irqsave+0x216/0x220
[10109.860661] ? __pfx_lru_add+0x10/0x10
[10109.860665] folio_batch_move_lru+0xc8/0x450
[10109.860670] ? lock_acquire+0xc4/0x2d0
[10109.860674] ? __folio_batch_add_and_move+0x60/0x2e0
[10109.860677] ? folio_migrate_mapping+0xa6/0x110
[10109.860679] ? folio_migrate_flags+0x13b/0x1b0
[10109.860681] ? __pfx_lru_add+0x10/0x10
[10109.860683] __folio_batch_add_and_move+0xe7/0x2e0
[10109.860685] ? dma_iova_try_alloc+0xb0/0x140
[10109.860689] folio_add_lru+0x64/0x80
[10109.860691] __migrate_device_finalize+0x12c/0x270
[10109.860695] migrate_device_finalize+0x10/0x20
[10109.860698] drm_pagemap_evict_to_ram+0x185/0x370 [drm_gpusvm_helper]
[10109.860704] ? drm_pagemap_evict_to_ram+0x96/0x370 [drm_gpusvm_helper]
[10109.860709] xe_svm_bo_evict+0x15/0x20 [xe]
[10109.860819] ? xe_svm_bo_evict+0x15/0x20 [xe]
[10109.860921] xe_bo_move+0x107e/0x1570 [xe]
[10109.860992] ? xe_ttm_tt_create+0x168/0x340 [xe]
[10109.861059] ? __up_read+0x98/0x2b0
[10109.861061] ? lock_is_held_type+0xa3/0x130
[10109.861067] ttm_bo_handle_move_mem+0xe8/0x1e0 [ttm]
[10109.861075] ttm_bo_evict+0x141/0x1c0 [ttm]
[10109.861081] ttm_bo_evict_cb+0x9f/0x100 [ttm]
[10109.861086] ttm_lru_walk_for_evict+0x84/0x190 [ttm]
[10109.861091] ? xe_ttm_vram_mgr_new+0x258/0x3a0 [xe]
[10109.861198] ttm_bo_alloc_resource+0x219/0x750 [ttm]
[10109.861203] ? ttm_bo_alloc_resource+0xa9/0x750 [ttm]
[10109.861208] ? lock_acquire+0xc4/0x2d0
[10109.861214] ttm_bo_validate+0x94/0x1c0 [ttm]
[10109.861218] ? ww_mutex_trylock+0x19d/0x3d0
[10109.861219] ? _raw_write_unlock+0x22/0x50
[10109.861223] ttm_bo_init_reserved+0x17d/0x1f0 [ttm]
[10109.861228] xe_bo_init_locked+0x20a/0x620 [xe]
[10109.861294] ? __pfx_xe_ttm_bo_destroy+0x10/0x10 [xe]
[10109.861359] ? mark_held_locks+0x46/0x90
[10109.861361] ? __create_object+0x68/0xc0
[10109.861366] __xe_bo_create_locked+0x384/0xa20 [xe]
[10109.861432] ? lock_acquire+0xc4/0x2d0
[10109.861434] ? xe_drm_pagemap_populate_mm+0xd3/0x340 [xe]
[10109.861542] xe_bo_create_locked+0x23/0x40 [xe]
[10109.861609] xe_drm_pagemap_populate_mm+0x12e/0x340 [xe]
[10109.861707] ? __lock_acquire+0x43e/0x2930
[10109.861716] drm_pagemap_populate_mm+0x74/0xe0 [drm_gpusvm_helper]
[10109.861720] xe_svm_alloc_vram+0xb5/0x2c0 [xe]
[10109.861817] ? seqcount_lockdep_reader_access.constprop.0+0x9f/0xc0
[10109.861819] ? ktime_get+0x23/0x130
[10109.861821] ? trace_hardirqs_on+0x22/0xe0
[10109.861823] ? seqcount_lockdep_reader_access.constprop.0+0x9f/0xc0
[10109.861826] __xe_svm_handle_pagefault+0x77d/0xbf0 [xe]
[10109.861924] ? rwsem_down_write_slowpath+0x43a/0x9a0
[10109.861926] ? _raw_spin_unlock_irq+0x27/0x70
[10109.861928] ? rwsem_down_write_slowpath+0x43a/0x9a0
[10109.861929] ? trace_hardirqs_on+0x22/0xe0
[10109.861931] ? _raw_spin_unlock_irq+0x27/0x70
[10109.861933] ? rwsem_down_write_slowpath+0x459/0x9a0
[10109.861937] xe_svm_handle_pagefault+0x3d/0xb0 [xe]
[10109.862030] xe_pagefault_queue_work+0x1a9/0x520 [xe]
[10109.862122] process_one_work+0x239/0x730
[10109.862127] worker_thread+0x200/0x3f0
[10109.862130] ? __pfx_worker_thread+0x10/0x10
[10109.862132] kthread+0x10d/0x150
[10109.862133] ? __pfx_kthread+0x10/0x10
[10109.862135] ret_from_fork+0x3bd/0x470
[10109.862138] ? __pfx_kthread+0x10/0x10
[10109.862140] ret_from_fork_asm+0x1a/0x30
[10109.862146] </TASK>
Fixes:
|
||
|---|---|---|
| .. | ||
| accel | ||
| accessibility | ||
| acpi | ||
| amba | ||
| android | ||
| ata | ||
| atm | ||
| auxdisplay | ||
| base | ||
| bcma | ||
| block | ||
| bluetooth | ||
| bus | ||
| cache | ||
| cdrom | ||
| cdx | ||
| char | ||
| clk | ||
| clocksource | ||
| comedi | ||
| connector | ||
| counter | ||
| cpufreq | ||
| cpuidle | ||
| crypto | ||
| cxl | ||
| dax | ||
| dca | ||
| devfreq | ||
| dibs | ||
| dio | ||
| dma | ||
| dma-buf | ||
| dpll | ||
| edac | ||
| eisa | ||
| extcon | ||
| firewire | ||
| firmware | ||
| fpga | ||
| fsi | ||
| fwctl | ||
| gnss | ||
| gpib | ||
| gpio | ||
| gpu | ||
| greybus | ||
| hid | ||
| hsi | ||
| hte | ||
| hv | ||
| hwmon | ||
| hwspinlock | ||
| hwtracing | ||
| i2c | ||
| i3c | ||
| idle | ||
| iio | ||
| infiniband | ||
| input | ||
| interconnect | ||
| iommu | ||
| ipack | ||
| irqchip | ||
| leds | ||
| macintosh | ||
| mailbox | ||
| mcb | ||
| md | ||
| media | ||
| memory | ||
| memstick | ||
| message | ||
| mfd | ||
| misc | ||
| mmc | ||
| most | ||
| mtd | ||
| mux | ||
| net | ||
| nfc | ||
| ntb | ||
| nubus | ||
| nvdimm | ||
| nvme | ||
| nvmem | ||
| of | ||
| opp | ||
| parisc | ||
| parport | ||
| pci | ||
| pcmcia | ||
| peci | ||
| perf | ||
| phy | ||
| pinctrl | ||
| platform | ||
| pmdomain | ||
| pnp | ||
| power | ||
| powercap | ||
| pps | ||
| ps3 | ||
| ptp | ||
| pwm | ||
| rapidio | ||
| ras | ||
| regulator | ||
| remoteproc | ||
| resctrl | ||
| reset | ||
| rpmsg | ||
| rtc | ||
| s390 | ||
| sbus | ||
| scsi | ||
| sh | ||
| siox | ||
| slimbus | ||
| soc | ||
| soundwire | ||
| spi | ||
| spmi | ||
| ssb | ||
| staging | ||
| target | ||
| tc | ||
| tee | ||
| thermal | ||
| thunderbolt | ||
| tty | ||
| ufs | ||
| uio | ||
| usb | ||
| vdpa | ||
| vfio | ||
| vhost | ||
| video | ||
| virt | ||
| virtio | ||
| w1 | ||
| watchdog | ||
| xen | ||
| zorro | ||
| Kconfig | ||
| Makefile | ||