mirror of
https://github.com/torvalds/linux.git
synced 2026-07-27 17:47:41 +02:00
a3fdf74ffa
ttm_pool_split_for_swap() unconditionally splits high-order pool pages
into order-0 pages before backup, so every compound the shrinker
touches is shattered even when the rest of the system would prefer it
stay intact. Under sustained kswapd pressure this fragments memory
enough to drive other parts of MM into recovery loops.
Back up each compound at its native order instead. In
ttm_pool_backup(), hand the full compound to the new
ttm_backup_backup_folio(), which backs up subpages to a contiguous
range of shmem indices and returns the base handle plus the number of
subpages actually backed up (@nr_backed). On full success, free the
compound once at its native order -- no split_page(), no per-4K
refcount juggling.
A per-folio backup can't be made fully atomic under memory pressure:
ttm_backup_backup_folio() must allocate shmem folios before source
subpages can be released, so under true OOM any subpage may fail
while the rest of the compound is still live. Two mechanisms handle
this without regressing reclaim behaviour:
- alloc_gfp gets __GFP_NOMEMALLOC whenever order > 0 (cleared again
for order-0), so a high-order backup fails fast with -ENOMEM
instead of draining kernel reserves, leaving them for other
allocations under the same pressure.
- If ttm_backup_backup_folio() still returns a short @nr_backed with
a valid handle for the successfully-backed prefix, split the
source compound with ttm_pool_split_for_swap(), free the prefix as
order-0 pages (already safely in shmem), and retry the remaining
subpages at order 0, where __GFP_NOMEMALLOC is cleared and
reserves may be used as a last resort.
This preserves the original split-on-OOM fallback while keeping the
common case fragmentation-free, and preserves the "partial backup is
allowed" contract (shrunken is incremented per subpage backed up).
The restore-side leftover-page split in ttm_pool_restore_commit() is
left as-is: it's unreachable in practice and not worth complicating
the restore state machine to avoid.
Testing: the existing backup_fault_inject point only truncated
tt->num_pages, which never exercised the reactive split path above
since it never left a compound partially backed up. Wire fault
injection into ttm_backup_backup_folio() itself: past the first
subpage of a compound, synthesize a -ENOMEM in place of
shmem_read_folio_gfp() when should_fail() trips, producing the same
short @nr_pages_backed a real failure would and forcing
ttm_pool_backup() through the split-and-retry path. The fault_attr
stays private to ttm_pool.c; ttm_backup.c reaches it through
ttm_backup_fault_inject_folio(), declared in ttm_pool_internal.h.
While converting the writeback branch to operate on the whole folio,
the unlock condition after shmem_writeout() also changed from `if
(ret)` to `if (ret == AOP_WRITEPAGE_ACTIVATE)`, matching the actual
contract: shmem_writeout()/swap_writeout() only leave the folio locked
when returning AOP_WRITEPAGE_ACTIVATE; any other return (including a
hard error from arch_prepare_to_swap()) means the folio was already
unlocked internally. The old `if (ret)` check would have double-
unlocked in that hard-error case.
Cc: Christian Koenig <christian.koenig@amd.com>
Cc: Huang Rui <ray.huang@amd.com>
Cc: Matthew Auld <matthew.auld@intel.com>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Cc: Maxime Ripard <mripard@kernel.org>
Cc: Thomas Zimmermann <tzimmermann@suse.de>
Cc: David Airlie <airlied@gmail.com>
Cc: Simona Vetter <simona@ffwll.ch>
Cc: dri-devel@lists.freedesktop.org
Cc: linux-kernel@vger.kernel.org
Cc: stable@vger.kernel.org
Fixes:
|
||
|---|---|---|
| .. | ||
| tests | ||
| Makefile | ||
| ttm_agp_backend.c | ||
| ttm_backup.c | ||
| ttm_bo_internal.h | ||
| ttm_bo_util.c | ||
| ttm_bo_vm.c | ||
| ttm_bo.c | ||
| ttm_device.c | ||
| ttm_execbuf_util.c | ||
| ttm_module.c | ||
| ttm_module.h | ||
| ttm_pool_internal.h | ||
| ttm_pool.c | ||
| ttm_range_manager.c | ||
| ttm_resource.c | ||
| ttm_sys_manager.c | ||
| ttm_tt.c | ||