github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-07-27 09:36:22 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

mm/page_alloc: fix defrag_mode for non-reclaimable allocations

Browse Source 
When defrag_mode is enabled, ALLOC_NOFRAGMENT is enforced to prevent
migratetype fallbacks and keep pageblocks clean.  The allocator relies on
reclaim and compaction to free pages of the correct type before allowing
fallback as a last resort.

However, non-reclaimable allocations such as GFP_ATOMIC cannot invoke
direct reclaim or compaction.  With defrag_mode=1, these allocations hit
the !can_direct_reclaim bailout in __alloc_pages_slowpath() with
ALLOC_NOFRAGMENT still set, and fail without ever attempting a fallback.

This causes a large number of SLUB allocation failures for
skbuff_head_cache under network-heavy workloads, despite free memory being
available in other migratetype freelists.

We observed it on a few of the Meta workloads that adopted
defrag_mode=1.

For the service under load there were 85509 SLUB allocation failures
messages in dmesg within 2 hours.  All of them are GFP_ATOMIC
allocations for skbuff_head_cache, despite free pages being available
in other migratetype freelists (~13 GB free).

Since it is networking path from the practical point of view, this
means dropped packets, failed RPC requests, tail latency spikes and
overall service degradation.

Clear ALLOC_NOFRAGMENT and retry for allocations that request kswapd
reclaim but cannot do direct reclaim themselves (GFP_ATOMIC).  Purely
speculative allocations like GFP_TRANSHUGE_LIGHT that don't set
__GFP_KSWAPD_RECLAIM are left to fail, since they have reasonable
fallbacks and should not cause fragmentation.

Link: https://lore.kernel.org/20260520122228.201550-1-d@ilvokhin.com
Fixes: e3aa7df331 ("mm: page_alloc: defrag_mode")
Signed-off-by: Dmitry Ilvokhin <d@ilvokhin.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
This commit is contained in:
Dmitry Ilvokhin 2026-05-20 12:22:28 +00:00 committed by Andrew Morton
parent 6414f790f2
commit 4c0ed883e0
1 changed files with 12 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
mm/page_alloc.c
View File

@ -4853,8 +4853,19 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
}
/* Caller is not willing to reclaim, we can't balance anything */
if (!can_direct_reclaim)
if (!can_direct_reclaim) {
/*
* Reclaim/compaction cannot run, so defrag_mode's strategy
* of enforcing ALLOC_NOFRAGMENT cannot be fulfilled. Allow
* fallbacks rather than failing the allocation outright.
*/
if (defrag_mode && (alloc_flags & ALLOC_NOFRAGMENT) &&
(gfp_mask & __GFP_KSWAPD_RECLAIM)) {
alloc_flags &= ~ALLOC_NOFRAGMENT;
goto retry;
}
goto nopage;
}
/* Avoid recursion of direct reclaim */
if (current->flags & PF_MEMALLOC)