github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

mm/huge_memory: fix list corruption when deferred_split_scan() races with deferred_split_folio()

Browse Source 
Since the deferred split queue became a list_lru, the isolate callback
moves the folios it took a reference on to an on-stack list, linked
through folio->_deferred_list, and deferred_split_scan() takes them off
that list with list_del_init() without holding the list_lru lock.

deferred_split_folio() does not know about the on-stack list. It only
holds the list_lru lock when it looks at the same list_head, and it can
run for such a folio: a THP is queued when it is mapped, as a candidate
for the underused check, and is queued "again" when a part of it gets
unmapped:

  CPU0 deferred_split_scan()          CPU1 deferred_split_folio()
  list_del_init(&folio->_deferred_list)
    next->prev = prev
    prev->next = next
    entry->next = entry
                                      list_lru_lock_irqsave()
                                      folio_set_partially_mapped()
                                      __list_lru_add()
                                        list_empty(item)  -> true
                                        list_add_tail(item, &l->list)
                                          item->next = &l->list
                                          item->prev = tail
    entry->prev = entry

The folio now sits on the list_lru with ->prev pointing to itself. The
next removal leaves the neighbours pointing to it, and they still do so
after the folio has been freed. With CONFIG_DEBUG_LIST:

  list_del corruption. prev->next should be ffd4000000270090, but was
  ffd4000000278090. (prev=ffd4000000270090)

Fix it by grabbing the lock around list_del_init() so
deferred_split_folio() cannot race with it.

Fixes: fafaeceb89 ("mm: switch deferred split shrinker to list_lru")
Cc: stable@vger.kernel.org
Reported-by: Josef Bacik <josef@toxicpanda.com>
Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Mani Tofigh <mt4022@columbia.edu>
Tested-by: Mani Tofigh <mt4022@columbia.edu>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Link: https://patch.msgid.link/20261031-fixes_core-v3-10-d6437171ecc3@kernel.org
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
This commit is contained in:
David Hildenbrand (Arm) 2026-10-31 10:39:39 +01:00
parent f3c493d2f4
commit f09171b67e
1 changed files with 13 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
mm/huge_memory.c
View File

@ -4522,8 +4522,7 @@ int split_folio_to_list(struct folio *folio, struct list_head *list)
*
* It is unsafe to call folio_unqueue_deferred_split() until folio refcount is
* zero: because even when the list_lru lock is held, a non-empty
* _deferred_list might be in use on deferred_split_scan()'s unlocked
* on-stack list.
* _deferred_list might be in use on deferred_split_scan()'s on-stack list.
*
* The list_lru sublist is determined by folio's memcg: it is therefore
* important to unqueue deferred split before changing folio memcg.
@ -4665,10 +4664,22 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
deferred_split_isolate, &dispose);
list_for_each_entry_safe(folio, next, &dispose, _deferred_list) {
struct mem_cgroup *memcg;
struct list_lru_one *lru;
bool did_split = false;
bool underused = false;
/*
* Sync with deferred_split_folio(), which can requeue the
* folio once list_empty() is true, and sets PG_partially_mapped.
*/
rcu_read_lock();
memcg = folio_memcg(folio);
lru = list_lru_lock_irq(&deferred_split_lru, folio_nid(folio),
&memcg);
list_del_init(&folio->_deferred_list);
list_lru_unlock_irq(lru);
rcu_read_unlock();
if (!folio_test_partially_mapped(folio)) {
/*