github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

mm/slab: sample slab_state once in kmem_cache_destroy()

Browse Source 
kmem_cache_release() tests slab_state >= FULL after kmem_cache_destroy()
has dropped slab_mutex and unlinked the cache, and for a
SLAB_TYPESAFE_BY_RCU cache after rcu_barrier(). If slab_late_init() runs
before that test it sets slab_state to FULL but skips the unlinked cache,
which never got a sysfs entry, so sysfs_slab_release() calls kobject_put()
on a kobject that was never initialized:

  WARNING: lib/kobject.c:734 at kobject_put+0x64/0x2c0, CPU#1: kworker/u8:3/55
  kobject: '(null)' ((____ptrval____)): is not initialized, yet kobject_put() is being called.
  refcount_t: underflow; use-after-free.
  kobject_put+0x64/0x2c0
  sysfs_slab_release+0xc/0x20
  kmem_cache_destroy+0x104/0x1e0
  bioset_exit+0x13c/0x1e0
  disk_release+0x54/0x140
  put_disk+0x18/0x40
  floppy_async_init+0xbec/0xd10

slab_state >= FULL asks whether the cache has a kobject, which cache
creation skips while slab_state <= UP. UP is the state just below FULL,
so a false sample cannot skip a kobject_put() that was owed.

Seen on sparc64 at boot, where the asynchronous floppy init tears down
its bio slab while the late initcalls are running.

Sample slab_state once, under slab_mutex which slab_late_init() holds
when it sets FULL, and use the result for both the unlink and the
release.

The history of this problem dates back to v3.2:

  Commit 69cb8e6b7c ("slub: free slabs without holding locks") in
  v3.2 moved sysfs_slab_remove() outside slub_lock and introduced
  a race condition between kmem_cache_destroy() and slab_sysfs_init().

  Commit 945cf2b619 ("mm/sl[aou]b: Extract a common function for
  kmem_cache_destroy") in v3.7 accidentally closed the race while
  extracting a common implementation for kmem_cache_destroy().

  Commit 5413dfba88 ("slub: drop mutex before deleting sysfs entry")
  in v3.8 re-opened the race by dropping slab_mutex around
  sysfs_slab_remove() in __kmem_cache_shutdown(), to avoid a lockdep
  splat between slab_mutex and the sysfs lock.

  Commit 657dc2f972 ("slab: remove synchronous rcu_barrier() call in
  memcg cache release path") in v4.11 shifted from synchronous
  rcu_barrier() to asynchronous RCU work for SLAB_TYPESAFE_BY_RCU caches.
  That commit, again, accidentally closed the race by moving
  sysfs_slab_release() to shutdown_cache(), but only for
  !SLAB_TYPESAFE_BY_RCU caches because destruction of
  SLAB_TYPESAFE_BY_RCU caches is done in an RCU work.

  Commit 0495e337b7 ("mm/slab_common: Deleting kobject in
  kmem_cache_destroy() without holding slab_mutex/cpu_hotplug_lock") in
  v6.0 re-introduced the race for !SLAB_TYPESAFE_BY_RCU caches.

Fixes: 69cb8e6b7c ("slub: free slabs without holding locks")
Fixes: 5413dfba88 ("slub: drop mutex before deleting sysfs entry")
Fixes: 0495e337b7 ("mm/slab_common: Deleting kobject in kmem_cache_destroy() without holding slab_mutex/cpu_hotplug_lock")
Cc: stable@vger.kernel.org
Signed-off-by: Imre Kaloz <kaloz@kernel.org>
Reviewed-by: Jose A. Perez de Azpillaga <azpijr@gmail.com>
Reviewed-by: Harry Yoo (Meta) <harry@kernel.org>
Link: https://patch.msgid.link/20261014150844.8536-1-kaloz@kernel.org
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Signed-off-by: Harry Yoo (Meta) <harry@kernel.org>
This commit is contained in:
Imre Kaloz 2026-10-14 17:08:43 +02:00 committed by Harry Yoo (Meta)
parent c57fbbe42e
commit 088bfdc0d7
1 changed files with 13 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
mm/slab_common.c
View File

@ -515,10 +515,10 @@ EXPORT_SYMBOL(kmem_buckets_create);
* and release of the kobject does not need slab_mutex or cpu_hotplug_lock
* protection. So they are now done without holding those locks.
*/
static void kmem_cache_release(struct kmem_cache *s)
static void kmem_cache_release(struct kmem_cache *s, bool sysfs_ready)
{
kfence_shutdown_cache(s);
if (__is_defined(SLAB_SUPPORTS_SYSFS) && slab_state >= FULL)
if (__is_defined(SLAB_SUPPORTS_SYSFS) && sysfs_ready)
sysfs_slab_release(s);
else
slab_kmem_cache_release(s);
@ -533,6 +533,7 @@ void slab_kmem_cache_release(struct kmem_cache *s)
void kmem_cache_destroy(struct kmem_cache *s)
{
bool sysfs_ready;
int err;
if (unlikely(!s) || !kasan_check_byte(s))
@ -580,10 +581,18 @@ void kmem_cache_destroy(struct kmem_cache *s)
list_del(&s->list);
/*
* slab_late_init() sets slab_state to FULL under slab_mutex and adds
* sysfs entries only for caches still on the list. Sample the state
* here, so that a cache unlinked before that point is not handed to
* sysfs_slab_release() with an uninitialized kobject.
*/
sysfs_ready = slab_state >= FULL;
mutex_unlock(&slab_mutex);
cpus_read_unlock();
if (slab_state >= FULL)
if (sysfs_ready)
sysfs_slab_unlink(s);
debugfs_slab_release(s);
@ -593,7 +602,7 @@ void kmem_cache_destroy(struct kmem_cache *s)
if (s->flags & SLAB_TYPESAFE_BY_RCU)
rcu_barrier();
kmem_cache_release(s);
kmem_cache_release(s, sysfs_ready);
}
EXPORT_SYMBOL(kmem_cache_destroy);