github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-13 06:23:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

mm/memcontrol: avoid false sharing between vmstats and events

Browse Source 
Moving v1 userspace eventfd handling into memcontrol-v1.c shrank struct
vmpressure from 112 to 24 bytes when CONFIG_MEMCG_V1 is disabled.  This
moved memory_events_local[MEMCG_SWAP_FAIL] and the hot vmstats_percpu
pointer onto the same cacheline.

The stress-ng mremap stressor exercises MADV_PAGEOUT with swap disabled,
generating about 20 million MEMCG_SWAP_FAIL updates per 60-second run on a
176-CPU test system.  Those writes bounce the line while memcg statistics
paths load vmstats_percpu.

Move cgwb_list into the existing alignment gap and cacheline-align
vmstats_percpu.  This separates the pointer from the event counters
without increasing the size of struct mem_cgroup in the tested
configuration.

The blamed commit reduced median mremap throughput by 4.38% on the test
system with one socket.  The patched kernel brings the performance to
within 0.5% of the parent which is within the observed boot-to-boot spread
(up to 1.2%).

Link: https://lore.kernel.org/20260817103835.2937733-1-usama.arif@linux.dev
Fixes: ea928e9e18 ("mm/vmpressure: move v1 userspace eventfd code into memcontrol-v1.c")
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Reported-by: kernel test robot <yi1.lai@intel.com>
Closes: https://lore.kernel.org/oe-lkp/202608131743.c6a7dda4-lkp@intel.com
Tested-by: kernel test robot <yi1.lai@intel.com>
  Link: http://lore.kernel.org/aoAABX59IzUXz/Rv@ly-workstation
Acked-by: Shakeel Butt <shakeel.butt@linux.dev>
Acked-by: Michal Hocko <mhocko@suse.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Yi Lai <yi1.lai@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
This commit is contained in:
Usama Arif 2026-08-17 03:38:35 -07:00 committed by Andrew Morton
parent f2b1cb39d5
commit c1afbd5de1
1 changed files with 7 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
include/linux/memcontrol.h
View File

@ -268,10 +268,15 @@ struct mem_cgroup {
#endif
int kmemcg_id;
struct memcg_vmstats_percpu __percpu *vmstats_percpu;
#ifdef CONFIG_CGROUP_WRITEBACK
struct list_head cgwb_list;
#endif
/* Keep the hot per-CPU stats pointer away from memory event counters. */
struct memcg_vmstats_percpu __percpu *vmstats_percpu
____cacheline_aligned_in_smp;
#ifdef CONFIG_CGROUP_WRITEBACK
struct wb_domain cgwb_domain;
struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT];
#endif