Linux kernel source tree
Go to file
Shakeel Butt 7182935bd5 memcg: flush stats only if updated
commit 11192d9c12 upstream.

At the moment, the kernel flushes the memcg stats on every refault and
also on every reclaim iteration.  Although rstat maintains per-cpu
update tree but on the flush the kernel still has to go through all the
cpu rstat update tree to check if there is anything to flush.  This
patch adds the tracking on the stats update side to make flush side more
clever by skipping the flush if there is no update.

The stats update codepath is very sensitive performance wise for many
workloads and benchmarks.  So, we can not follow what the commit
aa48e47e39 ("memcg: infrastructure to flush memcg stats") did which
was triggering async flush through queue_work() and caused a lot
performance regression reports.  That got reverted by the commit
1f828223b7 ("memcg: flush lruvec stats in the refault").

In this patch we kept the stats update codepath very minimal and let the
stats reader side to flush the stats only when the updates are over a
specific threshold.  For now the threshold is (nr_cpus * CHARGE_BATCH).

To evaluate the impact of this patch, an 8 GiB tmpfs file is created on
a system with swap-on-zram and the file was pushed to swap through
memory.force_empty interface.  On reading the whole file, the memcg stat
flush in the refault code path is triggered.  With this patch, we
observed 63% reduction in the read time of 8 GiB file.

Link: https://lkml.kernel.org/r/20211001190040.48086-1-shakeelb@google.com
Signed-off-by: Shakeel Butt <shakeelb@google.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Reviewed-by: "Michal Koutný" <mkoutny@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Ivan Babrou <ivan@cloudflare.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-01-29 10:58:24 +01:00
arch arm64: dts: qcom: msm8996: drop not documented adreno properties 2022-01-27 11:05:39 +01:00
block block: Fix fsync always failed if once failed 2022-01-27 11:05:25 +01:00
certs certs: Add support for using elliptic curve keys for signing modules 2021-08-23 19:55:42 +03:00
crypto crypto: jitter - consider 32 LSB for APT 2022-01-27 11:04:31 +01:00
Documentation dt-bindings: watchdog: Require samsung,syscon-phandle for Exynos7 2022-01-27 11:05:42 +01:00
drivers bnx2x: Invalidate fastpath HSI version for VFs 2022-01-29 10:58:24 +01:00
fs io_uring: fix not released cached task refs 2022-01-29 10:58:24 +01:00
include net_sched: restore "mpu xxx" handling 2022-01-27 11:05:40 +01:00
init init: make unknown command line param message clearer 2021-11-18 19:17:11 +01:00
ipc shm: extend forced shm destroy to support objects from several IPC nses 2021-11-25 09:48:42 +01:00
kernel taskstats: Cleanup the use of task->exit_code 2022-01-27 11:05:35 +01:00
lib lib/test_meminit: destroy cache in kmem_cache_alloc_bulk() test 2022-01-27 11:05:44 +01:00
LICENSES LICENSES/dual/CC-BY-4.0: Git rid of "smart quotes" 2021-07-15 06:31:24 -06:00
mm memcg: flush stats only if updated 2022-01-29 10:58:24 +01:00
net sch_api: Don't skip qdisc attach on ingress 2022-01-27 11:05:42 +01:00
samples samples: bpf: Fix 'unknown warning group' build warning on Clang 2022-01-27 11:03:29 +01:00
scripts scripts: sphinx-pre-install: Fix ctex support on Debian 2022-01-27 11:05:44 +01:00
security selinux: fix potential memleak in selinux_add_opt() 2022-01-27 11:03:43 +01:00
sound ASoC: mediatek: mt8183: fix device_node leak 2022-01-27 11:05:04 +01:00
tools mm/hmm.c: allow VM_MIXEDMAP to work with hmm_range_fault 2022-01-27 11:05:43 +01:00
usr .gitignore: prefix local generated files with a slash 2021-05-02 00:43:35 +09:00
virt KVM: downgrade two BUG_ONs to WARN_ON_ONCE 2021-12-22 09:32:34 +01:00
.clang-format clang-format: Update with the latest for_each macro list 2021-05-12 23:32:39 +02:00
.cocciconfig
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore .gitignore: ignore only top-level modules.builtin 2021-05-02 00:43:35 +09:00
.mailmap mailmap: add Andrej Shadura 2021-10-18 20:22:03 -10:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Move Daniel Drake to credits 2021-09-21 08:34:58 +03:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS drm fixes for 5.15 final 2021-10-28 12:17:01 -07:00
Makefile Linux 5.15.17 2022-01-27 11:05:44 +01:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.