github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-06-05 21:15:53 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph
Linux kernel source tree
1,220,421 Commits 1 Branch 933 Tags 20 GiB
C 97%
Assembly 1%
Shell 0.6%
Rust 0.5%
Python 0.4%
Other 0.3%
9d23fab89a
Go to file
Baolin Wang 9d23fab89a mm: migrate: record the mlocked page status to remove unnecessary lru drain 
[ Upstream commit eebb3dabbb ]

When doing compaction, I found the lru_add_drain() is an obvious hotspot
when migrating pages. The distribution of this hotspot is as follows:
   - 18.75% compact_zone
      - 17.39% migrate_pages
         - 13.79% migrate_pages_batch
            - 11.66% migrate_folio_move
               - 7.02% lru_add_drain
                  + 7.02% lru_add_drain_cpu
               + 3.00% move_to_new_folio
                 1.23% rmap_walk
            + 1.92% migrate_folio_unmap
         + 3.20% migrate_pages_sync
      + 0.90% isolate_migratepages

The lru_add_drain() was added by commit c3096e6782 ("mm/migrate:
__unmap_and_move() push good newpage to LRU") to drain the newpage to LRU
immediately, to help to build up the correct newpage->mlock_count in
remove_migration_ptes() for mlocked pages.  However, if there are no
mlocked pages are migrating, then we can avoid this lru drain operation,
especailly for the heavy concurrent scenarios.

So we can record the source pages' mlocked status in
migrate_folio_unmap(), and only drain the lru list when the mlocked status
is set in migrate_folio_move().

In addition, the page was already isolated from lru when migrating, so
checking the mlocked status is stable by folio_test_mlocked() in
migrate_folio_unmap().

After this patch, I can see the hotpot of the lru_add_drain() is gone:
   - 9.41% migrate_pages_batch
      - 6.15% migrate_folio_move
         - 3.64% move_to_new_folio
            + 1.80% migrate_folio_extra
            + 1.70% buffer_migrate_folio
         + 1.41% rmap_walk
         + 0.62% folio_add_lru
      + 3.07% migrate_folio_unmap

Meanwhile, the compaction latency shows some improvements when running
thpscale:
                            base                   patched
Amean     fault-both-1      1131.22 (   0.00%)     1112.55 *   1.65%*
Amean     fault-both-3      2489.75 (   0.00%)     2324.15 *   6.65%*
Amean     fault-both-5      3257.37 (   0.00%)     3183.18 *   2.28%*
Amean     fault-both-7      4257.99 (   0.00%)     4079.04 *   4.20%*
Amean     fault-both-12     6614.02 (   0.00%)     6075.60 *   8.14%*
Amean     fault-both-18    10607.78 (   0.00%)     8978.86 *  15.36%*
Amean     fault-both-24    14911.65 (   0.00%)    11619.55 *  22.08%*
Amean     fault-both-30    14954.67 (   0.00%)    14925.66 *   0.19%*
Amean     fault-both-32    16654.87 (   0.00%)    15580.31 *   6.45%*

Link: https://lkml.kernel.org/r/06e9153a7a4850352ec36602df3a3a844de45698.1697859741.git.baolin.wang@linux.alibaba.com
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: "Huang, Ying" <ying.huang@intel.com>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Mel Gorman <mgorman@techsingularity.net>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Yin Fengwei <fengwei.yin@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Stable-dep-of: d1adb25df7 ("mm: migrate: fix getting incorrect page mapping during page migration")
Signed-off-by: Sasha Levin <sashal@kernel.org>
2024-01-31 16:19:10 -08:00
arch LoongArch/smp: Call rcutree_report_cpu_starting() earlier 2024-01-31 16:18:57 -08:00
block block: ensure we hold a queue reference when using queue limits 2024-01-25 15:35:57 -08:00
certs certs: Reference revocation list for all keyrings 2023-08-17 20:12:41 +00:00
crypto crypto: api - Disallow identical driver names 2024-01-31 16:18:49 -08:00
Documentation drm: Allow drivers to indicate the damage helpers to ignore damage clips 2024-01-31 16:19:08 -08:00
drivers thermal: gov_power_allocator: avoid inability to reset a cdev 2024-01-31 16:19:10 -08:00
fs pipe: wakeup wr_wait after setting max_usage 2024-01-31 16:19:09 -08:00
include media: v4l2-cci: Add support for little-endian encoded registers 2024-01-31 16:19:10 -08:00
init rootfs: Fix support for rootfstype= when root= is given 2024-01-25 15:35:46 -08:00
io_uring io_uring: adjust defer tw counting 2024-01-25 15:36:00 -08:00
ipc Add x86 shadow stack support 2023-08-31 12:20:12 -07:00
kernel bpf: Add bpf_sock_addr_set_sun_path() to allow writing unix sockaddr from bpf 2024-01-31 16:19:04 -08:00
lib crypto: lib/mpi - Fix unexpected pointer access in mpi_ec_init 2024-01-31 16:18:49 -08:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm mm: migrate: record the mlocked page status to remove unnecessary lru drain 2024-01-31 16:19:10 -08:00
net net/bpf: Avoid unused "sin_addr_len" warning when CONFIG_CGROUP_BPF is not set 2024-01-31 16:19:09 -08:00
rust rust: Ignore preserve-most functions 2024-01-25 15:35:41 -08:00
samples vfio/mtty: Overhaul mtty interrupt handling 2024-01-10 17:16:55 +01:00
scripts scripts/get_abi: fix source path leak 2024-01-31 16:18:55 -08:00
security lsm: new security_file_ioctl_compat() hook 2024-01-31 16:18:54 -08:00
sound soundwire: fix initializing sysfs for same devices on different buses 2024-01-31 16:18:47 -08:00
tools selftests: bonding: do not test arp/ns target with mode balance-alb/tlb 2024-01-31 16:19:05 -08:00
usr initramfs: Encode dependency on KBUILD_BUILD_TIMESTAMP 2023-06-06 17:54:49 +09:00
virt ARM: 2023-09-07 13:52:20 -07:00
.clang-format iommu: Add for_each_group_device() 2023-05-23 08:15:51 +02:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore kbuild: rpm-pkg: rename binkernel.spec to kernel.spec 2023-07-25 00:59:33 +09:00
.mailmap 20 hotfixes. 12 are cc:stable and the remainder address post-6.5 issues 2023-10-24 09:52:16 -10:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS USB: Remove Wireless USB and UWB documentation 2023-08-09 14:17:32 +02:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS Char/Misc driver fixes for 6.6-final 2023-10-28 07:51:27 -10:00
Makefile Linux 6.6.14 2024-01-25 15:36:01 -08:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.