Linux kernel source tree
Go to file
Lance Yang dce7d10be4 mm/madvise: optimize lazyfreeing with mTHP in madvise_free
This patch optimizes lazyfreeing with PTE-mapped mTHP[1] (Inspired by
David Hildenbrand[2]).  We aim to avoid unnecessary folio splitting if the
large folio is fully mapped within the target range.

If a large folio is locked or shared, or if we fail to split it, we just
leave it in place and advance to the next PTE in the range.  But note that
the behavior is changed; previously, any failure of this sort would cause
the entire operation to give up.  As large folios become more common,
sticking to the old way could result in wasted opportunities.

On an Intel I5 CPU, lazyfreeing a 1GiB VMA backed by PTE-mapped folios of
the same size results in the following runtimes for madvise(MADV_FREE) in
seconds (shorter is better):

Folio Size |   Old    |   New    | Change
------------------------------------------
      4KiB | 0.590251 | 0.590259 |    0%
     16KiB | 2.990447 | 0.185655 |  -94%
     32KiB | 2.547831 | 0.104870 |  -95%
     64KiB | 2.457796 | 0.052812 |  -97%
    128KiB | 2.281034 | 0.032777 |  -99%
    256KiB | 2.230387 | 0.017496 |  -99%
    512KiB | 2.189106 | 0.010781 |  -99%
   1024KiB | 2.183949 | 0.007753 |  -99%
   2048KiB | 0.002799 | 0.002804 |    0%

[1] https://lkml.kernel.org/r/20231207161211.2374093-5-ryan.roberts@arm.com
[2] https://lore.kernel.org/linux-mm/20240214204435.167852-1-david@redhat.com

Link: https://lkml.kernel.org/r/20240418134435.6092-5-ioworker0@gmail.com
Signed-off-by: Lance Yang <ioworker0@gmail.com>
Reviewed-by: Ryan Roberts <ryan.roberts@arm.com>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: Barry Song <21cnbao@gmail.com>
Cc: Jeff Xie <xiehuan09@gmail.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Muchun Song <songmuchun@bytedance.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Yang Shi <shy828301@gmail.com>
Cc: Yin Fengwei <fengwei.yin@intel.com>
Cc: Zach O'Keefe <zokeefe@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2024-05-05 17:53:43 -07:00
arch mm/arm64: override clear_young_dirty_ptes() batch helper 2024-05-05 17:53:42 -07:00
block block-6.9-20240412 2024-04-12 10:22:33 -07:00
certs This update includes the following changes: 2023-11-02 16:15:30 -10:00
crypto This push fixes a regression that broke iwd as well as a divide by 2024-03-25 10:48:23 -07:00
Documentation mm/page_table_check: support userfault wr-protect entries 2024-05-05 17:53:41 -07:00
drivers mm: pass VMA instead of MM to follow_pte() 2024-05-05 17:53:27 -07:00
fs fs/proc/task_mmu: convert smaps_hugetlb_range() to work on folios 2024-05-05 17:53:41 -07:00
include mm/madvise: introduce clear_young_dirty_ptes() batch helper 2024-05-05 17:53:42 -07:00
init mm: introduce slabobj_ext to support slab object extensions 2024-04-25 20:55:51 -07:00
io_uring mm: switch mm->get_unmapped_area() to a flag 2024-04-25 20:56:25 -07:00
ipc sysctl changes for v6.9-rc1 2024-03-18 14:59:13 -07:00
kernel mm/treewide: rename CONFIG_HAVE_FAST_GUP to CONFIG_HAVE_GUP_FAST 2024-04-25 20:56:41 -07:00
lib xarray: inline xas_descend to improve performance 2024-05-05 17:53:38 -07:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm mm/madvise: optimize lazyfreeing with mTHP in madvise_free 2024-05-05 17:53:43 -07:00
net mm: change inlined allocation helpers to account at the call site 2024-04-25 20:55:59 -07:00
rust rust: add a rust helper for krealloc() 2024-04-25 20:55:55 -07:00
samples Tracing updates for 6.9: 2024-03-18 15:11:44 -07:00
scripts lib: add allocation tagging support for memory allocation profiling 2024-04-25 20:55:52 -07:00
security mm: init_mlocked_on_free_v3 2024-04-25 20:56:29 -07:00
sound fix missing vmalloc.h includes 2024-04-25 20:55:49 -07:00
tools selftests/mm: fix additional build errors for selftests 2024-04-25 20:56:42 -07:00
usr Kbuild updates for v6.8 2024-01-18 17:57:07 -08:00
virt mm: pass VMA instead of MM to follow_pte() 2024-05-05 17:53:27 -07:00
.clang-format clang-format: Update with v6.7-rc4's for_each macro list 2023-12-08 23:54:38 +01:00
.cocciconfig
.editorconfig Add .editorconfig file for basic formatting 2023-12-28 16:22:47 +09:00
.get_maintainer.ignore Add Jeff Kirsher to .get_maintainer.ignore 2024-03-08 11:36:54 +00:00
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore kbuild: create a list of all built DTB files 2024-02-19 18:20:39 +09:00
.mailmap MAINTAINERS: update Naoya Horiguchi's email address 2024-04-16 15:39:51 -07:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING
CREDITS MAINTAINERS: Drop Gustavo Pimentel as PCI DWC Maintainer 2024-03-27 13:41:02 -05:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS MAINTAINERS: add entries for code tagging and memory allocation profiling 2024-04-25 20:55:58 -07:00
Makefile Linux 6.9-rc4 2024-04-14 13:38:39 -07:00
README README: Fix spelling 2024-03-18 03:36:32 -06:00

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the reStructuredText markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.