mirror of
https://github.com/torvalds/linux.git
synced 2026-07-27 17:47:41 +02:00
Linux kernel source tree
dc795d4c02
Allocations from a fresh slab can consume all of its objects, and the freelist built during slab allocation is discarded immediately as a result. Instead of special-casing the whole-slab bulk refill case, defer freelist construction until after objects are emitted from a fresh slab. new_slab() now only allocates the slab and initializes its metadata. refill_objects() then obtains a fresh slab and lets alloc_from_new_slab() emit objects directly, building a freelist only for the objects left unallocated; the same change is applied to alloc_single_from_new_slab(). To keep CONFIG_SLAB_FREELIST_RANDOM=y/n on the same path, introduce a small iterator abstraction for walking free objects in allocation order. The iterator is used both for filling the sheaf and for building the freelist of the remaining objects. Also mark setup_object() inline. After this optimization, the compiler no longer consistently inlines this helper in the hot path, which can hurt performance. Explicitly marking it inline restores the expected code generation. This reduces per-object overhead when allocating from a fresh slab. The most direct benefit is in the paths that allocate objects first and only build a freelist for the remainder afterward: bulk allocation from a new slab in refill_objects(), single-object allocation from a new slab in ___slab_alloc(), and the corresponding early-boot paths that now use the same deferred-freelist scheme. Since refill_objects() is also used to refill sheaves, the optimization is not limited to the small set of kmem_cache_alloc_bulk()/kmem_cache_free_bulk() users; regular allocation workloads may benefit as well when they refill from a fresh slab. In slub_bulk_bench, the time per object drops by about 42% to 70% with CONFIG_SLAB_FREELIST_RANDOM=n, and by about 58% to 69% with CONFIG_SLAB_FREELIST_RANDOM=y. This benchmark is intended to isolate the cost removed by this change: each iteration allocates exactly slab->objects from a fresh slab. That makes it a near best-case scenario for deferred freelist construction, because the old path still built a full freelist even when no objects remained, while the new path avoids that work. Realistic workloads may see smaller end-to-end gains depending on how often allocations reach this fresh-slab refill path. Benchmark results (slub_bulk_bench): Machine: qemu-system-x86 -m 1024M -smp 8 -enable-kvm -cpu host Kernel: Linux 7.1.0-rc1-next-20260429 Config: x86_64_defconfig Cpu: 0 Rounds: 20 Total: 256MB - CONFIG_SLAB_FREELIST_RANDOM=n - obj_size=16, batch=256: before: 5.44 +- 0.07 ns/object after: 3.12 +- 0.03 ns/object delta: -42.6% obj_size=32, batch=128: before: 7.57 +- 0.32 ns/object after: 3.79 +- 0.07 ns/object delta: -49.9% obj_size=64, batch=64: before: 11.27 +- 0.09 ns/object after: 4.83 +- 0.06 ns/object delta: -57.2% obj_size=128, batch=32: before: 19.38 +- 0.13 ns/object after: 6.43 +- 0.08 ns/object delta: -66.8% obj_size=256, batch=32: before: 23.59 +- 0.18 ns/object after: 6.97 +- 0.07 ns/object delta: -70.5% obj_size=512, batch=32: before: 21.06 +- 0.14 ns/object after: 7.12 +- 0.17 ns/object delta: -66.2% - CONFIG_SLAB_FREELIST_RANDOM=y - obj_size=16, batch=256: before: 9.42 +- 0.11 ns/object after: 4.36 +- 0.19 ns/object delta: -53.7% obj_size=32, batch=128: before: 12.19 +- 0.62 ns/object after: 4.93 +- 0.07 ns/object delta: -59.6% obj_size=64, batch=64: before: 17.01 +- 0.73 ns/object after: 6.14 +- 0.12 ns/object delta: -63.9% obj_size=128, batch=32: before: 23.71 +- 1.10 ns/object after: 8.35 +- 0.18 ns/object delta: -64.8% obj_size=256, batch=32: before: 29.20 +- 0.35 ns/object after: 9.44 +- 1.32 ns/object delta: -67.7% obj_size=512, batch=32: before: 29.35 +- 0.79 ns/object after: 9.21 +- 0.34 ns/object delta: -68.6% Link: https://github.com/HSM6236/slub_bulk_test.git Suggested-by: Harry Yoo (Oracle) <harry@kernel.org> Reviewed-by: Harry Yoo (Oracle) <harry@kernel.org> Reviewed-by: Hao Li <hao.li@linux.dev> Tested-by: Hao Li <hao.li@linux.dev> Signed-off-by: Shengming Hu <hu.shengming@zte.com.cn> Link: https://patch.msgid.link/202604302204413066CxdJnJ3RAGH_7iE4EBIO@zte.com.cn Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> |
||
|---|---|---|
| arch | ||
| block | ||
| certs | ||
| crypto | ||
| Documentation | ||
| drivers | ||
| fs | ||
| include | ||
| init | ||
| io_uring | ||
| ipc | ||
| kernel | ||
| lib | ||
| LICENSES | ||
| mm | ||
| net | ||
| rust | ||
| samples | ||
| scripts | ||
| security | ||
| sound | ||
| tools | ||
| usr | ||
| virt | ||
| .clang-format | ||
| .clippy.toml | ||
| .cocciconfig | ||
| .editorconfig | ||
| .get_maintainer.ignore | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| .pylintrc | ||
| .rustfmt.toml | ||
| COPYING | ||
| CREDITS | ||
| Kbuild | ||
| Kconfig | ||
| MAINTAINERS | ||
| Makefile | ||
| README | ||
Linux kernel ============ The Linux kernel is the core of any Linux operating system. It manages hardware, system resources, and provides the fundamental services for all other software. Quick Start ----------- * Report a bug: See Documentation/admin-guide/reporting-issues.rst * Get the latest kernel: https://kernel.org * Build the kernel: See Documentation/admin-guide/quickly-build-trimmed-linux.rst * Join the community: https://lore.kernel.org/ Essential Documentation ----------------------- All users should be familiar with: * Building requirements: Documentation/process/changes.rst * Code of Conduct: Documentation/process/code-of-conduct.rst * License: See COPYING Documentation can be built with make htmldocs or viewed online at: https://www.kernel.org/doc/html/latest/ Who Are You? ============ Find your role below: * New Kernel Developer - Getting started with kernel development * Academic Researcher - Studying kernel internals and architecture * Security Expert - Hardening and vulnerability analysis * Backport/Maintenance Engineer - Maintaining stable kernels * System Administrator - Configuring and troubleshooting * Maintainer - Leading subsystems and reviewing patches * Hardware Vendor - Writing drivers for new hardware * Distribution Maintainer - Packaging kernels for distros * AI Coding Assistant - LLMs and AI-powered development tools For Specific Users ================== New Kernel Developer -------------------- Welcome! Start your kernel development journey here: * Getting Started: Documentation/process/development-process.rst * Your First Patch: Documentation/process/submitting-patches.rst * Coding Style: Documentation/process/coding-style.rst * Build System: Documentation/kbuild/index.rst * Development Tools: Documentation/dev-tools/index.rst * Kernel Hacking Guide: Documentation/kernel-hacking/hacking.rst * Core APIs: Documentation/core-api/index.rst Academic Researcher ------------------- Explore the kernel's architecture and internals: * Researcher Guidelines: Documentation/process/researcher-guidelines.rst * Memory Management: Documentation/mm/index.rst * Scheduler: Documentation/scheduler/index.rst * Networking Stack: Documentation/networking/index.rst * Filesystems: Documentation/filesystems/index.rst * RCU (Read-Copy Update): Documentation/RCU/index.rst * Locking Primitives: Documentation/locking/index.rst * Power Management: Documentation/power/index.rst Security Expert --------------- Security documentation and hardening guides: * Security Documentation: Documentation/security/index.rst * LSM Development: Documentation/security/lsm-development.rst * Self Protection: Documentation/security/self-protection.rst * Reporting Vulnerabilities: Documentation/process/security-bugs.rst * CVE Procedures: Documentation/process/cve.rst * Embargoed Hardware Issues: Documentation/process/embargoed-hardware-issues.rst * Security Features: Documentation/userspace-api/seccomp_filter.rst Backport/Maintenance Engineer ----------------------------- Maintain and stabilize kernel versions: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * Backporting Guide: Documentation/process/backporting.rst * Applying Patches: Documentation/process/applying-patches.rst * Subsystem Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git for Maintainers: Documentation/maintainer/configure-git.rst System Administrator -------------------- Configure, tune, and troubleshoot Linux systems: * Admin Guide: Documentation/admin-guide/index.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Sysctl Tuning: Documentation/admin-guide/sysctl/index.rst * Tracing/Debugging: Documentation/trace/index.rst * Performance Security: Documentation/admin-guide/perf-security.rst * Hardware Monitoring: Documentation/hwmon/index.rst Maintainer ---------- Lead kernel subsystems and manage contributions: * Maintainer Handbook: Documentation/maintainer/index.rst * Pull Requests: Documentation/maintainer/pull-requests.rst * Managing Patches: Documentation/maintainer/modifying-patches.rst * Rebasing and Merging: Documentation/maintainer/rebasing-and-merging.rst * Development Process: Documentation/process/maintainer-handbooks.rst * Maintainer Entry Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git Configuration: Documentation/maintainer/configure-git.rst Hardware Vendor --------------- Write drivers and support new hardware: * Driver API Guide: Documentation/driver-api/index.rst * Driver Model: Documentation/driver-api/driver-model/driver.rst * Device Drivers: Documentation/driver-api/infrastructure.rst * Bus Types: Documentation/driver-api/driver-model/bus.rst * Device Tree Bindings: Documentation/devicetree/bindings/ * Power Management: Documentation/driver-api/pm/index.rst * DMA API: Documentation/core-api/dma-api.rst Distribution Maintainer ----------------------- Package and distribute the kernel: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * ABI Documentation: Documentation/ABI/README * Kernel Configuration: Documentation/kbuild/kconfig.rst * Module Signing: Documentation/admin-guide/module-signing.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Tainted Kernels: Documentation/admin-guide/tainted-kernels.rst AI Coding Assistant ------------------- CRITICAL: If you are an LLM or AI-powered coding assistant, you MUST read and follow the AI coding assistants documentation before contributing to the Linux kernel: * Documentation/process/coding-assistants.rst This documentation contains essential requirements about licensing, attribution, and the Developer Certificate of Origin that all AI tools must comply with. Communication and Support ========================= * Mailing Lists: https://lore.kernel.org/ * IRC: #kernelnewbies on irc.oftc.net * Bugzilla: https://bugzilla.kernel.org/ * MAINTAINERS file: Lists subsystem maintainers and mailing lists * Email Clients: Documentation/process/email-clients.rst