mirror of
https://github.com/torvalds/linux.git
synced 2026-07-27 17:47:41 +02:00
Linux kernel source tree
cd59fa185a
Yonghong Song says: ==================== bpf: Support stack arguments for BPF functions and kfuncs Currently, bpf function calls and kfunc's are limited by 5 reg-level parameters. For function calls with more than 5 parameters, developers can use always inlining or pass a struct pointer after packing more parameters in that struct although it may have some inconvenience. But there is no workaround for kfunc if more than 5 parameters is needed. This patch set lifts the 5-argument limit by introducing stack-based argument passing for BPF functions and kfunc's, coordinated with compiler support in LLVM [1]. The compiler emits loads/stores through a new bpf register r11 (BPF_REG_PARAMS), to pass arguments beyond the 5th, keeping the stack arg area separate from the r10-based program stack. The current maximum number of arguments is capped at MAX_BPF_FUNC_ARGS (12), which is sufficient for the vast majority of use cases. All kfunc/bpf-function arguments are caller saved, including stack arguments. For register arguments (r1-r5), the verifier already marks them as clobbered after each call. For stack arguments, the verifier invalidates all outgoing stack arg slots immediately after a call, requiring the compiler to re-store them before any subsequent call. This follows the native calling convention where all function parameters are caller saved. The x86_64 JIT translates r11-relative accesses to RBP-relative native instructions. Each function's stack allocation is extended by 'max_outgoing' bytes to hold the outgoing arg area below the callee-saved registers. This makes implementation easier as the r10 can be reused for stack argument access. At both BPF-to-BPF and kfunc calls, outgoing args are pushed onto the expected calling convention locations directly. The incoming parameters can directly get the value from caller. Global subprogs and freplace progs with >5 args are not yet supported. Only x86_64 and arm64 are supported for now. Same selftests are tested by both x86_64 and arm64. Please see each individual patch for details. [1] https://github.com/llvm/llvm-project/pull/189060 Changelogs: v3 -> v4: - v3: https://lore.kernel.org/bpf/20260511053301.1878610-1-yonghong.song@linux.dev/ - Added no_stack_arg_load comparison in func_states_equal() to ensure correctness of pruning. - Shrink bpf_jmp_history_entry.flags to 4bit to match the number of flags. - Instead of passing bpf_subprog_info to JIT, use prog->aux->func_idx to find corresponding bpf_subprog_info from 'env'. - For patch 'bpf: Reject stack arguments if tail call reachable', use stack_arg_cnt instead of just incoming stack arg cnt. - Tighten invalidate_outgoing_stack_args() for kfunc/helper/bpf-to-bpf calls. - Disable private stack in verifier for x86_64 instead of in JIT. v2 -> v3: - v2: https://lore.kernel.org/bpf/20260507212942.1122000-1-yonghong.song@linux.dev/ - In do_check_common() and for main prog, if btf does not match with actual parameter, the verification will continue and will ignore arg_cnt. Make arg_cnt=1 explictly to prevent any incoming stack arguments. - Remove the loop which clear current frame stack slot and set the upper level frame stack slot. This is not needed unless there is a bug. Add a verifier_bug if the bug happens. - For liveness, avoid r11 based load/stores mixing with r10 based stack tracking. Also, print out stack arguments properly. - Pass bpf_subprog_info the JIT so we can avoid copy bpf_subprog_info fields to bpf_prog_aux. - Fix the missed allocation free for test infra BTF fixup. - Remove selftest result for precision backtracking test since the result would be change (two possible output). v1 -> v2: - v1: https://lore.kernel.org/bpf/20260424171433.2034470-1-yonghong.song@linux.dev/ - Several refactoring (convert bpf_get_spilled_reg macro to static inline func, Remove copy_register_state(), Refactor jmp history, Refactor record_call_access(), etc), suggested by Eduard. - Use incoming_stack_arg_cnt/stack_arg_cnt instead of incoming_stack_arg_depth/stack_arg_depth, suggested by Eduard. - Fix a stack arg pruning bug, from Eduard. - Fix a bug for precision marking and backtracking, basically callee needs to get the stack arg value from callers, helped from Eduard. - Set sub->arg_cnt earlier in btf_prepare_func_args(), this will avoid having incoming_stack_arg_cnt in bpf_subprog_info. - Do stack-arg liveness analysis together with r10 based liveness analysis, suggested by Eduard. - Fix a few tests to ensure that r11-based loads cannot be ahead of r11-based stores, and r11-based loads cannot be after kfunc/helper/bpf-function. ==================== Link: https://patch.msgid.link/20260513044949.2382019-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org> |
||
|---|---|---|
| arch | ||
| block | ||
| certs | ||
| crypto | ||
| Documentation | ||
| drivers | ||
| fs | ||
| include | ||
| init | ||
| io_uring | ||
| ipc | ||
| kernel | ||
| lib | ||
| LICENSES | ||
| mm | ||
| net | ||
| rust | ||
| samples | ||
| scripts | ||
| security | ||
| sound | ||
| tools | ||
| usr | ||
| virt | ||
| .clang-format | ||
| .clippy.toml | ||
| .cocciconfig | ||
| .editorconfig | ||
| .get_maintainer.ignore | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| .pylintrc | ||
| .rustfmt.toml | ||
| COPYING | ||
| CREDITS | ||
| Kbuild | ||
| Kconfig | ||
| MAINTAINERS | ||
| Makefile | ||
| README | ||
Linux kernel ============ The Linux kernel is the core of any Linux operating system. It manages hardware, system resources, and provides the fundamental services for all other software. Quick Start ----------- * Report a bug: See Documentation/admin-guide/reporting-issues.rst * Get the latest kernel: https://kernel.org * Build the kernel: See Documentation/admin-guide/quickly-build-trimmed-linux.rst * Join the community: https://lore.kernel.org/ Essential Documentation ----------------------- All users should be familiar with: * Building requirements: Documentation/process/changes.rst * Code of Conduct: Documentation/process/code-of-conduct.rst * License: See COPYING Documentation can be built with make htmldocs or viewed online at: https://www.kernel.org/doc/html/latest/ Who Are You? ============ Find your role below: * New Kernel Developer - Getting started with kernel development * Academic Researcher - Studying kernel internals and architecture * Security Expert - Hardening and vulnerability analysis * Backport/Maintenance Engineer - Maintaining stable kernels * System Administrator - Configuring and troubleshooting * Maintainer - Leading subsystems and reviewing patches * Hardware Vendor - Writing drivers for new hardware * Distribution Maintainer - Packaging kernels for distros * AI Coding Assistant - LLMs and AI-powered development tools For Specific Users ================== New Kernel Developer -------------------- Welcome! Start your kernel development journey here: * Getting Started: Documentation/process/development-process.rst * Your First Patch: Documentation/process/submitting-patches.rst * Coding Style: Documentation/process/coding-style.rst * Build System: Documentation/kbuild/index.rst * Development Tools: Documentation/dev-tools/index.rst * Kernel Hacking Guide: Documentation/kernel-hacking/hacking.rst * Core APIs: Documentation/core-api/index.rst Academic Researcher ------------------- Explore the kernel's architecture and internals: * Researcher Guidelines: Documentation/process/researcher-guidelines.rst * Memory Management: Documentation/mm/index.rst * Scheduler: Documentation/scheduler/index.rst * Networking Stack: Documentation/networking/index.rst * Filesystems: Documentation/filesystems/index.rst * RCU (Read-Copy Update): Documentation/RCU/index.rst * Locking Primitives: Documentation/locking/index.rst * Power Management: Documentation/power/index.rst Security Expert --------------- Security documentation and hardening guides: * Security Documentation: Documentation/security/index.rst * LSM Development: Documentation/security/lsm-development.rst * Self Protection: Documentation/security/self-protection.rst * Reporting Vulnerabilities: Documentation/process/security-bugs.rst * CVE Procedures: Documentation/process/cve.rst * Embargoed Hardware Issues: Documentation/process/embargoed-hardware-issues.rst * Security Features: Documentation/userspace-api/seccomp_filter.rst Backport/Maintenance Engineer ----------------------------- Maintain and stabilize kernel versions: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * Backporting Guide: Documentation/process/backporting.rst * Applying Patches: Documentation/process/applying-patches.rst * Subsystem Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git for Maintainers: Documentation/maintainer/configure-git.rst System Administrator -------------------- Configure, tune, and troubleshoot Linux systems: * Admin Guide: Documentation/admin-guide/index.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Sysctl Tuning: Documentation/admin-guide/sysctl/index.rst * Tracing/Debugging: Documentation/trace/index.rst * Performance Security: Documentation/admin-guide/perf-security.rst * Hardware Monitoring: Documentation/hwmon/index.rst Maintainer ---------- Lead kernel subsystems and manage contributions: * Maintainer Handbook: Documentation/maintainer/index.rst * Pull Requests: Documentation/maintainer/pull-requests.rst * Managing Patches: Documentation/maintainer/modifying-patches.rst * Rebasing and Merging: Documentation/maintainer/rebasing-and-merging.rst * Development Process: Documentation/process/maintainer-handbooks.rst * Maintainer Entry Profile: Documentation/maintainer/maintainer-entry-profile.rst * Git Configuration: Documentation/maintainer/configure-git.rst Hardware Vendor --------------- Write drivers and support new hardware: * Driver API Guide: Documentation/driver-api/index.rst * Driver Model: Documentation/driver-api/driver-model/driver.rst * Device Drivers: Documentation/driver-api/infrastructure.rst * Bus Types: Documentation/driver-api/driver-model/bus.rst * Device Tree Bindings: Documentation/devicetree/bindings/ * Power Management: Documentation/driver-api/pm/index.rst * DMA API: Documentation/core-api/dma-api.rst Distribution Maintainer ----------------------- Package and distribute the kernel: * Stable Kernel Rules: Documentation/process/stable-kernel-rules.rst * ABI Documentation: Documentation/ABI/README * Kernel Configuration: Documentation/kbuild/kconfig.rst * Module Signing: Documentation/admin-guide/module-signing.rst * Kernel Parameters: Documentation/admin-guide/kernel-parameters.rst * Tainted Kernels: Documentation/admin-guide/tainted-kernels.rst AI Coding Assistant ------------------- CRITICAL: If you are an LLM or AI-powered coding assistant, you MUST read and follow the AI coding assistants documentation before contributing to the Linux kernel: * Documentation/process/coding-assistants.rst This documentation contains essential requirements about licensing, attribution, and the Developer Certificate of Origin that all AI tools must comply with. Communication and Support ========================= * Mailing Lists: https://lore.kernel.org/ * IRC: #kernelnewbies on irc.oftc.net * Bugzilla: https://bugzilla.kernel.org/ * MAINTAINERS file: Lists subsystem maintainers and mailing lists * Email Clients: Documentation/process/email-clients.rst