github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-12 04:23:03 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

x86/bpf: Make arch_bpf_trampoline_size allocate from EXECMEM_MODULE_DATA

Browse Source 
Jiri Olsa reports slowdown of tracing_multi benchmark that allocates huge
number of trampolines [1].

The slowdown caused by extra protection changes in execmem_alloc_rw() and
execmem_free().

With ROX caches enabled, all execmem allocations except EXECMEM_MODULE_DATA
are ROX after the allocation. execmem_alloc_rw() temporarily sets them to
W+NX and execmem_free() resets them back to ROX.

The only user of bpf_jit_alloc_exec_rw() is x86::arch_bpf_trampoline_size()
that only needs a temporary writable buffer in the modules address space.

On x86 executable memory and module data are constrained to the same
address range, so x86::arch_bpf_trampoline_size() can directly use
execmem_alloc(EXECMEM_MODULE_DATA)

Replace the call to bpf_jit_alloc_exec_rw() with a call to
execmem_alloc(EXECMEM_MODULE_DATA) in x86::arch_bpf_trampoline_size() and
drop bpf_jit_alloc_exec_rw() helper.

Fixes: 5bf02dbf39 ("bpf, x86: Make sure allocation in arch_bpf_trampoline_size() is writable")
Reported-by: Jiri Olsa <olsajiri@gmail.com>
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Tested-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/all/an8r7EODLIL-bZM3@krava
Link: https://lore.kernel.org/bpf/20260818130510.3110054-1-rppt@kernel.org
This commit is contained in:
Mike Rapoport (Microsoft) 2026-08-18 16:05:10 +03:00 committed by Daniel Borkmann
parent 91ec203513
commit c7a2a36182
3 changed files with 5 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
arch/x86/net/bpf_jit_comp.c
View File

@ -13,6 +13,7 @@
#include <linux/bpf_verifier.h>
#include <linux/memory.h>
#include <linux/sort.h>
#include <linux/execmem.h>
#include <asm/extable.h>
#include <asm/ftrace.h>
#include <asm/set_memory.h>
@ -3818,15 +3819,16 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
*
* We cannot use kvmalloc here, because we need image to be in
* module memory range.
* Since it must be writable use bpf_jit_alloc_exec_rw().
* Since it must be writable use execmem_alloc(EXECMEM_MODULE_DATA)
* that returns writable memory in the module address space.
*/
image = bpf_jit_alloc_exec_rw(PAGE_SIZE);
image = execmem_alloc(EXECMEM_MODULE_DATA, PAGE_SIZE);
if (!image)
return -ENOMEM;
ret = __arch_prepare_bpf_trampoline(&im, image, image + PAGE_SIZE, image,
m, flags, tnodes, func_addr);
bpf_jit_free_exec(image);
execmem_free(image);
return ret;
}

1
include/linux/filter.h
View File

@ -1376,7 +1376,6 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr,
void bpf_jit_binary_free(struct bpf_binary_header *hdr);
u64 bpf_jit_alloc_exec_limit(void);
void *bpf_jit_alloc_exec(unsigned long size);
void *bpf_jit_alloc_exec_rw(unsigned long size);
void bpf_jit_free_exec(void *addr);
void bpf_jit_free(struct bpf_prog *fp);
struct bpf_binary_header *

5
kernel/bpf/core.c
View File

@ -1128,11 +1128,6 @@ void *bpf_jit_alloc_exec(unsigned long size)
return execmem_alloc(EXECMEM_BPF, size);
}
void *bpf_jit_alloc_exec_rw(unsigned long size)
{
return execmem_alloc_rw(EXECMEM_BPF, size);
}
void bpf_jit_free_exec(void *addr)
{
execmem_free(addr);