There is no LoongArch support for get_preempt_count() currently and its
fallback path always returns 0, just add it so that bpf_in_interrupt(),
bpf_in_nmi(), bpf_in_hardirq(), bpf_in_serving_softirq(), bpf_in_task()
work for LoongArch as well.
The latest kernels select CONFIG_THREAD_INFO_IN_TASK, it can just read
preempt_count from the thread_info which is embedded within task_struct.
With this patch, "./test_progs -t exe_ctx" passes on LoongArch.
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Add missing clocks and clock-frequency properties to i2c nodes for LS2K2000.
Signed-off-by: Hongliang Wang <wanghongliang@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Add missing clocks and clock-frequency properties to i2c nodes for LS2K1000.
Signed-off-by: Hongliang Wang <wanghongliang@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Add missing clocks and clock-frequency properties to i2c nodes for LS2K0500.
Signed-off-by: Hongliang Wang <wanghongliang@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The pointer to thread_info is always available in the $tp register, so
the call to bpf_get_smp_processor_id() can be inlined into a single load
instruction.
(1) Here is the sample test.c:
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
SEC("raw_tp/sys_enter")
int test_cpuid(void *ctx)
{
return bpf_get_smp_processor_id();
}
char _license[] SEC("license") = "GPL";
(2) Here are the test steps:
sudo yum install libbpf-devel kernel-devel bpftool
clang -target bpf -O2 -c test.c -o test.o
sudo sysctl -w net.core.bpf_jit_enable=1
sudo bpftool prog show name test_cpuid
sudo rm -f /sys/fs/bpf/test_cpuid
sudo bpftool prog load test.o /sys/fs/bpf/test_cpuid
ID=$(sudo bpftool prog show pinned /sys/fs/bpf/test_cpuid | grep -oE '^[0-9]+')
sudo bpftool prog dump jited id $ID
(3) Here are the test results:
Before: 6 instructions
...
64: lu12i.w $t1, 1213
68: ori $t1, $t1, 1680
6c: lu32i.d $t1, 0
70: lu52i.d $t1, $t1, -1792
74: jirl $ra, $t1, 0
78: move $a5, $a0
...
After: 1 instruction
...
64: ld.wu $a5, $tp, 16
...
This is similar with commit 2ddec2c80b ("riscv, bpf: inline
bpf_get_smp_processor_id()").
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The pointer to task_struct is always available in the $tp register, so
the calls to bpf_get_current_task() and bpf_get_current_task_btf() can
be inlined into a single move instruction.
(1) Here is the sample test.c:
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>
SEC("raw_tp/sys_enter")
long test_task(void *ctx)
{
return (long)bpf_get_current_task();
}
char _license[] SEC("license") = "GPL";
(2) Here are the test steps:
sudo yum install libbpf-devel kernel-devel bpftool
clang -target bpf -O2 -c test.c -o test.o
sudo sysctl -w net.core.bpf_jit_enable=1
sudo bpftool prog show name test_task
sudo rm -f /sys/fs/bpf/test_task
sudo bpftool prog load test.o /sys/fs/bpf/test_task
ID=$(sudo bpftool prog show pinned /sys/fs/bpf/test_task | grep -oE '^[0-9]+')
sudo bpftool prog dump jited id $ID
(3) Here are the test results:
Before: 6 instructions
...
64: lu12i.w $t1, 1093
68: ori $t1, $t1, 3320
6c: lu32i.d $t1, 0
70: lu52i.d $t1, $t1, -1792
74: jirl $ra, $t1, 0
78: move $a5, $a0
...
After: 1 instruction
...
64: move $a5, $tp
...
This is similar with commit 2bb138cb20 ("bpf, arm64: Inline
bpf_get_current_task/_btf() helpers").
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The current code updates the tail call counter (TCC) using a pre-increment
approach, it stores the incremented value back to memory before performing
any boundary or target validation checks.
This causes two major issues:
1. When a tail call fails because the target program is NULL, the TCC is
incorrectly incremented and saved in memory anyway.
2. This dummy increment implicitly consumes one slot of the allowed tail
call budget. As a result, the subsequent loop reaches the maximum limit
prematurely, leading to a test failure where the actual loop count is
32 instead of the expected 33.
Fix this by deferring the counter update. Change the branch condition to
BPF_JSGE (greater or equal) so that we check the boundary first. The TCC
is only incremented and stored back to memory after the boundary check
and the NULL-target check both pass.
Before:
$ sudo ./test_progs -t tailcalls/tailcall_3
...
test_tailcall_count:FAIL:tailcall count unexpected tailcall count: actual 32 != expected 33
...
#465/3 tailcalls/tailcall_3:FAIL
#465 tailcalls:FAIL
After:
$ sudo ./test_progs -t tailcalls/tailcall_3
#465/3 tailcalls/tailcall_3:OK
#465 tailcalls:OK
Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
Fixes: c0fcc955ff ("LoongArch: BPF: Fix the tailcall hierarchy")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The current LoongArch BPF JIT implementation hardcodes the number of
prologue instructions skipped during a tail call as a magic number '7'
in the jirl instruction. However, the accompanying comment explaining
this offset is completely outdated. It inaccurately states that only
a single TCC initialization instruction is bypassed, but in reality,
multiple setup slots are skipped, so fix these outdated comments in
__build_epilogue().
While at it, refine the comments in build_prologue() to describe the
skipped setup slots (RA saving, fentry nops, and the TCC register slot)
using proper dynamic tracing context. Also, remove the magic number '7'
by introducing descriptive macros to formally define the prologue layout
and make the tail call jump offset self-documenting.
Fixes: 61319d15a5 ("LoongArch: BPF: Adjust the jump offset of tail calls")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
The vDSO needs to have a unique build id in a similar manner to the
kernel and modules. Use the build salt macro.
Signed-off-by: Bastian Blank <waldi@debian.org>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
set_direct_map_valid_noflush() incorrectly passes 1 to __set_memory()
instead of nr. This causes only the first page's attr to be updated when
nr > 1.
Other architectures all pass nr correctly.
Cc: stable@vger.kernel.org
Fixes: 0c6378a715 ("arch: introduce set_direct_map_valid_noflush()")
Signed-off-by: Xuewen Wang <wangxuewen@kylinos.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
When hardware page table walker (PTW) is enabled on LoongArch, the CPU
may set _PAGE_DIRTY directly in the page table entry during a write TLB
miss, without going through the software TLB store handler. The software
TLB store handler (tlbex.S:254) sets both _PAGE_DIRTY and_PAGE_MODIFIED
together:
ori t0, t0, (_PAGE_VALID | _PAGE_DIRTY | _PAGE_MODIFIED)
Since hardware PTW only sets _PAGE_DIRTY, the software-only bit, i.e.
_PAGE_MODIFIED is left unchanged. This creates a window where a PTE has
_PAGE_DIRTY set (hardware knows the page is dirty) but _PAGE_MODIFIED
clear (software is unaware).
When fork()/clone() triggers copy-on-write, __copy_present_ptes() calls
pte_wrprotect(), which unconditionally clears both the _PAGE_WRITE and
_PAGE_DIRTY bits:
pte_val(pte) &= ~(_PAGE_WRITE | _PAGE_DIRTY);
Since _PAGE_MODIFIED was never set, the dirtiness information is lost
completely. Subsequently, when memory pressure triggers page reclaim,
page_mkclean() / try_to_unmap() sees the page as clean (i.e. pte_dirty()
returns false) and the page may be freed without writeback, causing data
corruption.
Fix this by propagating the _PAGE_DIRTY bit to the _PAGE_MODIFIED bit in
both pte_wrprotect() and pmd_wrprotect() before clearing writeable bits:
if (pte_val(pte) & _PAGE_DIRTY)
pte_val(pte) |= _PAGE_MODIFIED;
The pmd_wrprotect() fix handles the CONFIG_TRANSPARENT_HUGEPAGE case,
where pmd entries need the same treatment.
This ensures the software dirty tracking bit (checked by pte_dirty() and
pmd_dirty(), which read both the _PAGE_DIRTY and _PAGE_MODIFIED bits) is
preserved across fork COW write-protection.
The issue was found by the LTP madvise09 test case, which exercises page
reclaim after "madvise(MADV_FREE), write and fork" operation sequence on
private anonymous mappings.
Cc: stable@vger.kernel.org
Fixes: 09cfefb7fa ("LoongArch: Add memory management")
Co-developed-by: Tianyang Zhang <zhangtianyang@loongson.cn>
Signed-off-by: Tianyang Zhang <zhangtianyang@loongson.cn>
Signed-off-by: Hongchen Zhang <zhanghongchen@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
arch_kimage_file_post_load_cleanup() and load_other_segments(), both
inside the CONFIG_KEXEC_FILE block, take a struct kimage pointer before
the forward declaration appears. Move the forward declaration above so
it precedes its first use instead of relying on a transitive include.
Fixes: d162feec6b ("LoongArch: Add preparatory infrastructure for kexec_file")
Signed-off-by: 谢致邦 (XIE Zhibang) <Yeking@Red54.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
This is a port of MIPS commit 9f3f3bdc6d ("MIPS: smp: report dying
CPU to RCU in stop_this_cpu()"). smp_send_stop() parks all secondary
CPUs in stop_this_cpu(). And the function marks the CPU offline for the
scheduler via set_cpu_online(false) but never informs RCU, so RCU keeps
expecting a quiescent state from CPUs that are now spinning forever with
interrupts disabled.
As long as nothing waits for an RCU grace period after smp_send_stop()
this is harmless, which is why it went unnoticed. However, since commit
91840be8f7 ("irq_work: Fix use-after-free in irq_work_single() on
PREEMPT_RT"), irq_work_sync() calls synchronize_rcu() on architectures
without an irq_work self-IPI, i.e. where arch_irq_work_has_interrupt()
returns false. Any irq_work_sync() issued in the reboot/shutdown/halt
path after smp_send_stop() then blocks on a grace period that can never
complete, hanging the reboot:
WARNING: CPU: 0 PID: 15 at kernel/irq_work.c:144 irq_work_queue_on
...
rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
rcu: Offline CPU 1 blocking current GP.
rcu: Offline CPU 2 blocking current GP.
rcu: Offline CPU 3 blocking current GP.
This issue needs some hacks to reproduce, and it was not noticed on
LoongArch because arch_irq_work_has_interrupt() usually returns true.
Call rcutree_report_cpu_dead() once interrupts are disabled, mirroring
the generic CPU-hotplug offline path, so RCU stops waiting on the parked
CPUs and grace periods can still complete. LoongArch shuts down all CPUs
here without going through the CPU-hotplug mechanism, so this report is
not otherwise issued.
Cc: <stable@vger.kernel.org>
Fixes: 91840be8f7 ("irq_work: Fix use-after-free in irq_work_single() on PREEMPT_RT")
Reviewed-by: Guo Ren <guoren@kernel.org>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
For ACPI system we suppose the ISA/LPC PIO range is registered together
with PCI root bridge. But the fact is there may be some early access to
the ISA/LPC PIO range before ACPI PCI root register (most of them are
due to abnormal BIOS). Unconditionally register the ISA/LPC PIO range
usually causes ACPI PCI root register fail because of the address range
confliction. So we add a pair of helpers: acpi_add_early_pio() to add
PIO for early access, and acpi_remove_early_pio() to remove PIO before
PCI root register. Since acpi_remove_early_pio() may be called multiple
times, we add an acpi_pio flag to ensure PIO be removed only once.
Cc: <stable@vger.kernel.org>
Tested-by: Yuanzhen Gan <elysia-best@simplelinux.cn.eu.org>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Like other architectures such as x86, arm64, riscv, powerpc and s390,
select THREAD_INFO_IN_TASK for LoongArch to move thread_info off the
stack into task_struct. This follows modern kernel standards and also
makes the system more secure.
With this patch, thread_info is included in task_struct at an offset
of 0 instead of being placed at the bottom of the kernel stack. Thus,
the $tp register points to both thread_info and task_struct.
To support this, introduce a per-CPU variable cpu_tasks to store the
pointer to the current task_struct. This decouples the recovery of the
$tp register from the stack pointer during exception entry.
Then initialize cpu_tasks for the primary and secondary CPUs during
arch-specific setup and SMP boot paths. To eliminate the dangerous
windows during the early initialization where the cpu_tasks remains
uninitialized, set_current() is invoked as early as possible in both
setup_arch() and start_secondary(). This ensures the $tp recovery
barrier is armed in case any early boot exceptions or kernel panics
occur.
Modify SAVE_SOME and handle_syscall to restore the $tp register from
cpu_tasks, and also use the la_abs absolute addressing for cpu_tasks
access in assembly to bypass the relocation limits within exception
handling sections. By advancing the preservation of u0 in SAVE_SOME,
we reuse the PERCPU_BASE_KS value in u0 for the cpu_tasks calculation,
effectively eliminating a duplicate csrrd instruction execution on SMP
platforms.
Update <asm/switch_to.h> and <kernel/switch.S> to fully support the
CONFIG_THREAD_INFO_IN_TASK feature.
Remove the obsolete next_ti argument from __switch_to(), which shifts
the remaining arguments ahead in the calling convention (sched_ra from
a3 to a2, and sched_cfa from a4 to a3). Under the new configuration,
__switch_to() now directly derives the thread pointer ($tp) from the
next task_struct pointer in a1.
To preserve the optimal and clean "move tp, a1" path for 64-bit kernels,
the thread pointer ($tp) is assigned directly from a1 in the core path.
For 32-bit kernels, where a1 carries a 2000-byte structural pointer bias
at entry, an explicit adjustment "PTR_ADDI tp, tp, -TASK_STRUCT_OFFSET"
is introduced at the function exit.
In the context of __switch_to(), local interrupts are disabled, and the
kernel is in a critical switching phase where handling any synchronous
exception is practically impossible and prohibited.
If any synchronous exception or watchpoint does trigger in this narrow
window, it constitutes a fatal double fault and the kernel is expected
to die/panic immediately anyway. Therefore, the temporary biased value
in $tp is safe and acceptable here.
Additionally, evaluate the stack lookup as a single load instruction
"LONG_LPTR t0, a1, (TASK_STACK - TASK_STRUCT_OFFSET)", this perfectly
satisfies both 32-bit and 64-bit kernels. Using the "next" pointer in
a1 as the base register, rather than $tp, effectively unchains the data
dependency (RAW hazard) from the preceding move instruction, maximizing
the instruction-level parallelism and superscalar execution efficiency
while naturally adapting the structural shift.
With CONFIG_THREAD_INFO_IN_TASK enabled, the kernel stack life cycle is
decoupled from task_struct and can be freed concurrently.
Currently, show_stacktrace() reads raw stack data via __get_addr() and
subsequently calls show_backtrace() to unwind the frame, without holding
any reference to the target task's stack. If show_stacktrace() is called
on a concurrently exiting task, it could attempt to read from a freed or
reallocated kernel stack. This introduces a severe use-after-free (UAF)
read risk or kernel panics.
Wrap the entire stack inspection process inside show_stacktrace() with
a try_get_task_stack() and put_task_stack() pair. This ensures the task
stack remains pinned safely during both the raw stack data dump loop and
the subsequent stack unwinding phase.
Also, ensure that the task pointer is initialized to "current" early if
it is NULL, so that try_get_task_stack() always operates on a valid task
reference.
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
clang 23 fails to build crypto_bench.c and crypto_sanity.c with
"BPF stack limit exceeded". The progs fill a 408-byte
bpf_crypto_params on the stack and pass it to bpf_crypto_ctx_create().
clang 23 copies the byte-aligned cipher/key globals into it one byte at
a time through the stack, and keeps more than one copy of the struct
around. Together that blows the 512-byte limit.
Align the source arrays to 8 bytes so the copy is word-wise, and move
params off the stack into a static .bss var. static keeps it out of the
skeleton, where bpf_crypto_params is an incomplete type. Either change
alone is not enough.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Jiayuan Chen says:
====================
bpf, skmsg: some fixes for skmsg
All fixes are from previous patches sent by Weiming Shi, Zhang Cen,
Kuniyuki and Sechang Lim, which have already been reviewed by me and John and Jakub.
https://lore.kernel.org/bpf/20260610081218.506709-2-rhkrqnwk98@gmail.com/https://lore.kernel.org/bpf/20260520102715.3033936-1-rollkingzzc@gmail.com/https://lore.kernel.org/bpf/20260424191602.1522411-3-bestswngs@gmail.com/https://lore.kernel.org/bpf/20260423155807.1245644-2-bestswngs@gmail.com/https://lore.kernel.org/bpf/20260221233234.3814768-4-kuniyu@google.com/
The automated reviewer (sashiko) may still flag a few other potential
issues on top of this series. After looking into them, they are either
already covered by the patches here, are the BPF program's own
responsibility (e.g. initializing the payload it pushes) and intentionally
left out, or only reachable under very narrow conditions that require a
specially crafted BPF program and an unusual sk_msg ring state, so they are
not practical to trigger and are left out of this series. I'm collecting
these fixes together because the same
problems have been re-sent many times in slightly different forms, and I
hope this series can be prioritized for merging so the duplicates can
finally settle. With so many AI-generated patches floating around for
these spots, leaving them unmerged just keeps wasting maintainer review
cycles on the same issues.
v3->v4: Carry Kuniyuki Iwashima's reviewed-by tag.
Drop the __GFP_ZERO patch; initializing the pushed payload is the
BPF program's responsibility, not the kernel's (per maintainer
feedback).
https://lore.kernel.org/bpf/20260612130919.299124-1-jiayuan.chen@linux.dev/
v2->v3: Target to bpf-next and carry Emil's reviewed-by tag.
Reverse xmas tree style is used suggested by Cong.
(not all code match reverse xmas tree due to variable dependency)
v1->v2: fix problem when fix the conflict.
====================
Link: https://patch.msgid.link/20260615021959.140010-1-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a test in sockmap_basic.c that calls bpf_msg_pop_data() with a length
close to U32_MAX, which overflows the start + len bounds check. The sk_msg
program records the return value over a sendmsg and the test checks that
the call is rejected with -EINVAL.
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Cc: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260615021959.140010-7-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
start and len are u32, so
u64 last = start + len;
evaluates start + len in 32-bit and wraps before storing it in last.
The bounds check
if (start >= offset + l || last > msg->sg.size)
return -EINVAL;
can then be passed with an out-of-range start/len, after which the pop
loop runs off the end of the scatterlist and sk_msg_shift_left() calls
put_page() on the empty msg->sg.end slot:
Oops: general protection fault, probably for non-canonical address
0xdffffc0000000001: 0000 [#1] SMP KASAN PTI
KASAN: null-ptr-deref in range [0x0000000000000008-0x000000000000000f]
RIP: 0010:sk_msg_shift_left net/core/filter.c:2957 [inline]
RIP: 0010:____bpf_msg_pop_data net/core/filter.c:3103 [inline]
RIP: 0010:bpf_msg_pop_data+0x753/0x1a10 net/core/filter.c:2984
Call Trace:
<TASK>
bpf_prog_4cc92c278f4d5d56+0x1b1/0x1e8
bpf_prog_run_pin_on_cpu+0x107/0x320 include/linux/filter.h:746
sk_psock_msg_verdict+0x357/0x7f0 net/core/skmsg.c:934
tcp_bpf_send_verdict net/ipv4/tcp_bpf.c:420 [inline]
tcp_bpf_sendmsg+0x766/0x1ae0 net/ipv4/tcp_bpf.c:583
__sock_sendmsg+0x153/0x1c0 net/socket.c:802
__sys_sendto+0x326/0x430 net/socket.c:2265
__x64_sys_sendto+0xe3/0x100 net/socket.c:2268
do_syscall_64+0x14c/0x480
entry_SYSCALL_64_after_hwframe+0x77/0x7f
</TASK>
Widen the addition with a (u64) cast so the bound is evaluated in
64-bit and a len near U32_MAX no longer wraps below msg->sg.size.
While here, change pop from int to u32. It counts bytes against the
unsigned scatterlist lengths and can never be negative, so the signed
type only invites sign-confusion in the pop loop.
Fixes: 7246d8ed4d ("bpf: helper to pop data from messages")
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260615021959.140010-6-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
SK_MSG uses msg->sg.copy as per-scatterlist-entry provenance. Entries
with this bit set are copied before data/data_end are exposed to SK_MSG
BPF programs for direct packet access.
bpf_msg_pull_data(), bpf_msg_push_data(), and bpf_msg_pop_data()
rewrite the sk_msg scatterlist ring by collapsing, splitting, and
shifting entries. These operations move msg->sg.data[] entries, but the
parallel copy bitmap can be left behind on the old slot. A copied entry
can then return to msg->sg.start with its copy bit clear and be exposed
as directly writable packet data.
This corruption path requires an attached SK_MSG BPF program that calls
the mutating helpers; ordinary sockmap/TLS traffic that never runs
push/pop/pull helper sequences is not affected.
Keep msg->sg.copy synchronized with scatterlist entry moves, preserve
the copy bit when an entry is split, clear it when a helper replaces an
entry with a private page, and clear slots vacated by pull-data
compaction.
Fixes: 015632bb30 ("bpf: sk_msg program helper bpf_sk_msg_pull_data")
Fixes: 6fff607e2f ("bpf: sk_msg program helper bpf_msg_push_data")
Fixes: 7246d8ed4d ("bpf: helper to pop data from messages")
Cc: stable@vger.kernel.org
Co-developed-by: Han Guidong <2045gemini@gmail.com>
Reviewed-by: John Fastabend <john.fastabend@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Han Guidong <2045gemini@gmail.com>
Signed-off-by: Zhang Cen <rollkingzzc@gmail.com>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260615021959.140010-4-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When bpf_msg_push_data() splits a scatterlist element into head and
tail, the tail's page offset is advanced by `start` (absolute message
byte offset) instead of `start - offset` (byte position within the
element). This makes rsge.offset overshoot by `offset` bytes, pointing
to the wrong location within the page or beyond its boundary. Consumers
of the corrupted entry either silently read wrong data or trigger an
out-of-bounds access.
BUG: KASAN: slab-use-after-free in bpf_msg_pull_data (net/core/filter.c:2728)
Read of size 32752 at addr ffff8881042f0010 by task poc/130
Call Trace:
__asan_memcpy (mm/kasan/shadow.c:105)
bpf_msg_pull_data (net/core/filter.c:2728)
bpf_prog_run_pin_on_cpu (include/linux/bpf.h:1402)
sk_psock_msg_verdict (net/core/skmsg.c:934)
tcp_bpf_send_verdict (net/ipv4/tcp_bpf.c:421)
sock_sendmsg_nosec (net/socket.c:727)
Fixes: 6fff607e2f ("bpf: sk_msg program helper bpf_msg_push_data")
Reported-by: Xiang Mei <xmei5@asu.edu>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260615021959.140010-3-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When the scatterlist ring is full or nearly full, bpf_msg_push_data()
enters a copy fallback path and computes copy + len for the page
allocation size. Since len comes from BPF with arg3_type = ARG_ANYTHING
and both are u32, a crafted len can wrap the sum to a small value,
causing an undersized allocation followed by an out-of-bounds memcpy.
BUG: unable to handle page fault for address: ffffed104089a402
Oops: Oops: 0000 [#1] SMP KASAN NOPTI
Call Trace:
__asan_memcpy (mm/kasan/shadow.c:105)
bpf_msg_push_data (net/core/filter.c:2852 net/core/filter.c:2788)
bpf_prog_9ed8b5711920a7d7+0x2e/0x36
sk_psock_msg_verdict (net/core/skmsg.c:934)
tcp_bpf_sendmsg (net/ipv4/tcp_bpf.c:421 net/ipv4/tcp_bpf.c:584)
__sys_sendto (net/socket.c:2206)
do_syscall_64 (arch/x86/entry/syscall_64.c:94)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
Add an overflow check before the allocation.
Link: https://lore.kernel.org/all/20260424155913.A19FDC19425@smtp.kernel.org
Fixes: 6fff607e2f ("bpf: sk_msg program helper bpf_msg_push_data")
Tested-by: Xiang Mei <xmei5@asu.edu>
Tested-by: Xinyu Ma <mmmxny@gmail.com>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Link: https://lore.kernel.org/r/20260615021959.140010-2-jiayuan.chen@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
helper_fill_hashmap() is used also on parallel and stress map tests.
Those are consistently failing with ENOMEM on kernels built with
PREEMPT_RT if preallocation is disabled. The failure is transient and
only called by the memory cache refill running in a preemptible
irq_work, which can easily stall in case of contention.
Use a retriable update in those cases to handle transient ENOMEM and
make the test more stable also on PREEMPT_RT.
Also fix the sign of the value printed in case of error (strerror()
expects a positive errno while updates return it negative).
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/r/20260611150704.95133-1-gmonaco@redhat.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Verify the fix by:
1. Attach cgroup sockops prog.
2. Build a tcp connection using ipv4 addr in ipv6 socket.
3. Verify the return value of bpf_setsockopt() helper.
Assisted-by: Codex:gpt-5.5-xhigh
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/r/20260613162443.60515-3-leon.hwang@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
When TCP over IPv4 via INET6 API, bpf_get/setsockopt with ipv4 will
fail, because sk->sk_family is AF_INET6. With ipv6 will success, not
take effect, because inet_csk(sk)->icsk_af_ops is ipv6_mapped and
use ip_queue_xmit, inet_sk(sk)->tos.
To relax this restriction, allow getting/setting tos for those possible
ipv4-mapped ipv6 sockets.
Fixes: ee7f1e1302 ("bpf: Change bpf_setsockopt(SOL_IP) to reuse do_ip_setsockopt()")
Signed-off-by: Feng Zhou <zhoufeng.zf@bytedance.com>
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://lore.kernel.org/r/20260613162443.60515-2-leon.hwang@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The BPF selftests prefer static LLVM linking, which works for native
builds but can break cross builds. Its --link-static output may include
host-only libraries that are unavailable for the cross compilation,
causing link failures.
Avoid static LLVM linking for cross builds and use shared LLVM libraries
instead. Native builds keep the existing behavior.
Signed-off-by: Leo Yan <leo.yan@arm.com>
Link: https://lore.kernel.org/r/20260602-tools_build_fix_zero_init_bpf_only-v2-8-c76e5250ea1c@arm.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
The urandom_read helper and its shared library are built with $(CLANG)
directly rather than through the normal selftest $(CC) rules.
The CFLAGS variable can contain specific flags only for $(CC) but might
be imcompatible for $(CLANG) and those flags are not necessarily valid
for the clang-only urandom_read build.
Split the BPF selftest local flags into COMMON_CFLAGS and append them to
CFLAGS for the normal build path. Use COMMON_CFLAGS directly for
urandom_read and liburandom_read.so, while still filtering out -static as
before.
Signed-off-by: Leo Yan <leo.yan@arm.com>
Link: https://lore.kernel.org/r/20260602-tools_build_fix_zero_init_bpf_only-v2-7-c76e5250ea1c@arm.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
ASAN reports stack-buffer-overflow due to the uninitialized op_name.
Initialize it to fix the issue.
Fixes: 054b6c7866 ("selftests/bpf: Add verifier log tests for BPF_BTF_LOAD command")
Signed-off-by: Leo Yan <leo.yan@arm.com>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260602-tools_build_fix_zero_init_bpf_only-v2-6-c76e5250ea1c@arm.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
tools/scripts/Makefile.include may expand EXTRA_CFLAGS in a future
change. This could alter the initialization of CFLAGS, as the default
options "-g -O2" would never be set once EXTRA_CFLAGS is expanded.
Prepare for this by moving the CFLAGS initialization before including
tools/scripts/Makefile.include, so it is not affected by the extended
EXTRA_CFLAGS.
Append EXTRA_CFLAGS to CFLAGS only after including Makefile.include and
place it last so that the extra flags propagate properly and can
override the default options.
tools/scripts/Makefile.include already appends $(CLANG_CROSS_FLAGS) to
CFLAGS, the Makefile appends $(CLANG_CROSS_FLAGS) again, remove the
redundant append.
Signed-off-by: Leo Yan <leo.yan@arm.com>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/20260602-tools_build_fix_zero_init_bpf_only-v2-4-c76e5250ea1c@arm.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Prepare for future changes where EXTRA_CFLAGS may include flags not
applicable to the host compiler.
Move the HOST_CFLAGS assignment before appending EXTRA_CFLAGS to
CFLAGS so that HOST_CFLAGS does not inherit flags from EXTRA_CFLAGS.
Acked-by: Quentin Monnet <qmo@kernel.org>
Signed-off-by: Leo Yan <leo.yan@arm.com>
Link: https://lore.kernel.org/r/20260602-tools_build_fix_zero_init_bpf_only-v2-2-c76e5250ea1c@arm.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
bpftool builds a bootstrap libbpf with HOSTCC, but the libbpf submake can
still inherit target build flags through CFLAGS. This can break cross
builds when host objects are compiled with target-only options.
Since HOST_CFLAGS contains warning options that are not suitable for
building libbpf, use LIBBPF_BOOTSTRAP_CFLAGS with the warning options
removed to build the bootstrap libbpf. Clear EXTRA_CFLAGS so target
extra flags are not mixed into the host bootstrap libbpf build.
Signed-off-by: Leo Yan <leo.yan@arm.com>
Acked-by: Quentin Monnet <qmo@kernel.org>
Link: https://lore.kernel.org/r/20260602-tools_build_fix_zero_init_bpf_only-v2-1-c76e5250ea1c@arm.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
A comment in tools/testing/selftests/bpf/progs/test_fill_link_info.c
incorrectly refers to CONFIG_PPC6 instead of CONFIG_PPC64. Correct it.
Discovered while searching for CONFIG_* symbols referenced in code but
not defined in any Kconfig file.
Signed-off-by: Ethan Nelson-Moore <enelsonmoore@gmail.com>
Link: https://lore.kernel.org/r/20260610044023.225820-1-enelsonmoore@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
We verify info.uprobe_multi.flags against wrong kprobe-multi flag
(BPF_F_KPROBE_MULTI_RETURN). It's the same value as the correct
flag (BPF_F_UPROBE_MULTI_RETURN), so there's not functional change.
Fixes: 147c69307b ("selftests/bpf: Add link_info test for uprobe_multi link")
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260611114230.950379-8-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Add a uprobe_multi link API selftest that opens /proc/self/exe and passes
the resulting descriptor through opts.uprobe_multi.path_fd with
BPF_F_UPROBE_MULTI_PATH_FD set.
Assisted-by: Codex:GPT-5.4
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260611114230.950379-6-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Adding the path_fd field to struct bpf_link_create_opts and passing it
through kernel attr interface.
Assisted-by: Codex:GPT-5.4
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260611114230.950379-5-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Allow uprobe_multi link to identify the target binary by an already
opened file descriptor.
Adding new BPF_F_UPROBE_MULTI_PATH_FD flag and the path_fd field for
the attr.link_create.uprobe_multi struct.
When the flag is set, we resolve the target from path_fd, without the
flag, we keep the existing string path behavior.
I don't see a use case for supporting O_PATH file descriptors, because
we need to read the binary first to get probes offsets, so I'm using
the CLASS(fd, f), which fails for O_PATH fds.
Assisted-by: Codex:GPT-5.4
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260611114230.950379-4-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Resolve the uprobe_multi user path with user_path_at() instead of copying
the string with strndup_user() and passing it to kern_path(). This removes
the temporary allocation and keeps the lookup logic in one helper.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/r/20260611114230.950379-3-jolsa@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This comment was thoughtlessly copied from the x86 version and doesn't
apply to arm64. Remove it.
Reported-by: Will Deacon <will@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
Link: https://lore.kernel.org/r/20260614210209.2371030-1-tj@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
