github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-24 06:24:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph
master
linux/arch/arm64/include/asm
History
Mark Rutland 8cf2093f53 arm64: percpu: Fix LSE operations on {8,16}-bit types 
The assembly for __percpu_##name##_case_##sz() and
__percpu_##name##_return_case_##sz() doesn't use the 'sfx' macro
argument to form the LSE instruction. Without 'sfx', a W register
argument will imply a 32-bit memory location, and consequently
{8,16}-bit ops will erroneously read and write 32 bits of memory when
the LSE instruction is used.

Fix this by appending 'sfx' to 'op_lse' to LSE instruction. It is not
necessary (and not valid) to append 'sfx' to 'op_llsc', as 'op_llsc' is
a register-register operation which does not access memory (and does not
take a size suffix).

Fixes: 959bf2fd03 ("arm64: percpu: Rewrite per-cpu ops to allow use of LSE atomics")
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Cc: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Vladimir Murzin <vladimir.murzin@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Yang Shi <yang@os.amperecomputing.com>
Cc: stable@vger.kernel.org
Reviewed-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
2026-09-11 12:32:22 +00:00
..
stacktrace arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
vdso arm64: vDSO: compat_gettimeofday: Add explicit includes 2026-03-11 15:12:22 +01:00
xen arm64: ptrace: Replace interrupts_enabled() with regs_irqs_disabled() 2025-09-11 15:55:34 +01:00
acenv.h
acpi.h ACPI: PPTT: Use acpi_get_cpu_uid() and remove get_acpi_id_for_cpu() 2026-04-06 16:55:16 +02:00
alternative-macros.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
alternative.h arm64 updates for 6.19: 2025-12-02 17:03:55 -08:00
apple_m1_pmu.h drivers/perf: apple_m1: Support host/guest event filtering 2025-03-11 12:52:32 -07:00
arch_gicv3.h irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs 2026-08-11 11:33:38 +00:00
arch_timer.h arm64: arch_timer: reuse arch_timer_read_cnt{p,v}ct_el0() helpers 2026-06-05 12:04:40 +01:00
archrandom.h arm64: head: Move early kernel mapping routines into C code 2024-02-16 12:42:33 +00:00
arm_dsu_pmu.h
arm_pmuv3.h perf/arm_pmuv3: Add PMUv3.9 per counter EL0 access control 2024-10-28 17:27:15 +00:00
arm-cci.h
asm_pointer_auth.h
asm-bug.h arm64/bug: Add ARCH_WARN_ASM macro for BUG/WARN asm code sharing with Rust 2025-07-23 00:04:53 +02:00
asm-extable.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
asm-offsets.h
asm-prototypes.h
asm-uaccess.h arm64/mm: Directly use TTBRx_EL1_ASID_MASK 2026-03-14 16:12:27 +00:00
assembler.h arm64: entry: mask DAIF before returning from C EL1 handlers 2026-08-06 13:07:46 +00:00
atomic_ll_sc.h
atomic_lse.h arm64: atomics: lse: Remove unused parameters from ATOMIC_FETCH_OP_AND macros 2025-11-27 18:15:24 +00:00
atomic.h
barrier.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
bitops.h
bitrev.h
boot.h
brk-imm.h arm64: kretprobes: acquire the regs via a BRK exception 2024-02-20 18:13:57 +00:00
bug.h bugs/core: Extend __WARN_FLAGS() with the 'cond_str' parameter 2025-06-13 10:20:52 +02:00
cache.h dma-mapping: Separate DMA sync issuing and completion waiting 2026-03-13 23:47:31 +01:00
cacheflush.h arm64: Provide dcache_inval_poc_nosync helper 2026-03-13 23:47:16 +01:00
cfi.h arm64/cfi,bpf: Support kCFI + BPF on arm64 2025-07-31 18:23:54 -07:00
checksum.h
clocksource.h
cmpxchg.h arm64: Silence sparse warnings caused by the type casting in (cmp)xchg 2026-03-04 14:31:27 +00:00
compat.h
compiler.h
cpu_ops.h
cpu.h arm64: Don't read GMID_EL1 when MTE is disabled 2026-09-03 13:03:23 +00:00
cpucaps.h arm64: Clarify ARM64_WORKAROUND_REPEAT_TLBI semantics 2026-07-21 21:59:52 +00:00
cpufeature.h arm64: Don't read GMID_EL1 when MTE is disabled 2026-09-03 13:03:23 +00:00
cpuidle.h
cputype.h arm64: cputype: Add C1-Nano definitions 2026-08-06 12:15:29 +00:00
crash_reserve.h kexec: split crashkernel reservation code out from crash_core.c 2024-02-23 17:48:21 -08:00
current.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
daifflags.h arm64/daifflags: Make local_daif_*() helpers __always_inline 2026-05-19 11:38:44 +01:00
dcc.h
debug-monitors.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
dmi.h
efi.h arm64/efi: Don't fail check current_in_efi() if preemptible 2026-01-06 11:54:31 +00:00
el2_setup.h arm64: cpufeature: Add support for the MPAM v0.1 architecture version 2026-05-19 13:04:28 +01:00
elf.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
entry-common.h irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs 2026-08-11 11:33:38 +00:00
esr.h arm64: Correct value returned by ESR_ELx_FSC_ADDRSZ_nL() 2026-07-21 15:13:10 +00:00
exception.h arm64: entry: Switch to generic IRQ entry 2025-09-11 15:55:35 +01:00
exec.h
extable.h arm64: mm: Handle PAN faults on uaccess CPY* instructions 2025-03-07 18:28:29 +00:00
fixmap.h arm64: remove early_ioremap_reset() call and __late_* macros 2026-08-06 18:57:22 -07:00
fpsimd.h arm64: fpsimd: Move SME save/restore inline 2026-06-03 16:50:49 +01:00
fpu.h arm64/fpsimd: Allocate kernel mode FP/SIMD buffers on the stack 2025-11-12 09:52:03 +01:00
ftrace.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
futex.h arm64: futex: Consolidate 'old == new' check in __lsui_cmpxchg32() 2026-07-26 14:24:52 +00:00
gcs.h arm64: uaccess: Add additional userspace GCS accessors 2025-09-16 21:32:53 +01:00
gpr-num.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
hardirq.h arm64: errata: Work around AmpereOne's erratum AC04_CPU_23 2025-05-19 12:46:26 +01:00
hugetlb.h arm64: mm: Refactor __flush_tlb_range() to take flags 2026-03-13 17:23:04 +00:00
hw_breakpoint.h arm64/hw_breakpoint: Directly use ESR_ELx_WNR for an watchpoint exception 2024-03-01 17:36:51 +00:00
hwcap.h arm64/hwcap: Generate the KERNEL_HWCAP_ definitions for the hwcaps 2026-04-09 16:50:23 +01:00
hyp_image.h
hypervisor.h arm64/coco: Add pKVM as a CC platform 2026-07-21 22:27:47 +00:00
image.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
insn-def.h
insn.h arm64: insn: Add encoder for ADD/SUB (extended register) 2026-08-14 05:55:01 +02:00
io.h arm64: errata: work around NVIDIA Olympus device store/load ordering 2026-07-22 20:34:56 +00:00
irq_work.h arch: consolidate arch_irq_work_raise prototypes 2023-11-23 11:32:29 +01:00
irq.h arm64: irq: include <linux/cpumask.h> 2024-01-12 12:44:18 +00:00
irqflags.h arm64/irqflags: __always_inline the arch_local_irq_*() helpers 2026-04-27 13:13:36 +01:00
jump_label.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
kasan.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
Kbuild arm64/hwcap: Include kernel-hwcap.h in list of generated files 2026-04-14 16:23:40 +01:00
kernel-pgtable.h arm64: Reserve an extra page for early kernel mapping 2026-05-01 16:20:35 +01:00
kexec.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
kfence.h arm64: mm: Don't sleep in split_kernel_leaf_mapping() when in atomic context 2025-11-07 14:43:15 +00:00
kgdb.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
kprobes.h arm64: kprobes: Allow reentering kprobes while single-stepping 2026-07-17 19:03:08 +01:00
kvm_arm.h Merge branch 'kvm-arm64/feat-nv3' into next 2026-08-19 12:27:53 -07:00
kvm_asm.h Merge branch 'kvm-arm64/misc-7.3' into next 2026-08-19 14:08:24 -07:00
kvm_define_hypevents.h KVM: arm64: Add event support to the nVHE/pKVM hyp and trace remote 2026-03-11 08:51:17 +00:00
kvm_emulate.h Merge branch 'kvm-arm64/feat-nv3' into next 2026-08-19 12:27:53 -07:00
kvm_host.h Merge branch 'kvm-arm64/vncr-fixes' into next 2026-08-19 12:28:16 -07:00
kvm_hyp.h Arm: 2026-06-19 08:56:49 -07:00
kvm_hypevents.h KVM: arm64: Add missing hyp_enter when trapping sysreg 2026-07-23 09:56:57 +01:00
kvm_hyptrace.h KVM: arm64: Add event support to the nVHE/pKVM hyp and trace remote 2026-03-11 08:51:17 +00:00
kvm_mmu.h KVM: arm64: Remove @arch from __load_stage2() 2026-05-28 18:52:55 +01:00
kvm_mte.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
kvm_nested.h KVM: arm64: Correctly cap TLBI Range to the architural limit 2026-08-12 23:48:53 -07:00
kvm_pgtable.h KVM: arm64: Introduce hypercall to force reclaim of a protected page 2026-03-30 16:58:09 +01:00
kvm_pkvm.h KVM: arm64: Don't advertise eager page splitting under pKVM 2026-07-31 00:11:57 -07:00
kvm_ptrauth.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
kvm_types.h
linkage.h Merge branch 'for-next/mm' into for-next/core 2026-06-14 12:17:33 +01:00
lse.h arm64: Unconditionally enable LSE support 2026-01-22 10:16:41 +00:00
lsui.h arm64: futex: Support futex with FEAT_LSUI 2026-03-27 12:52:04 +00:00
mem_encrypt.h arm64/coco: Add pKVM as a CC platform 2026-07-21 22:27:47 +00:00
memory.h arm64: Add MT_S2{,_FWB}_AS_S1 encodings 2026-01-25 16:17:21 +00:00
mman.h mm: update architecture and driver code to use vm_flags_t 2025-07-09 22:42:14 -07:00
mmu_context.h arm64/mm: Directly use TTBRx_EL1_ASID_MASK 2026-03-14 16:12:27 +00:00
mmu.h arm64: Rename page table BSS section to .bss..pgtbl 2026-06-05 11:44:09 +01:00
module.h arm64: ftrace: fix unreachable PLT for ftrace_caller in init_module with CONFIG_DYNAMIC_FTRACE 2025-09-05 16:56:20 +01:00
module.lds.h kbuild: Use --force-group-allocation when linking modules 2026-07-06 08:03:13 +02:00
mpam.h arm_mpam: resctrl: Add CDP emulation 2026-03-27 15:30:10 +00:00
mshyperv.h arm64/hyperv: Add some missing functions to arm64 2025-03-20 21:23:03 +00:00
mte-def.h
mte-kasan.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
mte.h arm64: mte: Skip TFSR_EL1 checks and barriers in synchronous tag check mode 2026-04-09 18:56:22 +01:00
neon-intrinsics.h
neon.h arm64/fpsimd: Allocate kernel mode FP/SIMD buffers on the stack 2025-11-12 09:52:03 +01:00
nmi.h arm64: escalate smp_send_stop() to an SDEI NMI as a last resort 2026-08-02 09:21:06 +00:00
numa.h
page-def.h vdso: Introduce vdso/page.h 2024-10-16 00:13:04 +02:00
page.h mm/page_alloc: fix initialization of tags of the huge zero folio with init_on_free 2026-05-13 17:40:02 -07:00
paravirt.h arm64/paravirt: Use common code for paravirt_steal_clock() 2026-01-12 16:30:00 +01:00
pci.h
percpu.h arm64: percpu: Fix LSE operations on {8,16}-bit types 2026-09-11 12:32:22 +00:00
perf_event.h perf/core: Correct perf sampling with guest VMs 2024-11-14 10:40:01 +01:00
pgalloc.h arm64: pgtable: use mmu gather to free p4d level page table 2025-01-25 20:22:21 -08:00
pgtable-hwdef.h arm64/mm: Directly use TTBRx_EL1_CnP 2026-03-14 16:12:27 +00:00
pgtable-prot.h mm: rename uffd-wp PTE bit macros to uffd 2026-08-04 19:18:36 -07:00
pgtable-types.h arm64/mm: Rename ptdesc_t 2026-06-05 11:44:36 +01:00
pgtable.h mm.git review status for mm-hotfixes-stable..mm-stable 2026-08-20 18:17:08 -07:00
pkeys.h mm/pkeys: remove unused tsk parameter from arch_set_user_pkey_access() 2026-04-05 13:52:57 -07:00
pointer_auth.h
por.h arm64/sysreg: Move POR_EL0_INIT to asm/por.h 2025-03-10 18:10:41 +00:00
preempt.h preempt: Introduce __preempt_count_{sub,add}_return() 2026-08-08 22:44:06 +02:00
probes.h arm64: probes: Remove probe_opcode_t 2024-10-15 18:16:20 +01:00
proc-fns.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
processor.h arm64: process: Fix context switching MTE store-only tag check 2026-08-24 14:11:25 +00:00
ptdump.h arm64 fixes for -rc1 2026-08-28 09:02:04 -07:00
ptrace.h arm64: ptrace: Remove INIT_PSTATE_EL2 2026-08-06 13:07:45 +00:00
pvclock-abi.h
resctrl.h arm64: mpam: Select ARCH_HAS_CPU_RESCTRL 2026-03-27 15:32:11 +00:00
ring_buffer.h ring-buffer: Flush and stop persistent ring buffer on panic 2026-05-21 08:20:58 -04:00
rqspinlock.h bpf: Use architecture provided res_smp_cond_load_acquire 2025-04-10 12:47:07 -07:00
rsi_cmds.h arm64: RSI: fix field-spanning write warning in attestation token init 2026-07-26 14:22:46 +00:00
rsi_smc.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
rsi.h arm64: realm: ioremap: Allow mapping memory as encrypted 2025-09-19 10:12:01 +01:00
runtime-const.h arm64/runtime-const: Introduce runtime_const_mask_32() 2026-07-28 13:07:14 +02:00
rwonce.h arm64, compiler-context-analysis: Permit alias analysis through __READ_ONCE() with CONFIG_LTO=y 2026-02-26 18:03:07 +00:00
scs.h arm64: scs: Remove redundant save/restore of SCS SP on entry to/from EL0 2026-03-13 17:17:58 +00:00
sdei.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
seccomp.h arm64: Remove duplicate included header 2025-01-10 13:44:22 +00:00
sections.h KVM: arm64: Add .hyp.data section 2025-05-06 09:56:18 +01:00
semihost.h
set_memory.h - The series "zram: optimal post-processing target selection" from 2024-11-23 09:58:07 -08:00
setup.h arm64: mm: Rework the 'rodata=' options 2025-09-16 20:53:13 +01:00
shmparam.h
signal.h
signal32.h
simd.h arm64/simd: Avoid pointless clearing of FP/SIMD buffer 2025-12-14 10:18:22 -08:00
smp_plat.h
smp.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
sparsemem.h arm64/mm: Reduce PA space to 48 bits when LPA2 is not enabled 2024-12-19 17:23:52 +00:00
spectre.h arm64 updates for 6.19: 2025-12-02 17:03:55 -08:00
spinlock_types.h Improve consistency of '#error' directive messages 2024-11-11 17:17:04 -08:00
spinlock.h
stack_pointer.h
stackprotector.h
stacktrace.h arm64: remove CONFIG_VMAP_STACK checks from stacktrace overflow logic 2025-07-08 13:41:09 +01:00
stage2_pgtable.h KVM: arm64: Move VTCR_EL2 into struct s2_mmu 2023-10-23 18:48:46 +00:00
stat.h
static_call.h arm64: Use static call trampolines when kCFI is enabled 2026-04-01 15:29:59 +01:00
string.h
suspend.h arm64: Fix cleared E0POE bit after cpu_suspend()/resume() 2026-01-09 10:41:45 +00:00
sync_bitops.h
syscall_wrapper.h arm64: syscall: Pass 'orig_x0' as first argument to native system call 2026-08-02 09:52:36 +00:00
syscall.h arm64: Avoid memcpy() for syscall_get_arguments() 2026-01-05 16:47:48 +00:00
sysreg.h KVM: arm64: Make EL2 exception entry and exit context-synchronization events 2026-05-07 14:12:41 +01:00
system_misc.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
text-patching.h asm-generic: introduce text-patching.h 2024-11-07 14:25:15 -08:00
thread_info.h arm64: mpam: Context switch the MPAM registers 2026-03-27 15:27:59 +00:00
timex.h
tlb.h arm64: tlb: Flush walk cache when unsharing PMD tables 2026-05-22 11:13:51 +01:00
tlbbatch.h arm64: Avoid eager DVMSync reclaim batches with C1-Pro SME erratum 2026-06-29 12:00:37 +01:00
tlbflush.h arm64: Clarify ARM64_WORKAROUND_REPEAT_TLBI semantics 2026-07-21 21:59:52 +00:00
topology.h arch_topology: move parse_acpi_topology() to common code 2025-10-22 08:10:57 +02:00
trans_pgd.h
traps.h KVM: arm64: Dump instruction on hyp panic 2025-09-15 13:04:22 +01:00
uaccess.h uaccess: unify inline vs outline copy_{from,to}_user() selection 2026-05-28 21:24:44 -07:00
unistd.h arm64: rework compat syscall macros 2024-07-10 14:23:38 +02:00
unistd32.h arm64: convert unistd_32.h to syscall.tbl format 2024-07-10 14:23:38 +02:00
uprobes.h arm64: debug: call step handlers statically 2025-07-08 13:27:41 +01:00
vdso.h arm64: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headers 2025-11-11 19:35:59 +00:00
vectors.h arm64: Avoid cpus_have_const_cap() for ARM64_UNMAP_KERNEL_AT_EL0 2023-10-16 14:17:06 +01:00
vermagic.h
virt.h KVM: arm64: Reclaim faulting page from pKVM in spurious fault handler 2026-03-30 16:58:09 +01:00
vmalloc.h arm64: Enable vmalloc-huge with ptdump 2025-09-22 11:53:24 +01:00
vmap_stack.h arm64: Remove assertion on CONFIG_VMAP_STACK 2025-11-07 19:46:39 +00:00
vncr_mapping.h KVM: arm64: Make HCR_EL2 a non-VNCR register 2026-07-31 00:17:04 -07:00
word-at-a-time.h arm64: word-at-a-time: improve byte count calculations for LE 2024-06-19 12:35:19 -07:00