github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-09-23 22:14:03 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

arm64: percpu: Fix this_cpu_and() mask generation

Browse Source 
The arm64 implementation of this_cpu_and(pcp, val) is built in terms of
ANDNOT operations, which requires the 'val' argument to be bitwise
negated. The bitwise negation is not implemented correctly, with two
bugs described below.

(1) The bitwise negation is performed as '~val' rather than '~(val)'.
    This won't always generate the expected value when 'val' is an
    expression.

    For example, for this_cpu_and(pcp, 1 - 1):

    * 'val'    is  '1 - 1'   ===> (int) 0x00000000
    * '~val'   is '~1 - 1'   ===> (int) 0xfffffffd
    * '~(val)' is '~(1 - 1)' ===> (int) 0xffffffff

    ... and thus bit[1] of 'pcp' would be preserved unexpectedly by the
    ANDNOT operation.

(2) The bitwise negation is performed on 'val' before it has been cast
    to (at least) the width of 'pcp'. This won't always generate the
    expected value for the upper bits.

    For example, for this_cpu_and(pcp, zero), where 'pcp' is a u64 and
    'zero' is a u32:

    * 'zero'           ===> (u32) 0x00000000
    * '~(zero)'        ===> (u32) 0xffffffff
    * '(u64)~(zero)'   ===> (u64) 0x00000000ffffffff
    * '~((u64)(zero))' ===> (u64) 0xffffffffffffffff

    ... and thus bits[63:32] of 'pcp' would be preserved unexpectedly by
    the ANDNOT operation.

Fix these issues by adding brackets around 'val', and by casting 'val'
to an appropriately-sized type before bitwise negation.

Fixes: 959bf2fd03 ("arm64: percpu: Rewrite per-cpu ops to allow use of LSE atomics")
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Acked-by: Christopher Lameter (Ampere) <cl@gentwo.org>
Cc: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: James Morse <james.morse@arm.com>
Cc: Marc Zyngier <maz@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Vladimir Murzin <vladimir.murzin@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Yang Shi <yang@os.amperecomputing.com>
Cc: stable@vger.kernel.org
Signed-off-by: Will Deacon <will@kernel.org>
This commit is contained in:
Mark Rutland 2026-09-08 16:17:22 +01:00 committed by Will Deacon
parent 885bff055a
commit 44274c6572
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
arch/arm64/include/asm/percpu.h
View File

@ -206,13 +206,13 @@ PERCPU_RET_OP(add, add, ldadd)
_pcp_protect_return(__percpu_add_return_case_64, pcp, val)
#define this_cpu_and_1(pcp, val) \
_pcp_protect(__percpu_andnot_case_8, pcp, ~val)
_pcp_protect(__percpu_andnot_case_8, pcp, ~(u8)(val))
#define this_cpu_and_2(pcp, val) \
_pcp_protect(__percpu_andnot_case_16, pcp, ~val)
_pcp_protect(__percpu_andnot_case_16, pcp, ~(u16)(val))
#define this_cpu_and_4(pcp, val) \
_pcp_protect(__percpu_andnot_case_32, pcp, ~val)
_pcp_protect(__percpu_andnot_case_32, pcp, ~(u32)(val))
#define this_cpu_and_8(pcp, val) \
_pcp_protect(__percpu_andnot_case_64, pcp, ~val)
_pcp_protect(__percpu_andnot_case_64, pcp, ~(u64)(val))
#define this_cpu_or_1(pcp, val) \
_pcp_protect(__percpu_or_case_8, pcp, val)