github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-07-29 10:41:49 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

GICv5 initial host support

Browse Source 
Add host kernel support for the new arm64 GICv5 architecture, which is
 quite a departure from the previous ones.
 
 Include support for the full gamut of the architecture (interrupt
 routing and delivery to CPUs, wired interrupts, MSIs, and interrupt
 translation).
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEn9UcU+C1Yxj9lZw9I9DQutE9ekMFAmh45MYACgkQI9DQutE9
 ekPa3w//b5FfQAXwSco2+zqfR80a914CkBchHWJ50S1XHxymikI0VWin+4nsFXz1
 90/k52hz4a1rhjpMA0Z0rnEpzTpvyPckrfKDzUqf2Q8aAmfHMRw91kYvl2BII39O
 iWqEQKFRIxK5QR3mRt6C7mV8xth8zUbk/jPBdFbuB7iS/s8+Ayrxul9H4gHQsZqL
 f8fFZmFMKIIoshnWSr604510j0/jhj2lTXyesXGoNa/bBpPYsjOZeZByPaw+3RLS
 wGluBhMsbRk3gPzplVuPzMtQYLMinf2i08bhg4113zVvF1nvi1cs8ah28+HRH33X
 ZFIzClvWmCOu1zsYes49X8A6U2iJ4BL5Ndh9W6M3E7iH+pnzmYPsSuKL69welyvz
 7qRJnoAkIooaWrgES+TVCDGqC4gBTClBWUZKMRa21GMwyyPLaPQZBnAmHzqbbFO1
 k8WMcOVtvStc/Hd4Jc8GgbdWn5IRI6YAqIOEht1vYP9bKka8oj0nEt4I275bUlJP
 K2Qife4C6If8oAG+5Qu0dD6pAh7Pp6wylPm0EQ9AE5KCR4wWONOluvrSvU0WaAw6
 2uk5H/lTl0l9onO84YKP2dkYNawkKLVWeYnKFtpT1HKRUt1OkF01NsGKYivE5xp3
 qdsgyOYXR6r/MKa0ymfQ58y0txqTY7IQ/GSl44Sjh2WVU94Sp8A=
 =pB67
 -----END PGP SIGNATURE-----

Merge tag 'irqchip-gic-v5-host' into kvmarm/next

GICv5 initial host support

Add host kernel support for the new arm64 GICv5 architecture, which is
quite a departure from the previous ones.

Include support for the full gamut of the architecture (interrupt
routing and delivery to CPUs, wired interrupts, MSIs, and interrupt
translation).

* tag 'irqchip-gic-v5-host': (32 commits)
  arm64: smp: Fix pNMI setup after GICv5 rework
  arm64: Kconfig: Enable GICv5
  docs: arm64: gic-v5: Document booting requirements for GICv5
  irqchip/gic-v5: Add GICv5 IWB support
  irqchip/gic-v5: Add GICv5 ITS support
  irqchip/msi-lib: Add IRQ_DOMAIN_FLAG_FWNODE_PARENT handling
  irqchip/gic-v3: Rename GICv3 ITS MSI parent
  PCI/MSI: Add pci_msi_map_rid_ctlr_node() helper function
  of/irq: Add of_msi_xlate() helper function
  irqchip/gic-v5: Enable GICv5 SMP booting
  irqchip/gic-v5: Add GICv5 LPI/IPI support
  irqchip/gic-v5: Add GICv5 IRS/SPI support
  irqchip/gic-v5: Add GICv5 PPI support
  arm64: Add support for GICv5 GSB barriers
  arm64: smp: Support non-SGIs for IPIs
  arm64: cpucaps: Add GICv5 CPU interface (GCIE) capability
  arm64: cpucaps: Rename GICv3 CPU interface capability
  arm64: Disable GICv5 read/write/instruction traps
  arm64/sysreg: Add ICH_HFGITR_EL2
  arm64/sysreg: Add ICH_HFGWTR_EL2
  ...

Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
This commit is contained in:
Oliver Upton 2025-07-26 08:49:15 -07:00
commit ccd73c5782
32 changed files with 5200 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
Documentation/arch/arm64/booting.rst
View File

@ -223,6 +223,47 @@ Before jumping into the kernel, the following conditions must be met:
- SCR_EL3.HCE (bit 8) must be initialised to 0b1.
For systems with a GICv5 interrupt controller to be used in v5 mode:
- If the kernel is entered at EL1 and EL2 is present:
- ICH_HFGRTR_EL2.ICC_PPI_ACTIVERn_EL1 (bit 20) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_PPI_PRIORITYRn_EL1 (bit 19) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_PPI_PENDRn_EL1 (bit 18) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_PPI_ENABLERn_EL1 (bit 17) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_PPI_HMRn_EL1 (bit 16) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_IAFFIDR_EL1 (bit 7) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_ICSR_EL1 (bit 6) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_PCR_EL1 (bit 5) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_HPPIR_EL1 (bit 4) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_HAPR_EL1 (bit 3) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_CR0_EL1 (bit 2) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_IDRn_EL1 (bit 1) must be initialised to 0b1.
- ICH_HFGRTR_EL2.ICC_APR_EL1 (bit 0) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_PPI_ACTIVERn_EL1 (bit 20) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_PPI_PRIORITYRn_EL1 (bit 19) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_PPI_PENDRn_EL1 (bit 18) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_PPI_ENABLERn_EL1 (bit 17) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_ICSR_EL1 (bit 6) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_PCR_EL1 (bit 5) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_CR0_EL1 (bit 2) must be initialised to 0b1.
- ICH_HFGWTR_EL2.ICC_APR_EL1 (bit 0) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICRCDNMIA (bit 10) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICRCDIA (bit 9) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDDI (bit 8) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDEOI (bit 7) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDHM (bit 6) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDRCFG (bit 5) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDPEND (bit 4) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDAFF (bit 3) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDPRI (bit 2) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDDIS (bit 1) must be initialised to 0b1.
- ICH_HFGITR_EL2.GICCDEN (bit 0) must be initialised to 0b1.
- The DT or ACPI tables must describe a GICv5 interrupt controller.
For systems with a GICv3 interrupt controller to be used in v3 mode:
- If EL3 is present:

78
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v5-iwb.yaml Normal file
View File

@ -0,0 +1,78 @@
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/interrupt-controller/arm,gic-v5-iwb.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: ARM Generic Interrupt Controller, version 5 Interrupt Wire Bridge (IWB)
maintainers:
- Lorenzo Pieralisi <lpieralisi@kernel.org>
- Marc Zyngier <maz@kernel.org>
description: |
The GICv5 architecture defines the guidelines to implement GICv5
compliant interrupt controllers for AArch64 systems.
The GICv5 specification can be found at
https://developer.arm.com/documentation/aes0070
GICv5 has zero or more Interrupt Wire Bridges (IWB) that are responsible
for translating wire signals into interrupt messages to the GICv5 ITS.
allOf:
- $ref: /schemas/interrupt-controller.yaml#
properties:
compatible:
const: arm,gic-v5-iwb
reg:
items:
- description: IWB control frame
"#address-cells":
const: 0
"#interrupt-cells":
description: |
The 1st cell corresponds to the IWB wire.
The 2nd cell is the flags, encoded as follows:
bits[3:0] trigger type and level flags.
1 = low-to-high edge triggered
2 = high-to-low edge triggered
4 = active high level-sensitive
8 = active low level-sensitive
const: 2
interrupt-controller: true
msi-parent:
maxItems: 1
required:
- compatible
- reg
- "#interrupt-cells"
- interrupt-controller
- msi-parent
additionalProperties: false
examples:
- |
interrupt-controller@2f000000 {
compatible = "arm,gic-v5-iwb";
reg = <0x2f000000 0x10000>;
#address-cells = <0>;
#interrupt-cells = <2>;
interrupt-controller;
msi-parent = <&its0 64>;
};
...

267
Documentation/devicetree/bindings/interrupt-controller/arm,gic-v5.yaml Normal file
View File

@ -0,0 +1,267 @@
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/interrupt-controller/arm,gic-v5.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: ARM Generic Interrupt Controller, version 5
maintainers:
- Lorenzo Pieralisi <lpieralisi@kernel.org>
- Marc Zyngier <maz@kernel.org>
description: |
The GICv5 architecture defines the guidelines to implement GICv5
compliant interrupt controllers for AArch64 systems.
The GICv5 specification can be found at
https://developer.arm.com/documentation/aes0070
The GICv5 architecture is composed of multiple components:
- one or more IRS (Interrupt Routing Service)
- zero or more ITS (Interrupt Translation Service)
The architecture defines:
- PE-Private Peripheral Interrupts (PPI)
- Shared Peripheral Interrupts (SPI)
- Logical Peripheral Interrupts (LPI)
allOf:
- $ref: /schemas/interrupt-controller.yaml#
properties:
compatible:
const: arm,gic-v5
"#address-cells":
enum: [ 1, 2 ]
"#size-cells":
enum: [ 1, 2 ]
ranges: true
"#interrupt-cells":
description: |
The 1st cell corresponds to the INTID.Type field in the INTID; 1 for PPI,
3 for SPI. LPI interrupts must not be described in the bindings since
they are allocated dynamically by the software component managing them.
The 2nd cell contains the interrupt INTID.ID field.
The 3rd cell is the flags, encoded as follows:
bits[3:0] trigger type and level flags.
1 = low-to-high edge triggered
2 = high-to-low edge triggered
4 = active high level-sensitive
8 = active low level-sensitive
const: 3
interrupt-controller: true
interrupts:
description:
The VGIC maintenance interrupt.
maxItems: 1
required:
- compatible
- "#address-cells"
- "#size-cells"
- ranges
- "#interrupt-cells"
- interrupt-controller
patternProperties:
"^irs@[0-9a-f]+$":
type: object
description:
GICv5 has one or more Interrupt Routing Services (IRS) that are
responsible for handling IRQ state and routing.
additionalProperties: false
properties:
compatible:
const: arm,gic-v5-irs
reg:
minItems: 1
items:
- description: IRS config frames
- description: IRS setlpi frames
reg-names:
description:
Describe config and setlpi frames that are present.
"ns-" stands for non-secure, "s-" for secure, "realm-" for realm
and "el3-" for EL3.
minItems: 1
maxItems: 8
items:
enum: [ ns-config, s-config, realm-config, el3-config, ns-setlpi,
s-setlpi, realm-setlpi, el3-setlpi ]
"#address-cells":
enum: [ 1, 2 ]
"#size-cells":
enum: [ 1, 2 ]
ranges: true
dma-noncoherent:
description:
Present if the GIC IRS permits programming shareability and
cacheability attributes but is connected to a non-coherent
downstream interconnect.
cpus:
description:
CPUs managed by the IRS.
arm,iaffids:
$ref: /schemas/types.yaml#/definitions/uint16-array
description:
Interrupt AFFinity ID (IAFFID) associated with the CPU whose
CPU node phandle is at the same index in the cpus array.
patternProperties:
"^its@[0-9a-f]+$":
type: object
description:
GICv5 has zero or more Interrupt Translation Services (ITS) that are
used to route Message Signalled Interrupts (MSI) to the CPUs. Each
ITS is connected to an IRS.
additionalProperties: false
properties:
compatible:
const: arm,gic-v5-its
reg:
items:
- description: ITS config frames
reg-names:
description:
Describe config frames that are present.
"ns-" stands for non-secure, "s-" for secure, "realm-" for realm
and "el3-" for EL3.
minItems: 1
maxItems: 4
items:
enum: [ ns-config, s-config, realm-config, el3-config ]
"#address-cells":
enum: [ 1, 2 ]
"#size-cells":
enum: [ 1, 2 ]
ranges: true
dma-noncoherent:
description:
Present if the GIC ITS permits programming shareability and
cacheability attributes but is connected to a non-coherent
downstream interconnect.
patternProperties:
"^msi-controller@[0-9a-f]+$":
type: object
description:
GICv5 ITS has one or more translate register frames.
additionalProperties: false
properties:
reg:
items:
- description: ITS translate frames
reg-names:
description:
Describe translate frames that are present.
"ns-" stands for non-secure, "s-" for secure, "realm-" for realm
and "el3-" for EL3.
minItems: 1
maxItems: 4
items:
enum: [ ns-translate, s-translate, realm-translate, el3-translate ]
"#msi-cells":
description:
The single msi-cell is the DeviceID of the device which will
generate the MSI.
const: 1
msi-controller: true
required:
- reg
- reg-names
- "#msi-cells"
- msi-controller
required:
- compatible
- reg
- reg-names
required:
- compatible
- reg
- reg-names
- cpus
- arm,iaffids
additionalProperties: false
examples:
- |
interrupt-controller {
compatible = "arm,gic-v5";
#interrupt-cells = <3>;
interrupt-controller;
#address-cells = <1>;
#size-cells = <1>;
ranges;
interrupts = <1 25 4>;
irs@2f1a0000 {
compatible = "arm,gic-v5-irs";
reg = <0x2f1a0000 0x10000>; // IRS_CONFIG_FRAME
reg-names = "ns-config";
#address-cells = <1>;
#size-cells = <1>;
ranges;
cpus = <&cpu0>, <&cpu1>, <&cpu2>, <&cpu3>, <&cpu4>, <&cpu5>, <&cpu6>, <&cpu7>;
arm,iaffids = /bits/ 16 <0 1 2 3 4 5 6 7>;
its@2f120000 {
compatible = "arm,gic-v5-its";
reg = <0x2f120000 0x10000>; // ITS_CONFIG_FRAME
reg-names = "ns-config";
#address-cells = <1>;
#size-cells = <1>;
ranges;
msi-controller@2f130000 {
reg = <0x2f130000 0x10000>; // ITS_TRANSLATE_FRAME
reg-names = "ns-translate";
#msi-cells = <1>;
msi-controller;
};
};
};
};
...

10
MAINTAINERS
View File

@ -1964,6 +1964,16 @@ F: drivers/irqchip/irq-gic*.[ch]
F: include/linux/irqchip/arm-gic*.h
F: include/linux/irqchip/arm-vgic-info.h
ARM GENERIC INTERRUPT CONTROLLER V5 DRIVERS
M: Lorenzo Pieralisi <lpieralisi@kernel.org>
M: Marc Zyngier <maz@kernel.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
F: Documentation/devicetree/bindings/interrupt-controller/arm,gic-v5*.yaml
F: drivers/irqchip/irq-gic-its-msi-parent.[ch]
F: drivers/irqchip/irq-gic-v5*.[ch]
F: include/linux/irqchip/arm-gic-v5.h
ARM HDLCD DRM DRIVER
M: Liviu Dudau <liviu.dudau@arm.com>
S: Supported

1
arch/arm64/Kconfig
View File

@ -129,6 +129,7 @@ config ARM64
select ARM_GIC_V2M if PCI
select ARM_GIC_V3
select ARM_GIC_V3_ITS if PCI
select ARM_GIC_V5
select ARM_PSCI_FW
select BUILDTIME_TABLE_SORT
select CLONE_BACKWARDS

3
arch/arm64/include/asm/barrier.h
View File

@ -44,6 +44,9 @@
SB_BARRIER_INSN"nop\n", \
ARM64_HAS_SB))
#define gsb_ack() asm volatile(GSB_ACK_BARRIER_INSN : : : "memory")
#define gsb_sys() asm volatile(GSB_SYS_BARRIER_INSN : : : "memory")
#ifdef CONFIG_ARM64_PSEUDO_NMI
#define pmr_sync() \
do { \

45
arch/arm64/include/asm/el2_setup.h
View File

@ -165,6 +165,50 @@
.Lskip_gicv3_\@:
.endm
/* GICv5 system register access */
.macro __init_el2_gicv5
mrs_s x0, SYS_ID_AA64PFR2_EL1
ubfx x0, x0, #ID_AA64PFR2_EL1_GCIE_SHIFT, #4
cbz x0, .Lskip_gicv5_\@
mov x0, #(ICH_HFGITR_EL2_GICRCDNMIA | \
ICH_HFGITR_EL2_GICRCDIA | \
ICH_HFGITR_EL2_GICCDDI | \
ICH_HFGITR_EL2_GICCDEOI | \
ICH_HFGITR_EL2_GICCDHM | \
ICH_HFGITR_EL2_GICCDRCFG | \
ICH_HFGITR_EL2_GICCDPEND | \
ICH_HFGITR_EL2_GICCDAFF | \
ICH_HFGITR_EL2_GICCDPRI | \
ICH_HFGITR_EL2_GICCDDIS | \
ICH_HFGITR_EL2_GICCDEN)
msr_s SYS_ICH_HFGITR_EL2, x0 // Disable instruction traps
mov_q x0, (ICH_HFGRTR_EL2_ICC_PPI_ACTIVERn_EL1 | \
ICH_HFGRTR_EL2_ICC_PPI_PRIORITYRn_EL1 | \
ICH_HFGRTR_EL2_ICC_PPI_PENDRn_EL1 | \
ICH_HFGRTR_EL2_ICC_PPI_ENABLERn_EL1 | \
ICH_HFGRTR_EL2_ICC_PPI_HMRn_EL1 | \
ICH_HFGRTR_EL2_ICC_IAFFIDR_EL1 | \
ICH_HFGRTR_EL2_ICC_ICSR_EL1 | \
ICH_HFGRTR_EL2_ICC_PCR_EL1 | \
ICH_HFGRTR_EL2_ICC_HPPIR_EL1 | \
ICH_HFGRTR_EL2_ICC_HAPR_EL1 | \
ICH_HFGRTR_EL2_ICC_CR0_EL1 | \
ICH_HFGRTR_EL2_ICC_IDRn_EL1 | \
ICH_HFGRTR_EL2_ICC_APR_EL1)
msr_s SYS_ICH_HFGRTR_EL2, x0 // Disable reg read traps
mov_q x0, (ICH_HFGWTR_EL2_ICC_PPI_ACTIVERn_EL1 | \
ICH_HFGWTR_EL2_ICC_PPI_PRIORITYRn_EL1 | \
ICH_HFGWTR_EL2_ICC_PPI_PENDRn_EL1 | \
ICH_HFGWTR_EL2_ICC_PPI_ENABLERn_EL1 | \
ICH_HFGWTR_EL2_ICC_ICSR_EL1 | \
ICH_HFGWTR_EL2_ICC_PCR_EL1 | \
ICH_HFGWTR_EL2_ICC_CR0_EL1 | \
ICH_HFGWTR_EL2_ICC_APR_EL1)
msr_s SYS_ICH_HFGWTR_EL2, x0 // Disable reg write traps
.Lskip_gicv5_\@:
.endm
.macro __init_el2_hstr
msr hstr_el2, xzr // Disable CP15 traps to EL2
.endm
@ -314,6 +358,7 @@
__init_el2_lor
__init_el2_stage2
__init_el2_gicv3
__init_el2_gicv5
__init_el2_hstr
__init_el2_nvhe_idregs
__init_el2_cptr

24
arch/arm64/include/asm/smp.h
View File

@ -50,10 +50,32 @@ struct seq_file;
*/
extern void smp_init_cpus(void);
enum ipi_msg_type {
IPI_RESCHEDULE,
IPI_CALL_FUNC,
IPI_CPU_STOP,
IPI_CPU_STOP_NMI,
IPI_TIMER,
IPI_IRQ_WORK,
NR_IPI,
/*
* Any enum >= NR_IPI and < MAX_IPI is special and not tracable
* with trace_ipi_*
*/
IPI_CPU_BACKTRACE = NR_IPI,
IPI_KGDB_ROUNDUP,
MAX_IPI
};
/*
* Register IPI interrupts with the arch SMP code
*/
extern void set_smp_ipi_range(int ipi_base, int nr_ipi);
extern void set_smp_ipi_range_percpu(int ipi_base, int nr_ipi, int ncpus);
static inline void set_smp_ipi_range(int ipi_base, int n)
{
set_smp_ipi_range_percpu(ipi_base, n, 0);
}
/*
* Called from the secondary holding pen, this is the secondary CPU entry point.

71
arch/arm64/include/asm/sysreg.h
View File

@ -113,10 +113,14 @@
/* Register-based PAN access, for save/restore purposes */
#define SYS_PSTATE_PAN sys_reg(3, 0, 4, 2, 3)
#define __SYS_BARRIER_INSN(CRm, op2, Rt) \
__emit_inst(0xd5000000 | sys_insn(0, 3, 3, (CRm), (op2)) | ((Rt) & 0x1f))
#define __SYS_BARRIER_INSN(op0, op1, CRn, CRm, op2, Rt) \
__emit_inst(0xd5000000 | \
sys_insn((op0), (op1), (CRn), (CRm), (op2)) | \
((Rt) & 0x1f))
#define SB_BARRIER_INSN __SYS_BARRIER_INSN(0, 7, 31)
#define SB_BARRIER_INSN __SYS_BARRIER_INSN(0, 3, 3, 0, 7, 31)
#define GSB_SYS_BARRIER_INSN __SYS_BARRIER_INSN(1, 0, 12, 0, 0, 31)
#define GSB_ACK_BARRIER_INSN __SYS_BARRIER_INSN(1, 0, 12, 0, 1, 31)
/* Data cache zero operations */
#define SYS_DC_ISW sys_insn(1, 0, 7, 6, 2)
@ -1078,6 +1082,67 @@
#define GCS_CAP(x) ((((unsigned long)x) & GCS_CAP_ADDR_MASK) | \
GCS_CAP_VALID_TOKEN)
/*
* Definitions for GICv5 instructions
*/
#define GICV5_OP_GIC_CDAFF sys_insn(1, 0, 12, 1, 3)
#define GICV5_OP_GIC_CDDI sys_insn(1, 0, 12, 2, 0)
#define GICV5_OP_GIC_CDDIS sys_insn(1, 0, 12, 1, 0)
#define GICV5_OP_GIC_CDHM sys_insn(1, 0, 12, 2, 1)
#define GICV5_OP_GIC_CDEN sys_insn(1, 0, 12, 1, 1)
#define GICV5_OP_GIC_CDEOI sys_insn(1, 0, 12, 1, 7)
#define GICV5_OP_GIC_CDPEND sys_insn(1, 0, 12, 1, 4)
#define GICV5_OP_GIC_CDPRI sys_insn(1, 0, 12, 1, 2)
#define GICV5_OP_GIC_CDRCFG sys_insn(1, 0, 12, 1, 5)
#define GICV5_OP_GICR_CDIA sys_insn(1, 0, 12, 3, 0)
/* Definitions for GIC CDAFF */
#define GICV5_GIC_CDAFF_IAFFID_MASK GENMASK_ULL(47, 32)
#define GICV5_GIC_CDAFF_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDAFF_IRM_MASK BIT_ULL(28)
#define GICV5_GIC_CDAFF_ID_MASK GENMASK_ULL(23, 0)
/* Definitions for GIC CDDI */
#define GICV5_GIC_CDDI_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDDI_ID_MASK GENMASK_ULL(23, 0)
/* Definitions for GIC CDDIS */
#define GICV5_GIC_CDDIS_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDDIS_TYPE(r) FIELD_GET(GICV5_GIC_CDDIS_TYPE_MASK, r)
#define GICV5_GIC_CDDIS_ID_MASK GENMASK_ULL(23, 0)
#define GICV5_GIC_CDDIS_ID(r) FIELD_GET(GICV5_GIC_CDDIS_ID_MASK, r)
/* Definitions for GIC CDEN */
#define GICV5_GIC_CDEN_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDEN_ID_MASK GENMASK_ULL(23, 0)
/* Definitions for GIC CDHM */
#define GICV5_GIC_CDHM_HM_MASK BIT_ULL(32)
#define GICV5_GIC_CDHM_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDHM_ID_MASK GENMASK_ULL(23, 0)
/* Definitions for GIC CDPEND */
#define GICV5_GIC_CDPEND_PENDING_MASK BIT_ULL(32)
#define GICV5_GIC_CDPEND_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDPEND_ID_MASK GENMASK_ULL(23, 0)
/* Definitions for GIC CDPRI */
#define GICV5_GIC_CDPRI_PRIORITY_MASK GENMASK_ULL(39, 35)
#define GICV5_GIC_CDPRI_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDPRI_ID_MASK GENMASK_ULL(23, 0)
/* Definitions for GIC CDRCFG */
#define GICV5_GIC_CDRCFG_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDRCFG_ID_MASK GENMASK_ULL(23, 0)
/* Definitions for GICR CDIA */
#define GICV5_GIC_CDIA_VALID_MASK BIT_ULL(32)
#define GICV5_GICR_CDIA_VALID(r) FIELD_GET(GICV5_GIC_CDIA_VALID_MASK, r)
#define GICV5_GIC_CDIA_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GIC_CDIA_ID_MASK GENMASK_ULL(23, 0)
#define gicr_insn(insn) read_sysreg_s(GICV5_OP_GICR_##insn)
#define gic_insn(v, insn) write_sysreg_s(v, GICV5_OP_GIC_##insn)
#define ARM64_FEATURE_FIELD_BITS 4

17
arch/arm64/kernel/cpufeature.c
View File

@ -2298,11 +2298,11 @@ static bool can_use_gic_priorities(const struct arm64_cpu_capabilities *entry,
int scope)
{
/*
* ARM64_HAS_GIC_CPUIF_SYSREGS has a lower index, and is a boot CPU
* ARM64_HAS_GICV3_CPUIF has a lower index, and is a boot CPU
* feature, so will be detected earlier.
*/
BUILD_BUG_ON(ARM64_HAS_GIC_PRIO_MASKING <= ARM64_HAS_GIC_CPUIF_SYSREGS);
if (!cpus_have_cap(ARM64_HAS_GIC_CPUIF_SYSREGS))
BUILD_BUG_ON(ARM64_HAS_GIC_PRIO_MASKING <= ARM64_HAS_GICV3_CPUIF);
if (!cpus_have_cap(ARM64_HAS_GICV3_CPUIF))
return false;
return enable_pseudo_nmi;
@ -2498,8 +2498,8 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = has_always,
},
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_GIC_CPUIF_SYSREGS,
.desc = "GICv3 CPU interface",
.capability = ARM64_HAS_GICV3_CPUIF,
.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
.matches = has_useable_gicv3_cpuif,
ARM64_CPUID_FIELDS(ID_AA64PFR0_EL1, GIC, IMP)
@ -3070,6 +3070,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = has_cpuid_feature,
ARM64_CPUID_FIELDS(ID_AA64MMFR3_EL1, SCTLRX, IMP)
},
{
.desc = "GICv5 CPU interface",
.type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
.capability = ARM64_HAS_GICV5_CPUIF,
.matches = has_cpuid_feature,
ARM64_CPUID_FIELDS(ID_AA64PFR2_EL1, GCIE, IMP)
},
{},
};

144
arch/arm64/kernel/smp.c
View File

@ -64,26 +64,18 @@ struct secondary_data secondary_data;
/* Number of CPUs which aren't online, but looping in kernel text. */
static int cpus_stuck_in_kernel;
enum ipi_msg_type {
IPI_RESCHEDULE,
IPI_CALL_FUNC,
IPI_CPU_STOP,
IPI_CPU_STOP_NMI,
IPI_TIMER,
IPI_IRQ_WORK,
NR_IPI,
/*
* Any enum >= NR_IPI and < MAX_IPI is special and not tracable
* with trace_ipi_*
*/
IPI_CPU_BACKTRACE = NR_IPI,
IPI_KGDB_ROUNDUP,
MAX_IPI
};
static int ipi_irq_base __ro_after_init;
static int nr_ipi __ro_after_init = NR_IPI;
static struct irq_desc *ipi_desc[MAX_IPI] __ro_after_init;
struct ipi_descs {
struct irq_desc *descs[MAX_IPI];
};
static DEFINE_PER_CPU_READ_MOSTLY(struct ipi_descs, pcpu_ipi_desc);
#define get_ipi_desc(__cpu, __ipi) (per_cpu_ptr(&pcpu_ipi_desc, __cpu)->descs[__ipi])
static bool percpu_ipi_descs __ro_after_init;
static bool crash_stop;
@ -844,7 +836,7 @@ int arch_show_interrupts(struct seq_file *p, int prec)
seq_printf(p, "%*s%u:%s", prec - 1, "IPI", i,
prec >= 4 ? " " : "");
for_each_online_cpu(cpu)
seq_printf(p, "%10u ", irq_desc_kstat_cpu(ipi_desc[i], cpu));
seq_printf(p, "%10u ", irq_desc_kstat_cpu(get_ipi_desc(cpu, i), cpu));
seq_printf(p, " %s\n", ipi_types[i]);
}
@ -917,9 +909,20 @@ static void __noreturn ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs
#endif
}
static void arm64_send_ipi(const cpumask_t *mask, unsigned int nr)
{
unsigned int cpu;
if (!percpu_ipi_descs)
__ipi_send_mask(get_ipi_desc(0, nr), mask);
else
for_each_cpu(cpu, mask)
__ipi_send_single(get_ipi_desc(cpu, nr), cpu);
}
static void arm64_backtrace_ipi(cpumask_t *mask)
{
__ipi_send_mask(ipi_desc[IPI_CPU_BACKTRACE], mask);
arm64_send_ipi(mask, IPI_CPU_BACKTRACE);
}
void arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu)
@ -944,7 +947,7 @@ void kgdb_roundup_cpus(void)
if (cpu == this_cpu)
continue;
__ipi_send_single(ipi_desc[IPI_KGDB_ROUNDUP], cpu);
__ipi_send_single(get_ipi_desc(cpu, IPI_KGDB_ROUNDUP), cpu);
}
}
#endif
@ -1013,14 +1016,16 @@ static void do_handle_IPI(int ipinr)
static irqreturn_t ipi_handler(int irq, void *data)
{
do_handle_IPI(irq - ipi_irq_base);
unsigned int ipi = (irq - ipi_irq_base) % nr_ipi;
do_handle_IPI(ipi);
return IRQ_HANDLED;
}
static void smp_cross_call(const struct cpumask *target, unsigned int ipinr)
{
trace_ipi_raise(target, ipi_types[ipinr]);
__ipi_send_mask(ipi_desc[ipinr], target);
arm64_send_ipi(target, ipinr);
}
static bool ipi_should_be_nmi(enum ipi_msg_type ipi)
@ -1046,11 +1051,15 @@ static void ipi_setup(int cpu)
return;
for (i = 0; i < nr_ipi; i++) {
if (ipi_should_be_nmi(i)) {
prepare_percpu_nmi(ipi_irq_base + i);
enable_percpu_nmi(ipi_irq_base + i, 0);
if (!percpu_ipi_descs) {
if (ipi_should_be_nmi(i)) {
prepare_percpu_nmi(ipi_irq_base + i);
enable_percpu_nmi(ipi_irq_base + i, 0);
} else {
enable_percpu_irq(ipi_irq_base + i, 0);
}
} else {
enable_percpu_irq(ipi_irq_base + i, 0);
enable_irq(irq_desc_get_irq(get_ipi_desc(cpu, i)));
}
}
}
@ -1064,44 +1073,77 @@ static void ipi_teardown(int cpu)
return;
for (i = 0; i < nr_ipi; i++) {
if (ipi_should_be_nmi(i)) {
disable_percpu_nmi(ipi_irq_base + i);
teardown_percpu_nmi(ipi_irq_base + i);
if (!percpu_ipi_descs) {
if (ipi_should_be_nmi(i)) {
disable_percpu_nmi(ipi_irq_base + i);
teardown_percpu_nmi(ipi_irq_base + i);
} else {
disable_percpu_irq(ipi_irq_base + i);
}
} else {
disable_percpu_irq(ipi_irq_base + i);
disable_irq(irq_desc_get_irq(get_ipi_desc(cpu, i)));
}
}
}
#endif
void __init set_smp_ipi_range(int ipi_base, int n)
static void ipi_setup_sgi(int ipi)
{
int err, irq, cpu;
irq = ipi_irq_base + ipi;
if (ipi_should_be_nmi(ipi)) {
err = request_percpu_nmi(irq, ipi_handler, "IPI", &irq_stat);
WARN(err, "Could not request IRQ %d as NMI, err=%d\n", irq, err);
} else {
err = request_percpu_irq(irq, ipi_handler, "IPI", &irq_stat);
WARN(err, "Could not request IRQ %d as IRQ, err=%d\n", irq, err);
}
for_each_possible_cpu(cpu)
get_ipi_desc(cpu, ipi) = irq_to_desc(irq);
irq_set_status_flags(irq, IRQ_HIDDEN);
}
static void ipi_setup_lpi(int ipi, int ncpus)
{
for (int cpu = 0; cpu < ncpus; cpu++) {
int err, irq;
irq = ipi_irq_base + (cpu * nr_ipi) + ipi;
err = irq_force_affinity(irq, cpumask_of(cpu));
WARN(err, "Could not force affinity IRQ %d, err=%d\n", irq, err);
err = request_irq(irq, ipi_handler, IRQF_NO_AUTOEN, "IPI",
NULL);
WARN(err, "Could not request IRQ %d, err=%d\n", irq, err);
irq_set_status_flags(irq, (IRQ_HIDDEN | IRQ_NO_BALANCING_MASK));
get_ipi_desc(cpu, ipi) = irq_to_desc(irq);
}
}
void __init set_smp_ipi_range_percpu(int ipi_base, int n, int ncpus)
{
int i;
WARN_ON(n < MAX_IPI);
nr_ipi = min(n, MAX_IPI);
for (i = 0; i < nr_ipi; i++) {
int err;
if (ipi_should_be_nmi(i)) {
err = request_percpu_nmi(ipi_base + i, ipi_handler,
"IPI", &irq_stat);
WARN(err, "Could not request IPI %d as NMI, err=%d\n",
i, err);
} else {
err = request_percpu_irq(ipi_base + i, ipi_handler,
"IPI", &irq_stat);
WARN(err, "Could not request IPI %d as IRQ, err=%d\n",
i, err);
}
ipi_desc[i] = irq_to_desc(ipi_base + i);
irq_set_status_flags(ipi_base + i, IRQ_HIDDEN);
}
percpu_ipi_descs = !!ncpus;
ipi_irq_base = ipi_base;
for (i = 0; i < nr_ipi; i++) {
if (!percpu_ipi_descs)
ipi_setup_sgi(i);
else
ipi_setup_lpi(i, ncpus);
}
/* Setup the boot CPU immediately */
ipi_setup(smp_processor_id());
}

3
arch/arm64/tools/cpucaps
View File

@ -35,7 +35,8 @@ HAS_GENERIC_AUTH
HAS_GENERIC_AUTH_ARCH_QARMA3
HAS_GENERIC_AUTH_ARCH_QARMA5
HAS_GENERIC_AUTH_IMP_DEF
HAS_GIC_CPUIF_SYSREGS
HAS_GICV3_CPUIF
HAS_GICV5_CPUIF
HAS_GIC_PRIO_MASKING
HAS_GIC_PRIO_RELAXED_SYNC
HAS_HCR_NV1

495
arch/arm64/tools/sysreg
View File

@ -1314,7 +1314,10 @@ UnsignedEnum 19:16 UINJ
0b0000 NI
0b0001 IMP
EndEnum
Res0 15:12
UnsignedEnum 15:12 GCIE
0b0000 NI
0b0001 IMP
EndEnum
UnsignedEnum 11:8 MTEFAR
0b0000 NI
0b0001 IMP
@ -3021,6 +3024,435 @@ Sysreg PMIAR_EL1 3 0 9 14 7
Field 63:0 ADDRESS
EndSysreg
SysregFields ICC_PPI_HMRx_EL1
Field 63 HM63
Field 62 HM62
Field 61 HM61
Field 60 HM60
Field 59 HM59
Field 58 HM58
Field 57 HM57
Field 56 HM56
Field 55 HM55
Field 54 HM54
Field 53 HM53
Field 52 HM52
Field 51 HM51
Field 50 HM50
Field 49 HM49
Field 48 HM48
Field 47 HM47
Field 46 HM46
Field 45 HM45
Field 44 HM44
Field 43 HM43
Field 42 HM42
Field 41 HM41
Field 40 HM40
Field 39 HM39
Field 38 HM38
Field 37 HM37
Field 36 HM36
Field 35 HM35
Field 34 HM34
Field 33 HM33
Field 32 HM32
Field 31 HM31
Field 30 HM30
Field 29 HM29
Field 28 HM28
Field 27 HM27
Field 26 HM26
Field 25 HM25
Field 24 HM24
Field 23 HM23
Field 22 HM22
Field 21 HM21
Field 20 HM20
Field 19 HM19
Field 18 HM18
Field 17 HM17
Field 16 HM16
Field 15 HM15
Field 14 HM14
Field 13 HM13
Field 12 HM12
Field 11 HM11
Field 10 HM10
Field 9 HM9
Field 8 HM8
Field 7 HM7
Field 6 HM6
Field 5 HM5
Field 4 HM4
Field 3 HM3
Field 2 HM2
Field 1 HM1
Field 0 HM0
EndSysregFields
Sysreg ICC_PPI_HMR0_EL1 3 0 12 10 0
Fields ICC_PPI_HMRx_EL1
EndSysreg
Sysreg ICC_PPI_HMR1_EL1 3 0 12 10 1
Fields ICC_PPI_HMRx_EL1
EndSysreg
Sysreg ICC_IDR0_EL1 3 0 12 10 2
Res0 63:12
UnsignedEnum 11:8 GCIE_LEGACY
0b0000 NI
0b0001 IMP
EndEnum
UnsignedEnum 7:4 PRI_BITS
0b0011 4BITS
0b0100 5BITS
EndEnum
UnsignedEnum 3:0 ID_BITS
0b0000 16BITS
0b0001 24BITS
EndEnum
EndSysreg
Sysreg ICC_ICSR_EL1 3 0 12 10 4
Res0 63:48
Field 47:32 IAFFID
Res0 31:16
Field 15:11 Priority
Res0 10:6
Field 5 HM
Field 4 Active
Field 3 IRM
Field 2 Pending
Field 1 Enabled
Field 0 F
EndSysreg
SysregFields ICC_PPI_ENABLERx_EL1
Field 63 EN63
Field 62 EN62
Field 61 EN61
Field 60 EN60
Field 59 EN59
Field 58 EN58
Field 57 EN57
Field 56 EN56
Field 55 EN55
Field 54 EN54
Field 53 EN53
Field 52 EN52
Field 51 EN51
Field 50 EN50
Field 49 EN49
Field 48 EN48
Field 47 EN47
Field 46 EN46
Field 45 EN45
Field 44 EN44
Field 43 EN43
Field 42 EN42
Field 41 EN41
Field 40 EN40
Field 39 EN39
Field 38 EN38
Field 37 EN37
Field 36 EN36
Field 35 EN35
Field 34 EN34
Field 33 EN33
Field 32 EN32
Field 31 EN31
Field 30 EN30
Field 29 EN29
Field 28 EN28
Field 27 EN27
Field 26 EN26
Field 25 EN25
Field 24 EN24
Field 23 EN23
Field 22 EN22
Field 21 EN21
Field 20 EN20
Field 19 EN19
Field 18 EN18
Field 17 EN17
Field 16 EN16
Field 15 EN15
Field 14 EN14
Field 13 EN13
Field 12 EN12
Field 11 EN11
Field 10 EN10
Field 9 EN9
Field 8 EN8
Field 7 EN7
Field 6 EN6
Field 5 EN5
Field 4 EN4
Field 3 EN3
Field 2 EN2
Field 1 EN1
Field 0 EN0
EndSysregFields
Sysreg ICC_PPI_ENABLER0_EL1 3 0 12 10 6
Fields ICC_PPI_ENABLERx_EL1
EndSysreg
Sysreg ICC_PPI_ENABLER1_EL1 3 0 12 10 7
Fields ICC_PPI_ENABLERx_EL1
EndSysreg
SysregFields ICC_PPI_ACTIVERx_EL1
Field 63 Active63
Field 62 Active62
Field 61 Active61
Field 60 Active60
Field 59 Active59
Field 58 Active58
Field 57 Active57
Field 56 Active56
Field 55 Active55
Field 54 Active54
Field 53 Active53
Field 52 Active52
Field 51 Active51
Field 50 Active50
Field 49 Active49
Field 48 Active48
Field 47 Active47
Field 46 Active46
Field 45 Active45
Field 44 Active44
Field 43 Active43
Field 42 Active42
Field 41 Active41
Field 40 Active40
Field 39 Active39
Field 38 Active38
Field 37 Active37
Field 36 Active36
Field 35 Active35
Field 34 Active34
Field 33 Active33
Field 32 Active32
Field 31 Active31
Field 30 Active30
Field 29 Active29
Field 28 Active28
Field 27 Active27
Field 26 Active26
Field 25 Active25
Field 24 Active24
Field 23 Active23
Field 22 Active22
Field 21 Active21
Field 20 Active20
Field 19 Active19
Field 18 Active18
Field 17 Active17
Field 16 Active16
Field 15 Active15
Field 14 Active14
Field 13 Active13
Field 12 Active12
Field 11 Active11
Field 10 Active10
Field 9 Active9
Field 8 Active8
Field 7 Active7
Field 6 Active6
Field 5 Active5
Field 4 Active4
Field 3 Active3
Field 2 Active2
Field 1 Active1
Field 0 Active0
EndSysregFields
Sysreg ICC_PPI_CACTIVER0_EL1 3 0 12 13 0
Fields ICC_PPI_ACTIVERx_EL1
EndSysreg
Sysreg ICC_PPI_CACTIVER1_EL1 3 0 12 13 1
Fields ICC_PPI_ACTIVERx_EL1
EndSysreg
Sysreg ICC_PPI_SACTIVER0_EL1 3 0 12 13 2
Fields ICC_PPI_ACTIVERx_EL1
EndSysreg
Sysreg ICC_PPI_SACTIVER1_EL1 3 0 12 13 3
Fields ICC_PPI_ACTIVERx_EL1
EndSysreg
SysregFields ICC_PPI_PENDRx_EL1
Field 63 Pend63
Field 62 Pend62
Field 61 Pend61
Field 60 Pend60
Field 59 Pend59
Field 58 Pend58
Field 57 Pend57
Field 56 Pend56
Field 55 Pend55
Field 54 Pend54
Field 53 Pend53
Field 52 Pend52
Field 51 Pend51
Field 50 Pend50
Field 49 Pend49
Field 48 Pend48
Field 47 Pend47
Field 46 Pend46
Field 45 Pend45
Field 44 Pend44
Field 43 Pend43
Field 42 Pend42
Field 41 Pend41
Field 40 Pend40
Field 39 Pend39
Field 38 Pend38
Field 37 Pend37
Field 36 Pend36
Field 35 Pend35
Field 34 Pend34
Field 33 Pend33
Field 32 Pend32
Field 31 Pend31
Field 30 Pend30
Field 29 Pend29
Field 28 Pend28
Field 27 Pend27
Field 26 Pend26
Field 25 Pend25
Field 24 Pend24
Field 23 Pend23
Field 22 Pend22
Field 21 Pend21
Field 20 Pend20
Field 19 Pend19
Field 18 Pend18
Field 17 Pend17
Field 16 Pend16
Field 15 Pend15
Field 14 Pend14
Field 13 Pend13
Field 12 Pend12
Field 11 Pend11
Field 10 Pend10
Field 9 Pend9
Field 8 Pend8
Field 7 Pend7
Field 6 Pend6
Field 5 Pend5
Field 4 Pend4
Field 3 Pend3
Field 2 Pend2
Field 1 Pend1
Field 0 Pend0
EndSysregFields
Sysreg ICC_PPI_CPENDR0_EL1 3 0 12 13 4
Fields ICC_PPI_PENDRx_EL1
EndSysreg
Sysreg ICC_PPI_CPENDR1_EL1 3 0 12 13 5
Fields ICC_PPI_PENDRx_EL1
EndSysreg
Sysreg ICC_PPI_SPENDR0_EL1 3 0 12 13 6
Fields ICC_PPI_PENDRx_EL1
EndSysreg
Sysreg ICC_PPI_SPENDR1_EL1 3 0 12 13 7
Fields ICC_PPI_PENDRx_EL1
EndSysreg
SysregFields ICC_PPI_PRIORITYRx_EL1
Res0 63:61
Field 60:56 Priority7
Res0 55:53
Field 52:48 Priority6
Res0 47:45
Field 44:40 Priority5
Res0 39:37
Field 36:32 Priority4
Res0 31:29
Field 28:24 Priority3
Res0 23:21
Field 20:16 Priority2
Res0 15:13
Field 12:8 Priority1
Res0 7:5
Field 4:0 Priority0
EndSysregFields
Sysreg ICC_PPI_PRIORITYR0_EL1 3 0 12 14 0
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR1_EL1 3 0 12 14 1
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR2_EL1 3 0 12 14 2
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR3_EL1 3 0 12 14 3
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR4_EL1 3 0 12 14 4
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR5_EL1 3 0 12 14 5
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR6_EL1 3 0 12 14 6
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR7_EL1 3 0 12 14 7
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR8_EL1 3 0 12 15 0
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR9_EL1 3 0 12 15 1
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR10_EL1 3 0 12 15 2
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR11_EL1 3 0 12 15 3
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR12_EL1 3 0 12 15 4
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR13_EL1 3 0 12 15 5
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR14_EL1 3 0 12 15 6
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg ICC_PPI_PRIORITYR15_EL1 3 0 12 15 7
Fields ICC_PPI_PRIORITYRx_EL1
EndSysreg
Sysreg PMSELR_EL0 3 3 9 12 5
Res0 63:5
Field 4:0 SEL
@ -3103,6 +3535,19 @@ Res0 14:12
Field 11:0 AFFINITY
EndSysreg
Sysreg ICC_CR0_EL1 3 1 12 0 1
Res0 63:39
Field 38 PID
Field 37:32 IPPT
Res0 31:1
Field 0 EN
EndSysreg
Sysreg ICC_PCR_EL1 3 1 12 0 2
Res0 63:5
Field 4:0 PRIORITY
EndSysreg
Sysreg CSSELR_EL1 3 2 0 0 0
Res0 63:5
Field 4 TnD
@ -3989,6 +4434,54 @@ Field 31:16 PhyPARTID29
Field 15:0 PhyPARTID28
EndSysreg
Sysreg ICH_HFGRTR_EL2 3 4 12 9 4
Res0 63:21
Field 20 ICC_PPI_ACTIVERn_EL1
Field 19 ICC_PPI_PRIORITYRn_EL1
Field 18 ICC_PPI_PENDRn_EL1
Field 17 ICC_PPI_ENABLERn_EL1
Field 16 ICC_PPI_HMRn_EL1
Res0 15:8
Field 7 ICC_IAFFIDR_EL1
Field 6 ICC_ICSR_EL1
Field 5 ICC_PCR_EL1
Field 4 ICC_HPPIR_EL1
Field 3 ICC_HAPR_EL1
Field 2 ICC_CR0_EL1
Field 1 ICC_IDRn_EL1
Field 0 ICC_APR_EL1
EndSysreg
Sysreg ICH_HFGWTR_EL2 3 4 12 9 6
Res0 63:21
Field 20 ICC_PPI_ACTIVERn_EL1
Field 19 ICC_PPI_PRIORITYRn_EL1
Field 18 ICC_PPI_PENDRn_EL1
Field 17 ICC_PPI_ENABLERn_EL1
Res0 16:7
Field 6 ICC_ICSR_EL1
Field 5 ICC_PCR_EL1
Res0 4:3
Field 2 ICC_CR0_EL1
Res0 1
Field 0 ICC_APR_EL1
EndSysreg
Sysreg ICH_HFGITR_EL2 3 4 12 9 7
Res0 63:11
Field 10 GICRCDNMIA
Field 9 GICRCDIA
Field 8 GICCDDI
Field 7 GICCDEOI
Field 6 GICCDHM
Field 5 GICCDRCFG
Field 4 GICCDPEND
Field 3 GICCDAFF
Field 2 GICCDPRI
Field 1 GICCDDIS
Field 0 GICCDEN
EndSysreg
Sysreg ICH_HCR_EL2 3 4 12 11 0
Res0 63:32
Field 31:27 EOIcount

12
drivers/irqchip/Kconfig
View File

@ -41,10 +41,14 @@ config ARM_GIC_V3
select HAVE_ARM_SMCCC_DISCOVERY
select IRQ_MSI_IOMMU
config ARM_GIC_ITS_PARENT
bool
config ARM_GIC_V3_ITS
bool
select GENERIC_MSI_IRQ
select IRQ_MSI_LIB
select ARM_GIC_ITS_PARENT
default ARM_GIC_V3
select IRQ_MSI_IOMMU
@ -54,6 +58,14 @@ config ARM_GIC_V3_ITS_FSL_MC
depends on FSL_MC_BUS
default ARM_GIC_V3_ITS
config ARM_GIC_V5
bool
select IRQ_DOMAIN_HIERARCHY
select GENERIC_IRQ_EFFECTIVE_AFF_MASK
select GENERIC_MSI_IRQ
select IRQ_MSI_LIB
select ARM_GIC_ITS_PARENT
config ARM_NVIC
bool
select IRQ_DOMAIN_HIERARCHY

5
drivers/irqchip/Makefile
View File

@ -33,9 +33,12 @@ obj-$(CONFIG_ARCH_REALVIEW) += irq-gic-realview.o
obj-$(CONFIG_IRQ_MSI_LIB) += irq-msi-lib.o
obj-$(CONFIG_ARM_GIC_V2M) += irq-gic-v2m.o
obj-$(CONFIG_ARM_GIC_V3) += irq-gic-v3.o irq-gic-v3-mbi.o irq-gic-common.o
obj-$(CONFIG_ARM_GIC_V3_ITS) += irq-gic-v3-its.o irq-gic-v4.o irq-gic-v3-its-msi-parent.o
obj-$(CONFIG_ARM_GIC_ITS_PARENT) += irq-gic-its-msi-parent.o
obj-$(CONFIG_ARM_GIC_V3_ITS) += irq-gic-v3-its.o irq-gic-v4.o
obj-$(CONFIG_ARM_GIC_V3_ITS_FSL_MC) += irq-gic-v3-its-fsl-mc-msi.o
obj-$(CONFIG_PARTITION_PERCPU) += irq-partition-percpu.o
obj-$(CONFIG_ARM_GIC_V5) += irq-gic-v5.o irq-gic-v5-irs.o irq-gic-v5-its.o \
irq-gic-v5-iwb.o
obj-$(CONFIG_HISILICON_IRQ_MBIGEN) += irq-mbigen.o
obj-$(CONFIG_ARM_NVIC) += irq-nvic.o
obj-$(CONFIG_ARM_VIC) += irq-vic.o

2
drivers/irqchip/irq-gic-common.h
View File

@ -29,8 +29,6 @@ void gic_enable_quirks(u32 iidr, const struct gic_quirk *quirks,
void gic_enable_of_quirks(const struct device_node *np,
const struct gic_quirk *quirks, void *data);
extern const struct msi_parent_ops gic_v3_its_msi_parent_ops;
#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)
#define RDIST_FLAGS_FORCE_NON_SHAREABLE (1 << 2)

168
drivers/irqchip/irq-gic-v3-its-msi-parent.c → drivers/irqchip/irq-gic-its-msi-parent.c
View File

@ -5,9 +5,10 @@
// Copyright (C) 2022 Intel
#include <linux/acpi_iort.h>
#include <linux/of_address.h>
#include <linux/pci.h>
#include "irq-gic-common.h"
#include "irq-gic-its-msi-parent.h"
#include <linux/irqchip/irq-msi-lib.h>
#define ITS_MSI_FLAGS_REQUIRED (MSI_FLAG_USE_DEF_DOM_OPS | \
@ -18,6 +19,23 @@
MSI_FLAG_PCI_MSIX | \
MSI_FLAG_MULTI_PCI_MSI)
static int its_translate_frame_address(struct device_node *msi_node, phys_addr_t *pa)
{
struct resource res;
int ret;
ret = of_property_match_string(msi_node, "reg-names", "ns-translate");
if (ret < 0)
return ret;
ret = of_address_to_resource(msi_node, ret, &res);
if (ret)
return ret;
*pa = res.start;
return 0;
}
#ifdef CONFIG_PCI_MSI
static int its_pci_msi_vec_count(struct pci_dev *pdev, void *data)
{
@ -82,8 +100,46 @@ static int its_pci_msi_prepare(struct irq_domain *domain, struct device *dev,
msi_info = msi_get_domain_info(domain->parent);
return msi_info->ops->msi_prepare(domain->parent, dev, nvec, info);
}
static int its_v5_pci_msi_prepare(struct irq_domain *domain, struct device *dev,
int nvec, msi_alloc_info_t *info)
{
struct device_node *msi_node = NULL;
struct msi_domain_info *msi_info;
struct pci_dev *pdev;
phys_addr_t pa;
u32 rid;
int ret;
if (!dev_is_pci(dev))
return -EINVAL;
pdev = to_pci_dev(dev);
rid = pci_msi_map_rid_ctlr_node(pdev, &msi_node);
if (!msi_node)
return -ENODEV;
ret = its_translate_frame_address(msi_node, &pa);
if (ret)
return -ENODEV;
of_node_put(msi_node);
/* ITS specific DeviceID */
info->scratchpad[0].ul = rid;
/* ITS translate frame physical address */
info->scratchpad[1].ul = pa;
/* Always allocate power of two vectors */
nvec = roundup_pow_of_two(nvec);
msi_info = msi_get_domain_info(domain->parent);
return msi_info->ops->msi_prepare(domain->parent, dev, nvec, info);
}
#else /* CONFIG_PCI_MSI */
#define its_pci_msi_prepare NULL
#define its_v5_pci_msi_prepare NULL
#endif /* !CONFIG_PCI_MSI */
static int of_pmsi_get_dev_id(struct irq_domain *domain, struct device *dev,
@ -118,6 +174,53 @@ static int of_pmsi_get_dev_id(struct irq_domain *domain, struct device *dev,
return ret;
}
static int of_v5_pmsi_get_msi_info(struct irq_domain *domain, struct device *dev,
u32 *dev_id, phys_addr_t *pa)
{
int ret, index = 0;
/*
* Retrieve the DeviceID and the ITS translate frame node pointer
* out of the msi-parent property.
*/
do {
struct of_phandle_args args;
ret = of_parse_phandle_with_args(dev->of_node,
"msi-parent", "#msi-cells",
index, &args);
if (ret)
break;
/*
* The IRQ domain fwnode is the msi controller parent
* in GICv5 (where the msi controller nodes are the
* ITS translate frames).
*/
if (args.np->parent == irq_domain_get_of_node(domain)) {
if (WARN_ON(args.args_count != 1))
return -EINVAL;
*dev_id = args.args[0];
ret = its_translate_frame_address(args.np, pa);
if (ret)
return -ENODEV;
break;
}
index++;
} while (!ret);
if (ret) {
struct device_node *np = NULL;
ret = of_map_id(dev->of_node, dev->id, "msi-map", "msi-map-mask", &np, dev_id);
if (np) {
ret = its_translate_frame_address(np, pa);
of_node_put(np);
}
}
return ret;
}
int __weak iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
{
return -1;
@ -148,6 +251,33 @@ static int its_pmsi_prepare(struct irq_domain *domain, struct device *dev,
dev, nvec, info);
}
static int its_v5_pmsi_prepare(struct irq_domain *domain, struct device *dev,
int nvec, msi_alloc_info_t *info)
{
struct msi_domain_info *msi_info;
phys_addr_t pa;
u32 dev_id;
int ret;
if (!dev->of_node)
return -ENODEV;
ret = of_v5_pmsi_get_msi_info(domain->parent, dev, &dev_id, &pa);
if (ret)
return ret;
/* ITS specific DeviceID */
info->scratchpad[0].ul = dev_id;
/* ITS translate frame physical address */
info->scratchpad[1].ul = pa;
/* Allocate always as a power of 2 */
nvec = roundup_pow_of_two(nvec);
msi_info = msi_get_domain_info(domain->parent);
return msi_info->ops->msi_prepare(domain->parent, dev, nvec, info);
}
static void its_msi_teardown(struct irq_domain *domain, msi_alloc_info_t *info)
{
struct msi_domain_info *msi_info;
@ -199,6 +329,32 @@ static bool its_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
return true;
}
static bool its_v5_init_dev_msi_info(struct device *dev, struct irq_domain *domain,
struct irq_domain *real_parent, struct msi_domain_info *info)
{
if (!msi_lib_init_dev_msi_info(dev, domain, real_parent, info))
return false;
switch (info->bus_token) {
case DOMAIN_BUS_PCI_DEVICE_MSI:
case DOMAIN_BUS_PCI_DEVICE_MSIX:
info->ops->msi_prepare = its_v5_pci_msi_prepare;
info->ops->msi_teardown = its_msi_teardown;
break;
case DOMAIN_BUS_DEVICE_MSI:
case DOMAIN_BUS_WIRED_TO_MSI:
info->ops->msi_prepare = its_v5_pmsi_prepare;
info->ops->msi_teardown = its_msi_teardown;
break;
default:
/* Confused. How did the lib return true? */
WARN_ON_ONCE(1);
return false;
}
return true;
}
const struct msi_parent_ops gic_v3_its_msi_parent_ops = {
.supported_flags = ITS_MSI_FLAGS_SUPPORTED,
.required_flags = ITS_MSI_FLAGS_REQUIRED,
@ -208,3 +364,13 @@ const struct msi_parent_ops gic_v3_its_msi_parent_ops = {
.prefix = "ITS-",
.init_dev_msi_info = its_init_dev_msi_info,
};
const struct msi_parent_ops gic_v5_its_msi_parent_ops = {
.supported_flags = ITS_MSI_FLAGS_SUPPORTED,
.required_flags = ITS_MSI_FLAGS_REQUIRED,
.chip_flags = MSI_CHIP_FLAG_SET_EOI,
.bus_select_token = DOMAIN_BUS_NEXUS,
.bus_select_mask = MATCH_PCI_MSI | MATCH_PLATFORM_MSI,
.prefix = "ITS-v5-",
.init_dev_msi_info = its_v5_init_dev_msi_info,
};

12
drivers/irqchip/irq-gic-its-msi-parent.h Normal file
View File

@ -0,0 +1,12 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2024 ARM Limited, All Rights Reserved.
*/
#ifndef _IRQ_GIC_ITS_MSI_PARENT_H
#define _IRQ_GIC_ITS_MSI_PARENT_H
extern const struct msi_parent_ops gic_v3_its_msi_parent_ops;
extern const struct msi_parent_ops gic_v5_its_msi_parent_ops;
#endif /* _IRQ_GIC_ITS_MSI_PARENT_H */

1
drivers/irqchip/irq-gic-v3-its.c
View File

@ -41,6 +41,7 @@
#include <asm/exception.h>
#include "irq-gic-common.h"
#include "irq-gic-its-msi-parent.h"
#include <linux/irqchip/irq-msi-lib.h>
#define ITS_FLAGS_CMDQ_NEEDS_FLUSHING (1ULL << 0)

822
drivers/irqchip/irq-gic-v5-irs.c Normal file
View File

@ -0,0 +1,822 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2024-2025 ARM Limited, All Rights Reserved.
*/
#define pr_fmt(fmt) "GICv5 IRS: " fmt
#include <linux/log2.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/irqchip.h>
#include <linux/irqchip/arm-gic-v5.h>
/*
* Hardcoded ID_BITS limit for systems supporting only a 1-level IST
* table. Systems supporting only a 1-level IST table aren't expected
* to require more than 2^12 LPIs. Tweak as required.
*/
#define LPI_ID_BITS_LINEAR 12
#define IRS_FLAGS_NON_COHERENT BIT(0)
static DEFINE_PER_CPU_READ_MOSTLY(struct gicv5_irs_chip_data *, per_cpu_irs_data);
static LIST_HEAD(irs_nodes);
static u32 irs_readl_relaxed(struct gicv5_irs_chip_data *irs_data,
const u32 reg_offset)
{
return readl_relaxed(irs_data->irs_base + reg_offset);
}
static void irs_writel_relaxed(struct gicv5_irs_chip_data *irs_data,
const u32 val, const u32 reg_offset)
{
writel_relaxed(val, irs_data->irs_base + reg_offset);
}
static u64 irs_readq_relaxed(struct gicv5_irs_chip_data *irs_data,
const u32 reg_offset)
{
return readq_relaxed(irs_data->irs_base + reg_offset);
}
static void irs_writeq_relaxed(struct gicv5_irs_chip_data *irs_data,
const u64 val, const u32 reg_offset)
{
writeq_relaxed(val, irs_data->irs_base + reg_offset);
}
/*
* The polling wait (in gicv5_wait_for_op_s_atomic()) on a GIC register
* provides the memory barriers (through MMIO accessors)
* required to synchronize CPU and GIC access to IST memory.
*/
static int gicv5_irs_ist_synchronise(struct gicv5_irs_chip_data *irs_data)
{
return gicv5_wait_for_op_atomic(irs_data->irs_base, GICV5_IRS_IST_STATUSR,
GICV5_IRS_IST_STATUSR_IDLE, NULL);
}
static int __init gicv5_irs_init_ist_linear(struct gicv5_irs_chip_data *irs_data,
unsigned int lpi_id_bits,
unsigned int istsz)
{
size_t l2istsz;
u32 n, cfgr;
void *ist;
u64 baser;
int ret;
/* Taken from GICv5 specifications 10.2.1.13 IRS_IST_BASER */
n = max(5, lpi_id_bits + 1 + istsz);
l2istsz = BIT(n + 1);
/*
* Check memory requirements. For a linear IST we cap the
* number of ID bits to a value that should never exceed
* kmalloc interface memory allocation limits, so this
* check is really belt and braces.
*/
if (l2istsz > KMALLOC_MAX_SIZE) {
u8 lpi_id_cap = ilog2(KMALLOC_MAX_SIZE) - 2 + istsz;
pr_warn("Limiting LPI ID bits from %u to %u\n",
lpi_id_bits, lpi_id_cap);
lpi_id_bits = lpi_id_cap;
l2istsz = KMALLOC_MAX_SIZE;
}
ist = kzalloc(l2istsz, GFP_KERNEL);
if (!ist)
return -ENOMEM;
if (irs_data->flags & IRS_FLAGS_NON_COHERENT)
dcache_clean_inval_poc((unsigned long)ist,
(unsigned long)ist + l2istsz);
else
dsb(ishst);
cfgr = FIELD_PREP(GICV5_IRS_IST_CFGR_STRUCTURE,
GICV5_IRS_IST_CFGR_STRUCTURE_LINEAR) |
FIELD_PREP(GICV5_IRS_IST_CFGR_ISTSZ, istsz) |
FIELD_PREP(GICV5_IRS_IST_CFGR_L2SZ,
GICV5_IRS_IST_CFGR_L2SZ_4K) |
FIELD_PREP(GICV5_IRS_IST_CFGR_LPI_ID_BITS, lpi_id_bits);
irs_writel_relaxed(irs_data, cfgr, GICV5_IRS_IST_CFGR);
gicv5_global_data.ist.l2 = false;
baser = (virt_to_phys(ist) & GICV5_IRS_IST_BASER_ADDR_MASK) |
FIELD_PREP(GICV5_IRS_IST_BASER_VALID, 0x1);
irs_writeq_relaxed(irs_data, baser, GICV5_IRS_IST_BASER);
ret = gicv5_irs_ist_synchronise(irs_data);
if (ret) {
kfree(ist);
return ret;
}
return 0;
}
static int __init gicv5_irs_init_ist_two_level(struct gicv5_irs_chip_data *irs_data,
unsigned int lpi_id_bits,
unsigned int istsz,
unsigned int l2sz)
{
__le64 *l1ist;
u32 cfgr, n;
size_t l1sz;
u64 baser;
int ret;
/* Taken from GICv5 specifications 10.2.1.13 IRS_IST_BASER */
n = max(5, lpi_id_bits - ((10 - istsz) + (2 * l2sz)) + 2);
l1sz = BIT(n + 1);
l1ist = kzalloc(l1sz, GFP_KERNEL);
if (!l1ist)
return -ENOMEM;
if (irs_data->flags & IRS_FLAGS_NON_COHERENT)
dcache_clean_inval_poc((unsigned long)l1ist,
(unsigned long)l1ist + l1sz);
else
dsb(ishst);
cfgr = FIELD_PREP(GICV5_IRS_IST_CFGR_STRUCTURE,
GICV5_IRS_IST_CFGR_STRUCTURE_TWO_LEVEL) |
FIELD_PREP(GICV5_IRS_IST_CFGR_ISTSZ, istsz) |
FIELD_PREP(GICV5_IRS_IST_CFGR_L2SZ, l2sz) |
FIELD_PREP(GICV5_IRS_IST_CFGR_LPI_ID_BITS, lpi_id_bits);
irs_writel_relaxed(irs_data, cfgr, GICV5_IRS_IST_CFGR);
/*
* The L2SZ determine bits required at L2 level. Number of bytes
* required by metadata is reported through istsz - the number of bits
* covered by L2 entries scales accordingly.
*/
gicv5_global_data.ist.l2_size = BIT(11 + (2 * l2sz) + 1);
gicv5_global_data.ist.l2_bits = (10 - istsz) + (2 * l2sz);
gicv5_global_data.ist.l1ist_addr = l1ist;
gicv5_global_data.ist.l2 = true;
baser = (virt_to_phys(l1ist) & GICV5_IRS_IST_BASER_ADDR_MASK) |
FIELD_PREP(GICV5_IRS_IST_BASER_VALID, 0x1);
irs_writeq_relaxed(irs_data, baser, GICV5_IRS_IST_BASER);
ret = gicv5_irs_ist_synchronise(irs_data);
if (ret) {
kfree(l1ist);
return ret;
}
return 0;
}
/*
* Alloc L2 IST entries on demand.
*
* Locking/serialization is guaranteed by irqdomain core code by
* taking the hierarchical domain struct irq_domain.root->mutex.
*/
int gicv5_irs_iste_alloc(const u32 lpi)
{
struct gicv5_irs_chip_data *irs_data;
unsigned int index;
u32 l2istr, l2bits;
__le64 *l1ist;
size_t l2size;
void *l2ist;
int ret;
if (!gicv5_global_data.ist.l2)
return 0;
irs_data = per_cpu(per_cpu_irs_data, smp_processor_id());
if (!irs_data)
return -ENOENT;
l2size = gicv5_global_data.ist.l2_size;
l2bits = gicv5_global_data.ist.l2_bits;
l1ist = gicv5_global_data.ist.l1ist_addr;
index = lpi >> l2bits;
if (FIELD_GET(GICV5_ISTL1E_VALID, le64_to_cpu(l1ist[index])))
return 0;
l2ist = kzalloc(l2size, GFP_KERNEL);
if (!l2ist)
return -ENOMEM;
l1ist[index] = cpu_to_le64(virt_to_phys(l2ist) & GICV5_ISTL1E_L2_ADDR_MASK);
if (irs_data->flags & IRS_FLAGS_NON_COHERENT) {
dcache_clean_inval_poc((unsigned long)l2ist,
(unsigned long)l2ist + l2size);
dcache_clean_poc((unsigned long)(l1ist + index),
(unsigned long)(l1ist + index) + sizeof(*l1ist));
} else {
dsb(ishst);
}
l2istr = FIELD_PREP(GICV5_IRS_MAP_L2_ISTR_ID, lpi);
irs_writel_relaxed(irs_data, l2istr, GICV5_IRS_MAP_L2_ISTR);
ret = gicv5_irs_ist_synchronise(irs_data);
if (ret) {
l1ist[index] = 0;
kfree(l2ist);
return ret;
}
/*
* Make sure we invalidate the cache line pulled before the IRS
* had a chance to update the L1 entry and mark it valid.
*/
if (irs_data->flags & IRS_FLAGS_NON_COHERENT) {
/*
* gicv5_irs_ist_synchronise() includes memory
* barriers (MMIO accessors) required to guarantee that the
* following dcache invalidation is not executed before the
* IST mapping operation has completed.
*/
dcache_inval_poc((unsigned long)(l1ist + index),
(unsigned long)(l1ist + index) + sizeof(*l1ist));
}
return 0;
}
/*
* Try to match the L2 IST size to the pagesize, and if this is not possible
* pick the smallest supported L2 size in order to minimise the requirement for
* physically contiguous blocks of memory as page-sized allocations are
* guaranteed to be physically contiguous, and are by definition the easiest to
* find.
*
* Fall back to the smallest supported size (in the event that the pagesize
* itself is not supported) again serves to make it easier to find physically
* contiguous blocks of memory.
*/
static unsigned int gicv5_irs_l2_sz(u32 idr2)
{
switch (PAGE_SIZE) {
case SZ_64K:
if (GICV5_IRS_IST_L2SZ_SUPPORT_64KB(idr2))
return GICV5_IRS_IST_CFGR_L2SZ_64K;
fallthrough;
case SZ_4K:
if (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(idr2))
return GICV5_IRS_IST_CFGR_L2SZ_4K;
fallthrough;
case SZ_16K:
if (GICV5_IRS_IST_L2SZ_SUPPORT_16KB(idr2))
return GICV5_IRS_IST_CFGR_L2SZ_16K;
break;
}
if (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(idr2))
return GICV5_IRS_IST_CFGR_L2SZ_4K;
return GICV5_IRS_IST_CFGR_L2SZ_64K;
}
static int __init gicv5_irs_init_ist(struct gicv5_irs_chip_data *irs_data)
{
u32 lpi_id_bits, idr2_id_bits, idr2_min_lpi_id_bits, l2_iste_sz, l2sz;
u32 l2_iste_sz_split, idr2;
bool two_levels, istmd;
u64 baser;
int ret;
baser = irs_readq_relaxed(irs_data, GICV5_IRS_IST_BASER);
if (FIELD_GET(GICV5_IRS_IST_BASER_VALID, baser)) {
pr_err("IST is marked as valid already; cannot allocate\n");
return -EPERM;
}
idr2 = irs_readl_relaxed(irs_data, GICV5_IRS_IDR2);
two_levels = !!FIELD_GET(GICV5_IRS_IDR2_IST_LEVELS, idr2);
idr2_id_bits = FIELD_GET(GICV5_IRS_IDR2_ID_BITS, idr2);
idr2_min_lpi_id_bits = FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, idr2);
/*
* For two level tables we are always supporting the maximum allowed
* number of IDs.
*
* For 1-level tables, we should support a number of bits that
* is >= min_lpi_id_bits but cap it to LPI_ID_BITS_LINEAR lest
* the level 1-table gets too large and its memory allocation
* may fail.
*/
if (two_levels) {
lpi_id_bits = idr2_id_bits;
} else {
lpi_id_bits = max(LPI_ID_BITS_LINEAR, idr2_min_lpi_id_bits);
lpi_id_bits = min(lpi_id_bits, idr2_id_bits);
}
/*
* Cap the ID bits according to the CPUIF supported ID bits
*/
lpi_id_bits = min(lpi_id_bits, gicv5_global_data.cpuif_id_bits);
if (two_levels)
l2sz = gicv5_irs_l2_sz(idr2);
istmd = !!FIELD_GET(GICV5_IRS_IDR2_ISTMD, idr2);
l2_iste_sz = GICV5_IRS_IST_CFGR_ISTSZ_4;
if (istmd) {
l2_iste_sz_split = FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, idr2);
if (lpi_id_bits < l2_iste_sz_split)
l2_iste_sz = GICV5_IRS_IST_CFGR_ISTSZ_8;
else
l2_iste_sz = GICV5_IRS_IST_CFGR_ISTSZ_16;
}
/*
* Follow GICv5 specification recommendation to opt in for two
* level tables (ref: 10.2.1.14 IRS_IST_CFGR).
*/
if (two_levels && (lpi_id_bits > ((10 - l2_iste_sz) + (2 * l2sz)))) {
ret = gicv5_irs_init_ist_two_level(irs_data, lpi_id_bits,
l2_iste_sz, l2sz);
} else {
ret = gicv5_irs_init_ist_linear(irs_data, lpi_id_bits,
l2_iste_sz);
}
if (ret)
return ret;
gicv5_init_lpis(BIT(lpi_id_bits));
return 0;
}
struct iaffid_entry {
u16 iaffid;
bool valid;
};
static DEFINE_PER_CPU(struct iaffid_entry, cpu_iaffid);
int gicv5_irs_cpu_to_iaffid(int cpuid, u16 *iaffid)
{
if (!per_cpu(cpu_iaffid, cpuid).valid) {
pr_err("IAFFID for CPU %d has not been initialised\n", cpuid);
return -ENODEV;
}
*iaffid = per_cpu(cpu_iaffid, cpuid).iaffid;
return 0;
}
struct gicv5_irs_chip_data *gicv5_irs_lookup_by_spi_id(u32 spi_id)
{
struct gicv5_irs_chip_data *irs_data;
u32 min, max;
list_for_each_entry(irs_data, &irs_nodes, entry) {
if (!irs_data->spi_range)
continue;
min = irs_data->spi_min;
max = irs_data->spi_min + irs_data->spi_range - 1;
if (spi_id >= min && spi_id <= max)
return irs_data;
}
return NULL;
}
static int gicv5_irs_wait_for_spi_op(struct gicv5_irs_chip_data *irs_data)
{
u32 statusr;
int ret;
ret = gicv5_wait_for_op_atomic(irs_data->irs_base, GICV5_IRS_SPI_STATUSR,
GICV5_IRS_SPI_STATUSR_IDLE, &statusr);
if (ret)
return ret;
return !!FIELD_GET(GICV5_IRS_SPI_STATUSR_V, statusr) ? 0 : -EIO;
}
static int gicv5_irs_wait_for_irs_pe(struct gicv5_irs_chip_data *irs_data,
bool selr)
{
bool valid = true;
u32 statusr;
int ret;
ret = gicv5_wait_for_op_atomic(irs_data->irs_base, GICV5_IRS_PE_STATUSR,
GICV5_IRS_PE_STATUSR_IDLE, &statusr);
if (ret)
return ret;
if (selr)
valid = !!FIELD_GET(GICV5_IRS_PE_STATUSR_V, statusr);
return valid ? 0 : -EIO;
}
static int gicv5_irs_wait_for_pe_selr(struct gicv5_irs_chip_data *irs_data)
{
return gicv5_irs_wait_for_irs_pe(irs_data, true);
}
static int gicv5_irs_wait_for_pe_cr0(struct gicv5_irs_chip_data *irs_data)
{
return gicv5_irs_wait_for_irs_pe(irs_data, false);
}
int gicv5_spi_irq_set_type(struct irq_data *d, unsigned int type)
{
struct gicv5_irs_chip_data *irs_data = d->chip_data;
u32 selr, cfgr;
bool level;
int ret;
/*
* There is no distinction between HIGH/LOW for level IRQs
* and RISING/FALLING for edge IRQs in the architecture,
* hence consider them equivalent.
*/
switch (type) {
case IRQ_TYPE_EDGE_RISING:
case IRQ_TYPE_EDGE_FALLING:
level = false;
break;
case IRQ_TYPE_LEVEL_HIGH:
case IRQ_TYPE_LEVEL_LOW:
level = true;
break;
default:
return -EINVAL;
}
guard(raw_spinlock)(&irs_data->spi_config_lock);
selr = FIELD_PREP(GICV5_IRS_SPI_SELR_ID, d->hwirq);
irs_writel_relaxed(irs_data, selr, GICV5_IRS_SPI_SELR);
ret = gicv5_irs_wait_for_spi_op(irs_data);
if (ret)
return ret;
cfgr = FIELD_PREP(GICV5_IRS_SPI_CFGR_TM, level);
irs_writel_relaxed(irs_data, cfgr, GICV5_IRS_SPI_CFGR);
return gicv5_irs_wait_for_spi_op(irs_data);
}
static int gicv5_irs_wait_for_idle(struct gicv5_irs_chip_data *irs_data)
{
return gicv5_wait_for_op_atomic(irs_data->irs_base, GICV5_IRS_CR0,
GICV5_IRS_CR0_IDLE, NULL);
}
void gicv5_irs_syncr(void)
{
struct gicv5_irs_chip_data *irs_data;
u32 syncr;
irs_data = list_first_entry_or_null(&irs_nodes, struct gicv5_irs_chip_data, entry);
if (WARN_ON_ONCE(!irs_data))
return;
syncr = FIELD_PREP(GICV5_IRS_SYNCR_SYNC, 1);
irs_writel_relaxed(irs_data, syncr, GICV5_IRS_SYNCR);
gicv5_wait_for_op(irs_data->irs_base, GICV5_IRS_SYNC_STATUSR,
GICV5_IRS_SYNC_STATUSR_IDLE);
}
int gicv5_irs_register_cpu(int cpuid)
{
struct gicv5_irs_chip_data *irs_data;
u32 selr, cr0;
u16 iaffid;
int ret;
ret = gicv5_irs_cpu_to_iaffid(cpuid, &iaffid);
if (ret) {
pr_err("IAFFID for CPU %d has not been initialised\n", cpuid);
return ret;
}
irs_data = per_cpu(per_cpu_irs_data, cpuid);
if (!irs_data) {
pr_err("No IRS associated with CPU %u\n", cpuid);
return -ENXIO;
}
selr = FIELD_PREP(GICV5_IRS_PE_SELR_IAFFID, iaffid);
irs_writel_relaxed(irs_data, selr, GICV5_IRS_PE_SELR);
ret = gicv5_irs_wait_for_pe_selr(irs_data);
if (ret) {
pr_err("IAFFID 0x%x used in IRS_PE_SELR is invalid\n", iaffid);
return -ENXIO;
}
cr0 = FIELD_PREP(GICV5_IRS_PE_CR0_DPS, 0x1);
irs_writel_relaxed(irs_data, cr0, GICV5_IRS_PE_CR0);
ret = gicv5_irs_wait_for_pe_cr0(irs_data);
if (ret)
return ret;
pr_debug("CPU %d enabled PE IAFFID 0x%x\n", cpuid, iaffid);
return 0;
}
static void __init gicv5_irs_init_bases(struct gicv5_irs_chip_data *irs_data,
void __iomem *irs_base,
struct fwnode_handle *handle)
{
struct device_node *np = to_of_node(handle);
u32 cr0, cr1;
irs_data->fwnode = handle;
irs_data->irs_base = irs_base;
if (of_property_read_bool(np, "dma-noncoherent")) {
/*
* A non-coherent IRS implies that some cache levels cannot be
* used coherently by the cores and GIC. Our only option is to mark
* memory attributes for the GIC as non-cacheable; by default,
* non-cacheable memory attributes imply outer-shareable
* shareability, the value written into IRS_CR1_SH is ignored.
*/
cr1 = FIELD_PREP(GICV5_IRS_CR1_VPED_WA, GICV5_NO_WRITE_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VPED_RA, GICV5_NO_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VMD_WA, GICV5_NO_WRITE_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VMD_RA, GICV5_NO_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VPET_RA, GICV5_NO_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VMT_RA, GICV5_NO_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_IST_WA, GICV5_NO_WRITE_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_IST_RA, GICV5_NO_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_IC, GICV5_NON_CACHE) |
FIELD_PREP(GICV5_IRS_CR1_OC, GICV5_NON_CACHE);
irs_data->flags |= IRS_FLAGS_NON_COHERENT;
} else {
cr1 = FIELD_PREP(GICV5_IRS_CR1_VPED_WA, GICV5_WRITE_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VPED_RA, GICV5_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VMD_WA, GICV5_WRITE_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VMD_RA, GICV5_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VPET_RA, GICV5_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_VMT_RA, GICV5_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_IST_WA, GICV5_WRITE_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_IST_RA, GICV5_READ_ALLOC) |
FIELD_PREP(GICV5_IRS_CR1_IC, GICV5_WB_CACHE) |
FIELD_PREP(GICV5_IRS_CR1_OC, GICV5_WB_CACHE) |
FIELD_PREP(GICV5_IRS_CR1_SH, GICV5_INNER_SHARE);
}
irs_writel_relaxed(irs_data, cr1, GICV5_IRS_CR1);
cr0 = FIELD_PREP(GICV5_IRS_CR0_IRSEN, 0x1);
irs_writel_relaxed(irs_data, cr0, GICV5_IRS_CR0);
gicv5_irs_wait_for_idle(irs_data);
}
static int __init gicv5_irs_of_init_affinity(struct device_node *node,
struct gicv5_irs_chip_data *irs_data,
u8 iaffid_bits)
{
/*
* Detect IAFFID<->CPU mappings from the device tree and
* record IRS<->CPU topology information.
*/
u16 iaffid_mask = GENMASK(iaffid_bits - 1, 0);
int ret, i, ncpus, niaffids;
ncpus = of_count_phandle_with_args(node, "cpus", NULL);
if (ncpus < 0)
return -EINVAL;
niaffids = of_property_count_elems_of_size(node, "arm,iaffids",
sizeof(u16));
if (niaffids != ncpus)
return -EINVAL;
u16 *iaffids __free(kfree) = kcalloc(niaffids, sizeof(*iaffids), GFP_KERNEL);
if (!iaffids)
return -ENOMEM;
ret = of_property_read_u16_array(node, "arm,iaffids", iaffids, niaffids);
if (ret)
return ret;
for (i = 0; i < ncpus; i++) {
struct device_node *cpu_node;
int cpu;
cpu_node = of_parse_phandle(node, "cpus", i);
if (WARN_ON(!cpu_node))
continue;
cpu = of_cpu_node_to_id(cpu_node);
of_node_put(cpu_node);
if (WARN_ON(cpu < 0))
continue;
if (iaffids[i] & ~iaffid_mask) {
pr_warn("CPU %d iaffid 0x%x exceeds IRS iaffid bits\n",
cpu, iaffids[i]);
continue;
}
per_cpu(cpu_iaffid, cpu).iaffid = iaffids[i];
per_cpu(cpu_iaffid, cpu).valid = true;
/* We also know that the CPU is connected to this IRS */
per_cpu(per_cpu_irs_data, cpu) = irs_data;
}
return ret;
}
static void irs_setup_pri_bits(u32 idr1)
{
switch (FIELD_GET(GICV5_IRS_IDR1_PRIORITY_BITS, idr1)) {
case GICV5_IRS_IDR1_PRIORITY_BITS_1BITS:
gicv5_global_data.irs_pri_bits = 1;
break;
case GICV5_IRS_IDR1_PRIORITY_BITS_2BITS:
gicv5_global_data.irs_pri_bits = 2;
break;
case GICV5_IRS_IDR1_PRIORITY_BITS_3BITS:
gicv5_global_data.irs_pri_bits = 3;
break;
case GICV5_IRS_IDR1_PRIORITY_BITS_4BITS:
gicv5_global_data.irs_pri_bits = 4;
break;
case GICV5_IRS_IDR1_PRIORITY_BITS_5BITS:
gicv5_global_data.irs_pri_bits = 5;
break;
default:
pr_warn("Detected wrong IDR priority bits value 0x%lx\n",
FIELD_GET(GICV5_IRS_IDR1_PRIORITY_BITS, idr1));
gicv5_global_data.irs_pri_bits = 1;
break;
}
}
static int __init gicv5_irs_init(struct device_node *node)
{
struct gicv5_irs_chip_data *irs_data;
void __iomem *irs_base;
u32 idr, spi_count;
u8 iaffid_bits;
int ret;
irs_data = kzalloc(sizeof(*irs_data), GFP_KERNEL);
if (!irs_data)
return -ENOMEM;
raw_spin_lock_init(&irs_data->spi_config_lock);
ret = of_property_match_string(node, "reg-names", "ns-config");
if (ret < 0) {
pr_err("%pOF: ns-config reg-name not present\n", node);
goto out_err;
}
irs_base = of_io_request_and_map(node, ret, of_node_full_name(node));
if (IS_ERR(irs_base)) {
pr_err("%pOF: unable to map GICv5 IRS registers\n", node);
ret = PTR_ERR(irs_base);
goto out_err;
}
gicv5_irs_init_bases(irs_data, irs_base, &node->fwnode);
idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR1);
iaffid_bits = FIELD_GET(GICV5_IRS_IDR1_IAFFID_BITS, idr) + 1;
ret = gicv5_irs_of_init_affinity(node, irs_data, iaffid_bits);
if (ret) {
pr_err("Failed to parse CPU IAFFIDs from the device tree!\n");
goto out_iomem;
}
idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR2);
if (WARN(!FIELD_GET(GICV5_IRS_IDR2_LPI, idr),
"LPI support not available - no IPIs, can't proceed\n")) {
ret = -ENODEV;
goto out_iomem;
}
idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR7);
irs_data->spi_min = FIELD_GET(GICV5_IRS_IDR7_SPI_BASE, idr);
idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR6);
irs_data->spi_range = FIELD_GET(GICV5_IRS_IDR6_SPI_IRS_RANGE, idr);
if (irs_data->spi_range) {
pr_info("%s detected SPI range [%u-%u]\n",
of_node_full_name(node),
irs_data->spi_min,
irs_data->spi_min +
irs_data->spi_range - 1);
}
/*
* Do the global setting only on the first IRS.
* Global properties (iaffid_bits, global spi count) are guaranteed to
* be consistent across IRSes by the architecture.
*/
if (list_empty(&irs_nodes)) {
idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR1);
irs_setup_pri_bits(idr);
idr = irs_readl_relaxed(irs_data, GICV5_IRS_IDR5);
spi_count = FIELD_GET(GICV5_IRS_IDR5_SPI_RANGE, idr);
gicv5_global_data.global_spi_count = spi_count;
gicv5_init_lpi_domain();
pr_debug("Detected %u SPIs globally\n", spi_count);
}
list_add_tail(&irs_data->entry, &irs_nodes);
return 0;
out_iomem:
iounmap(irs_base);
out_err:
kfree(irs_data);
return ret;
}
void __init gicv5_irs_remove(void)
{
struct gicv5_irs_chip_data *irs_data, *tmp_data;
gicv5_free_lpi_domain();
gicv5_deinit_lpis();
list_for_each_entry_safe(irs_data, tmp_data, &irs_nodes, entry) {
iounmap(irs_data->irs_base);
list_del(&irs_data->entry);
kfree(irs_data);
}
}
int __init gicv5_irs_enable(void)
{
struct gicv5_irs_chip_data *irs_data;
int ret;
irs_data = list_first_entry_or_null(&irs_nodes,
struct gicv5_irs_chip_data, entry);
if (!irs_data)
return -ENODEV;
ret = gicv5_irs_init_ist(irs_data);
if (ret) {
pr_err("Failed to init IST\n");
return ret;
}
return 0;
}
void __init gicv5_irs_its_probe(void)
{
struct gicv5_irs_chip_data *irs_data;
list_for_each_entry(irs_data, &irs_nodes, entry)
gicv5_its_of_probe(to_of_node(irs_data->fwnode));
}
int __init gicv5_irs_of_probe(struct device_node *parent)
{
struct device_node *np;
int ret;
for_each_available_child_of_node(parent, np) {
if (!of_device_is_compatible(np, "arm,gic-v5-irs"))
continue;
ret = gicv5_irs_init(np);
if (ret)
pr_err("Failed to init IRS %s\n", np->full_name);
}
return list_empty(&irs_nodes) ? -ENODEV : 0;
}

1228
drivers/irqchip/irq-gic-v5-its.c Normal file
View File

File diff suppressed because it is too large Load Diff

284
drivers/irqchip/irq-gic-v5-iwb.c Normal file
View File

@ -0,0 +1,284 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (C) 2024-2025 ARM Limited, All Rights Reserved.
*/
#define pr_fmt(fmt) "GICv5 IWB: " fmt
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/msi.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_platform.h>
#include <linux/irqchip.h>
#include <linux/irqchip/arm-gic-v5.h>
struct gicv5_iwb_chip_data {
void __iomem *iwb_base;
u16 nr_regs;
};
static u32 iwb_readl_relaxed(struct gicv5_iwb_chip_data *iwb_node, const u32 reg_offset)
{
return readl_relaxed(iwb_node->iwb_base + reg_offset);
}
static void iwb_writel_relaxed(struct gicv5_iwb_chip_data *iwb_node, const u32 val,
const u32 reg_offset)
{
writel_relaxed(val, iwb_node->iwb_base + reg_offset);
}
static int gicv5_iwb_wait_for_wenabler(struct gicv5_iwb_chip_data *iwb_node)
{
return gicv5_wait_for_op_atomic(iwb_node->iwb_base, GICV5_IWB_WENABLE_STATUSR,
GICV5_IWB_WENABLE_STATUSR_IDLE, NULL);
}
static int __gicv5_iwb_set_wire_enable(struct gicv5_iwb_chip_data *iwb_node,
u32 iwb_wire, bool enable)
{
u32 n = iwb_wire / 32;
u8 i = iwb_wire % 32;
u32 val;
if (n >= iwb_node->nr_regs) {
pr_err("IWB_WENABLER<n> is invalid for n=%u\n", n);
return -EINVAL;
}
/*
* Enable IWB wire/pin at this point
* Note: This is not the same as enabling the interrupt
*/
val = iwb_readl_relaxed(iwb_node, GICV5_IWB_WENABLER + (4 * n));
if (enable)
val |= BIT(i);
else
val &= ~BIT(i);
iwb_writel_relaxed(iwb_node, val, GICV5_IWB_WENABLER + (4 * n));
return gicv5_iwb_wait_for_wenabler(iwb_node);
}
static int gicv5_iwb_enable_wire(struct gicv5_iwb_chip_data *iwb_node,
u32 iwb_wire)
{
return __gicv5_iwb_set_wire_enable(iwb_node, iwb_wire, true);
}
static int gicv5_iwb_disable_wire(struct gicv5_iwb_chip_data *iwb_node,
u32 iwb_wire)
{
return __gicv5_iwb_set_wire_enable(iwb_node, iwb_wire, false);
}
static void gicv5_iwb_irq_disable(struct irq_data *d)
{
struct gicv5_iwb_chip_data *iwb_node = irq_data_get_irq_chip_data(d);
gicv5_iwb_disable_wire(iwb_node, d->hwirq);
irq_chip_disable_parent(d);
}
static void gicv5_iwb_irq_enable(struct irq_data *d)
{
struct gicv5_iwb_chip_data *iwb_node = irq_data_get_irq_chip_data(d);
gicv5_iwb_enable_wire(iwb_node, d->hwirq);
irq_chip_enable_parent(d);
}
static int gicv5_iwb_set_type(struct irq_data *d, unsigned int type)
{
struct gicv5_iwb_chip_data *iwb_node = irq_data_get_irq_chip_data(d);
u32 iwb_wire, n, wtmr;
u8 i;
iwb_wire = d->hwirq;
i = iwb_wire % 32;
n = iwb_wire / 32;
if (n >= iwb_node->nr_regs) {
pr_err_once("reg %u out of range\n", n);
return -EINVAL;
}
wtmr = iwb_readl_relaxed(iwb_node, GICV5_IWB_WTMR + (4 * n));
switch (type) {
case IRQ_TYPE_LEVEL_HIGH:
case IRQ_TYPE_LEVEL_LOW:
wtmr |= BIT(i);
break;
case IRQ_TYPE_EDGE_RISING:
case IRQ_TYPE_EDGE_FALLING:
wtmr &= ~BIT(i);
break;
default:
pr_debug("unexpected wire trigger mode");
return -EINVAL;
}
iwb_writel_relaxed(iwb_node, wtmr, GICV5_IWB_WTMR + (4 * n));
return 0;
}
static void gicv5_iwb_domain_set_desc(msi_alloc_info_t *alloc_info, struct msi_desc *desc)
{
alloc_info->desc = desc;
alloc_info->hwirq = (u32)desc->data.icookie.value;
}
static int gicv5_iwb_irq_domain_translate(struct irq_domain *d, struct irq_fwspec *fwspec,
irq_hw_number_t *hwirq,
unsigned int *type)
{
if (!is_of_node(fwspec->fwnode))
return -EINVAL;
if (fwspec->param_count < 2)
return -EINVAL;
/*
* param[0] is be the wire
* param[1] is the interrupt type
*/
*hwirq = fwspec->param[0];
*type = fwspec->param[1] & IRQ_TYPE_SENSE_MASK;
return 0;
}
static void gicv5_iwb_write_msi_msg(struct irq_data *d, struct msi_msg *msg) {}
static const struct msi_domain_template iwb_msi_template = {
.chip = {
.name = "GICv5-IWB",
.irq_mask = irq_chip_mask_parent,
.irq_unmask = irq_chip_unmask_parent,
.irq_enable = gicv5_iwb_irq_enable,
.irq_disable = gicv5_iwb_irq_disable,
.irq_eoi = irq_chip_eoi_parent,
.irq_set_type = gicv5_iwb_set_type,
.irq_write_msi_msg = gicv5_iwb_write_msi_msg,
.irq_set_affinity = irq_chip_set_affinity_parent,
.irq_get_irqchip_state = irq_chip_get_parent_state,
.irq_set_irqchip_state = irq_chip_set_parent_state,
.flags = IRQCHIP_SET_TYPE_MASKED |
IRQCHIP_SKIP_SET_WAKE |
IRQCHIP_MASK_ON_SUSPEND,
},
.ops = {
.set_desc = gicv5_iwb_domain_set_desc,
.msi_translate = gicv5_iwb_irq_domain_translate,
},
.info = {
.bus_token = DOMAIN_BUS_WIRED_TO_MSI,
.flags = MSI_FLAG_USE_DEV_FWNODE,
},
.alloc_info = {
.flags = MSI_ALLOC_FLAGS_FIXED_MSG_DATA,
},
};
static bool gicv5_iwb_create_device_domain(struct device *dev, unsigned int size,
struct gicv5_iwb_chip_data *iwb_node)
{
if (WARN_ON_ONCE(!dev->msi.domain))
return false;
return msi_create_device_irq_domain(dev, MSI_DEFAULT_DOMAIN,
&iwb_msi_template, size,
NULL, iwb_node);
}
static struct gicv5_iwb_chip_data *
gicv5_iwb_init_bases(void __iomem *iwb_base, struct platform_device *pdev)
{
u32 nr_wires, idr0, cr0;
unsigned int n;
int ret;
struct gicv5_iwb_chip_data *iwb_node __free(kfree) = kzalloc(sizeof(*iwb_node),
GFP_KERNEL);
if (!iwb_node)
return ERR_PTR(-ENOMEM);
iwb_node->iwb_base = iwb_base;
idr0 = iwb_readl_relaxed(iwb_node, GICV5_IWB_IDR0);
nr_wires = (FIELD_GET(GICV5_IWB_IDR0_IW_RANGE, idr0) + 1) * 32;
cr0 = iwb_readl_relaxed(iwb_node, GICV5_IWB_CR0);
if (!FIELD_GET(GICV5_IWB_CR0_IWBEN, cr0)) {
dev_err(&pdev->dev, "IWB must be enabled in firmware\n");
return ERR_PTR(-EINVAL);
}
iwb_node->nr_regs = FIELD_GET(GICV5_IWB_IDR0_IW_RANGE, idr0) + 1;
for (n = 0; n < iwb_node->nr_regs; n++)
iwb_writel_relaxed(iwb_node, 0, GICV5_IWB_WENABLER + (sizeof(u32) * n));
ret = gicv5_iwb_wait_for_wenabler(iwb_node);
if (ret)
return ERR_PTR(ret);
if (!gicv5_iwb_create_device_domain(&pdev->dev, nr_wires, iwb_node))
return ERR_PTR(-ENOMEM);
return_ptr(iwb_node);
}
static int gicv5_iwb_device_probe(struct platform_device *pdev)
{
struct gicv5_iwb_chip_data *iwb_node;
void __iomem *iwb_base;
struct resource *res;
int ret;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!res)
return -EINVAL;
iwb_base = devm_ioremap(&pdev->dev, res->start, resource_size(res));
if (!iwb_base) {
dev_err(&pdev->dev, "failed to ioremap %pR\n", res);
return -ENOMEM;
}
iwb_node = gicv5_iwb_init_bases(iwb_base, pdev);
if (IS_ERR(iwb_node)) {
ret = PTR_ERR(iwb_node);
goto out_unmap;
}
return 0;
out_unmap:
iounmap(iwb_base);
return ret;
}
static const struct of_device_id gicv5_iwb_of_match[] = {
{ .compatible = "arm,gic-v5-iwb" },
{ /* END */ }
};
MODULE_DEVICE_TABLE(of, gicv5_iwb_of_match);
static struct platform_driver gicv5_iwb_platform_driver = {
.driver = {
.name = "GICv5 IWB",
.of_match_table = gicv5_iwb_of_match,
.suppress_bind_attrs = true,
},
.probe = gicv5_iwb_device_probe,
};
module_platform_driver(gicv5_iwb_platform_driver);

1087
drivers/irqchip/irq-gic-v5.c Normal file
View File

File diff suppressed because it is too large Load Diff

2
drivers/irqchip/irq-gic.c
View File

@ -54,7 +54,7 @@
static void gic_check_cpu_features(void)
{
WARN_TAINT_ONCE(this_cpu_has_cap(ARM64_HAS_GIC_CPUIF_SYSREGS),
WARN_TAINT_ONCE(this_cpu_has_cap(ARM64_HAS_GICV3_CPUIF),
TAINT_CPU_OUT_OF_SPEC,
"GICv3 system registers enabled, broken firmware!\n");
}

5
drivers/irqchip/irq-msi-lib.c
View File

@ -133,11 +133,14 @@ int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec,
{
const struct msi_parent_ops *ops = d->msi_parent_ops;
u32 busmask = BIT(bus_token);
struct fwnode_handle *fwh;
if (!ops)
return 0;
if (fwspec->fwnode != d->fwnode || fwspec->param_count != 0)
fwh = d->flags & IRQ_DOMAIN_FLAG_FWNODE_PARENT ? fwnode_get_parent(fwspec->fwnode)
: fwspec->fwnode;
if (fwh != d->fwnode || fwspec->param_count != 0)
return 0;
/* Handle pure domain searches */

22
drivers/of/irq.c
View File

@ -670,8 +670,20 @@ void __init of_irq_init(const struct of_device_id *matches)
}
}
static u32 __of_msi_map_id(struct device *dev, struct device_node **np,
u32 id_in)
/**
* of_msi_xlate - map a MSI ID and find relevant MSI controller node
* @dev: device for which the mapping is to be done.
* @msi_np: Pointer to store the MSI controller node
* @id_in: Device ID.
*
* Walk up the device hierarchy looking for devices with a "msi-map"
* property. If found, apply the mapping to @id_in. @msi_np pointed
* value must be NULL on entry, if an MSI controller is found @msi_np is
* initialized to the MSI controller node with a reference held.
*
* Returns: The mapped MSI id.
*/
u32 of_msi_xlate(struct device *dev, struct device_node **msi_np, u32 id_in)
{
struct device *parent_dev;
u32 id_out = id_in;
@ -682,7 +694,7 @@ static u32 __of_msi_map_id(struct device *dev, struct device_node **np,
*/
for (parent_dev = dev; parent_dev; parent_dev = parent_dev->parent)
if (!of_map_id(parent_dev->of_node, id_in, "msi-map",
"msi-map-mask", np, &id_out))
"msi-map-mask", msi_np, &id_out))
break;
return id_out;
}
@ -700,7 +712,7 @@ static u32 __of_msi_map_id(struct device *dev, struct device_node **np,
*/
u32 of_msi_map_id(struct device *dev, struct device_node *msi_np, u32 id_in)
{
return __of_msi_map_id(dev, &msi_np, id_in);
return of_msi_xlate(dev, &msi_np, id_in);
}
/**
@ -719,7 +731,7 @@ struct irq_domain *of_msi_map_get_device_domain(struct device *dev, u32 id,
{
struct device_node *np = NULL;
__of_msi_map_id(dev, &np, id);
of_msi_xlate(dev, &np, id);
return irq_find_matching_host(np, bus_token);
}

20
drivers/pci/msi/irqdomain.c
View File

@ -427,6 +427,26 @@ u32 pci_msi_domain_get_msi_rid(struct irq_domain *domain, struct pci_dev *pdev)
return rid;
}
/**
* pci_msi_map_rid_ctlr_node - Get the MSI controller node and MSI requester id (RID)
* @pdev: The PCI device
* @node: Pointer to store the MSI controller device node
*
* Use the firmware data to find the MSI controller node for @pdev.
* If found map the RID and initialize @node with it. @node value must
* be set to NULL on entry.
*
* Returns: The RID.
*/
u32 pci_msi_map_rid_ctlr_node(struct pci_dev *pdev, struct device_node **node)
{
u32 rid = pci_dev_id(pdev);
pci_for_each_dma_alias(pdev, get_msi_id_cb, &rid);
return of_msi_xlate(&pdev->dev, node, rid);
}
/**
* pci_msi_get_device_domain - Get the MSI domain for a given PCI device
* @pdev: The PCI device

1
include/asm-generic/msi.h
View File

@ -33,6 +33,7 @@ typedef struct msi_alloc_info {
/* Device generating MSIs is proxying for another device */
#define MSI_ALLOC_FLAGS_PROXY_DEVICE (1UL << 0)
#define MSI_ALLOC_FLAGS_FIXED_MSG_DATA (1UL << 1)
#define GENERIC_MSI_DOMAIN_OPS 1

394
include/linux/irqchip/arm-gic-v5.h Normal file
View File

@ -0,0 +1,394 @@
/* SPDX-License-Identifier: GPL-2.0-only */
/*
* Copyright (C) 2025 ARM Limited, All Rights Reserved.
*/
#ifndef __LINUX_IRQCHIP_ARM_GIC_V5_H
#define __LINUX_IRQCHIP_ARM_GIC_V5_H
#include <linux/iopoll.h>
#include <asm/cacheflush.h>
#include <asm/smp.h>
#include <asm/sysreg.h>
#define GICV5_IPIS_PER_CPU MAX_IPI
/*
* INTID handling
*/
#define GICV5_HWIRQ_ID GENMASK(23, 0)
#define GICV5_HWIRQ_TYPE GENMASK(31, 29)
#define GICV5_HWIRQ_INTID GENMASK_ULL(31, 0)
#define GICV5_HWIRQ_TYPE_PPI UL(0x1)
#define GICV5_HWIRQ_TYPE_LPI UL(0x2)
#define GICV5_HWIRQ_TYPE_SPI UL(0x3)
/*
* Tables attributes
*/
#define GICV5_NO_READ_ALLOC 0b0
#define GICV5_READ_ALLOC 0b1
#define GICV5_NO_WRITE_ALLOC 0b0
#define GICV5_WRITE_ALLOC 0b1
#define GICV5_NON_CACHE 0b00
#define GICV5_WB_CACHE 0b01
#define GICV5_WT_CACHE 0b10
#define GICV5_NON_SHARE 0b00
#define GICV5_OUTER_SHARE 0b10
#define GICV5_INNER_SHARE 0b11
/*
* IRS registers and tables structures
*/
#define GICV5_IRS_IDR1 0x0004
#define GICV5_IRS_IDR2 0x0008
#define GICV5_IRS_IDR5 0x0014
#define GICV5_IRS_IDR6 0x0018
#define GICV5_IRS_IDR7 0x001c
#define GICV5_IRS_CR0 0x0080
#define GICV5_IRS_CR1 0x0084
#define GICV5_IRS_SYNCR 0x00c0
#define GICV5_IRS_SYNC_STATUSR 0x00c4
#define GICV5_IRS_SPI_SELR 0x0108
#define GICV5_IRS_SPI_CFGR 0x0114
#define GICV5_IRS_SPI_STATUSR 0x0118
#define GICV5_IRS_PE_SELR 0x0140
#define GICV5_IRS_PE_STATUSR 0x0144
#define GICV5_IRS_PE_CR0 0x0148
#define GICV5_IRS_IST_BASER 0x0180
#define GICV5_IRS_IST_CFGR 0x0190
#define GICV5_IRS_IST_STATUSR 0x0194
#define GICV5_IRS_MAP_L2_ISTR 0x01c0
#define GICV5_IRS_IDR1_PRIORITY_BITS GENMASK(22, 20)
#define GICV5_IRS_IDR1_IAFFID_BITS GENMASK(19, 16)
#define GICV5_IRS_IDR1_PRIORITY_BITS_1BITS 0b000
#define GICV5_IRS_IDR1_PRIORITY_BITS_2BITS 0b001
#define GICV5_IRS_IDR1_PRIORITY_BITS_3BITS 0b010
#define GICV5_IRS_IDR1_PRIORITY_BITS_4BITS 0b011
#define GICV5_IRS_IDR1_PRIORITY_BITS_5BITS 0b100
#define GICV5_IRS_IDR2_ISTMD_SZ GENMASK(19, 15)
#define GICV5_IRS_IDR2_ISTMD BIT(14)
#define GICV5_IRS_IDR2_IST_L2SZ GENMASK(13, 11)
#define GICV5_IRS_IDR2_IST_LEVELS BIT(10)
#define GICV5_IRS_IDR2_MIN_LPI_ID_BITS GENMASK(9, 6)
#define GICV5_IRS_IDR2_LPI BIT(5)
#define GICV5_IRS_IDR2_ID_BITS GENMASK(4, 0)
#define GICV5_IRS_IDR5_SPI_RANGE GENMASK(24, 0)
#define GICV5_IRS_IDR6_SPI_IRS_RANGE GENMASK(24, 0)
#define GICV5_IRS_IDR7_SPI_BASE GENMASK(23, 0)
#define GICV5_IRS_IST_L2SZ_SUPPORT_4KB(r) FIELD_GET(BIT(11), (r))
#define GICV5_IRS_IST_L2SZ_SUPPORT_16KB(r) FIELD_GET(BIT(12), (r))
#define GICV5_IRS_IST_L2SZ_SUPPORT_64KB(r) FIELD_GET(BIT(13), (r))
#define GICV5_IRS_CR0_IDLE BIT(1)
#define GICV5_IRS_CR0_IRSEN BIT(0)
#define GICV5_IRS_CR1_VPED_WA BIT(15)
#define GICV5_IRS_CR1_VPED_RA BIT(14)
#define GICV5_IRS_CR1_VMD_WA BIT(13)
#define GICV5_IRS_CR1_VMD_RA BIT(12)
#define GICV5_IRS_CR1_VPET_WA BIT(11)
#define GICV5_IRS_CR1_VPET_RA BIT(10)
#define GICV5_IRS_CR1_VMT_WA BIT(9)
#define GICV5_IRS_CR1_VMT_RA BIT(8)
#define GICV5_IRS_CR1_IST_WA BIT(7)
#define GICV5_IRS_CR1_IST_RA BIT(6)
#define GICV5_IRS_CR1_IC GENMASK(5, 4)
#define GICV5_IRS_CR1_OC GENMASK(3, 2)
#define GICV5_IRS_CR1_SH GENMASK(1, 0)
#define GICV5_IRS_SYNCR_SYNC BIT(31)
#define GICV5_IRS_SYNC_STATUSR_IDLE BIT(0)
#define GICV5_IRS_SPI_STATUSR_V BIT(1)
#define GICV5_IRS_SPI_STATUSR_IDLE BIT(0)
#define GICV5_IRS_SPI_SELR_ID GENMASK(23, 0)
#define GICV5_IRS_SPI_CFGR_TM BIT(0)
#define GICV5_IRS_PE_SELR_IAFFID GENMASK(15, 0)
#define GICV5_IRS_PE_STATUSR_V BIT(1)
#define GICV5_IRS_PE_STATUSR_IDLE BIT(0)
#define GICV5_IRS_PE_CR0_DPS BIT(0)
#define GICV5_IRS_IST_STATUSR_IDLE BIT(0)
#define GICV5_IRS_IST_CFGR_STRUCTURE BIT(16)
#define GICV5_IRS_IST_CFGR_ISTSZ GENMASK(8, 7)
#define GICV5_IRS_IST_CFGR_L2SZ GENMASK(6, 5)
#define GICV5_IRS_IST_CFGR_LPI_ID_BITS GENMASK(4, 0)
#define GICV5_IRS_IST_CFGR_STRUCTURE_LINEAR 0b0
#define GICV5_IRS_IST_CFGR_STRUCTURE_TWO_LEVEL 0b1
#define GICV5_IRS_IST_CFGR_ISTSZ_4 0b00
#define GICV5_IRS_IST_CFGR_ISTSZ_8 0b01
#define GICV5_IRS_IST_CFGR_ISTSZ_16 0b10
#define GICV5_IRS_IST_CFGR_L2SZ_4K 0b00
#define GICV5_IRS_IST_CFGR_L2SZ_16K 0b01
#define GICV5_IRS_IST_CFGR_L2SZ_64K 0b10
#define GICV5_IRS_IST_BASER_ADDR_MASK GENMASK_ULL(55, 6)
#define GICV5_IRS_IST_BASER_VALID BIT_ULL(0)
#define GICV5_IRS_MAP_L2_ISTR_ID GENMASK(23, 0)
#define GICV5_ISTL1E_VALID BIT_ULL(0)
#define GICV5_ISTL1E_L2_ADDR_MASK GENMASK_ULL(55, 12)
/*
* ITS registers and tables structures
*/
#define GICV5_ITS_IDR1 0x0004
#define GICV5_ITS_IDR2 0x0008
#define GICV5_ITS_CR0 0x0080
#define GICV5_ITS_CR1 0x0084
#define GICV5_ITS_DT_BASER 0x00c0
#define GICV5_ITS_DT_CFGR 0x00d0
#define GICV5_ITS_DIDR 0x0100
#define GICV5_ITS_EIDR 0x0108
#define GICV5_ITS_INV_EVENTR 0x010c
#define GICV5_ITS_INV_DEVICER 0x0110
#define GICV5_ITS_STATUSR 0x0120
#define GICV5_ITS_SYNCR 0x0140
#define GICV5_ITS_SYNC_STATUSR 0x0148
#define GICV5_ITS_IDR1_L2SZ GENMASK(10, 8)
#define GICV5_ITS_IDR1_ITT_LEVELS BIT(7)
#define GICV5_ITS_IDR1_DT_LEVELS BIT(6)
#define GICV5_ITS_IDR1_DEVICEID_BITS GENMASK(5, 0)
#define GICV5_ITS_IDR1_L2SZ_SUPPORT_4KB(r) FIELD_GET(BIT(8), (r))
#define GICV5_ITS_IDR1_L2SZ_SUPPORT_16KB(r) FIELD_GET(BIT(9), (r))
#define GICV5_ITS_IDR1_L2SZ_SUPPORT_64KB(r) FIELD_GET(BIT(10), (r))
#define GICV5_ITS_IDR2_XDMN_EVENTs GENMASK(6, 5)
#define GICV5_ITS_IDR2_EVENTID_BITS GENMASK(4, 0)
#define GICV5_ITS_CR0_IDLE BIT(1)
#define GICV5_ITS_CR0_ITSEN BIT(0)
#define GICV5_ITS_CR1_ITT_RA BIT(7)
#define GICV5_ITS_CR1_DT_RA BIT(6)
#define GICV5_ITS_CR1_IC GENMASK(5, 4)
#define GICV5_ITS_CR1_OC GENMASK(3, 2)
#define GICV5_ITS_CR1_SH GENMASK(1, 0)
#define GICV5_ITS_DT_CFGR_STRUCTURE BIT(16)
#define GICV5_ITS_DT_CFGR_L2SZ GENMASK(7, 6)
#define GICV5_ITS_DT_CFGR_DEVICEID_BITS GENMASK(5, 0)
#define GICV5_ITS_DT_BASER_ADDR_MASK GENMASK_ULL(55, 3)
#define GICV5_ITS_INV_DEVICER_I BIT(31)
#define GICV5_ITS_INV_DEVICER_EVENTID_BITS GENMASK(5, 1)
#define GICV5_ITS_INV_DEVICER_L1 BIT(0)
#define GICV5_ITS_DIDR_DEVICEID GENMASK_ULL(31, 0)
#define GICV5_ITS_EIDR_EVENTID GENMASK(15, 0)
#define GICV5_ITS_INV_EVENTR_I BIT(31)
#define GICV5_ITS_INV_EVENTR_ITT_L2SZ GENMASK(2, 1)
#define GICV5_ITS_INV_EVENTR_L1 BIT(0)
#define GICV5_ITS_STATUSR_IDLE BIT(0)
#define GICV5_ITS_SYNCR_SYNC BIT_ULL(63)
#define GICV5_ITS_SYNCR_SYNCALL BIT_ULL(32)
#define GICV5_ITS_SYNCR_DEVICEID GENMASK_ULL(31, 0)
#define GICV5_ITS_SYNC_STATUSR_IDLE BIT(0)
#define GICV5_DTL1E_VALID BIT_ULL(0)
/* Note that there is no shift for the address by design */
#define GICV5_DTL1E_L2_ADDR_MASK GENMASK_ULL(55, 3)
#define GICV5_DTL1E_SPAN GENMASK_ULL(63, 60)
#define GICV5_DTL2E_VALID BIT_ULL(0)
#define GICV5_DTL2E_ITT_L2SZ GENMASK_ULL(2, 1)
/* Note that there is no shift for the address by design */
#define GICV5_DTL2E_ITT_ADDR_MASK GENMASK_ULL(55, 3)
#define GICV5_DTL2E_ITT_DSWE BIT_ULL(57)
#define GICV5_DTL2E_ITT_STRUCTURE BIT_ULL(58)
#define GICV5_DTL2E_EVENT_ID_BITS GENMASK_ULL(63, 59)
#define GICV5_ITTL1E_VALID BIT_ULL(0)
/* Note that there is no shift for the address by design */
#define GICV5_ITTL1E_L2_ADDR_MASK GENMASK_ULL(55, 3)
#define GICV5_ITTL1E_SPAN GENMASK_ULL(63, 60)
#define GICV5_ITTL2E_LPI_ID GENMASK_ULL(23, 0)
#define GICV5_ITTL2E_DAC GENMASK_ULL(29, 28)
#define GICV5_ITTL2E_VIRTUAL BIT_ULL(30)
#define GICV5_ITTL2E_VALID BIT_ULL(31)
#define GICV5_ITTL2E_VM_ID GENMASK_ULL(47, 32)
#define GICV5_ITS_DT_ITT_CFGR_L2SZ_4k 0b00
#define GICV5_ITS_DT_ITT_CFGR_L2SZ_16k 0b01
#define GICV5_ITS_DT_ITT_CFGR_L2SZ_64k 0b10
#define GICV5_ITS_DT_ITT_CFGR_STRUCTURE_LINEAR 0
#define GICV5_ITS_DT_ITT_CFGR_STRUCTURE_TWO_LEVEL 1
#define GICV5_ITS_HWIRQ_DEVICE_ID GENMASK_ULL(31, 0)
#define GICV5_ITS_HWIRQ_EVENT_ID GENMASK_ULL(63, 32)
/*
* IWB registers
*/
#define GICV5_IWB_IDR0 0x0000
#define GICV5_IWB_CR0 0x0080
#define GICV5_IWB_WENABLE_STATUSR 0x00c0
#define GICV5_IWB_WENABLER 0x2000
#define GICV5_IWB_WTMR 0x4000
#define GICV5_IWB_IDR0_INT_DOMS GENMASK(14, 11)
#define GICV5_IWB_IDR0_IW_RANGE GENMASK(10, 0)
#define GICV5_IWB_CR0_IDLE BIT(1)
#define GICV5_IWB_CR0_IWBEN BIT(0)
#define GICV5_IWB_WENABLE_STATUSR_IDLE BIT(0)
/*
* Global Data structures and functions
*/
struct gicv5_chip_data {
struct fwnode_handle *fwnode;
struct irq_domain *ppi_domain;
struct irq_domain *spi_domain;
struct irq_domain *lpi_domain;
struct irq_domain *ipi_domain;
u32 global_spi_count;
u8 cpuif_pri_bits;
u8 cpuif_id_bits;
u8 irs_pri_bits;
struct {
__le64 *l1ist_addr;
u32 l2_size;
u8 l2_bits;
bool l2;
} ist;
};
extern struct gicv5_chip_data gicv5_global_data __read_mostly;
struct gicv5_irs_chip_data {
struct list_head entry;
struct fwnode_handle *fwnode;
void __iomem *irs_base;
u32 flags;
u32 spi_min;
u32 spi_range;
raw_spinlock_t spi_config_lock;
};
static inline int gicv5_wait_for_op_s_atomic(void __iomem *addr, u32 offset,
const char *reg_s, u32 mask,
u32 *val)
{
void __iomem *reg = addr + offset;
u32 tmp;
int ret;
ret = readl_poll_timeout_atomic(reg, tmp, tmp & mask, 1, 10 * USEC_PER_MSEC);
if (unlikely(ret == -ETIMEDOUT)) {
pr_err_ratelimited("%s timeout...\n", reg_s);
return ret;
}
if (val)
*val = tmp;
return 0;
}
static inline int gicv5_wait_for_op_s(void __iomem *addr, u32 offset,
const char *reg_s, u32 mask)
{
void __iomem *reg = addr + offset;
u32 val;
int ret;
ret = readl_poll_timeout(reg, val, val & mask, 1, 10 * USEC_PER_MSEC);
if (unlikely(ret == -ETIMEDOUT)) {
pr_err_ratelimited("%s timeout...\n", reg_s);
return ret;
}
return 0;
}
#define gicv5_wait_for_op_atomic(base, reg, mask, val) \
gicv5_wait_for_op_s_atomic(base, reg, #reg, mask, val)
#define gicv5_wait_for_op(base, reg, mask) \
gicv5_wait_for_op_s(base, reg, #reg, mask)
void __init gicv5_init_lpi_domain(void);
void __init gicv5_free_lpi_domain(void);
int gicv5_irs_of_probe(struct device_node *parent);
void gicv5_irs_remove(void);
int gicv5_irs_enable(void);
void gicv5_irs_its_probe(void);
int gicv5_irs_register_cpu(int cpuid);
int gicv5_irs_cpu_to_iaffid(int cpu_id, u16 *iaffid);
struct gicv5_irs_chip_data *gicv5_irs_lookup_by_spi_id(u32 spi_id);
int gicv5_spi_irq_set_type(struct irq_data *d, unsigned int type);
int gicv5_irs_iste_alloc(u32 lpi);
void gicv5_irs_syncr(void);
struct gicv5_its_devtab_cfg {
union {
struct {
__le64 *devtab;
} linear;
struct {
__le64 *l1devtab;
__le64 **l2ptrs;
} l2;
};
u32 cfgr;
};
struct gicv5_its_itt_cfg {
union {
struct {
__le64 *itt;
unsigned int num_ents;
} linear;
struct {
__le64 *l1itt;
__le64 **l2ptrs;
unsigned int num_l1_ents;
u8 l2sz;
} l2;
};
u8 event_id_bits;
bool l2itt;
};
void gicv5_init_lpis(u32 max);
void gicv5_deinit_lpis(void);
int gicv5_alloc_lpi(void);
void gicv5_free_lpi(u32 lpi);
void __init gicv5_its_of_probe(struct device_node *parent);
#endif

3
include/linux/irqdomain.h
View File

@ -212,6 +212,9 @@ enum {
/* Address and data pair is mutable when irq_set_affinity() */
IRQ_DOMAIN_FLAG_MSI_IMMUTABLE = (1 << 11),
/* IRQ domain requires parent fwnode matching */
IRQ_DOMAIN_FLAG_FWNODE_PARENT = (1 << 12),
/*
* Flags starting from IRQ_DOMAIN_FLAG_NONCORE are reserved
* for implementation specific purposes and ignored by the

1
include/linux/msi.h
View File

@ -705,6 +705,7 @@ struct irq_domain *pci_msi_create_irq_domain(struct fwnode_handle *fwnode,
struct msi_domain_info *info,
struct irq_domain *parent);
u32 pci_msi_domain_get_msi_rid(struct irq_domain *domain, struct pci_dev *pdev);
u32 pci_msi_map_rid_ctlr_node(struct pci_dev *pdev, struct device_node **node);
struct irq_domain *pci_msi_get_device_domain(struct pci_dev *pdev);
#else /* CONFIG_PCI_MSI */
static inline struct irq_domain *pci_msi_get_device_domain(struct pci_dev *pdev)

5
include/linux/of_irq.h
View File

@ -54,6 +54,7 @@ extern struct irq_domain *of_msi_map_get_device_domain(struct device *dev,
u32 id,
u32 bus_token);
extern void of_msi_configure(struct device *dev, const struct device_node *np);
extern u32 of_msi_xlate(struct device *dev, struct device_node **msi_np, u32 id_in);
u32 of_msi_map_id(struct device *dev, struct device_node *msi_np, u32 id_in);
#else
static inline void of_irq_init(const struct of_device_id *matches)
@ -100,6 +101,10 @@ static inline struct irq_domain *of_msi_map_get_device_domain(struct device *dev
static inline void of_msi_configure(struct device *dev, struct device_node *np)
{
}
static inline u32 of_msi_xlate(struct device *dev, struct device_node **msi_np, u32 id_in)
{
return id_in;
}
static inline u32 of_msi_map_id(struct device *dev,
struct device_node *msi_np, u32 id_in)
{