github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

KVM/arm64 fixes for 7.3, round #2

Browse Source 
- Take a reference on the last IRQ loaded into an LR to prevent it
    from being freed while running the guest (Marc Zyngier)
 
  - Ensure that the ITS MOVALL command only affects LPIs that were
    previously affined to the source redistributor (Marc Zyngier)
 
  - Fix + test for honoring the host's trap configuration when running
    non-protected VMs while KVM is in protected mode (Fuad Tabba)
 
  - Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
    stage-2 (Mostafa Saleh)
 -----BEGIN PGP SIGNATURE-----
 
 iI0EABYKADUWIQSNXHjWXuzMZutrKNKivnWIJHzdFgUCar6KFRccb2xpdmVyLnVw
 dG9uQGxpbnV4LmRldgAKCRCivnWIJHzdFoQxAP4lpoLN5+XlRshepgo+t/hZnhqj
 4X6A8t7gtKv1iSp9WgD9FOeekzRH8HunEyjTyez/SmIb4ZZ9yYSMd5wIXX7uqgE=
 =FD18
 -----END PGP SIGNATURE-----

Merge tag 'kvmarm-fixes-7.3-2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD

KVM/arm64 fixes for 7.3, round #2

 - Take a reference on the last IRQ loaded into an LR to prevent it
   from being freed while running the guest (Marc Zyngier)

 - Ensure that the ITS MOVALL command only affects LPIs that were
   previously affined to the source redistributor (Marc Zyngier)

 - Fix + test for honoring the host's trap configuration when running
   non-protected VMs while KVM is in protected mode (Fuad Tabba)

 - Use the host stage-1 mapping granularity for VM_PFNMAP mappings at
   stage-2 (Mostafa Saleh)
This commit is contained in:
Paolo Bonzini 2026-10-14 18:23:05 +02:00
commit e17f9dca0a
605 changed files with 9271 additions and 4242 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.mailmap
View File

@ -421,6 +421,7 @@ Jens Axboe <axboe@kernel.dk> <axboe@suse.de>
Jens Axboe <axboe@kernel.dk> <jens.axboe@oracle.com>
Jens Axboe <axboe@kernel.dk> <axboe@fb.com>
Jens Axboe <axboe@kernel.dk> <axboe@meta.com>
Jens Axboe <axboe@kernel.dk> <axboe@anthropic.com>
Jens Osterkamp <Jens.Osterkamp@de.ibm.com>
Jens Wiklander <jenswi@kernel.org> <jens.wiklander@linaro.org>
Jernej Skrabec <jernej.skrabec@gmail.com> <jernej.skrabec@siol.net>
@ -551,6 +552,9 @@ Li Yang <leoyang.li@nxp.com> <leo@zh-kernel.org>
Lior David <quic_liord@quicinc.com> <liord@codeaurora.org>
Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@linaro.org>
Loic Poulain <loic.poulain@oss.qualcomm.com> <loic.poulain@intel.com>
Lorenzo Bianconi <lorenzo@kernel.org> <lorenzo.bianconi@redhat.com>
Lorenzo Bianconi <lorenzo@kernel.org> <lorenzo.bianconi@st.com>
Lorenzo Bianconi <lorenzo@kernel.org> <lorenzo.bianconi83@gmail.com>
Lorenzo Pieralisi <lpieralisi@kernel.org> <lorenzo.pieralisi@arm.com>
Lorenzo Stoakes <ljs@kernel.org> <lstoakes@gmail.com>
Lorenzo Stoakes <ljs@kernel.org> <lorenzo.stoakes@oracle.com>

2
Documentation/ABI/testing/sysfs-fs-erofs
View File

@ -5,7 +5,7 @@ Description: Shows all enabled kernel features.
Supported features:
compr_cfgs, big_pcluster, chunked_file, device_table,
compr_head2, sb_chksum, ztailpacking, dedupe, fragments,
48bit, metabox.
xattr_prefixes, 48bit, metabox.
What: /sys/fs/erofs/<disk>/sync_decompress
Date: November 2021

37
Documentation/admin-guide/kernel-parameters.txt
View File

@ -4987,18 +4987,6 @@ Kernel parameters
Set to non-zero if a chip is present that snoops speed
changes. Disabled by default.
pata_legacy.ht6560a= [HW,LIBATA]
Format: <int>
Set to 1, 2, or 3 for HT 6560A on the primary channel,
the secondary channel, or both channels respectively.
Disabled by default.
pata_legacy.ht6560b= [HW,LIBATA]
Format: <int>
Set to 1, 2, or 3 for HT 6560B on the primary channel,
the secondary channel, or both channels respectively.
Disabled by default.
pata_legacy.iordy_mask= [HW,LIBATA]
Format: <int>
IORDY enable mask. Set individual bits to allow IORDY
@ -5011,18 +4999,6 @@ Kernel parameters
with the sequence. By default IORDY is allowed across
all channels.
pata_legacy.opti82c46x= [HW,LIBATA]
Format: <int>
Set to 1, 2, or 3 for Opti 82c611A on the primary
channel, the secondary channel, or both channels
respectively. Disabled by default.
pata_legacy.opti82c611a= [HW,LIBATA]
Format: <int>
Set to 1, 2, or 3 for Opti 82c465MV on the primary
channel, the secondary channel, or both channels
respectively. Disabled by default.
pata_legacy.pio_mask= [HW,LIBATA]
Format: <int>
PIO mode mask for autospeed devices. Set individual
@ -5046,19 +5022,6 @@ Kernel parameters
the first port in the list above (0x1f0), and so on.
By default all supported ports are probed.
pata_legacy.qdi= [HW,LIBATA]
Format: <int>
Set to non-zero to probe QDI controllers. By default
set to 1 if CONFIG_PATA_QDI_MODULE, 0 otherwise.
pata_legacy.winbond= [HW,LIBATA]
Format: <int>
Set to non-zero to probe Winbond controllers. Use
the standard I/O port (0x130) if 1, otherwise the
value given is the I/O port to use (typically 0x1b0).
By default set to 1 if CONFIG_PATA_WINBOND_VLB_MODULE,
0 otherwise.
pata_platform.pio_mask= [HW,LIBATA]
Format: <int>
Supported PIO mode mask. Set individual bits to allow

2
Documentation/admin-guide/sysctl/kernel.rst
View File

@ -1416,7 +1416,7 @@ Controls whether userspace may arm Syscall User Dispatch via
== ===================================================================
Only present when the kernel is built with ``CONFIG_SYSCALL_USER_DISPATCH``
and ``CONFIG_PROC_SYSCTL``.
and ``CONFIG_SYSCTL``.
sysctl_writes_strict

8
Documentation/arch/riscv/hwprobe.rst
View File

@ -155,7 +155,8 @@ The following keys are defined:
defined in version 1.0 of the RISC-V Cryptography Extensions Volume II.
* :c:macro:`RISCV_HWPROBE_EXT_ZFH`: The Zfh extension version 1.0 is supported
as defined in the RISC-V ISA manual.
as defined in the RISC-V ISA manual. Zfh is a superset of Zfhmin, so
RISCV_HWPROBE_EXT_ZFHMIN is reported whenever RISCV_HWPROBE_EXT_ZFH is.
* :c:macro:`RISCV_HWPROBE_EXT_ZFHMIN`: The Zfhmin extension version 1.0 is
supported as defined in the RISC-V ISA manual.
@ -164,8 +165,9 @@ The following keys are defined:
is supported as defined in the RISC-V ISA manual.
* :c:macro:`RISCV_HWPROBE_EXT_ZVFH`: The Zvfh extension is supported as
defined in the RISC-V Vector manual starting from commit e2ccd0548d6c
("Remove draft warnings from Zvfh[min]").
defined in the RISC-V Vector manual starting from commit e2ccd0548d6c
("Remove draft warnings from Zvfh[min]"). Zvfh is a superset of Zvfhmin,
so RISCV_HWPROBE_EXT_ZVFHMIN is reported whenever RISCV_HWPROBE_EXT_ZVFH is.
* :c:macro:`RISCV_HWPROBE_EXT_ZVFHMIN`: The Zvfhmin extension is supported as
defined in the RISC-V Vector manual starting from commit e2ccd0548d6c

1
Documentation/devicetree/bindings/regulator/fcs,fan53555.yaml
View File

@ -23,6 +23,7 @@ properties:
- silergy,syr827
- silergy,syr828
- tcs,tcs4525
- tcs,tcs4526
- items:
- const: rockchip,rk8601
- const: rockchip,rk8600

8
Documentation/devicetree/bindings/riscv/cpus.yaml
View File

@ -117,8 +117,8 @@ properties:
$ref: /schemas/types.yaml#/definitions/uint32
description:
VLEN/8, the vector register length in bytes. This property is required on
thead systems where the vector register length is not identical on all harts, or
the vlenb CSR is not available.
thead systems where the vector register length is not identical on all
harts, or the vlenb CSR is not available.
# RISC-V has multiple properties for cache op block sizes as the sizes
# differ between individual CBO extensions
@ -151,8 +151,8 @@ anyOf:
- riscv,isa-base
dependencies:
riscv,isa-base: [ "riscv,isa-extensions" ]
riscv,isa-extensions: [ "riscv,isa-base" ]
riscv,isa-base: ["riscv,isa-extensions"]
riscv,isa-extensions: ["riscv,isa-base"]
required:
- interrupt-controller

16
Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.yaml
View File

@ -44,6 +44,21 @@ allOf:
else:
properties:
starfive,sfc-filter-syscon: false
- if:
properties:
compatible:
contains:
const: ultrarisc,dp1000-spi
then:
properties:
reg:
maxItems: 1
clocks:
minItems: 2
clock-names:
minItems: 2
required:
- clock-names
properties:
compatible:
@ -63,6 +78,7 @@ properties:
- mscc,jaguar2-spi
- sophgo,sg2042-spi
- thead,th1520-spi
- ultrarisc,dp1000-spi
- const: snps,dw-apb-ssi
- description: Vendor controllers compatible with v1.01a
items:

2
Documentation/hwmon/gpd-fan.rst
View File

@ -67,7 +67,7 @@ pwm1_enable
at full speed. Write "1" to set to manual, write "2" to let the EC control
decide fan speed. Read this attribute to see current status.
NB：In consideration of the safety of the device, when setting to manual mode,
NB: In consideration of the safety of the device, when setting to manual mode,
the pwm speed will be set to the maximum value (255) by default. You can set
a different value by writing pwm1 later.

15
Documentation/hwmon/hwmon-kernel-api.rst
View File

@ -42,6 +42,9 @@ register/unregister functions::
char *devm_hwmon_sanitize_name(struct device *dev, const char *name);
int hwmon_notify_event(struct device *dev, enum hwmon_sensor_types type,
u32 attr, int channel);
void hwmon_lock(struct device *dev);
void hwmon_unlock(struct device *dev);
@ -90,6 +93,18 @@ implemented in the driver, or debugfs functions, hwmon_lock() and hwmon_unlock()
can be used to ensure that calls to those functions are serialized. Those
functions also support guard() and scoped_guard() variants.
Drivers can call hwmon_notify_event() to notify userspace and the thermal
subsystem when a hardware monitoring event (such as an alarm or a fault
condition) occurs or clears. The parameters are the hwmon device, the sensor
type, the attribute identifier associated with the event (such as
hwmon_temp_max_alarm or hwmon_fan_fault), and the sensor channel number.
hwmon_notify_event() generates a sysfs event (calling sysfs_notify()) and a
udev event with the attribute name passed in the NAME environment property
(e.g., "NAME=temp1_max_alarm"). If the event is for a temperature sensor and
the sensor is attached to a thermal zone, it also notifies the thermal
subsystem to update the thermal zone. hwmon_notify_event() returns 0 on
success or a negative error code on failure.
Using devm_hwmon_device_register_with_info()
--------------------------------------------

2
Documentation/netlink/specs/rt-link.yaml
View File

@ -898,6 +898,8 @@ attribute-sets:
-
name: txqlen
type: u32
checks:
max: 32767
-
name: map
type: binary

44
Documentation/translations/zh_CN/arch/riscv/patch-acceptance.rst
View File

@ -15,19 +15,41 @@ arch/riscv 开发者维护指南
概述
----
RISC-V指令集体系结构是公开开发的：
RISC-V 指令集体系结构是公开开发的：
正在进行的草案可供所有人查看和测试实现。新模块或者扩展草案可能会在开发过程中发
生更改---有时以不兼容的方式对以前的草案进行更改。这种灵活性可能会给RISC-V Linux
维护者带来挑战。Linux开发过程更喜欢经过良好检查和测试的代码，而不是试验代码。我
们希望推广同样的规则到即将被内核合并的RISC-V相关代码。
生更改 --- 有时以不兼容的方式对以前的草案进行更改。这种灵活性可能会给 RISC-V
Linux 维护者带来挑战。Linux 维护者不赞成频繁的变更，且 Linux 开发过程更喜欢经过
良好检查和测试的代码，而不是试验代码。我们希望推广同样的规则到即将被内核合并的
RISC-V 相关代码。
Patchwork
---------
RISC-V 有一个 patchwork 实例，可以在那里查看补丁的状态：
https://patchwork.kernel.org/project/linux-riscv/list/
如果你的补丁不在默认视图中出现，那么 RISC-V 维护者很有可能已要求修改，或者希望
将其应用到另一个代码树上。
自动化流程会在该 patchwork 实例上运行，在每个补丁到达时立刻对其进行构建/测试。
自动化流程会根据补丁是否被识别为修复，选用 RISC-V `for-next` 或 `fixes` 分支
当前的 HEAD；若上述均应用失败，则使用 RISC-V `master` 分支。补丁系列被应用到的具
体提交将标注在 patchwork 上。任何检查未通过的补丁通常不会被应用，并且在大多数情
况下将需要重新提交。
附加的提交检查单
----------------
我们仅接受相关标准已经被RISC-V基金会标准为“已批准”或“已冻结”的扩展或模块的补丁。
（开发者当然可以维护自己的Linux内核树，其中包含所需代码扩展草案的代码。）
我们仅接受针对新模块或扩展的补丁，前提是这些模块或扩展的规范被列为未来不太可能发
生不兼容的变更。对于来自 RISC-V 基金会的规范，这意味着“已冻结”或“已批准”，对于
UEFI 论坛的规范，这意味着已发布的 ECR。（开发者当然可以维护自己的 Linux 内核树，
其中包含他们所需的任何扩展草案的代码。）
此外，RISC-V规范允许爱好者创建自己的自定义扩展。这些自定义拓展不需要通过RISC-V
基金会的任何审核或批准。为了避免将爱好者一些特别的RISC-V拓展添加进内核代码带来
的维护复杂性和对性能的潜在影响，我们将只接受RISC-V基金会正式冻结或批准的的扩展
补丁。（开发者当然可以维护自己的Linux内核树，其中包含他们想要的任何自定义扩展
的代码。）
此外，RISC-V 规范允许实现者创建自己的自定义扩展。这些自定义扩展不需要通过 RISC-V
基金会的任何审核或批准流程。为了避免因添加实现者特定的 RISC-V 扩展带来的维护复杂
性和对性能的潜在影响，我们将只考虑符合以下任一条件的扩展补丁：
- 已由 RISC-V 基金会正式冻结或批准
- 已按照标准 Linux 惯例，在广泛可用的硬件中实现
（实现者当然可以维护自己的 Linux 内核树，其中包含他们所需的任何自定义扩展的代码。）

22
MAINTAINERS
View File

@ -4512,7 +4512,7 @@ AYANEO PLATFORM EC DRIVER
M: Antheas Kapenekakis <lkml@antheas.dev>
L: platform-driver-x86@vger.kernel.org
S: Maintained
F: Documentation/ABI/testing/sysfs-platform-ayaneo
F: Documentation/ABI/testing/sysfs-platform-ayaneo-ec
F: drivers/platform/x86/ayaneo-ec.c
AZ6007 DVB DRIVER
@ -15020,6 +15020,8 @@ M: Madhavan Srinivasan <maddy@linux.ibm.com>
R: Michael Ellerman <mpe@ellerman.id.au>
R: Nicholas Piggin <npiggin@gmail.com>
R: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
R: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
R: Shrikanth Hegde <sshegde@linux.ibm.com>
L: linuxppc-dev@lists.ozlabs.org
S: Supported
W: https://github.com/linuxppc/wiki/wiki
@ -25334,8 +25336,8 @@ F: Documentation/networking/timestamping.rst
F: include/linux/net_tstamp.h
F: include/uapi/linux/net_tstamp.h
F: tools/testing/selftests/bpf/*/net_timestamping*
F: tools/testing/selftests/drivers/net/so_txtime.*
F: tools/testing/selftests/net/*timestamp*
F: tools/testing/selftests/net/so_txtime.c
SOEKRIS NET48XX LED SUPPORT
M: Chris Boot <bootc@bootc.net>
@ -28607,7 +28609,6 @@ L: iommu@lists.linux.dev
S: Maintained
F: Documentation/devicetree/bindings/iommu/verisilicon,iommu.yaml
F: drivers/iommu/vsi-iommu.c
F: include/linux/vsi-iommu.h
VF610 NAND DRIVER
M: Stefan Agner <stefan@agner.ch>
@ -28867,8 +28868,8 @@ F: include/uapi/linux/virtio_console.h
VIRTIO CORE
M: "Michael S. Tsirkin" <mst@redhat.com>
M: Jason Wang <jasowangio@gmail.com>
M: Eugenio Pérez <eperezma@redhat.com>
R: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
R: Eugenio Pérez <eperezma@redhat.com>
L: virtualization@lists.linux.dev
S: Maintained
F: Documentation/ABI/testing/sysfs-bus-vdpa
@ -28945,7 +28946,7 @@ F: include/uapi/linux/virtio_gpu.h
VIRTIO HOST (VHOST)
M: "Michael S. Tsirkin" <mst@redhat.com>
M: Jason Wang <jasowangio@gmail.com>
R: Eugenio Pérez <eperezma@redhat.com>
M: Eugenio Pérez <eperezma@redhat.com>
L: kvm@vger.kernel.org
L: virtualization@lists.linux.dev
L: netdev@vger.kernel.org
@ -29000,8 +29001,8 @@ F: include/uapi/linux/virtio_mem.h
VIRTIO NET DRIVER
M: "Michael S. Tsirkin" <mst@redhat.com>
M: Jason Wang <jasowangio@gmail.com>
M: Eugenio Pérez <eperezma@redhat.com>
R: Xuan Zhuo <xuanzhuo@linux.alibaba.com>
R: Eugenio Pérez <eperezma@redhat.com>
L: netdev@vger.kernel.org
L: virtualization@lists.linux.dev
S: Maintained
@ -29017,6 +29018,13 @@ S: Maintained
F: drivers/nvdimm/nd_virtio.c
F: drivers/nvdimm/virtio_pmem.c
VIRTIO RNG DRIVER
M: Laurent Vivier <lvivier@redhat.com>
L: virtualization@lists.linux.dev
S: Maintained
F: drivers/char/hw_random/virtio-rng.c
F: include/uapi/linux/virtio_rng.h
VIRTIO RTC DRIVER
M: Peter Hilber <peter.hilber@oss.qualcomm.com>
L: virtualization@lists.linux.dev
@ -29348,7 +29356,7 @@ M: Guenter Roeck <linux@roeck-us.net>
L: linux-watchdog@vger.kernel.org
S: Maintained
W: http://www.linux-watchdog.org/
T: git git://www.linux-watchdog.org/linux-watchdog.git
T: git git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git
F: Documentation/devicetree/bindings/watchdog/
F: Documentation/watchdog/
F: drivers/watchdog/

2
Makefile
View File

@ -2,7 +2,7 @@
VERSION = 7
PATCHLEVEL = 3
SUBLEVEL = 0
EXTRAVERSION = -rc2
EXTRAVERSION = -rc3
NAME = Baby Opossum Posse
# *DOCUMENTATION*

2
arch/arm64/include/asm/kvm_host.h
View File

@ -325,7 +325,7 @@ struct kvm_arch {
* Stage 2 paging state for VMs with nested S2 using a virtual
* VMID.
*/
struct kvm_s2_mmu *nested_mmus;
struct kvm_s2_mmu **nested_mmus;
size_t nested_mmus_size;
int nested_mmus_next;

3
arch/arm64/include/asm/kvm_nested.h
View File

@ -66,7 +66,8 @@ static inline u64 translate_ttbr0_el2_to_ttbr0_el1(u64 ttbr0)
extern bool forward_smc_trap(struct kvm_vcpu *vcpu);
extern bool forward_debug_exception(struct kvm_vcpu *vcpu);
extern void kvm_init_nested(struct kvm *kvm);
extern int kvm_init_nested(struct kvm *kvm);
extern void kvm_destroy_nested(struct kvm *kvm);
extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm_vcpu *vcpu);

3
arch/arm64/include/asm/kvm_pkvm.h
View File

@ -65,8 +65,7 @@ static inline bool kvm_pkvm_ioctl_allowed(struct kvm *kvm, unsigned int ioctl)
int r;
r = kvm_get_cap_for_kvm_ioctl(ioctl, &ext);
if (WARN_ON_ONCE(r < 0))
if (r < 0)
return false;
return kvm_pkvm_ext_allowed(kvm, ext);

8
arch/arm64/kvm/arm.c
View File

@ -236,8 +236,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
mutex_unlock(&kvm->lock);
#endif
kvm_init_nested(kvm);
ret = kvm_share_hyp(kvm, kvm + 1);
if (ret)
return ret;
@ -252,6 +250,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
if (ret)
goto err_free_cpumask;
ret = kvm_init_nested(kvm);
if (ret)
goto err_uninit_mmu;
if (is_protected_kvm_enabled()) {
/*
* If any failures occur after this is successful, make sure to
@ -280,6 +282,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
err_uninit_mmu:
kvm_uninit_stage2_mmu(kvm);
kvm_destroy_nested(kvm);
err_free_cpumask:
free_cpumask_var(kvm->arch.supported_cpus);
err_unshare_kvm:
@ -337,6 +340,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
kvm_unshare_hyp(kvm, kvm + 1);
kvm_destroy_nested(kvm);
kvm_arm_teardown_hypercalls(kvm);
}

5
arch/arm64/kvm/emulate-nested.c
View File

@ -1445,7 +1445,7 @@ static const struct encoding_to_trap_config encoding_to_fgt[] __initconst = {
SR_FGT(OP_AT_S1E1A, HFGITR, ATS1E1A, 1),
SR_FGT(OP_COSP_RCTX, HFGITR, COSPRCTX, 1),
SR_FGT(OP_GCSPUSHX, HFGITR, nGCSEPP, 0),
SR_FGT(OP_GCSPOPX, HFGITR, nGCSEPP, 0),
SR_FGT(OP_GCSPOPCX, HFGITR, nGCSEPP, 0),
SR_FGT(OP_GCSPUSHM, HFGITR, nGCSPUSHM_EL1, 0),
SR_FGT(OP_BRB_IALL, HFGITR, nBRBIALL, 0),
SR_FGT(OP_BRB_INJ, HFGITR, nBRBINJ, 0),
@ -2386,9 +2386,6 @@ int __init populate_nv_trap_config(void)
print_nv_trap_error(fgt, "FGT bit is reserved", ret);
}
if (!cpus_have_final_cap(ARM64_HAS_FGT))
continue;
prev = xa_store(&sr_forward_xa, enc,
xa_mk_value(tc.val), GFP_KERNEL);

2
arch/arm64/kvm/hyp/include/nvhe/mem_protect.h
View File

@ -52,8 +52,10 @@ int __pkvm_host_test_clear_young_guest(u64 gfn, u64 nr_pages, bool mkold, struct
int __pkvm_host_mkyoung_guest(u64 gfn, struct pkvm_hyp_vcpu *vcpu);
bool addr_is_memory(phys_addr_t phys);
bool addr_is_hyp_text(phys_addr_t phys);
int host_stage2_idmap_locked(phys_addr_t addr, u64 size, enum kvm_pgtable_prot prot);
int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id);
bool host_stage2_pte_is_hyp_owned(kvm_pte_t pte);
int kvm_host_prepare_stage2(void *pgt_pool_base);
int kvm_guest_prepare_stage2(struct pkvm_hyp_vm *vm, void *pgd);
void kvm_guest_destroy_stage2(struct pkvm_hyp_vm *vm);

1
arch/arm64/kvm/hyp/include/nvhe/mm.h
View File

@ -29,6 +29,7 @@ int __pkvm_create_private_mapping(phys_addr_t phys, size_t size,
enum kvm_pgtable_prot prot,
unsigned long *haddr);
int pkvm_create_stack(phys_addr_t phys, unsigned long *haddr);
int pkvm_check_host_ownership(void);
int pkvm_alloc_private_va_range(size_t size, unsigned long *haddr);
#endif /* __KVM_HYP_MM_H */

9
arch/arm64/kvm/hyp/include/nvhe/pkvm.h
View File

@ -12,6 +12,15 @@
#include <nvhe/gfp.h>
#include <nvhe/spinlock.h>
/*
* HCR_EL2 bits EL2 takes from the host on each entry, per VM type. The rest
* are EL2's own and nothing the host sets there reaches the guest.
*/
#define PKVM_HCR_EL2_HOST_PVM (HCR_EL2_TWI | HCR_EL2_TWE | HCR_EL2_VSE)
#define PKVM_HCR_EL2_HOST_NPVM (PKVM_HCR_EL2_HOST_PVM | HCR_EL2_VI | HCR_EL2_VF | \
HCR_EL2_TVM | HCR_EL2_TID2 | HCR_EL2_TID4 | \
HCR_EL2_TID5 | HCR_EL2_TTLBOS)
/*
* Holds the relevant data for maintaining the vcpu state completely at hyp.
*/

7
arch/arm64/kvm/hyp/nvhe/hyp-main.c
View File

@ -216,6 +216,7 @@ static void sync_debug_state(struct pkvm_hyp_vcpu *hyp_vcpu)
static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
{
struct kvm_vcpu *host_vcpu = hyp_vcpu->host_vcpu;
u64 host_hcr_mask = PKVM_HCR_EL2_HOST_PVM;
fpsimd_sve_flush();
flush_debug_state(hyp_vcpu);
@ -228,6 +229,7 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
if (!pkvm_hyp_vcpu_is_protected(hyp_vcpu)) {
if (vcpu_get_flag(host_vcpu, PKVM_HOST_STATE_DIRTY))
flush_hyp_vcpu_state(hyp_vcpu);
host_hcr_mask = PKVM_HCR_EL2_HOST_NPVM;
} else {
hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt;
}
@ -241,9 +243,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
* trap-control bit, so it must flow to the hyp vCPU alongside TWI/TWE
* for the vSError to be delivered. sync_hyp_vcpu() reflects it back.
*/
hyp_vcpu->vcpu.arch.hcr_el2 &= ~(HCR_TWI | HCR_TWE | HCR_VSE);
hyp_vcpu->vcpu.arch.hcr_el2 |= READ_ONCE(host_vcpu->arch.hcr_el2) &
(HCR_TWI | HCR_TWE | HCR_VSE);
hyp_vcpu->vcpu.arch.hcr_el2 &= ~host_hcr_mask;
hyp_vcpu->vcpu.arch.hcr_el2 |= READ_ONCE(host_vcpu->arch.hcr_el2) & host_hcr_mask;
hyp_vcpu->vcpu.arch.iflags = host_vcpu->arch.iflags;

20
arch/arm64/kvm/hyp/nvhe/mem_protect.c
View File

@ -450,6 +450,14 @@ bool addr_is_memory(phys_addr_t phys)
return !!find_mem_range(phys, &range);
}
bool addr_is_hyp_text(phys_addr_t phys)
{
phys_addr_t start = ALIGN_DOWN(__hyp_pa(__hyp_text_start), PAGE_SIZE);
phys_addr_t end = PAGE_ALIGN(__hyp_pa(__hyp_text_end));
return phys >= start && phys < end;
}
static bool is_in_mem_range(u64 addr, struct kvm_mem_range *range)
{
return range->start <= addr && addr < range->end;
@ -633,6 +641,18 @@ int host_stage2_set_owner_locked(phys_addr_t addr, u64 size, u8 owner_id)
return ret;
}
bool host_stage2_pte_is_hyp_owned(kvm_pte_t pte)
{
if (kvm_pte_valid(pte))
return false;
if (FIELD_GET(KVM_INVALID_PTE_TYPE_MASK, pte) !=
KVM_HOST_INVALID_PTE_TYPE_DONATION)
return false;
return FIELD_GET(KVM_HOST_DONATION_PTE_OWNER_MASK, pte) == PKVM_ID_HYP;
}
#define KVM_HOST_PTE_OWNER_GUEST_HANDLE_MASK GENMASK(15, 0)
/* We need 40 bits for the GFN to cover a 52-bit IPA with 4k pages and LPA2 */
#define KVM_HOST_PTE_OWNER_GUEST_GFN_MASK GENMASK(55, 16)

79
arch/arm64/kvm/hyp/nvhe/mm.c
View File

@ -25,6 +25,7 @@ struct memblock_region hyp_memory[HYP_MEMBLOCK_REGIONS];
unsigned int hyp_memblock_nr;
static u64 __io_map_base;
static u64 __io_map_next;
struct hyp_fixmap_slot {
u64 addr;
@ -50,7 +51,7 @@ static int __pkvm_alloc_private_va_range(unsigned long start, size_t size)
hyp_assert_lock_held(&pkvm_pgd_lock);
if (!start || start < __io_map_base)
if (!start || start < __io_map_next)
return -EINVAL;
/* The allocated size is always a multiple of PAGE_SIZE */
@ -60,7 +61,7 @@ static int __pkvm_alloc_private_va_range(unsigned long start, size_t size)
if (cur > __hyp_vmemmap)
return -ENOMEM;
__io_map_base = cur;
__io_map_next = cur;
return 0;
}
@ -70,7 +71,7 @@ static int __pkvm_alloc_private_va_range(unsigned long start, size_t size)
* @size: The size of the VA range to reserve.
* @haddr: The hypervisor virtual start address of the allocation.
*
* The private virtual address (VA) range is allocated above __io_map_base
* The private virtual address (VA) range is allocated above __io_map_next
* and aligned based on the order of @size.
*
* Return: 0 on success or negative error code on failure.
@ -81,7 +82,7 @@ int pkvm_alloc_private_va_range(size_t size, unsigned long *haddr)
int ret;
hyp_spin_lock(&pkvm_pgd_lock);
addr = __io_map_base;
addr = __io_map_next;
ret = __pkvm_alloc_private_va_range(addr, size);
hyp_spin_unlock(&pkvm_pgd_lock);
@ -341,7 +342,7 @@ static int create_fixblock(void)
return -EINVAL;
hyp_spin_lock(&pkvm_pgd_lock);
addr = ALIGN(__io_map_base, PMD_SIZE);
addr = ALIGN(__io_map_next, PMD_SIZE);
ret = __pkvm_alloc_private_va_range(addr, PMD_SIZE);
if (ret)
goto unlock;
@ -426,6 +427,7 @@ int hyp_create_idmap(u32 hyp_va_bits)
*/
__io_map_base = start & BIT(hyp_va_bits - 2);
__io_map_base ^= BIT(hyp_va_bits - 2);
__io_map_next = __io_map_base;
__hyp_vmemmap = __io_map_base | BIT(hyp_va_bits - 3);
return __pkvm_create_mappings(start, end - start, start, PAGE_HYP_EXEC);
@ -433,19 +435,19 @@ int hyp_create_idmap(u32 hyp_va_bits)
int pkvm_create_stack(phys_addr_t phys, unsigned long *haddr)
{
unsigned long addr, prev_base;
unsigned long addr, prev_next;
size_t size;
int ret;
hyp_spin_lock(&pkvm_pgd_lock);
prev_base = __io_map_base;
prev_next = __io_map_next;
/*
* Efficient stack verification using the NVHE_STACK_SHIFT bit implies
* an alignment of our allocation on the order of the size.
*/
size = NVHE_STACK_SIZE * 2;
addr = ALIGN(__io_map_base, size);
addr = ALIGN(__io_map_next, size);
ret = __pkvm_alloc_private_va_range(addr, size);
if (!ret) {
@ -461,7 +463,7 @@ int pkvm_create_stack(phys_addr_t phys, unsigned long *haddr)
ret = kvm_pgtable_hyp_map(&pkvm_pgtable, addr + NVHE_STACK_SIZE,
NVHE_STACK_SIZE, phys, PAGE_HYP);
if (ret)
__io_map_base = prev_base;
__io_map_next = prev_next;
}
hyp_spin_unlock(&pkvm_pgd_lock);
@ -470,6 +472,65 @@ int pkvm_create_stack(phys_addr_t phys, unsigned long *haddr)
return ret;
}
static int check_page_ownership(phys_addr_t phys)
{
kvm_pte_t pte;
bool host_ok;
int ret;
if (addr_is_memory(phys)) {
struct hyp_page *page = hyp_phys_to_page(phys);
if (get_hyp_state(page) != PKVM_PAGE_OWNED ||
get_host_state(page) != PKVM_NOPAGE)
return -EPERM;
}
ret = kvm_pgtable_get_leaf(&host_mmu.pgt, phys, &pte, NULL);
if (ret)
return ret;
/* Hyp text may stay host-readable, see fix_host_ownership_walker(). */
if (kvm_pte_valid(pte) && addr_is_hyp_text(phys))
host_ok = !(kvm_pgtable_stage2_pte_prot(pte) & KVM_PGTABLE_PROT_W);
else
host_ok = host_stage2_pte_is_hyp_owned(pte);
return host_ok ? 0 : -EPERM;
}
static int check_host_ownership_walker(const struct kvm_pgtable_visit_ctx *ctx,
enum kvm_pgtable_walk_flags visit)
{
phys_addr_t phys, end;
int ret;
if (!kvm_pte_valid(ctx->old))
return 0;
phys = kvm_pte_to_phys(ctx->old);
end = phys + kvm_granule_size(ctx->level);
for (; phys < end; phys += PAGE_SIZE) {
ret = check_page_ownership(phys);
if (ret)
return ret;
}
return 0;
}
int pkvm_check_host_ownership(void)
{
struct kvm_pgtable_walker walker = {
.cb = check_host_ownership_walker,
.flags = KVM_PGTABLE_WALK_LEAF,
};
/* The private range and the vmemmap share one quarter of the VA space. */
return kvm_pgtable_walk(&pkvm_pgtable, __io_map_base,
BIT(pkvm_pgtable.ia_bits - 2), &walker);
}
static void *admit_host_page(void *arg)
{
struct kvm_hyp_memcache *host_mc = arg;

54
arch/arm64/kvm/hyp/nvhe/pkvm.c
View File

@ -30,6 +30,7 @@ unsigned int kvm_host_sve_max_vl;
*/
static DEFINE_PER_CPU(struct pkvm_hyp_vcpu *, loaded_hyp_vcpu);
/* The PKVM_HCR_EL2_HOST_{PVM,NPVM} bits of this value come from the host on each entry. */
static void pkvm_vcpu_reset_hcr(struct kvm_vcpu *vcpu)
{
vcpu->arch.hcr_el2 = HCR_GUEST_FLAGS;
@ -47,18 +48,17 @@ static void pkvm_vcpu_reset_hcr(struct kvm_vcpu *vcpu)
if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
vcpu->arch.hcr_el2 |= HCR_FWB;
if (cpus_have_final_cap(ARM64_HAS_EVT) &&
!cpus_have_final_cap(ARM64_MISMATCHED_CACHE_TYPE) &&
kvm_read_vm_id_reg(vcpu->kvm, SYS_CTR_EL0) == read_cpuid(CTR_EL0))
vcpu->arch.hcr_el2 |= HCR_TID4;
else
vcpu->arch.hcr_el2 |= HCR_TID2;
/*
* Without AArch32 EL1, leave RW set and let the entry fail with an
* illegal exception return: the *32_EL2 registers EL2 would otherwise
* switch are UNDEFINED there.
*/
if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT) &&
cpus_have_final_cap(ARM64_HAS_32BIT_EL1))
vcpu->arch.hcr_el2 &= ~HCR_EL2_RW;
if (vcpu_has_ptrauth(vcpu))
vcpu->arch.hcr_el2 |= (HCR_API | HCR_APK);
if (kvm_has_mte(vcpu->kvm))
vcpu->arch.hcr_el2 |= HCR_ATA;
}
static void pvm_init_traps_hcr(struct kvm_vcpu *vcpu)
@ -76,6 +76,13 @@ static void pvm_init_traps_hcr(struct kvm_vcpu *vcpu)
*/
val |= HCR_TACR | HCR_TIDCP | HCR_TID3 | HCR_TID1;
if (cpus_have_final_cap(ARM64_HAS_EVT) &&
!cpus_have_final_cap(ARM64_MISMATCHED_CACHE_TYPE) &&
kvm_read_vm_id_reg(kvm, SYS_CTR_EL0) == read_cpuid(CTR_EL0))
val |= HCR_EL2_TID4;
else
val |= HCR_EL2_TID2;
if (!kvm_has_feat(kvm, ID_AA64PFR0_EL1, RAS, IMP)) {
val |= HCR_TERR | HCR_TEA;
val &= ~(HCR_FIEN);
@ -360,7 +367,7 @@ static void pkvm_init_features_from_host(struct pkvm_hyp_vm *hyp_vm, const struc
if (test_bit(KVM_ARCH_FLAG_WRITABLE_IMP_ID_REGS, &host_arch_flags))
hyp_vm->kvm.arch.midr_el1 = host_kvm->arch.midr_el1;
return;
goto out;
}
if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_MTE))
@ -379,13 +386,14 @@ static void pkvm_init_features_from_host(struct pkvm_hyp_vm *hyp_vm, const struc
if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_PTRAUTH_GENERIC))
set_bit(KVM_ARM_VCPU_PTRAUTH_GENERIC, allowed_features);
if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_SVE)) {
if (kvm_pkvm_ext_allowed(kvm, KVM_CAP_ARM_SVE))
set_bit(KVM_ARM_VCPU_SVE, allowed_features);
kvm->arch.flags |= host_arch_flags & BIT(KVM_ARCH_FLAG_GUEST_HAS_SVE);
}
bitmap_and(kvm->arch.vcpu_features, host_kvm->arch.vcpu_features,
allowed_features, KVM_VCPU_MAX_FEATURES);
out:
__assign_bit(KVM_ARCH_FLAG_GUEST_HAS_SVE, &kvm->arch.flags,
kvm_vcpu_has_feature(kvm, KVM_ARM_VCPU_SVE));
}
static void unpin_host_vcpu(struct kvm_vcpu *host_vcpu)
@ -398,10 +406,10 @@ static void unpin_host_sve_state(struct pkvm_hyp_vcpu *hyp_vcpu)
{
void *sve_state;
if (!vcpu_has_feature(&hyp_vcpu->vcpu, KVM_ARM_VCPU_SVE))
sve_state = hyp_vcpu->vcpu.arch.sve_state;
if (!sve_state)
return;
sve_state = hyp_vcpu->vcpu.arch.sve_state;
hyp_unpin_shared_mem(sve_state,
sve_state + vcpu_sve_state_size(&hyp_vcpu->vcpu));
}
@ -450,7 +458,7 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_vcpu *h
unsigned int sve_max_vl;
size_t sve_state_size;
void *sve_state;
int ret = 0;
int ret;
if (!vcpu_has_feature(vcpu, KVM_ARM_VCPU_SVE)) {
vcpu_clear_flag(vcpu, VCPU_SVE_FINALIZED);
@ -459,25 +467,21 @@ static int pkvm_vcpu_init_sve(struct pkvm_hyp_vcpu *hyp_vcpu, struct kvm_vcpu *h
/* Limit guest vector length to the maximum supported by the host. */
sve_max_vl = min(READ_ONCE(host_vcpu->arch.sve_max_vl), kvm_host_sve_max_vl);
sve_state_size = sve_state_size_from_vl(sve_max_vl);
sve_state = kern_hyp_va(READ_ONCE(host_vcpu->arch.sve_state));
if (!sve_state || !sve_state_size) {
ret = -EINVAL;
goto err;
}
if (!sve_vl_valid(sve_max_vl) || !sve_state)
return -EINVAL;
sve_state_size = sve_state_size_from_vl(sve_max_vl);
ret = hyp_pin_shared_mem(sve_state, sve_state + sve_state_size);
if (ret)
goto err;
return ret;
vcpu->arch.sve_state = sve_state;
vcpu->arch.sve_max_vl = sve_max_vl;
return 0;
err:
clear_bit(KVM_ARM_VCPU_SVE, vcpu->kvm->arch.vcpu_features);
return ret;
}
static int vm_copy_id_regs(struct pkvm_hyp_vcpu *hyp_vcpu)

16
arch/arm64/kvm/hyp/nvhe/setup.c
View File

@ -217,7 +217,7 @@ static int fix_host_ownership_walker(const struct kvm_pgtable_visit_ctx *ctx,
case PKVM_PAGE_OWNED:
set_hyp_state(page, PKVM_PAGE_OWNED);
/* hyp text is RO in the host stage-2 to be inspected on panic. */
if (prot == PAGE_HYP_EXEC) {
if (addr_is_hyp_text(phys)) {
set_host_state(page, PKVM_NOPAGE);
return host_stage2_idmap_locked(phys, PAGE_SIZE, KVM_PGTABLE_PROT_R);
} else {
@ -269,6 +269,16 @@ static int fix_host_ownership(void)
return ret;
}
/* The stacks sit in the private VA range, not the linear map. */
for (i = 0; i < hyp_nr_cpus; i++) {
struct kvm_nvhe_init_params *params = per_cpu_ptr(&kvm_init_params, i);
u64 start = params->stack_hyp_va - NVHE_STACK_SIZE;
ret = kvm_pgtable_walk(&pkvm_pgtable, start, NVHE_STACK_SIZE, &walker);
if (ret)
return ret;
}
return 0;
}
@ -324,6 +334,10 @@ void __noreturn __pkvm_init_finalise(void)
if (ret)
goto out;
ret = pkvm_check_host_ownership();
if (ret)
goto out;
ret = hyp_ffa_init(ffa_proxy_pages);
if (ret)
goto out;

3
arch/arm64/kvm/hypercalls.c
View File

@ -185,7 +185,8 @@ static int kvm_smccc_set_filter(struct kvm *kvm, struct kvm_smccc_filter __user
start = filter.base;
end = start + filter.nr_functions - 1;
if (end < start || filter.action >= NR_SMCCC_FILTER_ACTIONS)
if (!filter.nr_functions || end < start ||
filter.action >= NR_SMCCC_FILTER_ACTIONS)
return -EINVAL;
mutex_lock(&kvm->arch.config_lock);

62
arch/arm64/kvm/mmu.c
View File

@ -59,27 +59,36 @@ static phys_addr_t stage2_range_addr_end(phys_addr_t addr, phys_addr_t end)
* long will also starve other vCPUs. We have to also make sure that the page
* tables are not freed while we released the lock.
*/
static int stage2_apply_range(struct kvm_s2_mmu *mmu, phys_addr_t addr,
static int stage2_apply_range(struct kvm_s2_mmu *mmu, phys_addr_t start,
phys_addr_t end,
int (*fn)(struct kvm_pgtable *, u64, u64),
bool resched)
{
struct kvm *kvm = kvm_s2_mmu_to_kvm(mmu);
bool lock_dropped = false;
phys_addr_t addr = start;
int ret;
u64 next;
do {
struct kvm_pgtable *pgt = mmu->pgt;
/*
* We may be raced on PGT teardown when we release the
* kvm->mmu_lock. That's fine as the PGT is legitimately no
* longer present.
*/
if (!pgt)
return -EINVAL;
return lock_dropped ? 0 : -EINVAL;
next = stage2_range_addr_end(addr, end);
ret = fn(pgt, addr, next - addr);
if (ret)
break;
if (resched && next != end)
if (resched && next != end) {
cond_resched_rwlock_write(&kvm->mmu_lock);
lock_dropped = true;
}
} while (addr = next, addr != end);
return ret;
@ -1459,32 +1468,11 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
return PAGE_SIZE;
}
static int get_vma_page_shift(struct vm_area_struct *vma, unsigned long hva)
static int get_vma_page_shift(struct vm_area_struct *vma)
{
unsigned long pa;
if (is_vm_hugetlb_page(vma) && !(vma->vm_flags & VM_PFNMAP))
if (is_vm_hugetlb_page(vma))
return huge_page_shift(hstate_vma(vma));
if (!(vma->vm_flags & VM_PFNMAP))
return PAGE_SHIFT;
VM_BUG_ON(is_vm_hugetlb_page(vma));
pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start);
#ifndef __PAGETABLE_PMD_FOLDED
if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) &&
ALIGN_DOWN(hva, PUD_SIZE) >= vma->vm_start &&
ALIGN(hva, PUD_SIZE) <= vma->vm_end)
return PUD_SHIFT;
#endif
if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) &&
ALIGN_DOWN(hva, PMD_SIZE) >= vma->vm_start &&
ALIGN(hva, PMD_SIZE) <= vma->vm_end)
return PMD_SHIFT;
return PAGE_SHIFT;
}
@ -1785,7 +1773,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd,
vma_shift = PAGE_SHIFT;
} else {
s2vi->max_map_size = PUD_SIZE;
vma_shift = get_vma_page_shift(vma, s2fd->hva);
vma_shift = get_vma_page_shift(vma);
}
switch (vma_shift) {
@ -1943,16 +1931,6 @@ static int kvm_s2_fault_pin_pfn(const struct kvm_s2_fault_desc *s2fd,
return -EFAULT;
}
} else {
/*
* If the page was identified as device early by looking at
* the VMA flags, vma_pagesize is already representing the
* largest quantity we can map. If instead it was mapped
* via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE
* and must not be upgraded.
*
* In both cases, we don't let transparent_hugepage_adjust()
* change things at the last minute.
*/
s2vi->map_non_cacheable = true;
}
@ -2042,10 +2020,10 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault_desc *s2fd,
/*
* If we are not forced to use page mapping, check if we are
* backed by a THP and thus use block mapping if possible.
* backed by a huge stage-1 mapping and thus use block mapping if
* possible.
*/
if (mapping_size == PAGE_SIZE &&
!(s2vi->max_map_size == PAGE_SIZE || s2vi->map_non_cacheable)) {
if (mapping_size == PAGE_SIZE && s2vi->max_map_size != PAGE_SIZE) {
if (perm_fault_granule > PAGE_SIZE) {
mapping_size = perm_fault_granule;
} else {
@ -2126,10 +2104,6 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
return ret;
}
/*
* Let's check if we will get back a huge page backed by hugetlbfs, or
* get block mapping for device MMIO region.
*/
ret = kvm_s2_fault_pin_pfn(s2fd, &s2vi);
if (ret != 1)
return ret;

115
arch/arm64/kvm/nested.c
View File

@ -45,11 +45,24 @@ struct vncr_tlb {
*/
#define S2_MMU_PER_VCPU 2
void kvm_init_nested(struct kvm *kvm)
int kvm_init_nested(struct kvm *kvm)
{
kvm->arch.nested_mmus = NULL;
kvm->arch.nested_mmus = kvmalloc_objs(struct kvm_s2_mmu *,
KVM_MAX_VCPUS * S2_MMU_PER_VCPU,
GFP_KERNEL_ACCOUNT);
kvm->arch.nested_mmus_size = 0;
atomic_set(&kvm->arch.vncr_tlb_count, 0);
return kvm->arch.nested_mmus ? 0 : -ENOMEM;
}
void kvm_destroy_nested(struct kvm *kvm)
{
for (int i = 0; i < kvm->arch.nested_mmus_size; i+= S2_MMU_PER_VCPU)
kvfree(kvm->arch.nested_mmus[i]);
kvm->arch.nested_mmus_size = 0;
kvfree(kvm->arch.nested_mmus);
}
static int init_nested_s2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
@ -70,8 +83,9 @@ static int init_nested_s2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
{
struct kvm *kvm = vcpu->kvm;
struct kvm_s2_mmu *tmp;
int num_mmus, ret = 0;
int num_mmus;
lockdep_assert_held(&kvm->arch.config_lock);
if (test_bit(KVM_ARM_VCPU_HAS_EL2_E2H0, kvm->arch.vcpu_features) &&
!cpus_have_final_cap(ARM64_HAS_HCR_NV1))
@ -84,51 +98,40 @@ int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu)
if (!vcpu->arch.ctxt.vncr_array)
return -ENOMEM;
/*
* Let's treat memory allocation failures as benign: If we fail to
* allocate anything, return an error and keep the allocated array
* alive. Userspace may try to recover by initializing the vcpu
* again, and there is no reason to affect the whole VM for this.
*/
num_mmus = atomic_read(&kvm->online_vcpus) * S2_MMU_PER_VCPU;
if (num_mmus > kvm->arch.nested_mmus_size) {
tmp = kvzalloc_objs(*tmp, num_mmus, GFP_KERNEL_ACCOUNT);
struct kvm_s2_mmu *tmp;
int i, ret = 0;
tmp = kvzalloc_objs(*tmp, S2_MMU_PER_VCPU, GFP_KERNEL_ACCOUNT);
if (!tmp)
return -ENOMEM;
ret = -ENOMEM;
write_lock(&kvm->mmu_lock);
if (kvm->arch.nested_mmus_size) {
memcpy(tmp, kvm->arch.nested_mmus,
size_mul(sizeof(*tmp), kvm->arch.nested_mmus_size));
for (int i = 0; i < kvm->arch.nested_mmus_size; i++)
tmp[i].pgt->mmu = &tmp[i];
for (i = 0; !ret && i < S2_MMU_PER_VCPU; i++) {
ret = init_nested_s2_mmu(kvm, &tmp[i]);
if (ret)
break;
}
swap(kvm->arch.nested_mmus, tmp);
if (ret) {
while (--i >= 0)
kvm_free_stage2_pgd(&tmp[i]);
write_unlock(&kvm->mmu_lock);
kvfree(tmp);
free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
vcpu->arch.ctxt.vncr_array = NULL;
return ret;
}
kvfree(tmp);
guard(write_lock)(&kvm->mmu_lock);
for (i = 0; i < S2_MMU_PER_VCPU; i++)
kvm->arch.nested_mmus[i + kvm->arch.nested_mmus_size] = &tmp[i];
kvm->arch.nested_mmus_size += S2_MMU_PER_VCPU;
}
for (int i = kvm->arch.nested_mmus_size; !ret && i < num_mmus; i++)
ret = init_nested_s2_mmu(kvm, &kvm->arch.nested_mmus[i]);
if (ret) {
for (int i = kvm->arch.nested_mmus_size; i < num_mmus; i++)
kvm_free_stage2_pgd(&kvm->arch.nested_mmus[i]);
free_page((unsigned long)vcpu->arch.ctxt.vncr_array);
vcpu->arch.ctxt.vncr_array = NULL;
return ret;
}
kvm->arch.nested_mmus_size = num_mmus;
return 0;
}
@ -742,7 +745,7 @@ void kvm_s2_mmu_iterate_by_vmid(struct kvm *kvm, u16 vmid,
write_lock(&kvm->mmu_lock);
for (int i = 0; i < kvm->arch.nested_mmus_size; i++) {
struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
struct kvm_s2_mmu *mmu = kvm->arch.nested_mmus[i];
if (!kvm_s2_mmu_valid(mmu))
continue;
@ -784,7 +787,7 @@ struct kvm_s2_mmu *lookup_s2_mmu(struct kvm_vcpu *vcpu)
* if S2 translation is disabled.
*/
for (int i = 0; i < kvm->arch.nested_mmus_size; i++) {
struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
struct kvm_s2_mmu *mmu = kvm->arch.nested_mmus[i];
if (!kvm_s2_mmu_valid(mmu))
continue;
@ -823,7 +826,7 @@ static struct kvm_s2_mmu *get_s2_mmu_nested(struct kvm_vcpu *vcpu)
for (i = kvm->arch.nested_mmus_next;
i < (kvm->arch.nested_mmus_size + kvm->arch.nested_mmus_next);
i++) {
s2_mmu = &kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
s2_mmu = kvm->arch.nested_mmus[i % kvm->arch.nested_mmus_size];
if (atomic_read(&s2_mmu->refcnt) == 0)
break;
@ -1260,6 +1263,17 @@ void kvm_handle_s1e2_tlbi(struct kvm_vcpu *vcpu, u32 inst, u64 val)
invalidate_vncr_va(vcpu->kvm, &scope);
}
static void kvm_invalidate_vncr_ipa_all(struct kvm *kvm)
{
struct kvm_pgtable *pgt = kvm->arch.mmu.pgt;
lockdep_assert_held_write(&kvm->mmu_lock);
/* if the mmu lock was dropped, pgt teardown may have raced. */
if (pgt)
kvm_invalidate_vncr_ipa(kvm, 0, BIT(pgt->ia_bits));
}
void kvm_nested_s2_wp(struct kvm *kvm)
{
int i;
@ -1270,13 +1284,13 @@ void kvm_nested_s2_wp(struct kvm *kvm)
return;
for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
struct kvm_s2_mmu *mmu = kvm->arch.nested_mmus[i];
if (kvm_s2_mmu_valid(mmu))
kvm_stage2_wp_range(mmu, 0, kvm_phys_size(mmu));
}
kvm_invalidate_vncr_ipa(kvm, 0, BIT(kvm->arch.mmu.pgt->ia_bits));
kvm_invalidate_vncr_ipa_all(kvm);
}
void kvm_nested_s2_unmap(struct kvm *kvm, bool may_block)
@ -1289,13 +1303,13 @@ void kvm_nested_s2_unmap(struct kvm *kvm, bool may_block)
return;
for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
struct kvm_s2_mmu *mmu = kvm->arch.nested_mmus[i];
if (kvm_s2_mmu_valid(mmu))
kvm_stage2_unmap_range(mmu, 0, kvm_phys_size(mmu), may_block);
}
kvm_invalidate_vncr_ipa(kvm, 0, BIT(kvm->arch.mmu.pgt->ia_bits));
kvm_invalidate_vncr_ipa_all(kvm);
}
void kvm_nested_s2_flush(struct kvm *kvm)
@ -1308,7 +1322,7 @@ void kvm_nested_s2_flush(struct kvm *kvm)
return;
for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
struct kvm_s2_mmu *mmu = kvm->arch.nested_mmus[i];
if (kvm_s2_mmu_valid(mmu))
kvm_stage2_flush_range(mmu, 0, kvm_phys_size(mmu));
@ -1317,17 +1331,12 @@ void kvm_nested_s2_flush(struct kvm *kvm)
void kvm_arch_flush_shadow_all(struct kvm *kvm)
{
int i;
for (i = 0; i < kvm->arch.nested_mmus_size; i++) {
struct kvm_s2_mmu *mmu = &kvm->arch.nested_mmus[i];
for (int i = 0; i < kvm->arch.nested_mmus_size; i++) {
struct kvm_s2_mmu *mmu = kvm->arch.nested_mmus[i];
if (!WARN_ON(atomic_read(&mmu->refcnt)))
kvm_free_stage2_pgd(mmu);
}
kvfree(kvm->arch.nested_mmus);
kvm->arch.nested_mmus = NULL;
kvm->arch.nested_mmus_size = 0;
kvm_uninit_stage2_mmu(kvm);
}

1
arch/arm64/kvm/sys_regs.c
View File

@ -926,6 +926,7 @@ static u64 *demux_wb_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd)
struct kvm_guest_debug_arch *dbg = &vcpu->arch.vcpu_debug_state;
switch (rd->Op2) {
case 0b001:
case 0b100:
return &dbg->dbg_bvr[rd->CRm];
case 0b101:

58
arch/arm64/kvm/vgic/vgic-its.c
View File

@ -319,12 +319,16 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
return ret;
}
static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
static int update_affinity(struct vgic_irq *irq,
struct kvm_vcpu *from_vcpu, struct kvm_vcpu *vcpu)
{
struct its_vlpi_map map;
int ret;
guard(raw_spinlock_irqsave)(&irq->irq_lock);
if (from_vcpu && irq->target_vcpu != from_vcpu)
return 0;
irq->target_vcpu = vcpu;
if (!irq->hw)
@ -362,7 +366,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
return;
vcpu = collection_to_vcpu(kvm, ite->collection);
update_affinity(ite->irq, vcpu);
update_affinity(ite->irq, NULL, vcpu);
}
/*
@ -856,7 +860,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
vgic_its_invalidate_cache(its);
return update_affinity(ite->irq, vcpu);
return update_affinity(ite->irq, NULL, vcpu);
}
static bool __is_visible_gfn_locked(struct vgic_its *its, gpa_t gpa)
@ -1383,7 +1387,7 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
if (!irq)
continue;
update_affinity(irq, vcpu2);
update_affinity(irq, vcpu1, vcpu2);
vgic_put_irq(kvm, irq);
}
@ -1658,7 +1662,7 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
unsigned long val)
{
const struct vgic_its_abi *abi = vgic_its_get_abi(its);
u64 entry_size, table_type;
u64 old, entry_size, table_type;
u64 reg, *regptr, clearbits = 0;
/* When GITS_CTLR.Enable is 1, we ignore write accesses. */
@ -1681,7 +1685,9 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
return;
}
reg = update_64bit_reg(*regptr, addr & 7, len, val);
old = *regptr;
reg = update_64bit_reg(old, addr & 7, len, val);
reg &= ~GITS_BASER_RO_MASK;
reg &= ~clearbits;
@ -1691,7 +1697,8 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
*regptr = reg;
if (!(reg & GITS_BASER_VALID)) {
/* The ITS driver rewrites an unchanged GITS_BASER<n> on resume. */
if (reg != old) {
/* Take the its_lock to prevent a race with a save/restore */
mutex_lock(&its->its_lock);
switch (table_type) {
@ -1702,6 +1709,8 @@ static void vgic_mmio_write_its_baser(struct kvm *kvm,
vgic_its_free_collection_list(kvm, its);
break;
}
/* A concurrent injection may have cached a translation. */
vgic_its_invalidate_cache(its);
mutex_unlock(&its->its_lock);
}
}
@ -2019,18 +2028,22 @@ static int vgic_its_attr_regs_access(struct kvm_device *dev,
return ret;
}
static u32 compute_next_devid_offset(struct list_head *h,
static u32 compute_next_devid_offset(struct vgic_its *its, u64 baser,
struct its_device *dev)
{
struct its_device *next;
u32 next_offset;
struct its_device *next = dev;
if (list_is_last(&dev->dev_list, h))
return 0;
next = list_next_entry(dev, dev_list);
next_offset = next->device_id - dev->device_id;
/*
* Point at the next device vgic_its_save_device_tables() saves. It
* sorts device_list first, so the subtraction cannot underflow.
*/
list_for_each_entry_continue(next, &its->device_list, dev_list) {
if (vgic_its_check_id(its, baser, next->device_id, NULL))
return min_t(u32, next->device_id - dev->device_id,
VITS_DTE_MAX_DEVID_OFFSET);
}
return min_t(u32, next_offset, VITS_DTE_MAX_DEVID_OFFSET);
return 0;
}
static u32 compute_next_eventid_offset(struct list_head *h, struct its_ite *ite)
@ -2271,17 +2284,18 @@ static int vgic_its_restore_itt(struct vgic_its *its, struct its_device *dev)
* vgic_its_save_dte - Save a device table entry at a given GPA
*
* @its: ITS handle
* @baser: GITS_BASER<dev> the caller is saving against
* @dev: ITS device
* @ptr: GPA
*/
static int vgic_its_save_dte(struct vgic_its *its, struct its_device *dev,
gpa_t ptr)
static int vgic_its_save_dte(struct vgic_its *its, u64 baser,
struct its_device *dev, gpa_t ptr)
{
u64 val, itt_addr_field;
u32 next_offset;
itt_addr_field = dev->itt_addr >> 8;
next_offset = compute_next_devid_offset(&its->device_list, dev);
next_offset = compute_next_devid_offset(its, baser, dev);
val = (1ULL << KVM_ITS_DTE_VALID_SHIFT |
((u64)next_offset << KVM_ITS_DTE_NEXT_SHIFT) |
(itt_addr_field << KVM_ITS_DTE_ITTADDR_SHIFT) |
@ -2380,15 +2394,16 @@ static int vgic_its_save_device_tables(struct vgic_its *its)
int ret;
gpa_t eaddr;
/* Don't fail a save that userspace must be able to issue. */
if (!vgic_its_check_id(its, baser,
dev->device_id, &eaddr))
return -EINVAL;
continue;
ret = vgic_its_save_itt(its, dev);
if (ret)
return ret;
ret = vgic_its_save_dte(its, dev, eaddr);
ret = vgic_its_save_dte(its, baser, dev, eaddr);
if (ret)
return ret;
}
@ -2541,9 +2556,6 @@ static int vgic_its_save_collection_table(struct vgic_its *its)
max_size = GITS_BASER_NR_PAGES(baser) * SZ_64K;
list_for_each_entry(collection, &its->collection_list, coll_list) {
if (!vgic_its_check_id(its, baser, collection->collection_id, NULL))
return -EINVAL;
ret = vgic_its_save_cte(its, collection, gpa);
if (ret)
return ret;

6
arch/arm64/kvm/vgic/vgic-v2.c
View File

@ -122,6 +122,10 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)
for (int lr = 0; lr < vgic_cpu->vgic_v2.used_lrs; lr++)
vgic_v2_fold_lr(vcpu, cpuif->vgic_lr[lr]);
cpuif->used_lrs = 0;
if (!irq)
return;
/* See the GICv3 equivalent for the EOIcount handling rationale */
list_for_each_entry_continue(irq, &vgic_cpu->ap_list_head, ap_list) {
u32 lr;
@ -144,8 +148,6 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)
vgic_v2_fold_lr(vcpu, lr);
eoicount--;
}
cpuif->used_lrs = 0;
}
void vgic_v2_deactivate(struct kvm_vcpu *vcpu, u32 val)

6
arch/arm64/kvm/vgic/vgic-v3.c
View File

@ -155,6 +155,10 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
for (int lr = 0; lr < cpuif->used_lrs; lr++)
vgic_v3_fold_lr(vcpu, cpuif->vgic_lr[lr]);
cpuif->used_lrs = 0;
if (!irq)
return;
/*
* EOIMode=0: use EOIcount to emulate deactivation. We are
* guaranteed to deactivate in reverse order of the activation, so
@ -188,8 +192,6 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
vgic_v3_fold_lr(vcpu, lr);
eoicount--;
}
cpuif->used_lrs = 0;
}
void vgic_v3_deactivate(struct kvm_vcpu *vcpu, u64 val)

21
arch/arm64/kvm/vgic/vgic.c
View File

@ -853,6 +853,17 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)
goto retry;
}
/*
* Fix the last_lr_irq refcount which was obtained while
* populating the LRs. This can also result in the LPI being
* deleted.
*/
irq = *host_data_ptr(last_lr_irq);
if (irq) {
deleted_lpis |= vgic_put_irq_norelease(vcpu->kvm, irq);
*host_data_ptr(last_lr_irq) = NULL;
}
raw_spin_unlock(&vgic_cpu->ap_list_lock);
if (unlikely(deleted_lpis))
@ -866,9 +877,6 @@ static void vgic_fold_state(struct kvm_vcpu *vcpu)
return;
}
if (!*host_data_ptr(last_lr_irq))
return;
if (kvm_vgic_global_state.type == VGIC_V2)
vgic_v2_fold_lr_state(vcpu);
else
@ -1021,11 +1029,14 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
scoped_guard(raw_spinlock, &irq->irq_lock) {
if (likely(vgic_target_oracle(irq) == vcpu)) {
vgic_populate_lr(vcpu, irq, count++);
*host_data_ptr(last_lr_irq) = irq;
if (count == kvm_vgic_global_state.nr_lr) {
vgic_get_irq_ref(irq);
*host_data_ptr(last_lr_irq) = irq;
}
}
}
if (count == kvm_vgic_global_state.nr_lr)
if (*host_data_ptr(last_lr_irq))
break;
}

4
arch/arm64/tools/sysreg
View File

@ -3761,7 +3761,9 @@ Sysreg ICC_CR0_EL1 3 1 12 0 1
Res0 63:39
Field 38 PID
Field 37:32 IPPT
Res0 31:1
Res0 31:3
Field 2 LINK_IDLE
Field 1 LINK
Field 0 EN
EndSysreg

14
arch/powerpc/include/asm/entry-common.h
View File

@ -222,8 +222,6 @@ static inline void arch_interrupt_enter_prepare(struct pt_regs *regs)
if (user_mode(regs)) {
kuap_lock();
account_cpu_user_entry();
account_stolen_time();
} else {
kuap_save_and_lock(regs);
/*
@ -270,7 +268,7 @@ static inline void arch_interrupt_exit_prepare(struct pt_regs *regs)
}
/* irqentry_exit expects to be called with interrupts disabled */
local_irq_disable();
hard_irq_disable();
}
static inline void arch_interrupt_async_enter_prepare(struct pt_regs *regs)
@ -515,8 +513,14 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs,
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
local_paca->tm_scratch = regs->msr;
#endif
/* Restore user access locks last */
kuap_user_restore(regs);
/*
* Do not restore KUAP here. Generic entry might treat this as the last
* arch step before userspace but PowerPC still has kernel work after
* irqentry_exit()/syscall_exit_to_user_mode() i.e. in
* interrupt_exit_user_prepare() / syscall_exit_prepare() may enable
* IRQs and retry. Those functions restore KUAP immediately before rfi,
* which is where it should belong.
*/
}
#define arch_exit_to_user_mode_prepare arch_exit_to_user_mode_prepare

2
arch/powerpc/kernel/eeh_driver.c
View File

@ -533,9 +533,7 @@ static void eeh_rmv_device(struct eeh_dev *edev, void *userdata)
if (rmv_data)
list_add(&edev->rmv_entry, &rmv_data->removed_vf_list);
} else {
pci_lock_rescan_remove();
pci_stop_and_remove_bus_device(dev);
pci_unlock_rescan_remove();
}
}

2
arch/powerpc/kernel/interrupt.c
View File

@ -175,7 +175,7 @@ notrace unsigned long syscall_exit_restart(unsigned long r3, struct pt_regs *reg
current_thread_info()->exit_flags &= ~_TIF_RESTOREALL;
regs->exit_result |= ret;
return ret;
return regs->exit_result;
}
#endif

1
arch/powerpc/kernel/prom.c
View File

@ -824,7 +824,6 @@ void __init early_init_devtree(void *params)
fadump_append_bootargs();
/* Scan memory nodes and rebuild MEMBLOCKs */
early_init_dt_scan_root();
early_init_dt_scan_memory_ppc();
/*

8
arch/powerpc/kernel/rtas_pci.c
View File

@ -54,6 +54,10 @@ int rtas_pci_dn_read_config(struct pci_dn *pdn, int where, int size, u32 *val)
if (!config_access_valid(pdn, where))
return PCIBIOS_BAD_REGISTER_NUMBER;
#ifdef CONFIG_EEH
if (pdn->edev &&
(pdn->edev->mode & EEH_DEV_REMOVED))
return PCIBIOS_DEVICE_NOT_FOUND;
if (pdn->edev && pdn->edev->pe &&
(pdn->edev->pe->state & EEH_PE_CFG_BLOCKED))
return PCIBIOS_SET_FAILED;
@ -105,6 +109,10 @@ int rtas_pci_dn_write_config(struct pci_dn *pdn, int where, int size, u32 val)
if (!config_access_valid(pdn, where))
return PCIBIOS_BAD_REGISTER_NUMBER;
#ifdef CONFIG_EEH
if (pdn->edev &&
(pdn->edev->mode & EEH_DEV_REMOVED))
return PCIBIOS_DEVICE_NOT_FOUND;
if (pdn->edev && pdn->edev->pe &&
(pdn->edev->pe->state & EEH_PE_CFG_BLOCKED))
return PCIBIOS_SET_FAILED;

4
arch/powerpc/kernel/syscall.c
View File

@ -18,8 +18,10 @@ notrace long system_call_exception(struct pt_regs *regs, unsigned long r0)
long ret;
syscall_fn f;
if (unlikely(!syscall_enter_from_user_mode_randomize_stack(regs, &r0)))
if (unlikely(!syscall_enter_from_user_mode_randomize_stack(regs, &r0))) {
clear_thread_flag(TIF_SYSCALL_RET);
return syscall_get_error(current, regs);
}
if (unlikely(test_and_clear_thread_flag(TIF_SYSCALL_RET)))
return syscall_get_error(current, regs);

2
arch/powerpc/kexec/elf_64.c
View File

@ -94,6 +94,8 @@ static void *elf64_load(struct kimage *image, char *kernel_buf,
cmdline = modified_cmdline;
}
kexec_dprintk("Command line: %s", cmdline ? cmdline : "");
if (initrd != NULL) {
kbuf.buffer = initrd;
kbuf.bufsz = kbuf.memsz = initrd_len;

10
arch/powerpc/kexec/file_load_64.c
View File

@ -57,7 +57,7 @@ int arch_check_excluded_range(struct kimage *image, unsigned long start,
emem = image->arch.exclude_ranges;
for (i = 0; i < emem->nr_ranges; i++)
if (start < emem->ranges[i].end && end > emem->ranges[i].start)
if (start <= emem->ranges[i].end && end >= emem->ranges[i].start)
return 1;
return 0;
@ -113,7 +113,7 @@ static int add_usable_mem(struct umem_info *um_info, u64 base, u64 end)
loc_end = um_info->ranges[i].end;
if (loc_base >= base && loc_end <= end)
add = true;
else if (base < loc_end && end > loc_base) {
else if (base <= loc_end && end >= loc_base) {
if (loc_base < base)
loc_base = base;
if (loc_end > end)
@ -377,16 +377,12 @@ static int load_backup_segment(struct kimage *image, struct kexec_buf *kbuf)
static unsigned int kdump_extra_elfcorehdr_size(struct crash_mem *cmem)
{
#if defined(CONFIG_CRASH_HOTPLUG) && defined(CONFIG_MEMORY_HOTPLUG)
unsigned int extra_sz = 0;
if (CONFIG_CRASH_MAX_MEMORY_RANGES > (unsigned int)PN_XNUM)
pr_warn("Number of Phdrs %u exceeds max\n", CONFIG_CRASH_MAX_MEMORY_RANGES);
else if (cmem->nr_ranges >= CONFIG_CRASH_MAX_MEMORY_RANGES)
pr_warn("Configured crash mem ranges may not be enough\n");
else
extra_sz = (CONFIG_CRASH_MAX_MEMORY_RANGES - cmem->nr_ranges) * sizeof(Elf64_Phdr);
return extra_sz;
return (CONFIG_CRASH_MAX_MEMORY_RANGES - cmem->nr_ranges) * sizeof(Elf64_Phdr);
#endif
return 0;
}

4
arch/powerpc/kvm/book3s_hv.c
View File

@ -6140,12 +6140,12 @@ static int kvmppc_irq_bypass_add_producer_hv(struct irq_bypass_consumer *cons,
struct kvm_kernel_irqfd *irqfd =
container_of(cons, struct kvm_kernel_irqfd, consumer);
irqfd->producer = prod;
ret = kvmppc_set_passthru_irq(irqfd->kvm, prod->irq, irqfd->gsi);
if (ret)
pr_info("kvmppc_set_passthru_irq (irq %d, gsi %d) fails: %d\n",
prod->irq, irqfd->gsi, ret);
else
irqfd->producer = prod;
return ret;
}

2
arch/powerpc/platforms/pasemi/setup.c
View File

@ -165,6 +165,8 @@ static int __init pas_setup_mce_regs(void)
while (dev && reg < MAX_MCE_REGS) {
mce_regs[reg].name = kasprintf(GFP_KERNEL,
"mc%d_mcdebug_errsta", reg);
if (!mce_regs[reg].name)
return -ENOMEM;
mce_regs[reg].addr = pasemi_pci_getcfgaddr(dev, 0x730);
dev = pci_get_device(PCI_VENDOR_ID_PASEMI, 0xa00a, dev);
reg++;

6
arch/powerpc/platforms/powernv/pci-ioda.c
View File

@ -1623,15 +1623,13 @@ int64_t pnv_opal_pci_msi_eoi(struct irq_data *d)
return opal_pci_msi_eoi(phb->opal_id, d->parent_data->hwirq);
}
static struct irq_chip pnv_pci_msi_irq_chip;
/*
* Returns true iff chip is something that we could call
* pnv_opal_pci_msi_eoi for.
*/
bool is_pnv_opal_msi(struct irq_chip *chip)
{
return chip == &pnv_pci_msi_irq_chip;
return chip && chip->name && str_has_prefix(chip->name, "PNV-");
}
EXPORT_SYMBOL_GPL(is_pnv_opal_msi);
@ -1728,7 +1726,7 @@ static const struct msi_parent_ops pnv_msi_parent_ops = {
.chip_flags = MSI_CHIP_FLAG_SET_EOI,
.bus_select_token = DOMAIN_BUS_NEXUS,
.bus_select_mask = MATCH_PCI_MSI,
.prefix = "PNV-",
.prefix = "PNV-", /* Note: is_pnv_opal_msi() uses this */
.init_dev_msi_info = pnv_init_dev_msi_info,
};

5
arch/powerpc/platforms/ps3/repository.c
View File

@ -6,6 +6,8 @@
* Copyright 2006 Sony Corp.
*/
#include <linux/minmax.h>
#include <asm/lv1call.h>
#include "platform.h"
@ -74,8 +76,9 @@ static void _dump_node(unsigned int lpar_id, u64 n1, u64 n2, u64 n3, u64 n4,
static u64 make_first_field(const char *text, u64 index)
{
u64 n = 0;
size_t len = min(strlen(text), sizeof(n));
memcpy((char *)&n, text, strnlen(text, sizeof(n)));
memcpy(&n, text, len);
return PS3_VENDOR_ID_NONE + (n >> 32) + index;
}

19
arch/powerpc/platforms/pseries/htmdump.c
View File

@ -527,28 +527,28 @@ static int htmdump_init_debugfs(void)
htm_status_buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!htm_status_buf) {
pr_err("Failed to allocate htmstatus buf\n");
return -ENOMEM;
goto htm_status_buf_err;
}
/* Debugfs interface file to present System Processor Configuration */
htm_info_buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!htm_info_buf) {
pr_err("Failed to allocate htm info buf\n");
return -ENOMEM;
goto htm_info_buf_err;
}
/* Debugfs interface file to present HTM capabilities */
htm_caps_buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!htm_caps_buf) {
pr_err("Failed to allocate htm caps buf\n");
return -ENOMEM;
goto htm_caps_buf_err;
}
/* Memory to present HTM system memory configuration */
htm_mem_buf = kmalloc(PAGE_SIZE, GFP_KERNEL);
if (!htm_mem_buf) {
pr_err("Failed to allocate htm mem buf\n");
return -ENOMEM;
goto htm_mem_buf_err;
}
debugfs_create_file("htmstatus", 0400, htmdump_debugfs_dir, htm_status_buf, &htmstatus_fops);
@ -557,6 +557,17 @@ static int htmdump_init_debugfs(void)
debugfs_create_file("htmsystem_mem", 0400, htmdump_debugfs_dir, htm_mem_buf, &htmsystem_mem_fops);
return 0;
htm_mem_buf_err:
kfree(htm_caps_buf);
htm_caps_buf_err:
kfree(htm_info_buf);
htm_info_buf_err:
kfree(htm_status_buf);
htm_status_buf_err:
debugfs_remove_recursive(htmdump_debugfs_dir);
kfree(htm_buf);
return -ENOMEM;
}
static int __init htmdump_init(void)

13
arch/powerpc/platforms/pseries/pci.c
View File

@ -132,11 +132,14 @@ static int pseries_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
/* First integer stores max config */
max_config_vfs = of_read_number(&max_vfs[0], 1);
if (max_config_vfs < num_vfs || num_vfs > MAX_VFS_FOR_MAP_PE) {
dev_err(&pdev->dev,
"Num VFs %x > %x Configurable VFs\n",
num_vfs, (num_vfs > MAX_VFS_FOR_MAP_PE) ?
MAX_VFS_FOR_MAP_PE : max_config_vfs);
if (max_config_vfs < num_vfs) {
dev_err(&pdev->dev, "Num VFs %x > %x Configurable VFs\n",
num_vfs, max_config_vfs);
return -EINVAL;
}
if (num_vfs > MAX_VFS_FOR_MAP_PE) {
dev_err(&pdev->dev, "Num VFs %x > %x PE mapping limit\n",
num_vfs, MAX_VFS_FOR_MAP_PE);
return -EINVAL;
}

2
arch/riscv/Kconfig
View File

@ -326,7 +326,7 @@ config STACKTRACE_SUPPORT
config GENERIC_BUG
def_bool y
depends on BUG
select GENERIC_BUG_RELATIVE_POINTERS if 64BIT
select GENERIC_BUG_RELATIVE_POINTERS
config GENERIC_BUG_RELATIVE_POINTERS
bool

5
arch/riscv/include/asm/bug.h
View File

@ -29,13 +29,8 @@
typedef u32 bug_insn_t;
#ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS
#define __BUG_ENTRY_ADDR RISCV_INT " 1b - ."
#define __BUG_ENTRY_FILE(file) RISCV_INT " " file " - ."
#else
#define __BUG_ENTRY_ADDR RISCV_PTR " 1b"
#define __BUG_ENTRY_FILE(file) RISCV_PTR " " file
#endif
#ifdef CONFIG_DEBUG_BUGVERBOSE
#define __BUG_ENTRY(file, line, flags) \

4
arch/riscv/include/asm/switch_to.h
View File

@ -61,8 +61,8 @@ static inline void __switch_to_fpu(struct task_struct *prev,
static __always_inline bool has_fpu(void)
{
return riscv_has_extension_likely(RISCV_ISA_EXT_F) ||
riscv_has_extension_likely(RISCV_ISA_EXT_D);
/* D extension depends on F, so checking D alone is sufficient. */
return riscv_has_extension_likely(RISCV_ISA_EXT_D);
}
#else
static __always_inline bool has_fpu(void) { return false; }

20
arch/riscv/kernel/cpufeature.c
View File

@ -412,6 +412,19 @@ static const unsigned int riscv_zvbb_exts[] = {
RISCV_ISA_EXT_ZVKB
};
/*
* The RISC-V ISA manual specifies that Zfh implies Zfhmin and Zvfh implies
* Zvfhmin. Report the implied subset extensions whenever the supersets are
* detected (see https://github.com/riscv/riscv-isa-manual/pull/3070).
*/
static const unsigned int riscv_zfh_exts[] = {
RISCV_ISA_EXT_ZFHMIN
};
static const unsigned int riscv_zvfh_exts[] = {
RISCV_ISA_EXT_ZVFHMIN
};
#define RISCV_ISA_EXT_ZVE64F_IMPLY_LIST \
RISCV_ISA_EXT_ZVE64X, \
RISCV_ISA_EXT_ZVE32F, \
@ -550,7 +563,8 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
__RISCV_ISA_EXT_DATA(zawrs, RISCV_ISA_EXT_ZAWRS),
__RISCV_ISA_EXT_DATA_VALIDATE(zfa, RISCV_ISA_EXT_ZFA, riscv_ext_f_depends),
__RISCV_ISA_EXT_DATA_VALIDATE(zfbfmin, RISCV_ISA_EXT_ZFBFMIN, riscv_ext_f_depends),
__RISCV_ISA_EXT_DATA_VALIDATE(zfh, RISCV_ISA_EXT_ZFH, riscv_ext_f_depends),
__RISCV_ISA_EXT_SUPERSET_VALIDATE(zfh, RISCV_ISA_EXT_ZFH,
riscv_zfh_exts, riscv_ext_f_depends),
__RISCV_ISA_EXT_DATA_VALIDATE(zfhmin, RISCV_ISA_EXT_ZFHMIN, riscv_ext_f_depends),
__RISCV_ISA_EXT_DATA(zca, RISCV_ISA_EXT_ZCA),
__RISCV_ISA_EXT_DATA_VALIDATE(zcb, RISCV_ISA_EXT_ZCB, riscv_ext_zca_depends),
@ -586,7 +600,9 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
__RISCV_ISA_EXT_SUPERSET_VALIDATE(zve64x, RISCV_ISA_EXT_ZVE64X, riscv_zve64x_exts, riscv_ext_vector_x_validate),
__RISCV_ISA_EXT_DATA_VALIDATE(zvfbfmin, RISCV_ISA_EXT_ZVFBFMIN, riscv_vector_f_validate),
__RISCV_ISA_EXT_DATA_VALIDATE(zvfbfwma, RISCV_ISA_EXT_ZVFBFWMA, riscv_ext_zvfbfwma_validate),
__RISCV_ISA_EXT_DATA(zvfh, RISCV_ISA_EXT_ZVFH),
__RISCV_ISA_EXT_SUPERSET_VALIDATE(zvfh, RISCV_ISA_EXT_ZVFH,
riscv_zvfh_exts,
riscv_ext_vector_float_validate),
__RISCV_ISA_EXT_DATA(zvfhmin, RISCV_ISA_EXT_ZVFHMIN),
__RISCV_ISA_EXT_DATA_VALIDATE(zvkb, RISCV_ISA_EXT_ZVKB, riscv_ext_vector_crypto_validate),
__RISCV_ISA_EXT_DATA_VALIDATE(zvkg, RISCV_ISA_EXT_ZVKG, riscv_ext_vector_crypto_validate),

2
arch/riscv/kernel/patch.c
View File

@ -45,6 +45,8 @@ static __always_inline void *patch_map(void *addr, const unsigned int fixmap)
phys_addr_t phys;
if (core_kernel_text(uintaddr) || is_kernel_exittext(uintaddr)) {
if (!IS_ENABLED(CONFIG_STRICT_KERNEL_RWX))
return addr;
phys = __pa_symbol(addr);
} else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX)) {
struct page *page = vmalloc_to_page(addr);

4
arch/riscv/kernel/process.c
View File

@ -349,10 +349,8 @@ long set_tagged_addr_ctrl(struct task_struct *task, unsigned long arg)
if (arg & PR_TAGGED_ADDR_ENABLE && (tagged_addr_disabled || !pmlen))
return -EINVAL;
if (!(arg & PR_TAGGED_ADDR_ENABLE)) {
if (!(arg & PR_TAGGED_ADDR_ENABLE))
pmlen = PMLEN_0;
pmm = ENVCFG_PMM_PMLEN_0;
}
if (mmap_write_lock_killable(mm))
return -EINTR;

5
arch/riscv/kernel/sys_hwprobe.c
View File

@ -297,6 +297,8 @@ static u64 hwprobe_vec_misaligned(const struct cpumask *cpus)
static void hwprobe_one_pair(struct riscv_hwprobe *pair,
const struct cpumask *cpus)
{
pair->value = 0;
switch (pair->key) {
case RISCV_HWPROBE_KEY_MVENDORID:
case RISCV_HWPROBE_KEY_MARCHID:
@ -331,17 +333,14 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair,
break;
case RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE:
pair->value = 0;
if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOZ))
pair->value = riscv_cboz_block_size;
break;
case RISCV_HWPROBE_KEY_ZICBOM_BLOCK_SIZE:
pair->value = 0;
if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOM))
pair->value = riscv_cbom_block_size;
break;
case RISCV_HWPROBE_KEY_ZICBOP_BLOCK_SIZE:
pair->value = 0;
if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOP))
pair->value = riscv_cbop_block_size;
break;

5
arch/riscv/kernel/usercfi.c
View File

@ -525,9 +525,8 @@ static int __init setup_global_riscv_enable(char *str)
if (riscv_nousercfi)
pr_info("RISC-V user CFI disabled via cmdline - shadow stack status : %s, landing pad status : %s\n",
(riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_BCFI) ? "disabled" :
"enabled", (riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_FCFI) ?
"disabled" : "enabled");
str_disabled_enabled(riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_BCFI),
str_disabled_enabled(riscv_nousercfi & CMDLINE_DISABLE_RISCV_USERCFI_FCFI));
return 1;
}

5
arch/riscv/lib/uaccess.S
View File

@ -76,6 +76,7 @@ SYM_FUNC_START(fallback_scalar_usercopy_sum_enabled)
li a3, 9*SZREG-1 /* size must >= (word_copy stride + SZREG-1) */
bltu a2, a3, .Lbyte_copy_tail
#if !defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
/*
* Copy first bytes until dst is aligned to word boundary.
* a0 - start of dst
@ -103,7 +104,7 @@ SYM_FUNC_START(fallback_scalar_usercopy_sum_enabled)
/* a1 - start of src */
andi a3, a1, SZREG-1
bnez a3, .Lshift_copy
#endif
.Lword_copy:
/*
* Both src and dst are aligned, unrolled word copy
@ -137,6 +138,7 @@ SYM_FUNC_START(fallback_scalar_usercopy_sum_enabled)
addi t0, t0, 8*SZREG /* revert to original value */
j .Lbyte_copy_tail
#if !defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
.Lshift_copy:
/*
@ -189,6 +191,7 @@ SYM_FUNC_START(fallback_scalar_usercopy_sum_enabled)
/* Revert src to original unaligned value */
add a1, a1, a3
#endif
.Lbyte_copy_tail:
/*

4
arch/riscv/mm/init.c
View File

@ -1465,7 +1465,9 @@ struct execmem_info __init *execmem_arch_setup(void)
[EXECMEM_KPROBES] = {
.start = VMALLOC_START,
.end = VMALLOC_END,
.pgprot = PAGE_KERNEL_READ_EXEC,
.pgprot = IS_ENABLED(CONFIG_STRICT_MODULE_RWX) ?
PAGE_KERNEL_READ_EXEC :
PAGE_KERNEL_EXEC,
.alignment = 1,
},
[EXECMEM_BPF] = {

2
arch/s390/crypto/Makefile
View File

@ -3,6 +3,8 @@
# Cryptographic API
#
CONTEXT_ANALYSIS := y
obj-$(CONFIG_CRYPTO_AES_S390) += aes_s390.o
obj-$(CONFIG_CRYPTO_PAES_S390) += paes_s390.o
obj-$(CONFIG_S390_PRNG) += prng.o

87
arch/s390/crypto/aes_s390.c
View File

@ -26,14 +26,14 @@
#include <linux/module.h>
#include <linux/cpufeature.h>
#include <linux/init.h>
#include <linux/mutex.h>
#include <linux/fips.h>
#include <linux/semaphore.h>
#include <linux/string.h>
#include <crypto/xts.h>
#include <asm/cpacf.h>
static u8 *ctrblk;
static DEFINE_MUTEX(ctrblk_lock);
static DEFINE_SEMAPHORE(ctrblk_sem, 1);
static cpacf_mask_t km_functions, kmc_functions, kmctr_functions,
kma_functions;
@ -129,7 +129,7 @@ static int ecb_aes_crypt(struct skcipher_request *req, unsigned long modifier)
return fallback_skcipher_crypt(sctx, req, modifier);
ret = skcipher_walk_virt(&walk, req, false);
while ((nbytes = walk.nbytes) != 0) {
while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_km(sctx->fc | modifier, sctx->key,
@ -233,7 +233,7 @@ static int cbc_aes_crypt(struct skcipher_request *req, unsigned long modifier)
return ret;
memcpy(param.iv, walk.iv, AES_BLOCK_SIZE);
memcpy(param.key, sctx->key, sctx->key_len);
while ((nbytes = walk.nbytes) != 0) {
while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_kmc(sctx->fc | modifier, &param,
@ -359,7 +359,7 @@ static int xts_aes_crypt(struct skcipher_request *req, unsigned long modifier)
memcpy(xts_param.key + offset, xts_ctx->key, xts_ctx->key_len);
memcpy(xts_param.init, pcc_param.xts, 16);
while ((nbytes = walk.nbytes) != 0) {
while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_km(xts_ctx->fc | modifier, xts_param.key + offset,
@ -487,7 +487,7 @@ static int fullxts_aes_crypt(struct skcipher_request *req, unsigned long modifi
memcpy(fxts_param.tweak, req->iv, AES_BLOCK_SIZE);
fxts_param.nap[0] = 0x01; /* initial alpha power (1, little-endian) */
while ((nbytes = walk.nbytes) != 0) {
while (!ret && ((nbytes = walk.nbytes) != 0)) {
/* only use complete blocks */
n = nbytes & ~(AES_BLOCK_SIZE - 1);
cpacf_km(xts_ctx->fc | modifier, fxts_param.key + offset,
@ -562,48 +562,64 @@ static unsigned int __ctrblk_init(u8 *ctrptr, u8 *iv, unsigned int nbytes)
return n;
}
static int __ctr_aes_crypt(struct s390_aes_ctx *sctx,
struct skcipher_walk *walk, bool locked)
{
unsigned int n, nbytes;
int ret = 0;
u8 *ctrptr;
while (!ret && ((nbytes = walk->nbytes) >= AES_BLOCK_SIZE)) {
n = AES_BLOCK_SIZE;
if (nbytes >= 2 * AES_BLOCK_SIZE && locked)
n = __ctrblk_init(ctrblk, walk->iv, nbytes);
ctrptr = (n > AES_BLOCK_SIZE) ? ctrblk : walk->iv;
cpacf_kmctr(sctx->fc, sctx->key, walk->dst.virt.addr,
walk->src.virt.addr, n, ctrptr);
if (ctrptr == ctrblk)
memcpy(walk->iv, ctrptr + n - AES_BLOCK_SIZE,
AES_BLOCK_SIZE);
crypto_inc(walk->iv, AES_BLOCK_SIZE);
ret = skcipher_walk_done(walk, nbytes - n);
}
return ret;
}
static int ctr_aes_crypt(struct skcipher_request *req)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
struct s390_aes_ctx *sctx = crypto_skcipher_ctx(tfm);
u8 buf[AES_BLOCK_SIZE], *ctrptr;
struct skcipher_walk walk;
unsigned int n, nbytes;
int ret, locked;
u8 buf[AES_BLOCK_SIZE];
int ret;
if (unlikely(!sctx->fc))
return fallback_skcipher_crypt(sctx, req, 0);
locked = mutex_trylock(&ctrblk_lock);
ret = skcipher_walk_virt(&walk, req, false);
while ((nbytes = walk.nbytes) >= AES_BLOCK_SIZE) {
n = AES_BLOCK_SIZE;
if (ret)
return ret;
if (nbytes >= 2*AES_BLOCK_SIZE && locked)
n = __ctrblk_init(ctrblk, walk.iv, nbytes);
ctrptr = (n > AES_BLOCK_SIZE) ? ctrblk : walk.iv;
cpacf_kmctr(sctx->fc, sctx->key, walk.dst.virt.addr,
walk.src.virt.addr, n, ctrptr);
if (ctrptr == ctrblk)
memcpy(walk.iv, ctrptr + n - AES_BLOCK_SIZE,
AES_BLOCK_SIZE);
crypto_inc(walk.iv, AES_BLOCK_SIZE);
ret = skcipher_walk_done(&walk, nbytes - n);
if (down_trylock(&ctrblk_sem) == 0) {
ret = __ctr_aes_crypt(sctx, &walk, true);
up(&ctrblk_sem);
} else {
ret = __ctr_aes_crypt(sctx, &walk, false);
}
if (locked)
mutex_unlock(&ctrblk_lock);
/*
* final block may be < AES_BLOCK_SIZE, copy only nbytes
*/
if (nbytes) {
if (!ret && walk.nbytes > 0) {
memset(buf, 0, AES_BLOCK_SIZE);
memcpy(buf, walk.src.virt.addr, nbytes);
memcpy(buf, walk.src.virt.addr, walk.nbytes);
cpacf_kmctr(sctx->fc, sctx->key, buf, buf,
AES_BLOCK_SIZE, walk.iv);
memcpy(walk.dst.virt.addr, buf, nbytes);
memcpy(walk.dst.virt.addr, buf, walk.nbytes);
crypto_inc(walk.iv, AES_BLOCK_SIZE);
ret = skcipher_walk_done(&walk, 0);
memzero_explicit(buf, sizeof(buf));
}
return ret;
@ -895,10 +911,14 @@ static int gcm_aes_crypt(struct aead_request *req, unsigned int flags)
gw_in.ptr, aad_bytes);
n = aad_bytes + pc_bytes;
if (gcm_in_walk_done(&gw_in, n) != n)
return -ENOMEM;
if (gcm_out_walk_done(&gw_out, n) != n)
return -ENOMEM;
if (gcm_in_walk_done(&gw_in, n) != n) {
ret = -ENOMEM;
goto out;
}
if (gcm_out_walk_done(&gw_out, n) != n) {
ret = -ENOMEM;
goto out;
}
aadlen -= aad_bytes;
pclen -= pc_bytes;
} while (aadlen + pclen > 0);
@ -910,7 +930,10 @@ static int gcm_aes_crypt(struct aead_request *req, unsigned int flags)
} else
scatterwalk_map_and_copy(param.t, req->dst, len, taglen, 1);
out:
memzero_explicit(&param, sizeof(param));
memzero_explicit(gw_in.buf, sizeof(gw_in.buf));
memzero_explicit(gw_out.buf, sizeof(gw_out.buf));
return ret;
}

188
arch/s390/crypto/paes_s390.c
View File

@ -19,7 +19,7 @@
#include <linux/init.h>
#include <linux/miscdevice.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/semaphore.h>
#include <linux/spinlock.h>
#include <crypto/aes.h>
#include <crypto/algapi.h>
@ -45,7 +45,7 @@ module_param_named(clrkey, pkey_clrkey_allowed, bool, 0444);
MODULE_PARM_DESC(clrkey, "Allow clear key material (default N)");
static u8 *ctrblk;
static DEFINE_MUTEX(ctrblk_lock);
static DEFINE_SEMAPHORE(ctrblk_sem, 1);
static cpacf_mask_t km_functions, kmc_functions, kmctr_functions;
@ -220,6 +220,10 @@ static inline int convert_key(const u8 *key, unsigned int keylen,
xflags);
}
/* But finally map -EBUSY to -EIO to indicate an IO failure */
if (rc == -EBUSY)
rc = -EIO;
out:
pr_debug("rc=%d\n", rc);
return rc;
@ -432,8 +436,11 @@ static int ecb_paes_do_crypt(struct s390_paes_ctx *ctx,
n = nbytes & ~(AES_BLOCK_SIZE - 1);
k = cpacf_km(ctx->fc | req_ctx->modifier, param,
walk->dst.virt.addr, walk->src.virt.addr, n);
if (k)
if (k) {
rc = skcipher_walk_done(walk, nbytes - k);
if (rc)
goto out;
}
if (k < n) {
if (!maysleep) {
rc = -EKEYEXPIRED;
@ -460,6 +467,7 @@ static int ecb_paes_crypt(struct skcipher_request *req, unsigned long modifier)
struct s390_paes_ctx *ctx = crypto_skcipher_ctx(tfm);
struct skcipher_walk *walk = &req_ctx->walk;
bool tested = crypto_skcipher_tested(tfm);
bool cleanup = true;
int rc;
/*
@ -491,15 +499,17 @@ static int ecb_paes_crypt(struct skcipher_request *req, unsigned long modifier)
if (rc == 0 || rc == -EKEYEXPIRED) {
atomic_inc(&ctx->via_engine_ctr);
rc = crypto_transfer_skcipher_request_to_engine(paes_crypto_engine, req);
if (rc != -EINPROGRESS)
if (rc == -EINPROGRESS || rc == -EBUSY)
cleanup = false;
else
atomic_dec(&ctx->via_engine_ctr);
}
if (rc != -EINPROGRESS)
if (cleanup && walk->nbytes)
skcipher_walk_done(walk, rc);
out:
if (rc != -EINPROGRESS)
if (cleanup)
memzero_explicit(&req_ctx->param, sizeof(req_ctx->param));
pr_debug("rc=%d\n", rc);
return rc;
@ -549,7 +559,7 @@ static int ecb_paes_do_one_request(struct crypto_engine *engine, void *areq)
rc = ecb_paes_do_crypt(ctx, req_ctx, tested, true);
if (rc == -EKEYEXPIRED) {
return pkey_handle_expired();
} else if (rc) {
} else if (rc && walk->nbytes) {
skcipher_walk_done(walk, rc);
}
@ -559,7 +569,7 @@ static int ecb_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
return rc;
return 0;
}
static struct skcipher_engine_alg ecb_paes_alg = {
@ -567,6 +577,7 @@ static struct skcipher_engine_alg ecb_paes_alg = {
.base.cra_name = "ecb(paes)",
.base.cra_driver_name = "ecb-paes-s390",
.base.cra_priority = 401, /* combo: aes + ecb + 1 */
.base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct s390_paes_ctx),
.base.cra_module = THIS_MODULE,
@ -690,6 +701,8 @@ static int cbc_paes_do_crypt(struct s390_paes_ctx *ctx,
if (k) {
memcpy(walk->iv, param->iv, AES_BLOCK_SIZE);
rc = skcipher_walk_done(walk, nbytes - k);
if (rc)
goto out;
}
if (k < n) {
if (!maysleep) {
@ -717,6 +730,7 @@ static int cbc_paes_crypt(struct skcipher_request *req, unsigned long modifier)
struct s390_paes_ctx *ctx = crypto_skcipher_ctx(tfm);
struct skcipher_walk *walk = &req_ctx->walk;
bool tested = crypto_skcipher_tested(tfm);
bool cleanup = true;
int rc;
/*
@ -748,15 +762,17 @@ static int cbc_paes_crypt(struct skcipher_request *req, unsigned long modifier)
if (rc == 0 || rc == -EKEYEXPIRED) {
atomic_inc(&ctx->via_engine_ctr);
rc = crypto_transfer_skcipher_request_to_engine(paes_crypto_engine, req);
if (rc != -EINPROGRESS)
if (rc == -EINPROGRESS || rc == -EBUSY)
cleanup = false;
else
atomic_dec(&ctx->via_engine_ctr);
}
if (rc != -EINPROGRESS)
if (cleanup && walk->nbytes)
skcipher_walk_done(walk, rc);
out:
if (rc != -EINPROGRESS)
if (cleanup)
memzero_explicit(&req_ctx->param, sizeof(req_ctx->param));
pr_debug("rc=%d\n", rc);
return rc;
@ -806,7 +822,7 @@ static int cbc_paes_do_one_request(struct crypto_engine *engine, void *areq)
rc = cbc_paes_do_crypt(ctx, req_ctx, tested, true);
if (rc == -EKEYEXPIRED) {
return pkey_handle_expired();
} else if (rc) {
} else if (rc && walk->nbytes) {
skcipher_walk_done(walk, rc);
}
@ -816,7 +832,7 @@ static int cbc_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
return rc;
return 0;
}
static struct skcipher_engine_alg cbc_paes_alg = {
@ -824,6 +840,7 @@ static struct skcipher_engine_alg cbc_paes_alg = {
.base.cra_name = "cbc(paes)",
.base.cra_driver_name = "cbc-paes-s390",
.base.cra_priority = 402, /* cbc-paes-s390 + 1 */
.base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct s390_paes_ctx),
.base.cra_module = THIS_MODULE,
@ -914,15 +931,62 @@ static inline unsigned int __ctrblk_init(u8 *ctrptr, u8 *iv, unsigned int nbytes
return n;
}
static int __ctr_paes_do_crypt(struct s390_paes_ctx *ctx,
struct ctr_param *param,
struct skcipher_walk *walk,
bool tested, bool maysleep, bool locked)
{
unsigned int nbytes, n, k;
u8 *ctrptr;
int rc = 0;
/*
* Note that in case of partial processing or failure the walk
* is NOT unmapped here. So a follow up task may reuse the walk
* or in case of unrecoverable failure needs to unmap it.
*/
while ((nbytes = walk->nbytes) >= AES_BLOCK_SIZE) {
n = AES_BLOCK_SIZE;
if (nbytes >= 2 * AES_BLOCK_SIZE && locked)
n = __ctrblk_init(ctrblk, walk->iv, nbytes);
ctrptr = (n > AES_BLOCK_SIZE) ? ctrblk : walk->iv;
k = cpacf_kmctr(ctx->fc, param, walk->dst.virt.addr,
walk->src.virt.addr, n, ctrptr);
if (k) {
if (ctrptr == ctrblk)
memcpy(walk->iv, ctrptr + k - AES_BLOCK_SIZE,
AES_BLOCK_SIZE);
crypto_inc(walk->iv, AES_BLOCK_SIZE);
rc = skcipher_walk_done(walk, nbytes - k);
if (rc)
goto out;
}
if (k < n) {
if (!maysleep) {
rc = -EKEYEXPIRED;
goto out;
}
rc = paes_convert_key(ctx, tested);
if (rc)
goto out;
spin_lock_bh(&ctx->pk_lock);
memcpy(param->key, ctx->pk.protkey, sizeof(param->key));
spin_unlock_bh(&ctx->pk_lock);
}
}
out:
return rc;
}
static int ctr_paes_do_crypt(struct s390_paes_ctx *ctx,
struct s390_pctr_req_ctx *req_ctx,
bool tested, bool maysleep)
{
struct ctr_param *param = &req_ctx->param;
struct skcipher_walk *walk = &req_ctx->walk;
u8 buf[AES_BLOCK_SIZE], *ctrptr;
unsigned int nbytes, n, k;
int pk_state, locked, rc = 0;
u8 buf[AES_BLOCK_SIZE];
int pk_state, rc = 0;
if (!req_ctx->param_init_done) {
/* fetch and check protected key state */
@ -948,52 +1012,17 @@ static int ctr_paes_do_crypt(struct s390_paes_ctx *ctx,
if (rc)
goto out;
locked = mutex_trylock(&ctrblk_lock);
/*
* Note that in case of partial processing or failure the walk
* is NOT unmapped here. So a follow up task may reuse the walk
* or in case of unrecoverable failure needs to unmap it.
*/
while ((nbytes = walk->nbytes) >= AES_BLOCK_SIZE) {
n = AES_BLOCK_SIZE;
if (nbytes >= 2 * AES_BLOCK_SIZE && locked)
n = __ctrblk_init(ctrblk, walk->iv, nbytes);
ctrptr = (n > AES_BLOCK_SIZE) ? ctrblk : walk->iv;
k = cpacf_kmctr(ctx->fc, param, walk->dst.virt.addr,
walk->src.virt.addr, n, ctrptr);
if (k) {
if (ctrptr == ctrblk)
memcpy(walk->iv, ctrptr + k - AES_BLOCK_SIZE,
AES_BLOCK_SIZE);
crypto_inc(walk->iv, AES_BLOCK_SIZE);
rc = skcipher_walk_done(walk, nbytes - k);
}
if (k < n) {
if (!maysleep) {
if (locked)
mutex_unlock(&ctrblk_lock);
rc = -EKEYEXPIRED;
goto out;
}
rc = paes_convert_key(ctx, tested);
if (rc) {
if (locked)
mutex_unlock(&ctrblk_lock);
goto out;
}
spin_lock_bh(&ctx->pk_lock);
memcpy(param->key, ctx->pk.protkey, sizeof(param->key));
spin_unlock_bh(&ctx->pk_lock);
}
if (down_trylock(&ctrblk_sem) == 0) {
rc = __ctr_paes_do_crypt(ctx, param, walk, tested, maysleep, true);
up(&ctrblk_sem);
} else {
rc = __ctr_paes_do_crypt(ctx, param, walk, tested, maysleep, false);
}
if (locked)
mutex_unlock(&ctrblk_lock);
/* final block may be < AES_BLOCK_SIZE, copy only nbytes */
if (nbytes) {
if (!rc && walk->nbytes > 0) {
memset(buf, 0, AES_BLOCK_SIZE);
memcpy(buf, walk->src.virt.addr, nbytes);
memcpy(buf, walk->src.virt.addr, walk->nbytes);
while (1) {
if (cpacf_kmctr(ctx->fc, param, buf,
buf, AES_BLOCK_SIZE,
@ -1010,12 +1039,13 @@ static int ctr_paes_do_crypt(struct s390_paes_ctx *ctx,
memcpy(param->key, ctx->pk.protkey, sizeof(param->key));
spin_unlock_bh(&ctx->pk_lock);
}
memcpy(walk->dst.virt.addr, buf, nbytes);
memcpy(walk->dst.virt.addr, buf, walk->nbytes);
crypto_inc(walk->iv, AES_BLOCK_SIZE);
rc = skcipher_walk_done(walk, 0);
}
out:
memzero_explicit(buf, sizeof(buf));
pr_debug("rc=%d\n", rc);
return rc;
}
@ -1027,6 +1057,7 @@ static int ctr_paes_crypt(struct skcipher_request *req)
struct s390_paes_ctx *ctx = crypto_skcipher_ctx(tfm);
struct skcipher_walk *walk = &req_ctx->walk;
bool tested = crypto_skcipher_tested(tfm);
bool cleanup = true;
int rc;
/*
@ -1057,15 +1088,17 @@ static int ctr_paes_crypt(struct skcipher_request *req)
if (rc == 0 || rc == -EKEYEXPIRED) {
atomic_inc(&ctx->via_engine_ctr);
rc = crypto_transfer_skcipher_request_to_engine(paes_crypto_engine, req);
if (rc != -EINPROGRESS)
if (rc == -EINPROGRESS || rc == -EBUSY)
cleanup = false;
else
atomic_dec(&ctx->via_engine_ctr);
}
if (rc != -EINPROGRESS)
if (cleanup && walk->nbytes)
skcipher_walk_done(walk, rc);
out:
if (rc != -EINPROGRESS)
if (cleanup)
memzero_explicit(&req_ctx->param, sizeof(req_ctx->param));
pr_debug("rc=%d\n", rc);
return rc;
@ -1105,7 +1138,7 @@ static int ctr_paes_do_one_request(struct crypto_engine *engine, void *areq)
rc = ctr_paes_do_crypt(ctx, req_ctx, tested, true);
if (rc == -EKEYEXPIRED) {
return pkey_handle_expired();
} else if (rc) {
} else if (rc && walk->nbytes) {
skcipher_walk_done(walk, rc);
}
@ -1115,7 +1148,7 @@ static int ctr_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
return rc;
return 0;
}
static struct skcipher_engine_alg ctr_paes_alg = {
@ -1123,6 +1156,7 @@ static struct skcipher_engine_alg ctr_paes_alg = {
.base.cra_name = "ctr(paes)",
.base.cra_driver_name = "ctr-paes-s390",
.base.cra_priority = 402, /* ecb-paes-s390 + 1 */
.base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = 1,
.base.cra_ctxsize = sizeof(struct s390_paes_ctx),
.base.cra_module = THIS_MODULE,
@ -1283,8 +1317,11 @@ static int xts_paes_do_crypt_fullkey(struct s390_pxts_ctx *ctx,
n = nbytes & ~(AES_BLOCK_SIZE - 1);
k = cpacf_km(ctx->fc | req_ctx->modifier, param->key + offset,
walk->dst.virt.addr, walk->src.virt.addr, n);
if (k)
if (k) {
rc = skcipher_walk_done(walk, nbytes - k);
if (rc)
goto out;
}
if (k < n) {
if (!maysleep) {
rc = -EKEYEXPIRED;
@ -1337,7 +1374,7 @@ static inline int __xts_2keys_prep_param(struct s390_pxts_ctx *ctx,
memcpy(param->init, pcc_param.xts, 16);
}
memzero_explicit(pcc_param.key, sizeof(pcc_param.key));
memzero_explicit(&pcc_param, sizeof(pcc_param));
return rc;
}
@ -1377,8 +1414,11 @@ static int xts_paes_do_crypt_2keys(struct s390_pxts_ctx *ctx,
n = nbytes & ~(AES_BLOCK_SIZE - 1);
k = cpacf_km(ctx->fc | req_ctx->modifier, param->key + offset,
walk->dst.virt.addr, walk->src.virt.addr, n);
if (k)
if (k) {
rc = skcipher_walk_done(walk, nbytes - k);
if (rc)
goto out;
}
if (k < n) {
if (!maysleep) {
rc = -EKEYEXPIRED;
@ -1450,6 +1490,7 @@ static inline int xts_paes_crypt(struct skcipher_request *req, unsigned long mod
struct s390_pxts_ctx *ctx = crypto_skcipher_ctx(tfm);
struct skcipher_walk *walk = &req_ctx->walk;
bool tested = crypto_skcipher_tested(tfm);
bool cleanup = true;
int rc;
/*
@ -1481,15 +1522,17 @@ static inline int xts_paes_crypt(struct skcipher_request *req, unsigned long mod
if (rc == 0 || rc == -EKEYEXPIRED) {
atomic_inc(&ctx->via_engine_ctr);
rc = crypto_transfer_skcipher_request_to_engine(paes_crypto_engine, req);
if (rc != -EINPROGRESS)
if (rc == -EINPROGRESS || rc == -EBUSY)
cleanup = false;
else
atomic_dec(&ctx->via_engine_ctr);
}
if (rc != -EINPROGRESS)
if (cleanup && walk->nbytes)
skcipher_walk_done(walk, rc);
out:
if (rc != -EINPROGRESS)
if (cleanup)
memzero_explicit(&req_ctx->param, sizeof(req_ctx->param));
pr_debug("rc=%d\n", rc);
return rc;
@ -1539,7 +1582,7 @@ static int xts_paes_do_one_request(struct crypto_engine *engine, void *areq)
rc = xts_paes_do_crypt(ctx, req_ctx, tested, true);
if (rc == -EKEYEXPIRED) {
return pkey_handle_expired();
} else if (rc) {
} else if (rc && walk->nbytes) {
skcipher_walk_done(walk, rc);
}
@ -1549,7 +1592,7 @@ static int xts_paes_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&ctx->via_engine_ctr);
crypto_finalize_skcipher_request(engine, req, rc);
local_bh_enable();
return rc;
return 0;
}
static struct skcipher_engine_alg xts_paes_alg = {
@ -1557,6 +1600,7 @@ static struct skcipher_engine_alg xts_paes_alg = {
.base.cra_name = "xts(paes)",
.base.cra_driver_name = "xts-paes-s390",
.base.cra_priority = 402, /* ecb-paes-s390 + 1 */
.base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NO_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct s390_pxts_ctx),
.base.cra_module = THIS_MODULE,

36
arch/s390/crypto/phmac_s390.c
View File

@ -62,8 +62,10 @@ static inline int hwh_prepare(struct ahash_request *req,
*/
static inline int hwh_advance(struct hash_walk_helper *hwh, int n)
{
if (n < 0)
if (n < 0) {
hwh->walkbytes = n;
return crypto_hash_walk_done(&hwh->walk, n);
}
hwh->walkbytes -= n;
hwh->walkaddr += n;
@ -339,6 +341,10 @@ static inline int convert_key(const u8 *key, unsigned int keylen,
xflags);
}
/* But finally map -EBUSY to -EIO to indicate an IO failure */
if (rc == -EBUSY)
rc = -EIO;
out:
pr_debug("rc=%d\n", rc);
return rc;
@ -606,6 +612,7 @@ static int phmac_update(struct ahash_request *req)
struct phmac_tfm_ctx *tfm_ctx = crypto_ahash_ctx(tfm);
struct kmac_sha2_ctx *kmac_ctx = &req_ctx->kmac_ctx;
struct hash_walk_helper *hwh = &req_ctx->hwh;
bool cleanup = true;
int rc;
/* prep the walk in the request context */
@ -629,12 +636,15 @@ static int phmac_update(struct ahash_request *req)
req_ctx->async_op = OP_UPDATE;
atomic_inc(&tfm_ctx->via_engine_ctr);
rc = crypto_transfer_hash_request_to_engine(phmac_crypto_engine, req);
if (rc != -EINPROGRESS)
if (rc == -EINPROGRESS || rc == -EBUSY)
cleanup = false;
else
atomic_dec(&tfm_ctx->via_engine_ctr);
}
if (rc != -EINPROGRESS) {
hwh_advance(hwh, rc);
if (cleanup) {
if (hwh->walkbytes > 0)
hwh_advance(hwh, rc);
memzero_explicit(kmac_ctx, sizeof(*kmac_ctx));
}
@ -649,6 +659,7 @@ static int phmac_final(struct ahash_request *req)
struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
struct phmac_tfm_ctx *tfm_ctx = crypto_ahash_ctx(tfm);
struct kmac_sha2_ctx *kmac_ctx = &req_ctx->kmac_ctx;
bool cleanup = true;
int rc = 0;
/* Try synchronous operation if no active engine usage */
@ -667,12 +678,14 @@ static int phmac_final(struct ahash_request *req)
req_ctx->async_op = OP_FINAL;
atomic_inc(&tfm_ctx->via_engine_ctr);
rc = crypto_transfer_hash_request_to_engine(phmac_crypto_engine, req);
if (rc != -EINPROGRESS)
if (rc == -EINPROGRESS || rc == -EBUSY)
cleanup = false;
else
atomic_dec(&tfm_ctx->via_engine_ctr);
}
out:
if (rc != -EINPROGRESS)
if (cleanup)
memzero_explicit(kmac_ctx, sizeof(*kmac_ctx));
pr_debug("rc=%d\n", rc);
return rc;
@ -685,6 +698,7 @@ static int phmac_finup(struct ahash_request *req)
struct phmac_tfm_ctx *tfm_ctx = crypto_ahash_ctx(tfm);
struct kmac_sha2_ctx *kmac_ctx = &req_ctx->kmac_ctx;
struct hash_walk_helper *hwh = &req_ctx->hwh;
bool cleanup = true;
int rc;
/* prep the walk in the request context */
@ -716,15 +730,17 @@ static int phmac_finup(struct ahash_request *req)
/* req->async_op has been set to either OP_FINUP or OP_FINAL */
atomic_inc(&tfm_ctx->via_engine_ctr);
rc = crypto_transfer_hash_request_to_engine(phmac_crypto_engine, req);
if (rc != -EINPROGRESS)
if (rc == -EINPROGRESS || rc == -EBUSY)
cleanup = false;
else
atomic_dec(&tfm_ctx->via_engine_ctr);
}
if (rc != -EINPROGRESS)
if (cleanup && hwh->walkbytes > 0)
hwh_advance(hwh, rc);
out:
if (rc != -EINPROGRESS)
if (cleanup)
memzero_explicit(kmac_ctx, sizeof(*kmac_ctx));
pr_debug("rc=%d\n", rc);
return rc;
@ -914,7 +930,7 @@ static int phmac_do_one_request(struct crypto_engine *engine, void *areq)
atomic_dec(&tfm_ctx->via_engine_ctr);
crypto_finalize_hash_request(engine, req, rc);
local_bh_enable();
return rc;
return 0;
}
#define S390_ASYNC_PHMAC_ALG(x) \

1
arch/s390/include/asm/pai.h
View File

@ -76,7 +76,6 @@ static __always_inline void pai_kernel_exit(struct pt_regs *regs)
}
#define PAI_SAVE_AREA(x) ((x)->hw.event_base)
#define PAI_CPU_MASK(x) ((x)->hw.addr_filters)
#define PAI_PMU_IDX(x) ((x)->hw.last_tag)
#define PAI_SWLIST(x) (&(x)->hw.tp_list)

54
arch/s390/kernel/debug.c
View File

@ -182,7 +182,7 @@ static struct debug_param_t {
static int debug_param_num;
/* functions */
static void debug_get_param(const char *name, int *level, int *pages)
static void debug_get_param(const char *name, int *level, int *pages, bool quiet)
{
struct debug_param_t *p;
int i;
@ -192,11 +192,13 @@ static void debug_get_param(const char *name, int *level, int *pages)
if (!glob_match(p->name, name))
continue;
if (level && p->level != PARAM_UNSET) {
pr_info("%s: override level to %d\n", name, p->level);
if (!quiet)
pr_info("%s: override level to %d\n", name, p->level);
*level = p->level;
}
if (pages && p->pages != PARAM_UNSET) {
pr_info("%s: override pages to %d\n", name, p->pages);
if (!quiet)
pr_info("%s: override pages to %d\n", name, p->pages);
*pages = p->pages;
}
}
@ -251,7 +253,7 @@ static int __init s390dbf_parse(char *arg)
* regular memory allocations are possible.
*/
for (i = 0, id = __s390dbf_info; &id[i] < __s390dbf_info_end; i++)
debug_get_param(id[i]->name, &id[i]->level, NULL);
debug_get_param(id[i]->name, &id[i]->level, NULL, false);
return rc;
}
@ -395,7 +397,7 @@ static debug_info_t *debug_info_create(const char *name, int pages_per_area,
int level = DEBUG_DEFAULT_LEVEL;
debug_info_t *rc;
debug_get_param(name, &level, &pages_per_area);
debug_get_param(name, &level, &pages_per_area, false);
rc = debug_info_alloc(name, pages_per_area, nr_areas, buf_size, level, ALL_AREAS);
if (!rc)
goto out;
@ -960,7 +962,7 @@ void debug_register_static(debug_info_t *id, int pages_per_area, int nr_areas)
return;
}
debug_get_param(id->name, &id->level, &pages_per_area);
debug_get_param(id->name, &id->level, &pages_per_area, false);
copy = debug_info_alloc("", pages_per_area, nr_areas, id->buf_size,
id->level, ALL_AREAS);
if (!copy) {
@ -1074,9 +1076,6 @@ static void _debug_set_level(debug_info_t *id, int new_level)
{
unsigned long flags;
if (!id)
return;
if (new_level == DEBUG_OFF_LEVEL) {
pr_info("%s: switched off\n", id->name);
} else if ((new_level > DEBUG_MAX_LEVEL) || (new_level < 0)) {
@ -1101,8 +1100,14 @@ static void _debug_set_level(debug_info_t *id, int new_level)
*/
void debug_set_level(debug_info_t *id, int new_level)
{
/* Level specified via kernel parameter takes precedence */
debug_get_param(id->name, &new_level, NULL);
if (!id)
return;
/*
* Level specified via kernel parameter takes precedence. The override
* was already announced during registration, so stay quiet here.
*/
debug_get_param(id->name, &new_level, NULL, true);
_debug_set_level(id, new_level);
}
@ -1278,7 +1283,7 @@ void debug_set_critical(void)
debug_entry_t *debug_event_common(debug_info_t *id, int level, const void *buf,
int len)
{
debug_entry_t *active;
debug_entry_t *active = NULL;
unsigned long flags;
if (!debug_active || !id->areas)
@ -1289,6 +1294,8 @@ debug_entry_t *debug_event_common(debug_info_t *id, int level, const void *buf,
} else {
raw_spin_lock_irqsave(&id->lock, flags);
}
if (!id->areas)
goto out;
do {
active = get_active_entry(id);
memcpy(DEBUG_DATA(active), buf, min(len, id->buf_size));
@ -1298,7 +1305,7 @@ debug_entry_t *debug_event_common(debug_info_t *id, int level, const void *buf,
len -= id->buf_size;
buf += id->buf_size;
} while (len > 0);
out:
raw_spin_unlock_irqrestore(&id->lock, flags);
return active;
}
@ -1311,7 +1318,7 @@ EXPORT_SYMBOL(debug_event_common);
debug_entry_t *debug_exception_common(debug_info_t *id, int level,
const void *buf, int len)
{
debug_entry_t *active;
debug_entry_t *active = NULL;
unsigned long flags;
if (!debug_active || !id->areas)
@ -1322,6 +1329,8 @@ debug_entry_t *debug_exception_common(debug_info_t *id, int level,
} else {
raw_spin_lock_irqsave(&id->lock, flags);
}
if (!id->areas)
goto out;
do {
active = get_active_entry(id);
memcpy(DEBUG_DATA(active), buf, min(len, id->buf_size));
@ -1331,7 +1340,7 @@ debug_entry_t *debug_exception_common(debug_info_t *id, int level,
len -= id->buf_size;
buf += id->buf_size;
} while (len > 0);
out:
raw_spin_unlock_irqrestore(&id->lock, flags);
return active;
}
@ -1357,7 +1366,7 @@ static inline int debug_count_numargs(char *string)
debug_entry_t *__debug_sprintf_event(debug_info_t *id, int level, char *string, ...)
{
debug_sprintf_entry_t *curr_event;
debug_entry_t *active;
debug_entry_t *active = NULL;
unsigned long flags;
int numargs, idx;
va_list ap;
@ -1372,6 +1381,8 @@ debug_entry_t *__debug_sprintf_event(debug_info_t *id, int level, char *string,
} else {
raw_spin_lock_irqsave(&id->lock, flags);
}
if (!id->areas)
goto out;
active = get_active_entry(id);
curr_event = (debug_sprintf_entry_t *) DEBUG_DATA(active);
va_start(ap, string);
@ -1380,6 +1391,7 @@ debug_entry_t *__debug_sprintf_event(debug_info_t *id, int level, char *string,
curr_event->args[idx] = va_arg(ap, long);
va_end(ap);
debug_finish_entry(id, active, level, 0);
out:
raw_spin_unlock_irqrestore(&id->lock, flags);
return active;
@ -1392,7 +1404,7 @@ EXPORT_SYMBOL(__debug_sprintf_event);
debug_entry_t *__debug_sprintf_exception(debug_info_t *id, int level, char *string, ...)
{
debug_sprintf_entry_t *curr_event;
debug_entry_t *active;
debug_entry_t *active = NULL;
unsigned long flags;
int numargs, idx;
va_list ap;
@ -1408,6 +1420,8 @@ debug_entry_t *__debug_sprintf_exception(debug_info_t *id, int level, char *stri
} else {
raw_spin_lock_irqsave(&id->lock, flags);
}
if (!id->areas)
goto out;
active = get_active_entry(id);
curr_event = (debug_sprintf_entry_t *)DEBUG_DATA(active);
va_start(ap, string);
@ -1416,6 +1430,7 @@ debug_entry_t *__debug_sprintf_exception(debug_info_t *id, int level, char *stri
curr_event->args[idx] = va_arg(ap, long);
va_end(ap);
debug_finish_entry(id, active, level, 1);
out:
raw_spin_unlock_irqrestore(&id->lock, flags);
return active;
@ -1658,9 +1673,11 @@ static void debug_flush(debug_info_t *id, int area)
unsigned long flags;
int i, j;
if (!id || !id->areas)
if (!id)
return;
raw_spin_lock_irqsave(&id->lock, flags);
if (!id->areas)
goto out;
if (area == DEBUG_FLUSH_ALL) {
id->active_area = 0;
memset(id->active_entries, 0, id->nr_areas * sizeof(int));
@ -1675,6 +1692,7 @@ static void debug_flush(debug_info_t *id, int area)
for (i = 0; i < id->pages_per_area; i++)
memset(id->areas[area][i], 0, PAGE_SIZE);
}
out:
raw_spin_unlock_irqrestore(&id->lock, flags);
}

196
arch/s390/kernel/perf_pai.c
View File

@ -67,6 +67,7 @@ struct pai_mapptr {
static struct pai_root { /* Anchor to per CPU data */
refcount_t refcnt; /* Overall active events */
atomic_t tskctx; /* Overall per-task events */
struct pai_mapptr __percpu *mapptr;
} pai_root[PAI_PMU_MAX];
@ -93,14 +94,15 @@ struct pai_pmu { /* Define PAI PMU characteristics */
static struct pai_pmu pai_pmu[]; /* Forward declaration */
/* Free per CPU data when the last event is removed. */
static void pai_root_free(int idx)
static void pai_root_free(int idx, int tasks)
{
if (refcount_dec_and_test(&pai_root[idx].refcnt)) {
if (refcount_sub_and_test(tasks, &pai_root[idx].refcnt)) {
free_percpu(pai_root[idx].mapptr);
pai_root[idx].mapptr = NULL;
}
debug_sprintf_event(paidbg, 5, "%s root[%d].refcount %d\n", __func__,
idx, refcount_read(&pai_root[idx].refcnt));
debug_sprintf_event(paidbg, 5, "%s root[%d].refcount %d tskctx %d\n",
__func__, idx, refcount_read(&pai_root[idx].refcnt),
atomic_read(&pai_root[idx].tskctx));
}
/*
@ -137,40 +139,54 @@ static void pai_free(struct pai_mapptr *mp)
mp->mapptr = NULL;
}
/* Adjust usage counters and remove allocated memory when all users are
* gone.
*/
static void pai_event_destroy_cpu(struct perf_event *event, int cpu)
/* Called under mutex_lock */
static void pai_event_destroy_cpu(int idx, int cpu, bool hotplug)
{
int idx = PAI_PMU_IDX(event);
struct pai_mapptr *mp = per_cpu_ptr(pai_root[idx].mapptr, cpu);
struct pai_map *cpump = mp->mapptr;
struct pai_mapptr *mp;
struct pai_map *cpump;
int tasks = 1;
mutex_lock(&pai_reserve_mutex);
debug_sprintf_event(paidbg, 5, "%s event %#llx idx %d cpu %d users %d "
"refcnt %u\n", __func__, event->attr.config, idx,
event->cpu, cpump->active_events,
refcount_read(&cpump->refcnt));
if (refcount_dec_and_test(&cpump->refcnt))
/* Check reference count and return when all gone.
* 1. An event is installed on online CPU X.
* 2. CPU x is offlined and the per-CPU data is removed.
* 3. Event is destroyed via close system call.
*/
if (!refcount_read(&pai_root[idx].refcnt))
return; /* No events at all */
mp = per_cpu_ptr(pai_root[idx].mapptr, cpu);
if (!mp || !mp->mapptr) /* No events on that CPU */
return;
/* When hotplug is true, invocation is from CPU hotplug callback.
* Delete per-CPU resource and adjust refcnt when per-task events
* are currently active. This can be more than one.
* In this case adjust counters.
*/
if (hotplug)
tasks = atomic_read(&pai_root[idx].tskctx);
cpump = mp->mapptr;
if (refcount_sub_and_test(tasks, &cpump->refcnt))
pai_free(mp);
pai_root_free(idx);
mutex_unlock(&pai_reserve_mutex);
pai_root_free(idx, tasks);
}
static void pai_event_destroy(struct perf_event *event)
{
int cpu;
int cpu = 0, idx = PAI_PMU_IDX(event);
free_page(PAI_SAVE_AREA(event));
cpus_read_lock();
mutex_lock(&pai_reserve_mutex);
if (event->cpu == -1) {
struct cpumask *mask = PAI_CPU_MASK(event);
for_each_cpu(cpu, mask)
pai_event_destroy_cpu(event, cpu);
kfree(mask);
atomic_dec(&pai_root[idx].tskctx);
for_each_online_cpu(cpu)
pai_event_destroy_cpu(idx, cpu, false);
} else {
pai_event_destroy_cpu(event, event->cpu);
pai_event_destroy_cpu(idx, event->cpu, false);
}
mutex_unlock(&pai_reserve_mutex);
cpus_read_unlock();
}
static void paicrypt_event_destroy(struct perf_event *event)
@ -234,25 +250,30 @@ static u64 paicrypt_getall(struct perf_event *event)
return sum;
}
/* Check concurrent access of counting and sampling for crypto events.
* This function is called in process context and it is save to block.
* When the event initialization functions fails, no other call back will
* be invoked.
*
* Allocate the memory for the event.
*/
static int pai_alloc_cpu(struct perf_event *event, int cpu)
/* Called under mutex_lock */
static int pai_alloc_cpu(int idx, int cpu, bool hotplug)
{
int rc, idx = PAI_PMU_IDX(event);
struct pai_map *cpump = NULL;
bool need_paiext_cb = false;
struct pai_mapptr *mp;
int tasks = 1, rc = 0;
/* When hotplug is true, invocation is from CPU hotplug callback.
* Allocate per-CPU resource when per-task events are currently active.
* This can be more than one. In this case adjust all reference
* counters. Otherwise return, this ensures memory is only allocated
* when needed.
*/
if (hotplug) {
tasks = atomic_read(&pai_root[idx].tskctx);
if (!tasks)
goto out;
}
mutex_lock(&pai_reserve_mutex);
/* Allocate root node */
rc = pai_root_alloc(idx);
if (rc)
goto unlock;
goto out;
/* Allocate node for this event */
mp = per_cpu_ptr(pai_root[idx].mapptr, cpu);
@ -296,28 +317,45 @@ static int pai_alloc_cpu(struct perf_event *event, int cpu)
goto undo;
}
INIT_LIST_HEAD(&cpump->syswide_list);
refcount_set(&cpump->refcnt, 1);
refcount_set(&cpump->refcnt, tasks);
rc = 0;
} else {
refcount_inc(&cpump->refcnt);
refcount_add(tasks, &cpump->refcnt);
}
/* If tasks is greater than 1, we are called from CPU hotplug path
* and need to adjust the pai_root[idx].refcnt by the number of
* per-process events. Function pai_root_alloc(idx) already
* incremented by one. Adjust for the rest.
*/
if (tasks > 1)
refcount_add(tasks - 1, &pai_root[idx].refcnt);
undo:
if (rc) {
/* Error in allocation of event, decrement anchor. Since
* the event in not created, its destroy() function is never
* invoked. Adjust the reference counter for the anchor.
* The failure happened in the case of variable
* cpump == NULL branch above. The pai_root[XXX].refcnt has
* been incremented by one. Then the per-CPU allocation
* failed, so decrement it by one, regardless of tasks.
*/
pai_root_free(idx);
pai_root_free(idx, 1);
}
unlock:
mutex_unlock(&pai_reserve_mutex);
out:
/* If rc is non-zero, no increment of counter/sampler was done. */
return rc;
}
/* Check concurrent access of counting and sampling for PAI events.
* This function is called in process context and it is safe to block.
* When the event initialization functions fails, no other call back will
* be invoked.
* Called under mutex_lock.
*/
static int pai_alloc(struct perf_event *event)
{
int idx = PAI_PMU_IDX(event);
struct cpumask *maskptr;
int cpu, rc = -ENOMEM;
@ -326,24 +364,20 @@ static int pai_alloc(struct perf_event *event)
goto out;
for_each_online_cpu(cpu) {
rc = pai_alloc_cpu(event, cpu);
rc = pai_alloc_cpu(idx, cpu, false);
if (rc) {
for_each_cpu(cpu, maskptr)
pai_event_destroy_cpu(event, cpu);
kfree(maskptr);
goto out;
pai_event_destroy_cpu(idx, cpu, false);
goto undo;
}
cpumask_set_cpu(cpu, maskptr);
}
/*
* On error all cpumask are freed and all events have been destroyed.
* Save of which CPUs data structures have been allocated for.
* Release them in pai_event_destroy call back function
* for this event.
*/
PAI_CPU_MASK(event) = maskptr;
rc = 0;
/* Trace per-task events for CPU hotplug. */
atomic_inc(&pai_root[idx].tskctx);
undo:
kfree(maskptr);
out:
return rc;
}
@ -391,10 +425,14 @@ static int pai_event_init(struct perf_event *event, int idx)
}
}
cpus_read_lock();
mutex_lock(&pai_reserve_mutex);
if (event->cpu >= 0)
rc = pai_alloc_cpu(event, event->cpu);
rc = pai_alloc_cpu(idx, event->cpu, false);
else
rc = pai_alloc(event);
mutex_unlock(&pai_reserve_mutex);
cpus_read_unlock();
if (rc) {
free_page(PAI_SAVE_AREA(event));
goto out;
@ -1239,8 +1277,35 @@ static int __init paipmu_setup(void)
return install_ok;
}
static int pai_online_cpu(unsigned int cpu)
{
int rc;
mutex_lock(&pai_reserve_mutex);
rc = pai_alloc_cpu(PAI_PMU_CRYPTO, cpu, true);
if (rc)
goto out;
rc = pai_alloc_cpu(PAI_PMU_EXT, cpu, true);
if (rc)
pai_event_destroy_cpu(PAI_PMU_CRYPTO, cpu, true);
out:
mutex_unlock(&pai_reserve_mutex);
return rc;
}
static int pai_offline_cpu(unsigned int cpu)
{
mutex_lock(&pai_reserve_mutex);
pai_event_destroy_cpu(PAI_PMU_CRYPTO, cpu, true);
pai_event_destroy_cpu(PAI_PMU_EXT, cpu, true);
mutex_unlock(&pai_reserve_mutex);
return 0;
}
static int __init pai_init(void)
{
int state, rc;
/* Setup s390dbf facility */
paidbg = debug_register("pai", 1, 1, 128);
if (!paidbg) {
@ -1249,13 +1314,24 @@ static int __init pai_init(void)
}
debug_register_view(paidbg, &debug_sprintf_view);
if (!paipmu_setup()) {
/* No PMU registration, no need for debug buffer */
debug_unregister_view(paidbg, &debug_sprintf_view);
debug_unregister(paidbg);
return -ENODEV;
}
/* CPUHP_BP_PREPARE_DYN --> before CPU is brought online */
state = cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "perf/pai:prepare",
pai_online_cpu, pai_offline_cpu);
rc = state < 0 ? state : 0;
if (rc < 0)
goto out_debug;
rc = -ENODEV;
if (!paipmu_setup())
goto out_cpuhp;
return 0;
out_cpuhp:
cpuhp_remove_state(state);
out_debug:
debug_unregister_view(paidbg, &debug_sprintf_view);
debug_unregister(paidbg);
return rc;
}
device_initcall(pai_init);

6
arch/x86/crypto/aria-aesni-avx2-asm_64.S
View File

@ -982,6 +982,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_encrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_encrypt_32way)
@ -1007,6 +1008,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_decrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_decrypt_32way)
@ -1209,6 +1211,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_ctr_crypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %r10);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_ctr_crypt_32way)
@ -1359,6 +1362,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_encrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_gfni_encrypt_32way)
@ -1384,6 +1388,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_decrypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %rax);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_gfni_decrypt_32way)
@ -1428,6 +1433,7 @@ SYM_TYPED_FUNC_START(aria_aesni_avx2_gfni_ctr_crypt_32way)
%ymm8, %ymm9, %ymm10, %ymm11, %ymm12, %ymm13, %ymm14,
%ymm15, %r10);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_aesni_avx2_gfni_ctr_crypt_32way)

3
arch/x86/crypto/aria-gfni-avx512-asm_64.S
View File

@ -800,6 +800,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_encrypt_64way)
%zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14,
%zmm15, %rax);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_gfni_avx512_encrypt_64way)
@ -825,6 +826,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_decrypt_64way)
%zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14,
%zmm15, %rax);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_gfni_avx512_decrypt_64way)
@ -966,6 +968,7 @@ SYM_TYPED_FUNC_START(aria_gfni_avx512_ctr_crypt_64way)
%zmm9, %zmm8, %zmm11, %zmm10, %zmm12, %zmm13, %zmm14,
%zmm15, %r10);
vzeroupper;
FRAME_END
RET;
SYM_FUNC_END(aria_gfni_avx512_ctr_crypt_64way)

33
arch/x86/events/intel/core.c
View File

@ -3125,6 +3125,27 @@ static void intel_pmu_del_event(struct perf_event *event)
this_cpu_ptr(&cpu_hw_events)->n_late_setup--;
}
int __intel_pmu_quiesce(void)
{
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
int pmu_enabled = cpuc->enabled;
cpuc->enabled = 0;
if (pmu_enabled)
intel_pmu_disable_all();
return pmu_enabled;
}
void __intel_pmu_resume(int pmu_enabled)
{
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
cpuc->enabled = pmu_enabled;
if (pmu_enabled)
intel_pmu_enable_all(0);
}
static int icl_set_topdown_event_period(struct perf_event *event)
{
struct hw_perf_event *hwc = &event->hw;
@ -3316,16 +3337,13 @@ static void intel_pmu_read_event(struct perf_event *event)
if (event->hw.flags & (PERF_X86_EVENT_AUTO_RELOAD | PERF_X86_EVENT_TOPDOWN) ||
is_pebs_counter_event_group(event)) {
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
bool pmu_enabled = cpuc->enabled;
int pmu_enabled;
/* Only need to call update_topdown_event() once for group read. */
if (is_metric_event(event) && (cpuc->txn_flags & PERF_PMU_TXN_READ))
return;
cpuc->enabled = 0;
if (pmu_enabled)
intel_pmu_disable_all();
pmu_enabled = __intel_pmu_quiesce();
/*
* If the PEBS counters snapshotting is enabled,
* the topdown event is available in PEBS records.
@ -3334,10 +3352,7 @@ static void intel_pmu_read_event(struct perf_event *event)
static_call(intel_pmu_update_topdown_event)(event, NULL);
else
intel_pmu_drain_pebs_buffer();
cpuc->enabled = pmu_enabled;
if (pmu_enabled)
intel_pmu_enable_all(0);
__intel_pmu_resume(pmu_enabled);
return;
}

18
arch/x86/events/intel/ds.c
View File

@ -1242,8 +1242,11 @@ int intel_pmu_drain_bts_buffer(void)
void intel_pmu_drain_pebs_buffer(void)
{
struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events);
struct perf_sample_data data;
WARN_ON_ONCE(cpuc->enabled);
static_call(x86_pmu_drain_pebs)(NULL, &data);
}
@ -1864,8 +1867,11 @@ static void intel_pmu_pebs_via_pt_enable(struct perf_event *event)
static inline void intel_pmu_drain_large_pebs(struct cpu_hw_events *cpuc)
{
if (cpuc->n_pebs == cpuc->n_large_pebs &&
cpuc->n_pebs != cpuc->n_pebs_via_pt)
cpuc->n_pebs != cpuc->n_pebs_via_pt) {
int enabled = __intel_pmu_quiesce();
intel_pmu_drain_pebs_buffer();
__intel_pmu_resume(enabled);
}
}
static void __intel_pmu_pebs_enable(struct perf_event *event)
@ -2432,7 +2438,7 @@ static inline void __setup_pebs_basic_group(struct perf_event *event,
{
/* The ip in basic is EventingIP */
set_linear_ip(regs, ip);
regs->flags = PERF_EFLAGS_EXACT;
regs->flags |= PERF_EFLAGS_EXACT;
setup_pebs_time(event, data, tsc);
if (sample_type & PERF_SAMPLE_WEIGHT_STRUCT)
@ -2444,9 +2450,17 @@ static inline void __setup_pebs_gpr_group(struct perf_event *event,
struct pebs_gprs *gprs,
u64 sample_type)
{
/*
* Update flags with PEBS data. PERF_EFLAGS_EXACT must be set
* in previous basic group handling.
*/
regs->flags = gprs->flags | PERF_EFLAGS_EXACT;
if (event->attr.precise_ip < 2) {
set_linear_ip(regs, gprs->ip);
regs->flags &= ~PERF_EFLAGS_EXACT;
} else if (regs->flags & X86_VM_MASK) {
regs->flags ^= (PERF_EFLAGS_VM | X86_VM_MASK);
}
if (sample_type & (PERF_SAMPLE_REGS_INTR | PERF_SAMPLE_REGS_USER))

3
arch/x86/events/perf_event.h
View File

@ -1638,6 +1638,9 @@ static __always_inline void __intel_pmu_lbr_disable(void)
wrmsrq(MSR_IA32_DEBUGCTLMSR, debugctl);
}
extern int __intel_pmu_quiesce(void);
extern void __intel_pmu_resume(int pmu_enabled);
int intel_pmu_save_and_restart(struct perf_event *event);
struct event_constraint *

19
arch/x86/kernel/amd_node.c
View File

@ -38,7 +38,6 @@ static struct pci_dev **amd_roots;
/* Protect the PCI config register pairs used for SMN. */
static DEFINE_MUTEX(smn_mutex);
static bool smn_exclusive;
#define SMN_INDEX_OFFSET 0x60
#define SMN_DATA_OFFSET 0x64
@ -91,11 +90,16 @@ static int __amd_smn_rw(u8 i_off, u8 d_off, u16 node, u32 address, u32 *value, b
if (node >= amd_num_nodes())
return err;
root = amd_roots[node];
if (!root)
/*
* Uninitialized amd_roots indicates pci_request_config_region_exclusive()
* didn't run or failed and thus the kernel cannot rely on having
* exclusive access to SMN registers so prevent that.
*/
if (!amd_roots)
return err;
if (!smn_exclusive)
root = amd_roots[node];
if (!root)
return err;
guard(mutex)(&smn_mutex);
@ -287,6 +291,11 @@ static int __init amd_smn_init(void)
return -ENOMEM;
roots_per_node = num_roots / num_nodes;
if (!roots_per_node) {
if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR))
pr_warn(FW_BUG "Error detecting roots per node.\n");
roots_per_node = 1;
}
count = 0;
node = 0;
@ -308,8 +317,6 @@ static int __init amd_smn_init(void)
debugfs_create_file("value", 0600, debugfs_dir, NULL, &smn_value_fops);
}
smn_exclusive = true;
return 0;
}

2
arch/x86/kernel/cpu/mce/amd.c
View File

@ -865,7 +865,7 @@ static void amd_deferred_error_interrupt(void)
void mce_amd_handle_storm(unsigned int bank, bool on)
{
threshold_restart_bank(bank, on);
threshold_restart_bank(bank, !on);
}
static void amd_reset_thr_limit(unsigned int bank)

1
drivers/accel/ethosu/ethosu_device.h
View File

@ -87,6 +87,7 @@ struct gen_pool;
#define PMU_EV_TYPE_IDLE 0x20
enum ethosu_cmds {
NPU_OP_STOP = 0x0,
NPU_OP_CONV = 0x2,
NPU_OP_DEPTHWISE = 0x3,
NPU_OP_POOL = 0x5,

3
drivers/accel/ethosu/ethosu_drv.c
View File

@ -281,8 +281,6 @@ static int ethosu_device_suspend(struct device *dev)
static int ethosu_sram_init(struct ethosu_device *ethosudev)
{
ethosudev->npu_info.sram_size = 0;
ethosudev->srampool = of_gen_pool_get(ethosudev->base.dev->of_node, "sram", 0);
if (!ethosudev->srampool)
return 0;
@ -293,6 +291,7 @@ static int ethosu_sram_init(struct ethosu_device *ethosudev)
ethosudev->npu_info.sram_size,
&ethosudev->sramphys);
if (!ethosudev->sram) {
ethosudev->npu_info.sram_size = 0;
dev_err(ethosudev->base.dev, "failed to allocate from SRAM pool\n");
return -ENOMEM;
}

9
drivers/accel/ethosu/ethosu_gem.c
View File

@ -390,6 +390,7 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,
struct ethosu_validated_cmdstream_info __free(kfree) *info = kzalloc_obj(*info);
struct ethosu_device *edev = to_ethosu_device(ddev);
u32 *bocmds = bo->base.vaddr;
bool ends_with_stop = false;
struct cmd_state st;
int i, ret;
@ -426,6 +427,11 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,
}
switch (cmd) {
case NPU_OP_STOP:
if (i != size / 4 - 1)
return -EINVAL;
ends_with_stop = true;
break;
case NPU_OP_DMA_START:
srclen = dma_length(info, &st.dma, &st.dma.src);
dstlen = dma_length(info, &st.dma, &st.dma.dst);
@ -688,6 +694,9 @@ static int ethosu_gem_cmdstream_copy_and_validate(struct drm_device *ddev,
}
}
if (!ends_with_stop)
return -EINVAL;
for (i = 0; i < NPU_BASEP_REGION_MAX; i++) {
if (!info->region_size[i])
continue;

16
drivers/accel/ethosu/ethosu_job.c
View File

@ -343,7 +343,7 @@ int ethosu_job_init(struct ethosu_device *edev)
ret = devm_request_threaded_irq(dev, edev->irq,
ethosu_job_irq_handler,
ethosu_job_irq_handler_thread,
IRQF_SHARED, KBUILD_MODNAME,
0, KBUILD_MODNAME,
edev);
if (ret) {
dev_err(dev, "failed to request irq\n");
@ -374,12 +374,10 @@ int ethosu_job_open(struct ethosu_file_priv *ethosu_priv)
{
struct ethosu_device *dev = ethosu_priv->edev;
struct drm_gpu_scheduler *sched = &dev->sched;
int ret;
ret = drm_sched_entity_init(&ethosu_priv->sched_entity,
DRM_SCHED_PRIORITY_NORMAL,
&sched, 1, NULL);
return WARN_ON(ret);
return drm_sched_entity_init(&ethosu_priv->sched_entity,
DRM_SCHED_PRIORITY_NORMAL,
&sched, 1, NULL);
}
void ethosu_job_close(struct ethosu_file_priv *ethosu_priv)
@ -449,13 +447,13 @@ static int ethosu_ioctl_submit_job(struct drm_device *dev, struct drm_file *file
if (!cmd_info->region_size[i])
continue;
if (i == ETHOSU_SRAM_REGION) {
if (cmd_info->region_size[i] <= edev->npu_info.sram_size)
if (cmd_info->region_size[i] <= ejob->sram_size)
continue;
dev_err(dev->dev,
"cmd stream region %d size greater than SRAM size (%llu > %u)\n",
"cmd stream region %d size greater than job SRAM size (%llu > %u)\n",
i, cmd_info->region_size[i],
edev->npu_info.sram_size);
ejob->sram_size);
ret = -EINVAL;
goto out_cleanup_job;
}

87
drivers/accel/ivpu/ivpu_fw_log.c
View File

@ -26,10 +26,17 @@ MODULE_PARM_DESC(fw_log_level,
" error=" __stringify(IVPU_FW_LOG_ERROR)
" fatal=" __stringify(IVPU_FW_LOG_FATAL));
struct ivpu_fw_log_desc {
struct vpu_tracing_buffer_header *log;
u32 header_size;
u32 size;
};
static int fw_log_from_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, u32 *offset,
struct vpu_tracing_buffer_header **out_log)
struct ivpu_fw_log_desc *desc)
{
struct vpu_tracing_buffer_header *log;
u32 header_size, size;
if ((*offset + sizeof(*log)) > ivpu_bo_size(bo))
return -EINVAL;
@ -39,26 +46,32 @@ static int fw_log_from_bo(struct ivpu_device *vdev, struct ivpu_bo *bo, u32 *off
if (log->vpu_canary_start != VPU_TRACING_BUFFER_CANARY)
return -EINVAL;
if (log->header_size < sizeof(*log) || log->header_size > 1024) {
ivpu_dbg(vdev, FW_BOOT, "Invalid header size 0x%x\n", log->header_size);
header_size = READ_ONCE(log->header_size);
size = READ_ONCE(log->size);
if (header_size < sizeof(*log) || header_size > 1024) {
ivpu_dbg(vdev, FW_BOOT, "Invalid header size 0x%x\n", header_size);
return -EINVAL;
}
if (log->size < log->header_size) {
ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", log->size);
if ((char *)log + size > (char *)ivpu_bo_vaddr(bo) + ivpu_bo_size(bo)) {
ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", size);
return -EINVAL;
}
if ((char *)log + log->size > (char *)ivpu_bo_vaddr(bo) + ivpu_bo_size(bo)) {
ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x\n", log->size);
if (size < header_size) {
ivpu_dbg(vdev, FW_BOOT, "Invalid log size 0x%x < header size 0x%x\n",
size, header_size);
return -EINVAL;
}
*out_log = log;
*offset += log->size;
desc->log = log;
desc->header_size = header_size;
desc->size = size;
*offset += size;
ivpu_dbg(vdev, FW_BOOT,
"FW log name \"%s\", write offset 0x%x size 0x%x, wrap count %d, hdr version %d size %d format %d, alignment %d",
log->name, log->write_index, log->size, log->wrap_count, log->header_version,
log->header_size, log->format, log->alignment);
"FW log name \"%.*s\", write offset 0x%x size 0x%x, wrap count %d, hdr version %d size %d format %d, alignment %d",
(int)ARRAY_SIZE(log->name), log->name, log->write_index, size, log->wrap_count,
log->header_version, header_size, log->format, log->alignment);
return 0;
}
@ -94,11 +107,12 @@ static void fw_log_print_lines(char *buffer, u32 size, struct drm_printer *p)
drm_printf(p, "%s", line);
}
static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const char *prefix,
static void fw_log_print_buffer(struct ivpu_fw_log_desc *desc, const char *prefix,
bool only_new_msgs, struct drm_printer *p)
{
char *log_data = (void *)log + log->header_size;
u32 data_size = log->size - log->header_size;
struct vpu_tracing_buffer_header *log = desc->log;
char *log_data = (void *)log + desc->header_size;
u32 data_size = desc->size - desc->header_size;
u32 log_start = only_new_msgs ? READ_ONCE(log->read_index) : 0;
u32 log_end = READ_ONCE(log->write_index);
@ -109,7 +123,8 @@ static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const cha
if (log->wrap_count == log->read_wrap_count) {
if (log_end <= log_start) {
drm_printf(p, "==== %s \"%s\" log empty ====\n", prefix, log->name);
drm_printf(p, "==== %s \"%.*s\" log empty ====\n", prefix,
(int)ARRAY_SIZE(log->name), log->name);
return;
}
} else if (log->wrap_count == log->read_wrap_count + 1) {
@ -119,7 +134,8 @@ static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const cha
log_start = log_end;
}
drm_printf(p, "==== %s \"%s\" log start ====\n", prefix, log->name);
drm_printf(p, "==== %s \"%.*s\" log start ====\n", prefix, (int)ARRAY_SIZE(log->name),
log->name);
if (log_end > log_start) {
fw_log_print_lines(log_data + log_start, log_end - log_start, p);
} else {
@ -127,18 +143,19 @@ static void fw_log_print_buffer(struct vpu_tracing_buffer_header *log, const cha
fw_log_print_lines(log_data, log_end, p);
}
drm_printf(p, "\n\x1b[0m"); /* add new line and clear formatting */
drm_printf(p, "==== %s \"%s\" log end ====\n", prefix, log->name);
drm_printf(p, "==== %s \"%.*s\" log end ====\n", prefix, (int)ARRAY_SIZE(log->name),
log->name);
}
static void
fw_log_print_all_in_bo(struct ivpu_device *vdev, const char *name,
struct ivpu_bo *bo, bool only_new_msgs, struct drm_printer *p)
{
struct vpu_tracing_buffer_header *log;
struct ivpu_fw_log_desc desc;
u32 next = 0;
while (fw_log_from_bo(vdev, bo, &next, &log) == 0)
fw_log_print_buffer(log, name, only_new_msgs, p);
while (fw_log_from_bo(vdev, bo, &next, &desc) == 0)
fw_log_print_buffer(&desc, name, only_new_msgs, p);
}
void ivpu_fw_log_print(struct ivpu_device *vdev, bool only_new_msgs, struct drm_printer *p)
@ -149,36 +166,36 @@ void ivpu_fw_log_print(struct ivpu_device *vdev, bool only_new_msgs, struct drm_
void ivpu_fw_log_mark_read(struct ivpu_device *vdev)
{
struct vpu_tracing_buffer_header *log;
struct ivpu_fw_log_desc desc;
u32 next;
next = 0;
while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &log) == 0) {
log->read_index = READ_ONCE(log->write_index);
log->read_wrap_count = READ_ONCE(log->wrap_count);
while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &desc) == 0) {
desc.log->read_index = READ_ONCE(desc.log->write_index);
desc.log->read_wrap_count = READ_ONCE(desc.log->wrap_count);
}
next = 0;
while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &log) == 0) {
log->read_index = READ_ONCE(log->write_index);
log->read_wrap_count = READ_ONCE(log->wrap_count);
while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &desc) == 0) {
desc.log->read_index = READ_ONCE(desc.log->write_index);
desc.log->read_wrap_count = READ_ONCE(desc.log->wrap_count);
}
}
void ivpu_fw_log_reset(struct ivpu_device *vdev)
{
struct vpu_tracing_buffer_header *log;
struct ivpu_fw_log_desc desc;
u32 next;
next = 0;
while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &log) == 0) {
log->read_index = 0;
log->read_wrap_count = 0;
while (fw_log_from_bo(vdev, vdev->fw->mem_log_crit, &next, &desc) == 0) {
desc.log->read_index = 0;
desc.log->read_wrap_count = 0;
}
next = 0;
while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &log) == 0) {
log->read_index = 0;
log->read_wrap_count = 0;
while (fw_log_from_bo(vdev, vdev->fw->mem_log_verb, &next, &desc) == 0) {
desc.log->read_index = 0;
desc.log->read_wrap_count = 0;
}
}

14
drivers/accel/ivpu/ivpu_gem.h
View File

@ -87,15 +87,23 @@ static inline bool ivpu_bo_is_resident(struct ivpu_bo *bo)
return !!bo->base.pages;
}
static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u32 vpu_addr)
static inline void *ivpu_to_cpu_addr(struct ivpu_bo *bo, u64 vpu_addr, u64 size)
{
u64 bo_size = ivpu_bo_size(bo);
u64 offset;
if (vpu_addr < bo->vpu_addr)
return NULL;
if (vpu_addr >= (bo->vpu_addr + ivpu_bo_size(bo)))
if (size > bo_size)
return NULL;
return ivpu_bo_vaddr(bo) + (vpu_addr - bo->vpu_addr);
offset = vpu_addr - bo->vpu_addr;
if (offset > bo_size - size)
return NULL;
return ivpu_bo_vaddr(bo) + offset;
}
static inline u32 cpu_to_vpu_addr(struct ivpu_bo *bo, void *cpu_addr)

7
drivers/accel/ivpu/ivpu_ipc.c
View File

@ -79,7 +79,7 @@ ivpu_ipc_tx_prepare(struct ivpu_device *vdev, struct ivpu_ipc_consumer *cons,
return -ENOMEM;
}
tx_buf = ivpu_to_cpu_addr(ipc->mem_tx, tx_buf_vpu_addr);
tx_buf = ivpu_to_cpu_addr(ipc->mem_tx, tx_buf_vpu_addr, sizeof(*tx_buf));
if (drm_WARN_ON(&vdev->drm, !tx_buf)) {
gen_pool_free(ipc->mm_tx, tx_buf_vpu_addr, sizeof(*tx_buf));
return -EIO;
@ -420,7 +420,7 @@ void ivpu_ipc_irq_handler(struct ivpu_device *vdev)
return;
}
ipc_hdr = ivpu_to_cpu_addr(ipc->mem_rx, vpu_addr);
ipc_hdr = ivpu_to_cpu_addr(ipc->mem_rx, vpu_addr, sizeof(*ipc_hdr));
if (!ipc_hdr) {
ivpu_warn_ratelimited(vdev, "IPC msg 0x%x out of range\n", vpu_addr);
continue;
@ -429,7 +429,8 @@ void ivpu_ipc_irq_handler(struct ivpu_device *vdev)
jsm_msg = NULL;
if (ipc_hdr->channel != IVPU_IPC_CHAN_BOOT_MSG) {
jsm_msg = ivpu_to_cpu_addr(ipc->mem_rx, ipc_hdr->data_addr);
jsm_msg = ivpu_to_cpu_addr(ipc->mem_rx, ipc_hdr->data_addr,
sizeof(*jsm_msg));
if (!jsm_msg) {
ivpu_warn_ratelimited(vdev, "JSM msg 0x%x out of range\n",
ipc_hdr->data_addr);

26
drivers/block/sunvdc.c
View File

@ -525,6 +525,23 @@ static int __send_request(struct request *req)
err = __vdc_tx_trigger(port);
if (err < 0) {
printk(KERN_ERR PFX "vdc_tx_trigger() failure, err=%d\n", err);
/*
* If the port was reset (-ENOTCONN), the dring and the
* LDC channel including all of its mappings are already
* torn down and reallocated - there is nothing to undo
* and @desc must not be touched.
*
* For any other failure the descriptor was never handed
* to the peer: unmap the cookies and free the descriptor
* again, so that a later retry of the request does not
* leak LDC map table entries.
*/
if (err != -ENOTCONN) {
ldc_unmap(port->vio.lp, desc->cookies,
desc->ncookies);
desc->hdr.state = VIO_DESC_FREE;
rqe->req = NULL;
}
} else {
port->req_id++;
dr->prod = vio_dring_next(dr, dr->prod);
@ -539,6 +556,7 @@ static blk_status_t vdc_queue_rq(struct blk_mq_hw_ctx *hctx,
struct vdc_port *port = hctx->queue->queuedata;
struct vio_dring_state *dr;
unsigned long flags;
int ret;
dr = &port->vio.drings[VIO_DRIVER_TX_RING];
@ -560,7 +578,13 @@ static blk_status_t vdc_queue_rq(struct blk_mq_hw_ctx *hctx,
return BLK_STS_DEV_RESOURCE;
}
if (__send_request(bd->rq) < 0) {
ret = __send_request(bd->rq);
if (ret == -EAGAIN) {
spin_unlock_irqrestore(&port->vio.lock, flags);
/* already spun for 10msec, defer 10msec and retry */
blk_mq_delay_kick_requeue_list(hctx->queue, 10);
return BLK_STS_DEV_RESOURCE;
} else if (ret < 0) {
spin_unlock_irqrestore(&port->vio.lock, flags);
return BLK_STS_IOERR;
}

1
drivers/block/ublk_drv.c
View File

@ -3030,6 +3030,7 @@ static void ublk_queue_reset_io_flags(struct ublk_queue *ubq)
ubq->canceling = false;
spin_unlock(&ubq->cancel_lock);
ubq->fail_io = false;
ubq->force_abort = false;
}
/* device can only be started after all IOs are ready */

6
drivers/bluetooth/btintel_pcie.c
View File

@ -1099,7 +1099,7 @@ static void btintel_pcie_msix_tx_handle(struct btintel_pcie_data *data)
urbd0 = &txq->urbd0s[cr_tia];
if (urbd0->tfd_index > txq->count)
if (urbd0->tfd_index >= txq->count)
return;
cr_tia = (cr_tia + 1) % txq->count;
@ -1599,7 +1599,9 @@ static int btintel_pcie_submit_rx_work(struct btintel_pcie_data *data, u8 status
rfh_hdr = buf;
len = rfh_hdr->packet_len;
if (len <= 0) {
if (len == 0 || len > BTINTEL_PCIE_BUFFER_SIZE - sizeof(*rfh_hdr)) {
bt_dev_err(data->hdev, "Invalid packet_len %d (max %zu)", len,
BTINTEL_PCIE_BUFFER_SIZE - sizeof(*rfh_hdr));
ret = -EINVAL;
goto resubmit;
}

11
drivers/bluetooth/btmtk.c
View File

@ -1374,16 +1374,6 @@ int btmtk_usb_setup(struct hci_dev *hdev)
break;
case 0x7922:
case 0x7925:
/*
* A remote wakeup could cause the device completely unresponsive, and
* recovering from such a state needs a power cycle.
*
* Since the remote wakeup capability is super broken, just disable it
* to get rid of the troubles. The device can still be autosuspended
* when the bluetooth interface is closed.
*/
device_set_wakeup_capable(&btmtk_data->udev->dev, false);
fallthrough;
case 0x7961:
case 0x7902:
case 0x6639:
@ -1587,5 +1577,6 @@ MODULE_FIRMWARE(FIRMWARE_MT7663);
MODULE_FIRMWARE(FIRMWARE_MT7668);
MODULE_FIRMWARE(FIRMWARE_MT7922);
MODULE_FIRMWARE(FIRMWARE_MT7961);
MODULE_FIRMWARE(FIRMWARE_MT7920);
MODULE_FIRMWARE(FIRMWARE_MT7925);
MODULE_FIRMWARE(FIRMWARE_MT7927);

1
drivers/bluetooth/btmtk.h
View File

@ -7,6 +7,7 @@
#define FIRMWARE_MT7922 "mediatek/BT_RAM_CODE_MT7922_1_1_hdr.bin"
#define FIRMWARE_MT7902 "mediatek/BT_RAM_CODE_MT7902_1_1_hdr.bin"
#define FIRMWARE_MT7961 "mediatek/BT_RAM_CODE_MT7961_1_2_hdr.bin"
#define FIRMWARE_MT7920 "mediatek/BT_RAM_CODE_MT7961_1a_2_hdr.bin"
#define FIRMWARE_MT7925 "mediatek/mt7925/BT_RAM_CODE_MT7925_1_1_hdr.bin"
#define FIRMWARE_MT7927 "mediatek/mt7927/BT_RAM_CODE_MT6639_2_1_hdr.bin"

6
drivers/bluetooth/btqcomsmd.c
View File

@ -188,7 +188,10 @@ static int btqcomsmd_probe(struct platform_device *pdev)
return 0;
hci_free_dev:
rpmsg_destroy_ept(btq->cmd_channel);
rpmsg_destroy_ept(btq->acl_channel);
hci_free_dev(hdev);
return ret;
destroy_cmd_channel:
rpmsg_destroy_ept(btq->cmd_channel);
destroy_acl_channel:
@ -202,10 +205,11 @@ static void btqcomsmd_remove(struct platform_device *pdev)
struct btqcomsmd *btq = platform_get_drvdata(pdev);
hci_unregister_dev(btq->hdev);
hci_free_dev(btq->hdev);
rpmsg_destroy_ept(btq->cmd_channel);
rpmsg_destroy_ept(btq->acl_channel);
hci_free_dev(btq->hdev);
}
static const struct of_device_id btqcomsmd_of_match[] = {

5
drivers/bluetooth/btrtl.c
View File

@ -591,7 +591,7 @@ static int rtlbt_parse_firmware_v2(struct hci_dev *hdev,
* headers.
*/
if (!key_id)
break;
continue;
rc = btrtl_parse_section(hdev, btrtl_dev, opcode,
ptr, section_len);
break;
@ -600,8 +600,7 @@ static int rtlbt_parse_firmware_v2(struct hci_dev *hdev,
ptr, section_len);
break;
default:
rc = 0;
break;
continue;
}
if (rc < 0) {
rtl_dev_err(hdev, "RTL: Parse section (%u) err %d",

105
drivers/bluetooth/btusb.c
View File

@ -6,6 +6,7 @@
* Copyright (C) 2005-2008 Marcel Holtmann <marcel@holtmann.org>
*/
#include <linux/cpufeature.h>
#include <linux/dmi.h>
#include <linux/module.h>
#include <linux/usb.h>
@ -980,6 +981,8 @@ struct btqca_data {
#define BTUSB_USE_ALT3_FOR_WBS 15
#define BTUSB_ALT6_CONTINUOUS_TX 16
#define BTUSB_HW_SSR_ACTIVE 17
#define BTUSB_WAKEUP_BROKEN 18
#define BTUSB_RESET 19
struct btusb_data {
struct hci_dev *hdev;
@ -1054,13 +1057,15 @@ static void btusb_reset(struct hci_dev *hdev)
int err;
data = hci_get_drvdata(hdev);
/* This is not an unbalanced PM reference since the device will reset */
err = usb_autopm_get_interface(data->intf);
if (err) {
bt_dev_err(hdev, "Failed usb_autopm_get_interface: %d", err);
return;
}
if (test_and_set_bit(BTUSB_RESET, &data->flags))
usb_autopm_put_interface_no_suspend(data->intf);
bt_dev_err(hdev, "Resetting usb device.");
usb_queue_reset_device(data->intf);
}
@ -2092,18 +2097,24 @@ static int btusb_close(struct hci_dev *hdev)
BT_DBG("%s", hdev->name);
cancel_delayed_work(&data->rx_work);
cancel_work_sync(&data->work);
cancel_work_sync(&data->waker);
skb_queue_purge(&data->acl_q);
clear_bit(BTUSB_ISOC_RUNNING, &data->flags);
clear_bit(BTUSB_BULK_RUNNING, &data->flags);
clear_bit(BTUSB_INTR_RUNNING, &data->flags);
clear_bit(BTUSB_DIAG_RUNNING, &data->flags);
btusb_stop_traffic(data);
/* rx_work must only be canceled once the URBs that can rearm it are
* gone, and it must be canceled synchronously since btusb_disconnect()
* frees the btusb_data it dereferences right after hci_unregister_dev().
*/
cancel_delayed_work_sync(&data->rx_work);
skb_queue_purge(&data->acl_q);
btusb_free_frags(data);
err = usb_autopm_get_interface(data->intf);
@ -2129,7 +2140,7 @@ static int btusb_flush(struct hci_dev *hdev)
BT_DBG("%s", hdev->name);
cancel_delayed_work(&data->rx_work);
cancel_delayed_work_sync(&data->rx_work);
skb_queue_purge(&data->acl_q);
@ -2923,8 +2934,11 @@ static int btusb_mtk_reset(struct hci_dev *hdev, void *rst_data)
}
err = usb_autopm_get_interface(data->intf);
if (err < 0)
if (err < 0) {
bt_dev_err(hdev, "Failed usb_autopm_get_interface: %d", err);
clear_bit(BTMTK_HW_RESET_ACTIVE, &btmtk_data->flags);
return err;
}
/* Release MediaTek ISO data interface */
btusb_mtk_release_iso_intf(hdev);
@ -2946,6 +2960,11 @@ static int btusb_mtk_reset(struct hci_dev *hdev, void *rst_data)
err = btmtk_usb_subsys_reset(hdev, btmtk_data->dev_id);
if (test_and_set_bit(BTUSB_RESET, &data->flags)) {
bt_dev_err(hdev, "last usb reset failed? Resetting again");
usb_autopm_put_interface_no_suspend(data->intf);
}
usb_queue_reset_device(data->intf);
clear_bit(BTMTK_HW_RESET_ACTIVE, &btmtk_data->flags);
@ -2969,10 +2988,25 @@ static int btusb_send_frame_mtk(struct hci_dev *hdev, struct sk_buff *skb)
}
}
static inline bool platform_is_ryzen(void)
{
#ifdef CONFIG_X86
return boot_cpu_has(X86_FEATURE_ZEN);
#else
return false;
#endif
}
static inline bool is_direct_child_of_root_hub(struct usb_device *udev)
{
return udev->parent == udev->bus->root_hub;
}
static int btusb_mtk_setup(struct hci_dev *hdev)
{
struct btusb_data *data = hci_get_drvdata(hdev);
struct btmtk_data *btmtk_data = hci_get_priv(hdev);
int err;
/* MediaTek WMT vendor cmd requiring below USB resources to
* complete the handshake.
@ -2989,7 +3023,40 @@ static int btusb_mtk_setup(struct hci_dev *hdev)
btusb_mtk_claim_iso_intf(data);
}
return btmtk_usb_setup(hdev);
err = btmtk_usb_setup(hdev);
if (err)
return err;
switch (btmtk_data->dev_id) {
case 0x7922:
case 0x7925:
/*
* All reports seen to be relevant to Ryzen-based laptops. These
* NICs are usually used as OEM components thanks to some sort
* of reference designs.
*
* Their popularity on other platforms is unclear. While there
* is still a chance that the quirk may exist on other
* platforms, be cautious and only apply the quirk to direct
* children of Ryzen platforms's root hubs for the time being.
*
* In most cases the root hub is on the SoC or PCH, which needs
* the quirk. Unfortunately, this can't distinguish root hubs on
* PCIe add-in cards. Such roughness should be acceptable, as
* PCIe USB controller add-in cards are less commonly used
* nowadays. On the other hand, applying the quirk doesn't hurt
* any functionalities either, as the device can still be used
* as a wakeup source if desired.
*
* Theoretically, we could retrieve the root hub's PCI vendor ID
* with some hierarchy magic, but that's too intrusive...
*/
if (platform_is_ryzen() && is_direct_child_of_root_hub(data->udev))
set_bit(BTUSB_WAKEUP_BROKEN, &data->flags);
break;
}
return 0;
}
static int btusb_mtk_shutdown(struct hci_dev *hdev)
@ -4540,6 +4607,9 @@ static void btusb_disconnect(struct usb_interface *intf)
if (data->reset_gpio)
gpiod_put(data->reset_gpio);
if (test_and_clear_bit(BTUSB_RESET, &data->flags))
usb_autopm_put_interface_no_suspend(data->intf);
if (intf == data->intf) {
if (data->isoc)
usb_driver_release_interface(&btusb_driver, data->isoc);
@ -4565,11 +4635,26 @@ static int btusb_suspend(struct usb_interface *intf, pm_message_t message)
BT_DBG("intf %p", intf);
/* Don't auto-suspend if there are connections or discovery in
* progress; external suspend calls shall never fail.
/*
* It is reported that remote wakeup events could sometimes cause some
* adapters completely unresponsive. Resetting the xHCI root hub doesn't
* help at all, and recovering from such a state needs a power cycle.
* Since disabling remote wakeup simply causes the USB core to gate
* runtime autosuspend as well due to needs_remote_wakeup == 1, let's do
* this ourselves to make our life easier. The interface can be safely
* autosuspended as long as remote wakeup is disabled, i.e., after
* closing the HCI device.
*
* Don't auto-suspend if there are connections or discovery in progress.
*
* External suspend calls shall never fail. Specifically, a device with
* broken remote wakeup may still take the advantage of remote wakeup in
* order to wake up the system from sleep if userspace has enabled it as
* a wakeup source.
*/
if (PMSG_IS_AUTO(message) &&
(hci_conn_count(data->hdev) || hci_discovery_active(data->hdev)))
((test_bit(BTUSB_WAKEUP_BROKEN, &data->flags) && data->intf->needs_remote_wakeup) ||
hci_conn_count(data->hdev) || hci_discovery_active(data->hdev)))
return -EBUSY;
if (data->suspend_count++)

21
drivers/char/virtio_console.c
View File

@ -1964,13 +1964,28 @@ static const struct file_operations portdev_fops = {
static void remove_vqs(struct ports_device *portdev)
{
struct virtqueue *vq;
bool multiport = use_multiport(portdev);
virtio_device_for_each_vq(portdev->vdev, vq) {
struct port_buffer *buf;
unsigned int len;
flush_bufs(vq, true);
while ((buf = virtqueue_detach_unused_buf(vq)))
free_buf(buf, true);
/*
* c_ovq cookies are &portdev->cpkt, not port_buffer.
* Detach them but do not free_buf().
*/
if (multiport && vq == portdev->c_ovq) {
spin_lock(&portdev->c_ovq_lock);
while (virtqueue_get_buf(vq, &len))
;
while (virtqueue_detach_unused_buf(vq))
;
spin_unlock(&portdev->c_ovq_lock);
} else {
flush_bufs(vq, true);
while ((buf = virtqueue_detach_unused_buf(vq)))
free_buf(buf, true);
}
cond_resched();
}
portdev->vdev->config->del_vqs(portdev->vdev);

6
drivers/cpufreq/cpufreq.c
View File

@ -1249,7 +1249,7 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu)
if (!policy)
return NULL;
if (!alloc_cpumask_var(&policy->cpus, GFP_KERNEL))
if (!zalloc_cpumask_var(&policy->cpus, GFP_KERNEL))
goto err_free_policy;
if (!zalloc_cpumask_var(&policy->related_cpus, GFP_KERNEL))
@ -1258,6 +1258,8 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu)
if (!zalloc_cpumask_var(&policy->real_cpus, GFP_KERNEL))
goto err_free_rcpumask;
init_rwsem(&policy->rwsem);
init_completion(&policy->kobj_unregister);
ret = kobject_init_and_add(&policy->kobj, &ktype_cpufreq,
cpufreq_global_kobject, "policy%u", cpu);
@ -1272,8 +1274,6 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu)
goto err_free_real_cpus;
}
init_rwsem(&policy->rwsem);
freq_constraints_init(&policy->constraints);
policy->nb_min.notifier_call = cpufreq_notifier_min;

10
drivers/dibs/dibs_main.c
View File

@ -251,13 +251,19 @@ static int __init dibs_init(void)
rc = class_register(&dibs_class);
if (rc)
return rc;
goto err;
rc = dibs_loopback_init();
if (rc)
pr_err("%s fails with %d\n", __func__, rc);
goto err_unregister;
return rc;
err_unregister:
class_unregister(&dibs_class);
err:
pr_err("%s fails with %d\n", __func__, rc);
return rc;
}
static void __exit dibs_exit(void)

10
drivers/edac/altera_edac.c
View File

@ -1534,7 +1534,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
altdev = dci->pvt_info;
*altdev = *device;
if (!devres_open_group(&altdev->ddev, altr_portb_setup, GFP_KERNEL))
if (!devres_open_group(device->edac->dev, altr_portb_setup, GFP_KERNEL))
return -ENOMEM;
/* Update PortB specific values */
@ -1562,7 +1562,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
rc = -ENODEV;
goto err_release_group_1;
}
rc = devm_request_irq(&altdev->ddev, altdev->sb_irq,
rc = devm_request_irq(device->edac->dev, altdev->sb_irq,
prv->ecc_irq_handler, IRQF_TRIGGER_HIGH,
ecc_name, altdev);
if (rc) {
@ -1584,7 +1584,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
rc = -ENODEV;
goto err_release_group_1;
}
rc = devm_request_irq(&altdev->ddev, altdev->db_irq,
rc = devm_request_irq(device->edac->dev, altdev->db_irq,
prv->ecc_irq_handler, IRQF_TRIGGER_HIGH,
ecc_name, altdev);
if (rc) {
@ -1604,13 +1604,13 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
list_add(&altdev->next, &altdev->edac->a10_ecc_devices);
devres_remove_group(&altdev->ddev, altr_portb_setup);
devres_remove_group(device->edac->dev, altr_portb_setup);
return 0;
err_release_group_1:
edac_device_free_ctl_info(dci);
devres_release_group(&altdev->ddev, altr_portb_setup);
devres_release_group(device->edac->dev, altr_portb_setup);
edac_printk(KERN_ERR, EDAC_DEVICE,
"%s:Error setting up EDAC device: %d\n", ecc_name, rc);
return rc;

1
drivers/gpu/drm/adp/Kconfig
View File

@ -6,7 +6,6 @@ config DRM_ADP
select DRM_KMS_HELPER
select DRM_BRIDGE_CONNECTOR
select DRM_DISPLAY_HELPER
select DRM_KMS_DMA_HELPER
select DRM_GEM_DMA_HELPER
select DRM_PANEL_BRIDGE
select VIDEOMODE_HELPERS

3
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
View File

@ -1701,9 +1701,8 @@ u64 amdgpu_bo_print_info(int id, struct amdgpu_bo *bo, struct seq_file *m)
if (dma_resv_trylock(bo->tbo.base.resv)) {
dma_resv_describe(bo->tbo.base.resv, m);
dma_resv_unlock(bo->tbo.base.resv);
} else {
seq_puts(m, "\n");
}
seq_puts(m, "\n");
return size;
}

20
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
View File

@ -191,6 +191,8 @@ amdgpu_ttm_job_submit(struct amdgpu_device *adev, struct amdgpu_ttm_buffer_entit
* @tmz: if we should setup a TMZ enabled mapping
* @size: in number of bytes to map, out number of bytes mapped
* @addr: resulting address inside the MC address space
* @vm_needs_flush: out, set true if a GART window was programmed (VMID 0 flush
* needed) or false for a direct address
*
* Setup one of the GART windows to access a specific piece of memory or return
* the physical address for local memory.
@ -200,7 +202,8 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity,
struct ttm_resource *mem,
struct amdgpu_res_cursor *mm_cur,
unsigned int window,
bool tmz, uint64_t *size, uint64_t *addr)
bool tmz, uint64_t *size, uint64_t *addr,
bool *vm_needs_flush)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->bdev);
unsigned int offset, num_pages, num_dw, num_bytes;
@ -221,9 +224,12 @@ static int amdgpu_ttm_map_buffer(struct amdgpu_ttm_buffer_entity *entity,
if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) {
*addr = amdgpu_ttm_domain_start(adev, mem->mem_type) +
mm_cur->start;
*vm_needs_flush = false;
return 0;
}
/* A GART window is programmed below, so its VMID 0 TLB needs a flush */
*vm_needs_flush = true;
/*
* If start begins at an offset inside the page, then adjust the size
@ -324,6 +330,7 @@ static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
while (src_mm.remaining) {
uint64_t from, to, cur_size, tiling_flags;
uint32_t num_type, data_format, max_com, write_compress_disable;
bool src_vm_flush, dst_vm_flush;
struct dma_fence *next;
/* Never copy more than 256MiB at once to avoid a timeout */
@ -331,12 +338,12 @@ static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
/* Map src to window 0 and dst to window 1. */
r = amdgpu_ttm_map_buffer(entity, src->bo, src->mem, &src_mm,
0, tmz, &cur_size, &from);
0, tmz, &cur_size, &from, &src_vm_flush);
if (r)
goto error;
r = amdgpu_ttm_map_buffer(entity, dst->bo, dst->mem, &dst_mm,
1, tmz, &cur_size, &to);
1, tmz, &cur_size, &to, &dst_vm_flush);
if (r)
goto error;
@ -364,7 +371,7 @@ static int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
}
r = amdgpu_copy_buffer(adev, entity, from, to, cur_size, resv,
&next, true, copy_flags);
&next, src_vm_flush || dst_vm_flush, copy_flags);
if (r)
goto error;
@ -2624,6 +2631,7 @@ int amdgpu_ttm_clear_buffer(struct amdgpu_ttm_buffer_entity *entity,
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
struct dma_fence *fence = NULL;
struct amdgpu_res_cursor dst;
bool vm_needs_flush = false;
int r;
if (!entity)
@ -2645,13 +2653,13 @@ int amdgpu_ttm_clear_buffer(struct amdgpu_ttm_buffer_entity *entity,
cur_size = min(dst.size, 256ULL << 20);
r = amdgpu_ttm_map_buffer(entity, &bo->tbo, bo->tbo.resource, &dst,
0, false, &cur_size, &to);
0, false, &cur_size, &to, &vm_needs_flush);
if (r)
goto error;
r = amdgpu_ttm_fill_mem(adev, entity,
0, to, cur_size, resv,
&next, true, k_job_id);
&next, vm_needs_flush, k_job_id);
if (r)
goto error;

8
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
View File

@ -747,9 +747,9 @@ static int amdgpu_dm_init(struct amdgpu_device *adev)
}
if (adev->dm.dc->caps.max_links > 0) {
adev->dm.hdmi_frl_status_polling_wq =
create_singlethread_workqueue("hdmi_frl_status_polling_workqueue");
create_singlethread_workqueue("hdmi_frl_status_polling_wq");
if (!adev->dm.hdmi_frl_status_polling_wq)
drm_err(adev_to_drm(adev), "failed to initialize hdmi_frl_status_polling_workqueue\n");
drm_err(adev_to_drm(adev), "failed to initialize hdmi_frl_status_polling_wq\n");
}
if (dc_is_dmub_outbox_supported(adev->dm.dc)) {
init_completion(&adev->dm.dmub_aux_transfer_done);
@ -1972,6 +1972,10 @@ static int dm_resume(struct amdgpu_ip_block *ip_block)
/* On resume we need to rewrite the MSTM control bits to enable MST*/
s3_handle_mst(ddev, false);
/* Exit IPS before the detection loop's first AUX/DDC access. */
scoped_guard(mutex, &dm->dc_lock)
dc_exit_ips_for_hw_access(dm->dc);
/* Do detection*/
drm_connector_list_iter_begin(ddev, &iter);
drm_for_each_connector_iter(connector, &iter) {

24
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_connector.c
View File

@ -756,10 +756,14 @@ amdgpu_dm_get_output_color_space(const struct dc_crtc_timing *dc_crtc_timing,
break;
case DRM_MODE_COLORIMETRY_BT2020_RGB:
case DRM_MODE_COLORIMETRY_BT2020_YCC:
if (dc_crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB)
color_space = COLOR_SPACE_2020_RGB_FULLRANGE;
else
if (dc_crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB) {
if (connector_state->hdmi.broadcast_rgb == DRM_HDMI_BROADCAST_RGB_LIMITED)
color_space = COLOR_SPACE_2020_RGB_LIMITEDRANGE;
else
color_space = COLOR_SPACE_2020_RGB_FULLRANGE;
} else {
color_space = COLOR_SPACE_2020_YCBCR_LIMITED;
}
break;
case DRM_MODE_COLORIMETRY_DEFAULT: /* ITU601 */
default:
@ -3937,17 +3941,15 @@ void amdgpu_dm_update_freesync_caps(struct drm_connector *connector,
}
/* Handle MCCS */
if (do_mccs)
if (do_mccs) {
dm_helpers_read_mccs_caps(adev->dm.dc->ctx, amdgpu_dm_connector->dc_link, sink);
if ((sink->sink_signal == SIGNAL_TYPE_HDMI_TYPE_A ||
as_type == FREESYNC_TYPE_PCON_IN_WHITELIST) &&
(!sink->edid_caps.freesync_vcp_code ||
(sink->edid_caps.freesync_vcp_code && !sink->mccs_caps.freesync_supported)))
freesync_capable = false;
if (sink->edid_caps.freesync_vcp_code && !sink->mccs_caps.freesync_supported)
freesync_capable = false;
if (do_mccs && sink->mccs_caps.freesync_supported && freesync_capable)
dm_helpers_mccs_vcp_set(adev->dm.dc->ctx, amdgpu_dm_connector->dc_link, sink);
if (sink->mccs_caps.freesync_supported && freesync_capable)
dm_helpers_mccs_vcp_set(adev->dm.dc->ctx, amdgpu_dm_connector->dc_link, sink);
}
update:
if (dm_con_state)

20
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
View File

@ -196,7 +196,6 @@ static ssize_t dp_link_settings_read(struct file *f, char __user *buf,
char *rd_buf_ptr = NULL;
const uint32_t rd_buf_size = 100;
uint32_t result = 0;
uint8_t str_len = 0;
int r;
if (*pos & 3 || size & 3)
@ -208,29 +207,26 @@ static ssize_t dp_link_settings_read(struct file *f, char __user *buf,
rd_buf_ptr = rd_buf;
str_len = strlen("Current: %d 0x%x %d ");
snprintf(rd_buf_ptr, str_len, "Current: %d 0x%x %d ",
rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
"Current: %d 0x%x %d ",
link->cur_link_settings.lane_count,
link->cur_link_settings.link_rate,
link->cur_link_settings.link_spread);
rd_buf_ptr += str_len;
str_len = strlen("Verified: %d 0x%x %d ");
snprintf(rd_buf_ptr, str_len, "Verified: %d 0x%x %d ",
rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
"Verified: %d 0x%x %d ",
link->verified_link_cap.lane_count,
link->verified_link_cap.link_rate,
link->verified_link_cap.link_spread);
rd_buf_ptr += str_len;
str_len = strlen("Reported: %d 0x%x %d ");
snprintf(rd_buf_ptr, str_len, "Reported: %d 0x%x %d ",
rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
"Reported: %d 0x%x %d ",
link->reported_link_cap.lane_count,
link->reported_link_cap.link_rate,
link->reported_link_cap.link_spread);
rd_buf_ptr += str_len;
str_len = strlen("Preferred: %d 0x%x %d ");
snprintf(rd_buf_ptr, str_len, "Preferred: %d 0x%x %d\n",
rd_buf_ptr += scnprintf(rd_buf_ptr, rd_buf_size - (rd_buf_ptr - rd_buf),
"Preferred: %d 0x%x %d\n",
link->preferred_link_setting.lane_count,
link->preferred_link_setting.link_rate,
link->preferred_link_setting.link_spread);

6
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
View File

@ -200,6 +200,7 @@ enum dc_edid_status dm_helpers_parse_edid_caps(
edid_caps->edid_hdmi = connector->display_info.is_hdmi;
if (edid_caps->edid_hdmi) {
edid_caps->qs_bit = connector->display_info.rgb_quant_range_selectable;
populate_hdmi_info_from_connector(link->dc->config.enable_frl, &connector->display_info.hdmi, edid_caps);
drm_dbg_driver(connector->dev, "%s: HDMI_FRL [%s] max_frl_rate %d\n", __func__, connector->name, edid_caps->max_frl_rate);
if (edid_caps->frl_dsc_support)
@ -1202,9 +1203,10 @@ void populate_hdmi_info_from_connector(bool enable_frl, struct drm_hdmi_info *hd
edid_caps->max_frl_rate = get_max_frl_rate(hdmi->max_lanes, hdmi->max_frl_rate_per_lane);
edid_caps->frl_dsc_support = hdmi->dsc_cap.v_1p2;
if (edid_caps->frl_dsc_support) {
if (hdmi->dsc_cap.bpc_supported == 10)
/* HF-VSDB DSC max bpc is cumulative: >=12 implies 10 and 8. */
if (hdmi->dsc_cap.bpc_supported >= 10)
edid_caps->frl_dsc_10bpc = true;
else if (hdmi->dsc_cap.bpc_supported == 12)
if (hdmi->dsc_cap.bpc_supported >= 12)
edid_caps->frl_dsc_12bpc = true;
edid_caps->frl_dsc_all_bpp = hdmi->dsc_cap.all_bpp;
edid_caps->frl_dsc_native_420 = hdmi->dsc_cap.native_420;

Some files were not shown because too many files have changed in this diff Show More